Automated hinting enhancement and engineering using ML automation in SQL query engine

Abstract

A database system generates prompts for LLMs or other machine learning (ML) models to narrow down a search space to highly relevant information about a database. Distinctively different instances of classifiers, clustering algorithms, or topic modeling models can be trained for each column or each table in a database, respectively, based on information from ML automation within the database system. The model instances can then be used to identify relevant data sources during generative LLM inference to answer a user's question. Thus, the prompt generation combines ML automation and other ML models or LLMs for topic modeling and schema description.

Description

Technical Field

[0001] This disclosure relates to the use of large language models (LLMs) in database systems and, more specifically, to the use of information extracted from machine learning automation within the database system to provide automated prompts for LLMs. Background Technology

[0002] Generative artificial intelligence (GenAI, or GAI) is the ability of generative models to generate text, images, videos, or other data, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data with similar characteristics. Generative AI can benefit a wide range of industries, including software development, healthcare, finance, entertainment, customer service, sales and marketing, art, writing, fashion, and product design.

[0003] Large Language Models (LLMs) are computational models capable of performing language generation or other natural language processing tasks. As language models, LLMs acquire these capabilities by learning statistical relationships from large amounts of text through self-supervised and semi-supervised training processes. The largest and most capable LLMs are artificial neural networks built with a decoder-only, transformer-based architecture, enabling efficient processing and generation of large-scale text data. Modern models can be fine-tuned for specific tasks or can be guided by cue engineering.

[0004] For LLM, state-of-the-art solutions involve handcrafting cues for specific tasks or use cases. For example, generating financial reports based on account activity would require a cue that cannot be reused for different use cases, such as a cue for restaurant food recommendations. In both use cases, the cue may lack configurability for dynamic details.

[0005] On the other hand, some solutions involve overgeneralized hints designed to accommodate reuse across different use cases. These hints can include all potential contextual information to handle all potential use cases by leveraging a small subset of the hints that is specific to a particular task or use case. However, using LLMs with long contexts (to accommodate multiple use cases) leads to a phenomenon known as “lost in the middle,” meaning that model performance is highest when relevant information appears at the beginning (primacy bias) or end (recency bias) of the input context, and degrades when relevant information is in the middle of the context. These examples illustrate the difficulty of leveraging generic hints applicable to all use cases. Therefore, unless used for narrow, predefined scenarios, state-of-the-art hints result in low semantic accuracy (e.g., incorrect or irrelevant information) and low task accuracy (e.g., incorrect format, type, or range of the generated output).

[0006] The methods described in this section are permissible but not necessarily methods previously conceived or adopted. Therefore, unless otherwise stated, no method described in this section should be assumed to be prior art simply because it is included here. Furthermore, no method should be considered easily understood, routine, or conventional simply because it is included here. Attached Figure Description

[0007] In the attached diagram: Figure 1 This is a block diagram illustrating a database system with an in-memory query acceleration engine according to an embodiment.

[0008] Figure 2 This is a flowchart illustrating the suggestive engineering and enhancement operations for using in-database generative AI to provide responses to natural language queries, according to an embodiment.

[0009] Figure 3 The illustration shows an example database schema according to an embodiment, which can be used to design prompts for user queries to improve the speed and accuracy of LLM generation.

[0010] Figure 4A A naive method for generating hints for an LLM, according to an embodiment, is described.

[0011] Figure 4B A method for generating prompts for an LLM based on information from ML automation, according to an embodiment, is described.

[0012] Figure 5 It is a block diagram illustrating a computer system on which various aspects of the illustrative embodiments can be implemented.

[0013] Figure 6 It is a block diagram of a basic software system that can be used to control the operation of a computer system on which various aspects of the illustrative embodiments can be implemented. Detailed Implementation

[0014] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the invention. However, it will be apparent, however, that the invention may be practiced without these specific details. In other instances, well-known structures and devices are illustrated in block diagram form to avoid unnecessarily obscuring the invention.

[0015] General Overview

[0016] Oracle® HeatWave TMThis database system is an example of a fully managed database service powered by an integrated in-memory query acceleration engine. The database service combines transaction, analytics, and machine learning services to deliver real-time, secure analytics without the complexity, latency, and cost of extract, transform, load (ETL) duplicates. Oracle® HeatWave TM Database systems also include HeatWave TM Lakehouse object repository appliance allows users to query data stored in object repository appliances in various file formats.

[0017] Oracle® HeatWave TM The database system also includes MySQL Autopilot. TM Machine Learning (ML) automation components are used to improve the performance and scalability of database systems and in-memory query acceleration engines. ML automation components address many important and often challenging aspects of achieving high query performance at scale, including feeding, data loading, query execution, and fault handling. ML automation components use advanced techniques to sample data, collect statistics about the data and queries, and build machine learning models to model memory usage, network load, and execution time. As more queries are executed, ML automation components make the in-memory query acceleration engine increasingly intelligent, thereby continuously improving system performance over time.

[0018] Oracle® HeatWave TM The database system also includes a generative artificial intelligence (AI) component, providing integrated and automated generative AI with a large in-database language model (LLM), automated in-database vector repository, horizontally scalable vector processing, and the ability to engage in contextual dialogue in natural language. Users can use the in-database LLM to help generate or summarize content based on unstructured documents. Users can ask questions in natural language via the application, and the LLM will process the requests and deliver content.

[0019] The embodiments use LLM or other machine learning (ML) models to narrow the search space to highly relevant information about the database. In one embodiment, distinct instances of a classifier, clustering algorithm, or topic modeling model can be trained for each column or table in the database, based on information from ML automation within the database system. These model instances can then be used during generative LLM inference to identify relevant data sources to answer user questions about the database. Therefore, the embodiments combine ML automation with other ML models or LLM for topic modeling and pattern description.

[0020] The embodiment uses Natural Language to SQL (NL2SQL) generation to translate a user's natural language query into an SQL query to obtain information about the database. In this embodiment, the user asks natural language questions about the database and tables, and the LLM provides responses about relevant tables and columns based on information received from the ML automation component. The embodiment uses information from the ML automation to dynamically engineer language prompts to combine the user's questions with schema, table, and column information. The user can then ask another question based on the answer generated from reasoning about the previous question. In this way, the user can ask a series of interrelated questions to investigate the database. In one example embodiment, a chatbot operates as an interactive expert providing understanding of the database.

[0021] According to embodiments, automated prompt enhancement and engineering can handle multiple use cases with high accuracy and speed. This approach uses metadata describing the database content provided by the ML automation component of the database system. Prompt engineering fully utilizes this information to generate language prompts. Information from metadata from the database and vector repository can be used together. Language prompts can also be enhanced using ML reasoning to categorize the user's question and the data sources from which the user's question should be answered.

[0022] In this embodiment, Retrieval Enhancement Generation (RAG) dynamically leverages a knowledge base of reference documents (such as technical guides and manuals) and proprietary application literature (such as functional specifications and design documents) to enhance the accuracy of language prompts. The vector repository associates each reference document with a fixed-size encoding representing the document, referred to as a reference encoding or reference embedding. A vector repository agent guides searches through the vector repository and provides context-enhanced prompts based on the search results. In one example embodiment, the vector repository agent performs a similarity search based on the document embedding and the query embedding of the user's query.

[0023] Database systems with in-memory query acceleration

[0024] Figure 1 This is a block diagram illustrating a database system with an in-memory query acceleration engine according to an embodiment. Database system 150 allows a user (such as user 105) to search database 110 or object repository 120. In this embodiment, database system 150 may be implemented as the open-source database MySQL or Oracle InnoDB. TMGeneral-purpose storage engine. MySQL is a relational database management system (RDBMS) that stores data in separate tables, rather than cramming all the data into one large repository. The database structure is organized into files optimized for speed. A logical data model with objects such as tables, views, rows, and columns provides a flexible programming environment. As the name suggests, MySQL uses the Structured Query Language (SQL), and users can directly enter SQL queries, embed SQL statements in code of another language, or use language-specific application programming interfaces (APIs) that hide SQL syntax.

[0025] In this embodiment, database system 150 includes an in-memory query acceleration component. A non-limiting example of a database system with an in-memory query acceleration engine is Oracle® HeatWave. TM Database system. Database system 150 includes an Online Transaction Processing (OLTP) component 151, an Online Analytical Processing (OLAP) component 152, an ML automation component 153, an ML modeling component 154, a hinting engineering component 155, a generative AI component 156, and a vector repository 157. OLTP is a type of data processing consisting of multiple transactions that occur concurrently. OLAP is a type of data processing used to answer multidimensional analysis (MDA) queries. OLTP component 151 allows users to run OLTP workloads on database 110, and OLAP component 152 allows users to run OLAP workloads.

[0026] ML Automation

[0027] ML Automation Component 153 analyzes data related to database operations, including information about queries, data loading, and resource utilization. It generates and analyzes intensive database data, including static information (such as schema details) and dynamic information (such as content statistics). ML Automation Component 153 also includes ML models for predicting resource usage and query performance. Therefore, ML Automation Component 153 has ample ML infrastructure that provides more or less direct access to database schemas, content statistics, and usage statistics. This integration makes ML automation suitable for implementing database ML innovation and insight models.

[0028] ML Automation Component 153 uses advanced machine learning techniques to accelerate and automate database systems 150 and in-memory queries, improving performance and scalability. A non-limiting example of an ML Automation Component is Oracle® HeatWave. TMAutopilot ML automation components. ML automation component 153 focuses on four aspects of the service lifecycle: system setup, data loading, query execution, and fault handling. ML automation component 153 includes the following capabilities: Automated provisioning predicts the number of compute nodes required to run a workload by adaptively sampling the table data to be analyzed. This means that customers no longer need to manually estimate the optimal size of the cluster.

[0029] Automatic parallel loading optimizes loading time and memory usage by predicting the optimal degree of parallelism for each table being loaded into database system 150.

[0030] Automatic data placement prediction tables should have their partitioned columns in memory to help achieve optimal query performance. It also predicts the expected performance gain from utilizing new column recommendations. This minimizes data movement across nodes caused by suboptimal choices made by operators when manually selecting columns.

[0031] Automatic encoding determines the optimal representation of the columns being loaded into database system 150, taking into account the query. This optimal representation provides best query performance and minimizes the size of the cluster to minimize cost.

[0032] Automatic query planning improves the system's performance by learning various statistics from query execution and refining the execution plan for future queries. This improves system performance as more queries are run.

[0033] Automatic query time estimation estimates the execution time of a query before it is executed. This provides a prediction of how long the query will take, allowing customers to decide if the query is taking too long and run a different query instead.

[0034] Automatic change propagation intelligently determines the optimal time to propagate changes in a MySQL database to the horizontally scaled data management layer of the database system. This helps ensure that changes are propagated at the correct, optimal pace.

[0035] Automatic scheduling identifies which queries in the queue are short-running and intelligently prioritizes them over long-running queries to reduce overall wait time. Most other database systems use a first-in, first-out (FIFO) scheduling mechanism.

[0036] If one or more database system nodes become unresponsive due to software or hardware failure, automatic error recovery will feed new nodes and reload the necessary data.

[0037] Therefore, the ML automation component 153 uses ML techniques to predict or classify system setup, data loading, query execution, and fault handling using statistical data and models. These statistical data may include, for example, user-specific workload statistics, overall workload statistics, database table statistics, query performance statistics, etc.

[0038] In-database ML model

[0039] ML Model Component 154 supports in-database machine learning (ML) to fully automate the ML lifecycle and store all trained models within a MySQL database 110, eliminating the need to move data or models to machine learning tools or services. Compared to other cloud database services, ML Model Component 154 offers the following capabilities: Fully automated model training: All different stages of model creation using ML model components 154 are fully automated and require no intervention from developers. This produces more accurate, well-tuned models, eliminates manual work, and ensures that training processing is always complete.

[0040] Model and Inference Explanation: Model interpretability helps developers understand the behavior of machine learning models. Predictive interpretability is a set of techniques that help answer the question of why a machine learning model makes a particular prediction. ML Model Component 154 integrates both model interpretation and predictive interpretation as part of its model training process.

[0041] Hyperparameter Tuning: ML Model Component 154 implements a novel gradient search-based reduction algorithm for hyperparameter tuning. This enables parallel execution of hyperparameter searches without compromising model accuracy. Hyperparameter tuning is the most time-consuming stage in ML model training, and this unique capability provides a significant performance advantage over other cloud services for building machine learning models.

[0042] Algorithm Selection: ML Model Component 154 uses the concept of a surrogate model to determine the optimal ML algorithm for training. The surrogate model is a simple model that exhibits the characteristics of a full, complex model. Using a simple surrogate model, algorithm selection is performed very efficiently without loss of accuracy.

[0043] Intelligent data sampling: During model training, the ML model component 154 samples a small subset of the data to improve performance. This sampling is done in a way that captures all representative data points in the sample dataset.

[0044] Feature selection: Feature selection helps determine the attributes of training data that influence the behavior of the machine learning model used to make predictions. The feature selection techniques used in ML model component 154 have been trained on a wide range of datasets across multiple domains and applications. Based on these collected statistics and meta-information, ML model component 154 is able to efficiently identify relevant features in new datasets.

[0045] Therefore, ML model component 154 provides the ability to train, tune, and implement ML models.

[0046] Generative AI

[0047] Generative AI component 156 is an integration platform that combines generative artificial intelligence (AI) with the existing in-memory database technology of database system 150. Specifically, generative AI component 156 is integrated with the MySQL database service. Generative AI component 156 fully leverages the in-memory architecture of database system 150 to provide efficient processing for large language models (LLMs) and the vector repository 157 that provides their generative AI capabilities.

[0048] In some embodiments, generative AI component 156 utilizes an in-database optimized LLM to instantly benefit from generative AI and employs natural language for contextual dialogue based on information provided by unstructured documents. Generative AI component 156 can achieve more accurate and context-sensitive answers by having the LLM search proprietary documents without AI expertise or moving data to a separate vector database. Vector repository 157 is integrated and automates embedding generation. Generative AI component 156 uses data from object storage device 120 and MySQL database 110 to build, train, and interpret ML models.

[0049] Vector repository

[0050] With support for generative AI, user 105 can interact with database system 150 using natural language. Both user queries and responses from the system can be generated in natural language using a large language model (LLM). In some embodiments, the LLM is trained on public data, and for organizations seeking to fully leverage the capabilities of the LLM for their enterprise data, the results can be inaccurate due to the illusion problem of the LLM and the lack of enterprise knowledge. To mitigate this problem, database system 150 includes a vector repository 157.

[0051] Vector repository 157 uses a language encoder to create vector embeddings from documents that can be stored in various formats. Vector repository 157 also takes the user's question to create vector embeddings and performs a similarity search in n-dimensional space. The output of the vector repository is context, which is included in a prompt along with the user's question, serving as input to the LLM. The LLM uses this information to generate a response that now includes proprietary information from documents in object repository 120.

[0052] In one embodiment, vector repository 157 represents each fragment of a document in object repository 120 as a vector that can be stored as rows in a database table. For example, each fragment can be a paragraph in a document, and the vector can include a document identifier, author, publication date, section identifier, page number, paragraph offset on the page, and the encoding of the fragment's text (also referred to herein as an embedding). Thus, for a given document, the rows of fragments for that given document will have unique embeddings for that fragment; however, the document identifier (or document embedding) will be the same. This allows the system to filter by document, date, author, etc., and then perform similarity searches on the fragments. The way fragments are represented can vary depending on the implementation. In one embodiment, the vector representation is stored in database 110. Vector repository 157 provides very fast searching of unstructured data by providing embeddings searched by similarity scores rather than pattern matching, which can be slow and resource-intensive. In some embodiments, fragments from vector repository 157 can be provided as context in prompts.

[0053] Tips for engineering and enhancement

[0054] In one embodiment, an LLM or other machine learning (ML) model is used to narrow the search space down to highly relevant information about the database. In one embodiment, distinct instances of a classifier, clustering algorithm, or topic modeling model can be trained for each column or table in the database, based on information from the ML automation component 153. The prompting engineering component 155 then uses the model instances during generative LLM inference to identify relevant data sources to answer natural language queries from user 105 about database 110.

[0055] In this embodiment, database system 150 is an elastic cloud database service, and ML automation component 153 analyzes data related to database operations, including information about queries, data loading, and resource utilization. In other words, ML automation component 153 generates and analyzes intensive data about database 110, including static information (such as schema details) and dynamic information (such as content statistics). ML automation component 153 uses machine learning models to predict resource usage and query performance. Therefore, ML automation component 153 has sufficient machine learning infrastructure that has more or less direct access to database schema, content statistics, and usage statistics. This integration makes ML automation component 153 suitable for enabling database machine learning innovation and insight models for suggestive engineering and enhancement.

[0056] The hint engineering component 155 leverages knowledge about the database, such as schema, table and column names, and aggregated column or table statistics, to improve the inference speed and accuracy of generative AI (e.g., LLM) 156 by designing and enhancing language hints provided to the LLM. A vector repository 157 of metadata is used to dynamically tailor hints and LLM responses for specific business lines and specific summarization tasks. Aggregated statistics (e.g., range, mean, null values, cardinality) and descriptions of tables and columns are collected and used for hint engineering. This approach eliminates the need for users to specify relevant tables and columns. For example, users may not know or understand the database schema or Structured Query Language (SQL), which is not an obstacle for users conducting database investigations in this paper. This approach accelerates LLM inference because it is not necessary to explore all tables during inference. This improves accuracy because irrelevant information is not reflected in the final hint to the LLM, thus solving the "lost in the middle" problem inherent in LLMs using hints with long contexts.

[0057] In this embodiment, the prompting engineering component 155 uses (1) information from the ML automation component 153, and (2) a database description, such as that inferred from the ML automation information by the insight model, to generate natural language to SQL (NL2SQL), which improves the accuracy of analysis and summarization as well as user ergonomics. State-of-the-art NL2SQL technology requires multiple examples and comprehensive (i.e., exhaustive and formal) schema information to generate correct SQL. Utilizing metadata from the ML automation component 153 eliminates or reduces the requirement to provide formal schema information or examples. Instead, the prompting engineering component 155 can dynamically generate or enhance language prompts to combine the user's questions with schema, table, and column information.

[0058] By fully leveraging the information contained in the metadata of the ML automation component 153, user interaction with the data using natural language is facilitated. User 105 can ask natural language questions about the database and tables, and due to the ML automation integration, the LLM can provide responses about the relevant tables and columns. User 105 can then ask another question based on the answer generated from reasoning about the previous question. In this way, user 105 can ask a series of interrelated questions to investigate the database 110. In an example embodiment, the generative AI component 156 may include a chatbot that operates as an interactive expert providing an understanding of the database 110.

[0059] Some embodiments provide automated prompt enhancement and engineering that can handle multiple use cases with high accuracy and speed. The prompt engineering component 155 utilizes metadata describing the content of a relational database provided by the ML automation component 153. The prompt engineering component 155 fully leverages this information to generate language prompts. Information from the metadata of the database and the vector repository 157 can be used together. ML inference can be used to further enhance the language prompts to categorize the user's questions and the data sources from which the user's questions should be answered.

[0060] Enhanced prompts can be particularly important when interacting with data, such as in Natural Language to SQL (NL2SQL) use cases. In those cases, user 105 presents a natural language question (referred to herein as an interactive question) to database system 150, intending to extract information from structured or unstructured data. For example, a manager in an organization might have questions about multiple employees, their individual salaries, and the date of each person's last raise. This would require retrieving information from at least three separate tables. Furthermore, only a few columns need to be selected from those tables. After automatically and dynamically discovering pattern data, the pattern data must be specified in the language prompt to generatively infer a natural language response (i.e., the answer) to be returned to the user. Needless to say, this is a complex task even for a single use case. Therefore, flexible automation of this task is crucial.

[0061] Retrieval Enhancement Generation (RAG) dynamically leverages a knowledge base of reference documents (such as technical guides and manuals) and proprietary application literature (such as functional specifications and design documents) to enhance the accuracy of language prompts. In the preparation phase before the generative AI component 156 is serviced, a knowledge index of existing structured (e.g., JSON, XML, and HTML) and unstructured (e.g., prose, word processor) reference documents is created in a vector repository 157 or an indexed database. Whether structured or unstructured, reference documents can contain natural language, such as multi-word terms, phrases, sentences, and paragraphs, either partially or entirely. Each reference document has a fixed-size dense semantic encoding, which, for example, can be inferred by a trained encoder model that uses natural language processing (NLP) to accept input documents as sequences of lexical tokens. For example, the encoder model can be a large language model (LLM), such as a bidirectional encoder representation (BERT) from a transformer. In an embodiment, the vector repository 157 associates each reference document with a fixed-size encoding representing that document (referred to herein as a reference encoding).

[0062] Following the preparation phase is the dialogue phase, in which the generative AI component 156 performs the service. In one embodiment, the dialogue phase is performed by a vector repository agent, which is a program or process that uses content retrieval to enhance analytics, decision-making, and / or task reasoning. In one embodiment, the vector repository 157 and the prompting engineering component 155 are parts of the vector repository agent. The dialogue phase includes a search phase using the vector repository 157, followed by the generative phase. The vector repository agent operates the search phase, and then operates the prompting engineering, which is an early part of the generative phase. Subsequently, the generative model operates the later parts of the generative phase.

[0063] The search phase can utilize instances of classifiers, clustering algorithms, or topic modeling models within the ML model component 154 to provide information about the user's question and the corresponding information about each column or table in the database. These ML models 154, referred to herein as insight models, help the vector repository agent determine which tables and columns are relevant to (i.e., implied by) the user's question. For example, the question model might infer that the user's question contains geographic names, and the column model might infer which table columns contain geographic names. The inferred output of any of those models is called a pattern insight, or, if it comes from the question model, a question insight. The combination of the user's question and all the inferred insights is called the search key.

[0064] The search phase can lead to the following sequence: a) generating a sequence of lexical tokens representing the search key; b) an encoder model taking the token sequence as input; c) the encoder model inferring a fixed-size encoding representing the token sequence, referred to herein as the search encoding; d) a knowledge index taking the search encoding as the lookup key; and d) the knowledge index selecting and returning one or more reference documents represented by the closest (i.e., most semantically similar to the search encoding) of one or more already stored reference encodings. In this embodiment, (d) is accelerated because the search key limits the scope of the search for matching reference documents.

[0065] Similarity can be measured using semantic distance, such as multidimensional spatial vector distance, e.g., Euclidean or Manhattan distance. For example, knowledge indexes can implement nearest neighbor search. The output of the search phase is a dynamically selected set of highly relevant (i.e., semantically similar) reference documents, referred to herein as matching documents, because they semantically match the search key. For example, matching documents can be ranked (i.e., sorted) by a similarity score such as a measured semantic distance, and this score (i.e., distance) is based on a comparison of the reference encoding with the search encoding, which does not result in access to the reference document represented by the reference encoding. However, alternative embodiments may not require ranking.

[0066] Figure 1 Generative models such as LLMs are referred to as generative AI components.156 Generative models are not encoder models. Figure 1 (Not shown in the text), but both can be LLMs, for example. The generative phase leads in sequence to: a) generating language hints based on search keys and matching documents (e.g., proxies by a vector repository), b) the generative model accepting the language hints as input, and c) the generative model inferring (i.e., generating) natural language. When inferring answers to user questions, the generative model can perform summarization or recommendation.

[0067] Figure 2 This is a flowchart illustrating the prompting engineering and enhancement operations of using in-database generative AI to provide responses to natural language queries according to an embodiment. The operation begins (box 200), and the database system receives a natural language query from the user (box 201). The database system retrieves metadata about the data source and the user from an ML automation component (box 202). The data source can be a relational database comprising one or more database tables or an object repository comprising one or more vector repositories representing multiple documents using semantic encoding. The ML automation component can include an ML model for predicting resource usage and query performance.

[0068] The database system then generates language hints for the generative AI model (e.g., LLM) based on the natural language query and metadata received from ML automation (box 203). In one embodiment, the generative AI model includes a large language model (LLM), and the natural language query is a query about a relational database.

[0069] In another embodiment, the generative AI model includes a Natural Language to Structured Query Language (NL2SQL) generative model, and the NL2SQL generative model is configured to generate one or more SQL queries based on language prompts to search a relational database. In some embodiments, the database system provides language prompts as input to the NL2SQL generative model to generate SQL queries. The database system can then execute SQL queries on one or more database tables to generate search results or can make the SQL queries displayed to a user. In one embodiment, the database system can provide search results as input to a large language model to generate a natural language description of the search results.

[0070] In another embodiment, the database system can generate language hints by selecting a subset of database tables at least partially based on metadata and add a schema description for that subset of database tables to the language hints. In one embodiment, the database system adds schema descriptions by generating query embeddings based on natural language queries and selecting a subset of database tables at least partially based on the similarity between one or more query embeddings and table-by-table topic modeling embeddings of the one or more database tables.

[0071] In another embodiment, the data source is an object repository that includes one or more vector repositories representing multiple documents using semantic encoding, and the database system generates language hints by filtering one or more vector repositories based on metadata, as discussed above.

[0072] The database system uses a generative AI model to generate responses based on language prompts and searches of data sources (box 204). The language prompts limit the scope of the search of the data source based on metadata. In one embodiment, the database system generates zero or more queries and executes them to extract relevant data. During this search phase, ML automation, such as Oracle® HeatWave, can be used. TM Autopilot ML automation component.

[0073] The database system responds to the user in natural language (box 205). In some embodiments, the database system is designed to prompt the user with data retrieved during a search of the data source in natural language. The operation then ends (box 206).

[0074] In one embodiment, the following pseudocode can implement this generative phase and can be implemented by, for example... Figure 1 The prompt shown is for project component 155 or is executed by the vector repository agent: 1. Classification must be queried to provide the user with one or more sources of data for the response. For this step, either an ML model (e.g., a classification or clustering model) or LLM inference (e.g., N-shot classification) can be utilized. Example: "Which of the following tables {tables extracted by Autopilot} would be useful in responding to this question {user's question}?" Response: {List of relevant tables} 2. Utilize the LLM model to identify the relevant tables and columns that must be queried to obtain relevant information. Table and column information comes from ML automation, which is aware of the table schema and aggregated statistics for each column. This information can be easily used to enhance cues for the LLM, enabling the extraction of relevant information. Furthermore, the same information can be used to structure cues for LLM usage to generate natural language responses to the user. Example: “From the following table {list of relevant tables}, which columns are relevant to the response to this question {user's question}?” Response: {list of columns}, for example, a list of columns from table A, a list of columns from table B, etc.

[0075] 3. Based on the determination in step 2, generate queries (or multiple queries) to extract relevant information from columns in the table or database. Response: {Extracted data}

[0076] 4. Design prompts to respond back to the user in natural language using the extracted data. Example: "Given the following information {extracted data} is extracted from these columns {list of columns} contained in these tables {list of related tables}, provide an informative response to the following question {user question}."

[0077] The pseudocode above is dynamic and contextual, which increases accuracy. The pseudocode contains placeholders where: a) the engineering component 155 is prompted to use dynamically acquired contextual information, and b) some of this context becomes available and a response containing some of this context (such as insights inferred from the insight model or query results from one or more databases). For example, a user's question may involve multiple databases, and this can be used to infer an answer to that question. That is, the generative phase can inferentially answer interactive questions about multiple databases, even those that a naive user might not be aware of.

[0078] Accurate inferences based on dynamic context help users quickly understand the database. This assisted automation of ergonomics is unprecedented. Here, ergonomics is a quantitative performance metric that can be based on one, some, or all of the following: a) the time spent reading and understanding the inferred answers, and b) the amount of data and metadata accessed during the generative phase. Both of these metrics, as well as the latency of the generative phase, are reduced through this generative AI application because one or more language cues increase the relevance (i.e., accuracy) of significantly different LLMs.

[0079] In various embodiments, the insight model, encoder model, and generative model may or may not all be LLMs. Such LLMs accept distinctly different dynamically generated language prompts as input, referred to herein as insight prompts, encoder prompts, and generative prompts, respectively. For example, with encoder and generative prompts, the database is explored more efficiently when prompt enhancement and generation are connected to ML automation and used with one or more insight prompts. In one use case, an insight prompt may ask natural language questions about the data to gather information about which tables and columns are relevant to the user's question. In this way, the generative model returns responses about database artifacts such as schemas, tables, and columns without requiring the generative model itself to query those artifacts or actually interact with the database data. For example, this can help database users quickly gain and understand a high-level view of the database and how to formulate strategies or solve problems that arise in the application domain.

[0080] Steps 1 and 2 generate insightful hints for internal operations to reduce the overall search space, making the generative model more accurate and faster. Step 3 is optional and generates database queries to retrieve relevant data from or about: a) relational databases, and b) structured or semi-structured data from object repositories, such as comma-separated value (CSV) spreadsheets, Apache Parquet, etc. TM Software and storage format, or Apache Avro TM Software and storage format, or c) unstructured (e.g., natural language) data from Vector Repository 157. Pseudocode step 4 generates a generative hint, based on which the generative model infers an answer to the user's question. Due to the accuracy of the generative hint, when the answer is subsequently inferred, it will contain highly accurate natural language, such as multi-word terms, phrases, sentences, and paragraphs.

[0081] Example of a project

[0082] The ML automation component 153 understands the user's workload by relying on machine learning models to make predictions that help the database system 150 automate supply, shape prediction, data placement, query optimization, and query execution. The ML automation component 153 samples data to obtain features and predicts the necessary parameters for optimization. Data sampling allows the ML automation component 153 to obtain accurate statistics for all columns in each table, in addition to schema and entity relationship knowledge. Workload knowledge, along with schema and table information, can significantly improve retrieval and generation accuracy and speed.

[0083] According to an embodiment, the ML automation component 153 improves natural language interaction with data by incorporating workload knowledge from the ML model into the prompting process. This knowledge may include: user workload awareness; knowledge of tables, schemas, and column-by-column statistics; and the user's previous interactions with data in different tables. This can be applied to relational databases 110 (such as Oracle InnoDB). TM Structured files in a MySQL database or Oracle Cloud Data Lakehouse, or unstructured files applied to an object repository 120 via vector repository 157. These enhancements improve the user experience, accuracy, and speed of data retrieval through enhanced generation and NL2SQL generation. They achieve faster and more accurate data retrieval by retrieving data only from relevant vector repositories and tables, by selecting tables, and by producing more accurate and user-friendly results through patterns in NL2SQL based on past user behavior.

[0084] Figure 3The illustration depicts an example database schema according to an embodiment, upon which suggestions can be designed for user queries to improve the speed and accuracy of LLM generation. This schema includes six tables: Department Employees (dept_emp) 310, Salaries (salaries) 320, Departments (departments) 330, Employees (employees) 340, Department Managers (dept_manager) 350, and Titles (titles) 360. During the operation of the database system 150, the ML automation component 153 can perform automated functions such as data placement, query plan improvement, and query time estimation. The ML automation component 153 performs these functions by learning the database schema, user load statistics, and performance statistics. Therefore, in the example implementation, the ML automation component 153 knows that the Employees table 340 is accessed 80% of the time, the Dept_emp table 310 is accessed 10% of the time, the Salaries table 320 is accessed 8% of the time, and the remaining tables combined account for 2% of the access time. Therefore, for example, ML automation component 153 can infer that the query is highly likely to access the employee table 340 and very unlikely to access the department table 330.

[0085] Consider the following natural language query submitted by user 105: How many female employees who joined after January 2020 earned between $200,000 and $220,000? Given Figure 3 Based on the database schema shown, it can be inferred that the query will require at least the employee table 340 and the salary table 320. Furthermore, the query will access the gender and hire date columns of the employee table 340, and the salary column of the salary table 320.

[0086] Figure 4A A naive approach to generating hints for LLM, based on an embodiment, is described. This approach uses a template that begins with "Generate an SQL query, providing the following tables and schema to answer this question:", in which the user's query and database schema are inserted. Without knowing which tables and columns might be accessed, the tables and schema include all tables in the database schema. Note that in a practical implementation, a complete schema description will be included, including columns, data types, lengths, indexes, etc. This results in very long contexts. As previously mentioned, LLM encounters a long context problem known as "lost in the middle." Due to primacy and recency bias, LLM performance degrades when relevant information is in the middle of the context. Including schema descriptions of tables and columns that are unlikely to be accessed increases the likelihood that LLM will provide results with low semantic accuracy and low task accuracy.

[0087] However, since the ML automation component 153 knows the tables most frequently used by the user or all users, the engineering component 155 can provide schemas for only the three or four most frequently used tables instead of all tables. Figure 4B A method for generating prompts for an LLM based on information from ML automation, according to an embodiment, is described. In this case, the context in the prompt is only... Figure 4A The length of the naive method shown is half that of the LLM. Furthermore, only relevant tables and schema descriptions will be provided to the LLM, making it faster and more accurate.

[0088] Furthermore, more complex databases can have dozens or even hundreds of tables. Such a large number of tables and columns makes the prompt size very large. The accuracy of LLM decreases with large prompt sizes because, as mentioned above, LLM loses the context of information embedded in the prompt. Typically, LLM is only good at extracting information from the beginning or end of the prompt, and if relevant information is in the middle of the prompt, a significant amount of accuracy can be lost. Moreover, for large prompts, the prompt processing and generation times become excessively long, making the model less user-friendly. In some use cases, prompts can exceed the maximum allowed number of input tokens, resulting in lost information due to truncation or even truncated output. This will render the generated response unusable.

[0089] In some embodiments, in addition to the most frequently used tables and columns, some tables can be added or removed based on topic modeling. In this approach, the user's input query embedding is matched against a table-by-table topic modeling embedding. Tables with high similarity to the user's query (e.g., cosine similarity score) can be added to the suggestions. Tables with low similarity can be discarded from the suggestions to further refine the suggestion engineering.

[0090] In this embodiment, the database system uses NL2SQL based on... Figure 4B The database system generates one or more SQL queries based on the prompts. In one embodiment, the database system may return the SQL queries to the user for review and editing. Alternatively, the database system may execute the SQL queries and generate query results. In one embodiment, the database system may return the query results to the user. In another embodiment, the database system then generates prompts for the LLM, which include the user's query and the query results as context. The LLM generates a natural language response that interprets the query results as an answer to the user's query.

[0091] Therefore, users can ask questions about one or more databases. For example, a user might ask which tables must be queried to answer a specific question. In this case, the user may not be particularly concerned with the answer to the question itself, but rather with the database schema. Thus, the user might be skilled at generating SQL queries but unfamiliar with a particular database. In another example, a user might ask what a query to answer a given question would look like. Again, the user might not care about the answer to the query, but rather want to explore the structure and syntax of SQL queries. In this case, the user might be new to SQL and learning to generate SQL queries by exploring examples using a specific database. Furthermore, users and database systems can engage in a back-and-forth dialogue, where the user asks questions based on answers previously provided by the database system.

[0092] DBMS Overview

[0093] A Database Management System (DBMS) manages databases. A DBMS may include one or more database servers. A database consists of database data and a database dictionary, stored on a persistent storage mechanism such as a collection of disks. Database data can be stored in one or more collections of records. The data within each record is organized into one or more attributes. In a relational DBMS, collections are called tables (or data frames), records are called records, and attributes are called attributes. In a document DBMS (“DOCS”), a collection of records is a collection of documents, each of which can be a data object marked up using a hierarchical markup language, such as a JSON object or an XML document. Attributes are called JSON fields or XML elements. Relational DBMSs can also store hierarchically marked up data objects; however, these data objects are contained within attributes of records, such as attributes of JSON type.

[0094] Users interact with the DBMS database server by submitting commands to it. These commands cause the database server to perform operations on the data stored in the database. A user can be one or more applications running on a client computer that interact with the database server. Multiple users may also be collectively referred to as "users" throughout this document.

[0095] Database commands can take the form of database statements conforming to a database language. The database language used to express database commands is Structured Query Language (SQL). SQL has many different versions; some are standard, some are proprietary, and there are various extensions. Data Definition Language (“DDL”) commands are issued to the database server to create or configure data objects, referred to herein as database objects, such as tables, views, or complex data types. SQL / XML is a common extension of SQL used when manipulating XML data in object-relational databases.

[0096] In a DBMS, changes to the database are made through transactions. A database transaction is a collection of operations that modify database data. In a DBMS, a database transaction is initiated in response to database commands that request changes, such as DML commands requesting updates, inserts, or deletes records, or CRUD object method calls requesting the creation, updating, or deletion of documents. DML commands and DDL specify changes to the data, such as INSERT and UPDATE statements. DML statements or commands are not necessarily statements or commands that only query the database. Committing a transaction makes the changes made permanent.

[0097] In transaction processing, all changes made to a transaction are performed atomically. When a transaction is committed, either all changes are committed, or the transaction is rolled back. These changes are recorded in a change record, which may include redo records and undo records. Redo records can be used to reapply changes made to data blocks. Undo records are used to reverse or undo changes made to data blocks by a transaction.

[0098] Examples of this type of transaction metadata include change logs that record the changes a transaction makes to database data. Another example of transaction metadata is embedded transaction metadata stored within the database data, which describes the transactions that modify the database data.

[0099] Undo records are used to provide transaction consistency by performing operations referred to herein as consistency operations. Each undo record is associated with a logical time. An example of a logical time is a System Change Number (SCN). For example, an Lamporting mechanism can be used to maintain SCNs. For a data block being read to compute a database command, the DBMS applies the required undo records to a copy of the data block to bring the copy to a state consistent with the snapshot time of the query. The DBMS determines which undo records to apply to the data block based on the corresponding logical time associated with the undo records.

[0100] In distributed transactions, multiple DBMSs use a two-phase commit method to commit the distributed transaction. Each DBMS executes its local transaction within a branch of the distributed transaction. A coordinating DBMS is responsible for coordinating the commit of transactions on one or more other database systems. These other DBMSs are referred to as participating DBMSs in this document.

[0101] Two-phase commit involves two phases: the prepare-to-commit phase and the commit phase. In the prepare-to-commit phase, branch transactions are prepared in each participating database system. When a branch transaction is prepared on a DBMS, the database is in a "ready state," ensuring that modifications made to the database data as part of the branch transaction can be committed. This guarantee may require persistent storage of the change records used for the branch transaction. Participating DBMSs confirm that they have completed the prepare-to-commit phase and have entered the ready state for the corresponding branch transaction in their respective participating DBMSs.

[0102] During the commit phase, the coordinating database system commits transactions on both the coordinating database system and the participating database systems. Specifically, the coordinating database system sends a message to the participants requesting them to commit the modifications specified by the transactions on the data in the participating database systems. Then, the participating and coordinating database systems commit the transactions.

[0103] On the other hand, if the participating database system is unable to prepare or the coordinating database system is unable to commit, then at least one database system cannot make the changes specified by the transaction. In this case, all modifications at every point in the participating and coordinating database systems are rolled back, thus restoring each database system to its state before the changes.

[0104] A client can send a series of requests to a DBMS, such as requests to execute queries, by establishing a database session. A database session includes a specific connection established between the client and the database server, through which the client can send a series of requests. Database session processes execute within the database session and process the requests sent by the client through the database session. The database session can generate an execution plan for the queries sent by the database session client and marshal subordinate processes to execute that execution plan.

[0105] Database servers can maintain session state data about database sessions. Session state data reflects the current state of the session and may include the identity of the user who created the session, the services used by the user, instances of object types, language and character set data, statistics on resource usage of the session, temporary variable values ​​generated by the processes executing the software within the session, and storage of cursors, variables, and other information.

[0106] A database server comprises multiple database processes. These processes run under the control of the database server (i.e., they can be created or terminated by the database server) and perform various database server functions. Database processes include those that run within the database session established for the client.

[0107] A database process is a unit of execution. A database process can be a computer system process, a thread, or a user-defined execution context, such as a user thread or fiber. Database processes can also include "database server system" processes that provide services and / or perform functions on behalf of the entire database server. Such database server system processes include listeners, garbage collectors, log writers, and recovery processes.

[0108] A multi-node database management system consists of interconnected computing nodes (“nodes”), each running a database server that shares access to the same database. Typically, nodes are interconnected via a network and share access to shared storage devices to varying degrees, such as a collection of disk drives and the blocks of data stored on them. Nodes in a multi-node database system can be in the form of a group of computers (e.g., workstations, personal computers) interconnected via a network. Alternatively, nodes can be nodes in a grid, where the grid consists of nodes in the form of server blades interconnected with other server blades on a rack.

[0109] In a multi-node database system, each node hosts a database server. A server, such as a database server, is a combination of integrated software components and computing resources, such as memory, nodes, and processes on the nodes used to execute the integrated software components on the processor. This combination of software and computing resources is dedicated to performing a specific function on behalf of one or more clients.

[0110] Resources from multiple nodes in a multi-node database system can be allocated to run the software of a specific database server. Each combination of software and the allocation of resources from the nodes constitutes a server, referred to herein as a "server instance" or "instance". A database server may include multiple database instances, some or all of which may run on separate computers, including separate server blades.

[0111] A database dictionary can include multiple data structures that store database metadata. For example, a database dictionary can include multiple files and tables. Parts of the data structures can be cached in the database server's main memory.

[0112] When a database object is defined by a database dictionary, the database dictionary contains metadata that defines the attributes of the database object. For example, the metadata in the database dictionary defining a database table may specify attribute names and data types of the attributes, as well as one or more files or portions thereof for storing the data of the table. The metadata in the database dictionary defining a procedure may specify the procedure name, the procedure's arguments, and the return data type and data types of the arguments, and may include the source code and its compiled version.

[0113] Database objects can be defined by a database dictionary, but the metadata in the database dictionary itself may only partially specify the attributes of the database object. Other attributes can be defined by data structures that are not considered part of the database dictionary. For example, a user-defined function implemented in a Java class can be partially defined by the database dictionary by specifying the name of the user-defined function and by specifying a reference to the file containing the Java class source code (i.e., the java file) and the compiled version of the class (i.e., the class file).

[0114] Primitive data types are data types that a DBMS supports "out of the box." On the other hand, non-primitive data types may not be supported by the DBMS out of the box. Non-primitive data types include user-defined abstract types or object classes. Non-primitive data types are only recognized and processed in DBMS database commands after they have been defined in the DBMS's database dictionary, for example, by issuing a DDL statement defining the non-primitive data type to the DBMS. Primitive data types do not need to be defined in the database dictionary to be recognized as valid data types and processed by the DBMS in database statements. Generally, DBMS database software is programmed to recognize and process primitive data types, rather than configuring the DBMS to do so by issuing DDL statements to define data types.

[0115] Hardware Overview

[0116] According to one embodiment, the techniques described herein are implemented by one or more dedicated computing devices. The dedicated computing device may be hardwired to execute these techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs) persistently programmed to execute these techniques, or may include one or more general-purpose hardware processors programmed to execute these techniques according to program instructions in firmware, memory, other storage devices, or a combination thereof. Such a dedicated computing device may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to implement these techniques. The dedicated computing device may be a desktop computer system, a portable computer system, a handheld device, a networking device, or any other device that combines hardwired and / or program logic to implement these techniques.

[0117] For example, Figure 5 This is a block diagram illustrating a computer system 500 on which various aspects of the illustrative embodiments may be implemented. The computer system 500 includes a bus 502 or other communication mechanism for transmitting information, and a hardware processor 504 coupled to the bus 502 for processing information. The hardware processor 504 may be, for example, a general-purpose microprocessor.

[0118] Computer system 500 also includes main memory 506 (such as random access memory (RAM) or other dynamic storage device) coupled to bus 502 for storing information and instructions to be executed by processor 504. Main memory 506 can also be used to store temporary variables or other intermediate information during the execution of instructions executed by processor 504. When these instructions are stored in non-transitory storage media accessible to processor 504, computer system 500 becomes a dedicated machine customized to perform the operations specified in the instructions.

[0119] Computer system 500 also includes a read-only memory (ROM) 508 or other static storage device coupled to bus 502 for storing static information and instructions for processor 504. Storage device 510 (such as a disk, optical disk, or solid-state drive) is provided and coupled to bus 502 for storing information and instructions.

[0120] Computer system 500 can be coupled to display 512 (such as a cathode ray tube (CRT)) via bus 502 for displaying information to the computer user. Input device 514, including alphanumeric keys and other keys, is coupled to bus 502 for transmitting information and command selections to processor 504. Another type of user input device is cursor control 516 (such as a mouse, trackball, or arrow keys) for transmitting directional information and command selections to processor 504 and for controlling cursor movement on display 512. Such input devices typically have two degrees of freedom on two axes (a first axis (e.g., x) and a second axis (e.g., y)), allowing the device to specify a position in a plane.

[0121] Computer system 500 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware and / or program logic (which, in conjunction with the computer system, enable or program the computer system 500 as a special-purpose machine). According to one embodiment, computer system 500 performs the techniques described herein in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. These instructions may be read into main memory 506 from another storage medium, such as storage device 510. Execution of the sequence of instructions contained in main memory 506 causes processor 504 to perform the processing steps described herein. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions.

[0122] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such storage media can include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs, magnetic disks, or solid-state drives, such as storage device 510. Volatile media include dynamic memory, such as main memory 506. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs, NVRAMs, any other memory chips, or magnetic tape cassettes.

[0123] Storage media differ from transmission media but can be used in conjunction with them. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including conductors containing bus 502. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0124] Various forms of media can be used to carry one or more sequences of one or more instructions to processor 504 for execution. For example, instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit them over a telephone line using a modem. A modem local to computer system 500 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector may receive the data carried in the infrared signal, and appropriate circuitry may place the data on bus 502. Bus 502 transfers the data to main memory 506, from which processor 504 retrieves and executes the instructions. Instructions received by main memory 506 may optionally be stored on storage device 510 before or after execution by processor 504.

[0125] Computer system 500 also includes a communication interface 518 coupled to bus 502. Communication interface 518 provides bidirectional data communication coupled to network link 520, which is connected to local network 522. For example, communication interface 518 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem providing data communication connectivity with a corresponding type of telephone line. As another example, communication interface 518 may be a Local Area Network (LAN) card to provide data communication connectivity with a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 518 transmits and receives electrical, electromagnetic, or optical signals carrying streams of digital data representing various types of information.

[0126] Network link 520 typically provides data communication to other data devices via one or more networks. For example, network link 520 may provide a connection via local network 522 to host computer 524 or to data devices operated by Internet Service Provider (ISP) 526. ISP 526 then provides data communication services via a global packet data communication network (now commonly referred to as the "Internet" 528). Both local network 522 and Internet 528 use electrical, electromagnetic, or optical signals carrying digital data streams. Signals through various networks, as well as signals on network link 520 and through communication interface 518 (which carries digital data to and from computer system 500), are example forms of transmission media.

[0127] Computer system 500 can send messages and receive data, including program code, through one or more networks, network links 520, and communication interfaces 518. In the Internet example, server 530 can send requested code to the application through the Internet 528, ISP 526, local network 522, and communication interface 518.

[0128] The received code can be executed by processor 504 upon receipt and / or stored in storage device 510 or other non-volatile memory for later execution.

[0129] Software Overview

[0130] Figure 6 This is a block diagram of a basic software system 600 that can be used to control the operation of a computer system 500 on which aspects of the illustrative embodiments can be implemented. The software system 600 and its components (including their connections, relationships, and functions) are merely exemplary and are not intended to limit the implementation of one or more of the example embodiments. Other software systems suitable for implementing one or more example embodiments may have different components, including components with different connections, relationships, and functions.

[0131] Software system 600 is provided to guide the operation of computer system 500. Software system 600, which may be stored on system memory (RAM) 506 and fixed storage devices (e.g., hard disk or flash memory) 510, includes a kernel or operating system (OS) 610.

[0132] OS 610 manages low-level aspects of computer operations, including managing process execution, memory allocation, file input and output (I / O), and device I / O. One or more applications, designated 602A, 602B, 602C...602N, can be "loaded" (e.g., transferred from fixed storage device 510 to memory 506) for execution by system 600. Applications or other software intended for use on computer system 500 can also be stored as downloadable computer-executable instruction sets, for example, for downloading and installing from Internet locations (e.g., web servers, app stores, or other online services).

[0133] Software system 600 includes a graphical user interface (GUI) 615 for receiving user commands and data graphically (e.g., "clicks" or "touch gestures"). These inputs can then be manipulated by system 600 according to instructions from operating system 610 and / or (one or more) applications 602. GUI 615 also displays the results of operations from OS 610 and (one or more) applications 602, allowing the user to provide additional input or terminate the session (e.g., log off).

[0134] OS 610 can execute directly on the bare hardware 620 of computer system 500 (e.g., one or more processors 504). Alternatively, a hypervisor or virtual machine monitor (VMM) 630 can be inserted between the bare hardware 620 and OS 610. In this configuration, VMM 630 acts as a software "buffer" or virtualization layer between OS 610 and the bare hardware 620 of computer system 500.

[0135] VMM 630 instantiates and runs one or more virtual machine instances (“guest machines”). Each guest machine includes a “guest” operating system (such as OS 610) and one or more applications (such as application(s) 602) designed to run on the guest operating system. VMM 630 presents a virtual operating platform to the guest operating system and manages the execution of the guest operating system.

[0136] In some instances, VMM 630 can allow a guest operating system to run as if it were running directly on the bare hardware 620 of computer system 600. In these instances, the same version of the guest operating system configured to run directly on the bare hardware 620 can also run on VMM 630 without modification or reconfiguration. In other words, VMM 630 can provide full hardware and CPU virtualization to the guest operating system in some instances.

[0137] In other instances, a guest operating system can be specifically designed or configured to run on VMM 630 for improved efficiency. In these instances, the guest operating system is "aware" that it is running on a virtual machine monitor. In other words, VMM 630 can provide paravirtualization to the guest operating system in some instances.

[0138] Computer system processes include the allocation of hardware processor time, as well as the allocation of memory (physical and / or virtual), the allocation of memory for storing instructions executed by the hardware processor, the allocation of memory for storing data generated by the execution of instructions by the hardware processor, and / or the allocation of hardware processor state (e.g., register contents) between hardware processor time allocations when the computer system process is not running. Computer system processes run under the control of the operating system and can also run under the control of other programs executing on the computer system.

[0139] cloud computing

[0140] This article generally uses the term "cloud computing" to describe a computing model that enables on-demand access to a shared pool of computing resources (such as computer networks, servers, software applications, and services) and allows for the rapid provisioning and release of resources with minimal management effort or service provider interaction.

[0141] Cloud computing environments (sometimes called cloud environments or the cloud itself) can be implemented in various ways to best suit different requirements. For example, in a public cloud environment, the underlying computing infrastructure is owned by an organization that makes its cloud services available to other organizations or the public. In contrast, private cloud environments are generally designed for use by a single organization or within a single organization. Community clouds are designed to be shared by several organizations within a community; while hybrid clouds include two or more types of clouds (e.g., private, community, or public) bound together by data and application portability.

[0142] Generally, the cloud computing model allows some of the responsibilities that might have previously been provided by an organization's own IT department to be delivered as service layers within a cloud environment for consumer use (depending on the public / private nature of the cloud, within or outside the organization). Depending on the specific implementation, the precise definition of the components or features provided by or within each cloud service layer can vary, but common examples include: Software as a Service (SaaS). SaaS ), where consumers use software applications running on cloud infrastructure, while SaaSProviders manage or control the underlying cloud infrastructure and applications. Platform as a Service (PaaS) allows consumers to develop, deploy, and otherwise control their own applications using software programming languages ​​and development tools supported by the PaaS provider, while the PaaS provider manages or controls other aspects of the cloud environment (i.e., everything in the runtime execution environment). Infrastructure as a Service (IaaS) allows consumers to deploy and run arbitrary software applications and / or provision processing, storage, networking, and other basic computing resources, while the IaaS provider manages or controls the underlying physical cloud infrastructure (i.e., everything below the operating system layer). Database as a Service (DBAS)... DBaaS ), where consumers use database servers or database management systems running on cloud infrastructure, while DbaaS The provider manages or controls the underlying cloud infrastructure, applications, and servers, including one or more database servers.

[0143] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details, which may vary depending on the implementation. Therefore, the specification and drawings should be viewed in an illustrative rather than restrictive sense. The sole and exclusive indication of the scope of the invention, and what the applicant intends to define as the scope of the invention, is the literal and equivalent scope of the claims generated in the specific form of the set of claims arising from this application, including any subsequent corrections.

Claims

1. A method comprising: Receive natural language queries from users; Retrieve metadata from the machine learning (ML) automation component of the database system, the metadata including at least one of the following: Workload statistics for one or more users in a database system. Data source schema details, or Dynamic content statistics data from the data source; Based on natural language queries received from the user and the metadata, language prompts are generated for the generative artificial intelligence (AI) model; A generative AI model generates a response based on language prompts and a search of data sources, where the language prompts limit the scope of the search of data sources based on the metadata. The method is performed by one or more computing devices.

2. The method of claim 1, wherein the metadata is generated by one or more machine learning models for predicting resource usage and query performance in a database system.

3. The method of claim 1, wherein: A data source is a relational database that includes one or more database tables. Generative AI models include large language models (LLMs), and Natural language queries are queries about relational databases.

4. The method of claim 1, wherein: A data source is a relational database that includes one or more database tables. Generative AI models include generative models for natural language to structured query language (NL2SQL), and... The NL2SQL generative model is configured to generate one or more SQL queries based on language prompts to search relational databases.

5. The method of claim 4, further comprising: Provide language prompts as input to the NL2SQL generative model to generate the one or more SQL queries.

6. The method of claim 5, further comprising: This enables the display of one or more SQL queries to the user.

7. The method of claim 5, further comprising: Execute the one or more SQL queries on the one or more database tables to generate search results.

8. The method of claim 7, further comprising: The search results are provided as input to a large language model to generate natural language descriptions of the search results.

9. The method of claim 1, wherein: A data source is a relational database that includes one or more database tables, and Generating language prompts involves adding a set of one or more pattern descriptions by selecting a subset of database tables from the one or more database tables, at least in part based on the metadata, and adding pattern descriptions of the subset of database tables to the language prompts.

10. The method of claim 9, wherein the set of adding one or more pattern descriptions further comprises: Generate one or more query embeddings based on natural language queries; as well as A subset of the database tables is selected at least in part based on the similarity between the one or more query embeddings and the table-by-table topic modeling embeddings of the one or more database tables.

11. The method of claim 1, wherein: The data source includes an object repository, which includes one or more vector repositories that represent multiple documents using semantic encoding, and Generating language hints involves filtering the one or more vector repositories based on the metadata.

12. The method of claim 1, wherein the ML automation component performs at least one of the following: Automatic supply Automatic parallel loading, Automatic data placement Automatic encoding Improvements to the automatic query plan Automatic query time estimation Automatically change propagation, Automatic scheduling, or Automatic error recovery.

13. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause to perform the following operations: Receive natural language queries from users; Retrieve metadata from the machine learning (ML) automation component of the database system, the metadata including at least one of the following: Workload statistics for one or more users in a database system. Data source schema details, or Dynamic content statistics data from the data source; Based on natural language queries received from the user and the metadata, language prompts are generated for the generative artificial intelligence (AI) model; Based on language prompts and searches of data sources, a generative AI model generates responses, and Language prompts limit the scope of the search for the data source based on the aforementioned metadata.

14. The one or more non-transitory computer-readable media of claim 13, wherein the metadata is generated by one or more machine learning models for predicting resource usage and query performance in a database system.

15. One or more non-transitory computer-readable media as claimed in claim 13, wherein: A data source is a relational database that includes one or more database tables. Generative AI models include large language models (LLMs), and Natural language queries are queries about relational databases.

16. One or more non-transitory computer-readable media as claimed in claim 13, wherein: A data source is a relational database that includes one or more database tables. Generative AI models include generative models for natural language to structured query language (NL2SQL), and... The NL2SQL generative model is configured to generate one or more SQL queries based on language prompts to search relational databases.

17. One or more non-transitory computer-readable media as claimed in claim 13, wherein: A data source is a relational database that includes one or more database tables, and Generating language prompts involves adding a set of one or more pattern descriptions by selecting a subset of database tables from the one or more database tables, at least in part based on the metadata, and adding pattern descriptions of the subset of database tables to the language prompts.

18. The one or more non-transitory computer-readable media of claim 17, wherein the set of added one or more pattern descriptions further comprises: Generate one or more query embeddings based on natural language queries; as well as A subset of the database tables is selected at least in part based on the similarity between the one or more query embeddings and the table-by-table topic modeling embeddings of the one or more database tables.

19. One or more non-transitory computer-readable media as claimed in claim 13, wherein: The data source includes an object repository, which includes one or more vector repositories that represent multiple documents using semantic encoding, and Generating language hints involves filtering the one or more vector repositories based on the metadata.

20. One or more non-transitory computer-readable media as claimed in claim 13, wherein the ML automation component performs at least one of the following: Automatic supply Automatic parallel loading, Automatic data placement Automatic encoding Improvements to the automatic query plan Automatic query time estimation Automatically change propagation, Automatic scheduling, or Automatic error recovery.

Images