Systems and methods of knowledge answering engine with user interface customizations

Abstract

A method may include displaying, on a user device, a prompt to a user. A method may include receiving, from the user device, an input from the user, wherein the input is a query. A method may include determining, using a semantic knowledge component, one or more context items of the query. A method may include determining, using a learning engine accessing one or more algorithms and one or more data sources, one or more responses. A method may include determining, using an insight, a format of one or more outputs from the one or more responses based on at least one of the one or more context items. A method may include outputting, on the user device, the one or more outputs and a summary based on the one or more outputs from the one or more responses.

Description

SYSTEMS AND METHODS OF KNOWLEDGE ANSWERING ENGINE WITH USER INTERFACE CUSTOMIZATIONSCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This patent application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 637,581 , filed on April 23, 2024, the entirety of which is incorporated herein by reference.TECHNICAL FIELD

[0002] Various embodiments of the present disclosure relate generally to machine learning techniques for generating summarizations and insights and, more particularly, to systems and methods for extracting and processing user inputs as they relate to business analytics.BACKGROUND

[0003] Generative artificial intelligence (Al) applications that exist today focus on the task of using text to generate an image, video, or audio (or a combination of video and audio). This is done by using generative Al techniques to learn the mapping from one modality to the other. A use of this technology is also found in a conversational aspect, where refinements on an initial description can occur to improve the output.

[0004] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art, or suggestions of the prior art, by inclusion in this section.SUMMARY OF THE DISCLOSURE

[0005] In some aspects, the techniques described herein relate to a method including: displaying, on a user device, a prompt to a user; receiving, from the user device, an input from the user, wherein the input is a query; determining, using a semantic knowledge component, one or more context items of the received query; determining, using a learning engine, one or more responses; determining, using an insight, an output from the one or more responses; and outputting, on the user device, a summary based on the output from the one or more responses.

[0006] In some aspects, the techniques described herein relate to method comprising: displaying, on a user device, a prompt to a user; receiving, from the user device, an input from the user, wherein the input is a query; determining, using a semantic knowledge component, one or more context items of the received query;determining, using a learning engine accessing one or more algorithms and one or more data sources , one or more responses; determining, using an insight , the format of an output from the one or more responses based on at least one of the context items; and outputting, on the user device, one or more of the output or a summary based on the output from the one or more responses. One advantage here is that free input from the user can be accepted and the output can be determined according to the context and also according to an insight used to determine the format / modality of the output.

[0007] In some aspects, the techniques described herein relate to a method, wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, and a business calendar. Of course the semantic knowledge component can include many other features.

[0008] In some aspects, the techniques described herein relate to a method, wherein the semantic knowledge component is configured to parse and analyse the query. The semantic knowledge component may determine any of intent, conversational context and content of the user query. Especially taken together, these characteristics of the query give a useful breadth of information to provide the response.

[0009] In some aspects, the techniques described herein relate to a method, wherein intent refers to the type of question asked: is it a what, a where, a how, or some other type of question.

[0010] In some aspects, the techniques described herein relate to a method, wherein the conversational context refers to the position of the query in the “conversation” between the user and the system. For example, if the user has already detailed in the conversation that sales are to be provided in dollars (rather than, for example, product units), then that interpretation of sales may be carried through to subsequent queries (for example in the same session) using the word “sales”. Hence in general, the conversational context may incorporate any further information previously entered by the user to clarify the query.

[0011] In some aspects, the techniques described herein relate to a method, wherein the context items indicate information requested in the user query in a form that is readable by the learning engine. The context items may thus “translate” thequery into something related to the databases and / or other resources available to the system.

[0012] In some aspects, the techniques described herein relate to a method, wherein one or more sets of information relating to the user query are determined using one or more of an entity identification LLM determining prompts to gather more information from the user, a conversational context LLM determining the context of the user query, or an agent classifier preferably including a retrieval-based agent classifier and an agent classification LLM and determining an agent to be used for the query. Of course, one or more different models may be used to determine relevant sets of information, but Large Language Models are a useful tool in this area.

[0013] In some aspects, the techniques described herein relate to a method, wherein the learning engine is configured to continuously update a set of rules and instructions for processing and determining portions of a user query. This enables the system to learn from previous queries and outputs.

[0014] In some aspects, the techniques described herein relate to a method, wherein the algorithms are provided within software agents, including both out of the box agents and custom agents. Hence, the system may integrate bought-in technology with in-house development to merge the advantages of both by using agents, in some aspects, the techniques described herein relate to a method, wherein the learning engine further includes accessing one or more database and accessing one or more algorithms which may be provided as software agents (as mentioned above).

[0015] In some aspects, the techniques described herein relate to a method, wherein the one or more algorithms include at least one of a time series model, a key driver analysis model, a LLM, and a transformer based model.

[0016] In some aspects, the techniques described herein relate to a method further comprising generating a description of how the output was arrived at, and displaying the description on user request. This is valuable for enhancing the user’s trust in the system.

[0017] In some aspects, the techniques described herein relate to a method, wherein the user includes a user profile, wherein the user profile includes data access permissions. The data permissions may be reviewed to determine access with respect to the user and with respect to the query content. The additional securitypermitted by these features may be important for the enterprise operating the system.

[0018] In some aspects, the techniques described herein relate to a method, further including using an adaptive question and answer (QA) engine component determining one or more automated questions to assist the user, displaying the automated question on the user device, receiving from the user device an automated question response, and using the automated question response for processing and determining portions of a user query. Such automated question may be valuable in tailoring the method to the user’s needs and / or avoiding unnecessary processing of queries which have not been well specified by the user.

[0019] In some aspects, the techniques described herein relate to a method, wherein an automated question is determined at one or more of: before analysis of the user query; or after outputting, on the user device, the one or more of the output and the summary based on the output from the one or more responses.

[0020] In some aspects, the techniques described herein relate to a method, wherein an autonomous Al engine includes one or more sets of rules and instructions and assists the adaptive QA engine in providing the one or more automated questions.

[0021] In some aspects, the techniques described herein relate to a method, wherein the insights further includes determining a format of the output based on at least one of the context items.

[0022] In some aspects, the techniques described herein relate to a method, further includes displaying an additional prompt to the user after outputting, on the user device, a summary based on the output from the one or more responses.

[0023] In some aspects, the techniques described herein relate to a system including: a memory configured to store instructions; and one or more processors configured to execute the instructions to perform operations including: displaying, on a user device, a prompt to a user; receiving, from the user device, an input from the user, wherein the input is a query; determining, using a semantic knowledge component, one or more context items of the received query; determining, using a learning engine, one or more responses; determining, using an insight, an output from the one or more responses; and outputting, on the user device, a summary based on the output from the one or more responses.

[0024] In some aspects, the techniques described herein relate to a system including: a memory configured to store instructions; and one or more processors configured to execute the instructions to perform operations according to any of the method aspects set out above.

[0025] In some aspects, the techniques described herein relate to a system, wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, and a business calendar.

[0026] In some aspects, the techniques described herein relate to a system, wherein the learning engine further includes accessing one or more database and accessing one or more algorithms.

[0027] In some aspects, the techniques described herein relate to a system, wherein the one or more algorithms include at least one of a time series model, a key driver analysis model, a LLM, and a transformer based model.

[0028] In some aspects, the techniques described herein relate to a system, wherein the user includes a user profile, wherein the user profile includes data access permissions.

[0029] In some aspects, the techniques described herein relate to a system, wherein the insights further includes determining a format of the output based on at least one of the context items.

[0030] In some aspects, the techniques described herein relate to a system, further includes displaying an additional prompt to the user after outputting, on the user device, a summary based on the output from the one or more responses.

[0031] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: displaying, on a user device, a prompt to a user; receiving, from the user device, an input from the user, wherein the input is a query; determining, using a semantic knowledge component, one or more context items of the received query; determining, using a learning engine, one or more responses; determining, using an insight, an output from the one or more responses; and outputting, on the user device, a summary based on the output from the one or more responses.

[0032] In some aspects, the techniques described herein relate to a computer program or non-transitory computer-readable medium storing instructions that, whenexecuted by one or more processors, cause the one or more processors to perform operations according to any of the method aspects set out above.

[0033] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, and a business calendar.

[0034] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the learning engine further includes accessing one or more database and accessing one or more algorithms.

[0035] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the one or more algorithms include at least one of a time series model, a key driver analysis model, a LLM, and a transformer based model.

[0036] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the user includes a user profile, wherein the user profile includes data access permissions.

[0037] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the insights further includes determining a format of the output based on at least one of the context items.

[0038] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, the operations further including displaying an additional prompt to the user after outputting, on the user device, a summary based on the output from the one or more responses.

[0039] In some aspects, the techniques described herein relate to a method including: displaying, on a user device, a prompt to a user; receiving, from the user device, an input from the user, wherein the input is a query; determining, using a semantic knowledge component, one or more context items of the query; determining, using a learning engine accessing one or more algorithms and one or more data sources, one or more responses; determining, using an insight, a format of one or more outputs from the one or more responses based on at least one of the one or more context items; and outputting, on the user device, the one or more outputs and a summary based on the one or more outputs from the one or more responses.

[0040] In some aspects, the techniques described herein relate to a method, wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, or a business calendar.

[0041] In some aspects, the techniques described herein relate to a method, wherein the semantic knowledge component is configured to parse and analyze the query to determine intent, conversational context and content of the query.

[0042] In some aspects, the techniques described herein relate to a method, where the intent of the query indicates a type of question within the query, and the conversational context incorporates any further information previously entered by the user to clarify the query.

[0043] In some aspects, the techniques described herein relate to a method, wherein the one or more context items indicate information requested in the query in a form that is readable by the learning engine.

[0044] In some aspects, the techniques described herein relate to a method, wherein the one or more context items are displayed to the user along with the query, the one or more outputs, and the summary based on the one or more outputs from the one or more responses.

[0045] In some aspects, the techniques described herein relate to a method, wherein one or more sets of information relating to the query are determined using one or more of an entity identification LLM determining prompts to gather more information from the user, a conversational context LLM determining the one or more context items of the query, or an agent classifier preferably including a retrievalbased agent classifier and an agent classification LLM and determining an agent to be used for the query.

[0046] In some aspects, the techniques described herein relate to a method, wherein the learning engine is configured to continuously update a set of rules and instructions for processing and determining portions of a user query.

[0047] In some aspects, the techniques described herein relate to a method, wherein the one or more algorithms are provided within software agents, including both out of box agents and custom agents.

[0048] In some aspects, the techniques described herein relate to a method, wherein the one or more algorithms include at least one of a time series model, a key driver analysis model, a LLM, or a transformer based model.

[0049] In some aspects, the techniques described herein relate to a method, further including generating a description of how the one or more outputs were arrived at, and displaying the description on user request.

[0050] In some aspects, the techniques described herein relate to a method, wherein the user includes a user profile, wherein the user profile includes data access permissions.

[0051] In some aspects, the techniques described herein relate to a method, further including an adaptive question and answer (QA) engine component determining one or more automated questions to assist the user, displaying the one or more automated questions on the user device, receiving from the user device an automated question response, and using the automated question response for processing and determining portions of a user query.

[0052] In some aspects, the techniques described herein relate to a method, wherein an automated question is determined at one or more of: before analysis of the user query; or after outputting, on the user device, the one or more outputs and the summary based on the one or more outputs from the one or more responses.

[0053] In some aspects, the techniques described herein relate to a method, wherein an autonomous Al engine includes one or more sets of rules and instructions and assists the adaptive QA engine component in providing the one or more automated questions.

[0054] In some aspects, the techniques described herein relate to a system including: a memory configured to store instructions; and one or more processors configured to execute the instructions to perform operations according to any of the preceding method described herein.

[0055] In some aspects, the techniques described herein relate to a computer program or a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations described herein.

[0056] In some aspects, the techniques described herein relate to a computer program or the non-transitory computer-readable medium, wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, or a business calendar.

[0057] In some aspects, the techniques described herein relate to a computer program or the non-transitory computer-readable medium, the operations furtherincluding displaying an additional prompt to the user after outputting, on the user device, the summary based on the one or more outputs from the one or more responses.

[0058] Additional objects and advantages of the disclosed embodiments will be set forth in part in the description that follows, and in part will be apparent from the description, or may be learned by practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. As will be apparent from the embodiments below, an advantage to the disclosed systems and methods is that multiple parties may fully utilize their data without allowing others to have direct access to raw data. The disclosed systems and methods discussed below may allow advertisers to understand users' online behaviors through the indirect use of raw data and may maintain privacy of the users and the data.

[0059] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed.BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and together with the description, serve to explain the principles of the disclosed embodiments.

[0061] FIG. 1 A depicts a flowchart of the operation of a knowledge answering engine with III customizations, according to one or more embodiments.

[0062] FIG. 1 B depicts an exemplary system architecture diagram of a knowledge answering engine with III customizations, according to one or more embodiments.

[0063] FIGs. 2A-2E depict exemplary architecture and flow diagrams of a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0064] FIGs. 3A-3D depict exemplary architecture diagrams of a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0065] FIG. 4 depicts an exemplary architecture diagram of a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0066] FIG. 5 depicts interaction on a user interface of a knowledge answering engine with III customizations, according to one or more embodiments.

[0067] FIG. 6 depicts further interaction on a user interface of a knowledge answering engine with III customizations, according to one or more embodiments.

[0068] FIG. 7 depicts display on a user interface of how an output was derived by a knowledge answering engine with III customizations, according to one or more embodiments.

[0069] FIG. 8 depicts assistance on a user interface of a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0070] FIG. 9 depicts display on a user interface requesting user assistance for a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0071] FIG. 10 depicts still further interaction on a user interface of a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0072] FIG. 11 depicts a user introduction scheme of a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0073] FIG. 12 depicts the screen of FIG. 11 with a query entered to a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0074] FIG. 13 depicts the interface of FIG. 11 with the user assistance requested to answer the query entered to a knowledge answering engine with Ul customizations, according to one or more embodiments

[0075] FIG. 14 depicts the interface of FIG. 11 with a different query entered to a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0076] FIG. 15 depicts the interface of FIG. 11 with a further different query entered to a knowledge answering engine with Ul customizations, according to one or more embodiments.

[0077] FIG. 16 illustrates an implementation of a computer system that executes techniques presented herein.DETAILED DESCRIPTION OF EMBODIMENTS

[0078] Various embodiments of the present disclosure relate generally to machine learning techniques for generating summarizations and results and, moreparticularly, to systems and methods for extracting and processing user inputs as they relate to business analytics

[0079] The terminology used below may be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the present disclosure. Indeed, certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section.

[0080] Performing business tasks such as data analysis, related result generation (e.g., including various insight modalities / formats ranging from text to graphs), and knowledge retrieval across massive enterprise data and documentations (e.g., search and answer) stored in various locations and systems for typical business associates, who do not have any or sufficient coding background or extensive analytic professional experience, may be extremely time consuming and challenging. It may often be a very difficult learning curve for business users to gain coding and professional analytic experience in order to proficiently and efficiently perform data analysis, even with various Business Intelligence (Bl) analytics tools. Adoption and training time may also be extensive and often notable to meet individual users’ needs, due to vast different result and granular format needs.

[0081] Supporting customized and / or personalized needs for tasks, as described above, from technical data analytics development teams may also be extremely time consuming to satisfy given the nature of extensive and vast different intents and needs from various business associates and stakeholders, demanding diverse types of queries, formats, result types, outcomes, granularities, etc.

[0082] In view of the above problems and shortcomings, the present disclosure discusses embodiments that may eliminate or mitigate these problems and shortcomings, and provide technical improvements over conventional approaches. The following paragraphs summarize some of those advantages and benefits, and the techniques utilized to achieve those advantages and benefits.

[0083] To help drive transparency and / or trust and enable validation from development with generative artificial intelligence (GenAI), the present disclosure contemplates a customized solution that may include both front-end and application programming interface (API) call features to disclose how the data analysis results are achieved, either by disclosing the structured query language (SQL) queries, aswell as various key selection criteria and reasoning from more complex models’ outputs, thus avoiding blackbox solutions.

[0084] Further, customization to provide guardrails for outputs (e.g., minimize hallucination) and drive trust may enable GenAI’s responses in the front-end to inform users on the questions it may not be able answer and / or have low confidence for results, as well as the reason behind. For example, “I cannot provide answers, because I cannot identify the data and / or the data does not exist”, etc. Customizations may include features to provide follow-up questions to help clarify ambiguity of questions and scope before running analysis.

[0085] Further customizations may include building specific new agents for advanced analytics capabilities for supporting various business problem spaces (e.g., leveraging causal inference data manipulation language (DML) for attribution, driver analysis, optimization, recommendation engine, forecasting, etc.). The frontend user interface (Ul) may also contain features to ask clarifying questions to confirm user questions, inform users questions may not be answered, and protect against abusive and non-business related prompt injections.

[0086] The present disclosure also discusses a tool designed to solve key business problems across different types of questions from diagnostic, prescriptive, and predictive. The tool may provide users with the ability to analyze different types of data and generate insights understanding how, what, why outcomes, which may be used to determine the format of the output. For example a “what” question may require an answer including a number, and a “trend” question may require a graph. The tool may also exercise the added option of triggering pre-trained machine learning models such as attribution, forecasting, and prediction using Natural Language Programming (NLP) prompts to provide business users with answers.

[0087] The tool may be able to ingest various types of data (e.g., relational data and text data). The tool may also harmonize and analyze types of data based on different business questions. The tool may be able to create and / or trigger SQL queries or machine learning models based on the context provided by the user question / prompt.

[0088] To help drive the outcome of solving different types of questions, the tool may use existing logics and agents to trigger complex SQL queries (e.g., involving multiple tables, conditions, and aggregations). The agent may perform a task autonomously through NLP prompts. The tool may use existing machinelearning models customized to domain specific business questions. The existing machine learning models may be fine-tuned to fit business specific needs and thoroughly tested and validated. The tool may generate different types of visualizations integrated with in-house processes. This may also include the use of internal metrics and key performance indicators (KPIs) for visualization.

[0089] Fig. 1A depicts an exemplary method, according to one or more embodiments. The method may comprise the following steps. In a displaying step s10, a prompt to a user may be displayed. The prompt may be displayed on, for example, a user device. In a receiving step s20, an input from the user may be received. The input may be a query. The input may be received from, for example, the user device. In a determining step s30, using a semantic knowledge component, one or more context items of the received query may be determined. The semantic knowledge component may include at least one of a measure relation, one or more synonyms, one or more analysis packs, and a business calendar.

[0090] The semantic knowledge component may be configured to parse and analyze the user query to determine intent, conversational context and content of the user query. The intent of the user query may indicate the type of question within the query. Further, the conversational context may incorporate any further information previously entered by the user to clarify the query.

[0091] The context items may indicate information requested in the user query in a form that is readable by the learning engine. For instance, the context items may be the (actual) information requested in the query and converted into items that can be understood by the system. An example query may be, “what are the overall sales in 2023?”. In this instance, the context items may be “Mars,” “sales,” “ytd,” etc. These context items may therefore indicate the information requested in the user query.

[0092] One or more sets of information relating to the user query may be determined using one or more of an entity identification large language model (LLM) determining prompts to gather more information from the user, a conversational context LLM determining the context of the user query, and an agent classification LLM determining an agent to be used for the query. The prompts (one or more prompts) may include automated questions determined by the system (i.e. , the entity identification LLM) to clarify the content identified in the user query.

[0093] In another determining step s40, one or more responses may be determined using a learning engine accessing one or more algorithms and one ormore data sources or databases. The learning engine may be configured to continuously update a set of rules and instructions for processing and determining portions of a user query. For example, the learning engine may include a machine learning model that may continuously update the set of rules and instructions for processing and determining portions of the user query. The algorithms accessed by the learning engine may be provided within software agents, including both out of the box agents and custom agents. The software agents may be comprised within an agents components and thus the out of the box agents and custom agents may be referred to as sub-components. The one or more algorithms may include at least one of a time series model, a key driver analysis model, a LLM, and a transformer-based model.

[0094] In yet another determining step s50, using an insight, the format of an output from the one or more responses based on at least one of the context items may be determined. Insights may be, for example, the decision of how to present the output. That is, the insights may determine the output to the user. The output may be in the form of text, audio, video, graphic, or a combination thereof.

[0095] In an outputting step s60, on the user device, one or more of the outputs and a summary based on the output(s) from the one or more response may be output. The context items may be displayed to the user along with the query and the one or more of the outputs and the summary based on the output from the one or more responses. The method may further comprise generating a description of how the output was arrived at, and displaying the description on user request.

[0096] The user may include a user profile. The user profile may include data access permissions. The prompt may include a series of one or more additional prompts to determine the user (e.g., the user profile and / or access permissions) corresponding to the queried information. The data permission may be reviewed to determine access with respect to the user and with respect to the query content. That is, the data permission review may determine whether the user has sufficient access and whether the user has sufficient access to view the requested content (e.g., requested content in the input query).

[0097] The method may further include an adaptive question and answer (QA) engine component determining one or more automated questions to assist the user, displaying the automated question on the user device, receiving from the user device an automated question response, and using the automated question response forprocessing and determining portions of a user query. An automated question may be determined at one or more of: before analysis of the user query; and after outputting, on the user device, the one or more of the output and the summary based on the output from the one or more responses. The adaptive QA engine may be performed at any time during the method (process) to further determine the content, context, output, etc. An autonomous Al engine may include one or more sets of rules and instructions and assists the adaptive QA engine in providing the one or more automated questions.

[0098] FIG. 1 B depicts an exemplary system architecture diagram, according to one or more embodiments. The system 100 may include one or more components working in tandem, the one or more components may be separate components or may be part of the same component or a combination thereof. The components may include administration and security 101 , an agent component 103 used to execute algorithms to answer queries, adaptive question and answer (QA) engine 105, autonomous Al engine 107, learning engine 109, knowledge API 111 , data API 113, semantic layer 115, data hub 117, datasource connectors 119, and large language models (LLM) 121 . The administration and security component may include information pertaining to users (e.g., user profile), permissions (e.g., access restrictions or limitations), and rules. The agent component may include one or more sub-components, the one or more sub-components may include an orchestrator 123, intent classification 125, conversational context 127, algorithm execution 129, and narrative summaries 131. The orchestrator sub-component 123 may perform functions relating to gathering and processing information from each of the one or more sub-components. The intent classification sub-component 125 may include data and rules for determining intent of a user query, for example using algorithms available in the industry, and support the format of the output against a response. The conversation context sub-component 127 may include one or more sets of rules and programs to determine the context of the user query. The context of the query may be determined using one or more of NLP, LLM, machine learning models, or a combination thereof. The algorithm execution sub-component 129 may perform the execution of the one or more sets of rules and instructions of the one or more subcomponents. The narrative summaries sub-component 131 may include one or more sets of rules and instructions to generate summaries of the user query, and the classifications generated from one or more sub-components.

[0099] The adaptive question and answer (QA) engine component 105 may include one or more sets of rules and instructions to determine a set of automated questions and answers to assist the user. For example, one or more prompts may be displayed to the user to further gather information regarding the user query to further define the information requested by the user. The adaptive QA engine 105 may be executed at any time during the process to further determine the content, context, output, etc. The autonomous Al engine 107 may include one or more sets of rules and instructions to assist the adaptive QA engine 105 in performing the user prompts. The learning engine 109 provides responses to the queries and may include one or more sets of rules and instructions to modify and update itself based on the one or more processes being performed by the other components. For example, the learning engine 109 may include a machine learning model that may continuously update a set of rules and instructions for processing and determining portions of a user query.

[0100] The knowledge API 111 may enable the system to search, view, and / or fetch information regarding the user query. For example, the knowledge API 111 may fetch information from the administration and security component with respect to the user submitting the query (e.g. does the user have sufficient access) and the query content (e.g., does the user have sufficient access to the view the requested content). The data API 113 component may facilitate the exchange of information between one or more applications and databases, enabling the system to seamlessly integrate functionalities securely and swiftly. For example, the data API 113 may assist the knowledge API 111 to exchange information between the administration and security component as described above. The semantic layer 115 may connect the data hub component 117 to the knowledge API and data API components 111 , 113 to assist in access to analytics, reports, dashboards, etc. The data hub component 117 may include a central repository for storing and managing data from one or more data sources. The data hub 117 may serves as a single point of access for the system data, allowing for easier component access, sharing, and analysis of the information from the one or more data sources. The datasource connectors 119 component may include one or more connections to the one or more data sources available within the system 100. The LLM component 121 may include one or more LLMs that may be incorporated into the system 100. The one or moreLLMs may assist in the process of summarizing, translating, predicting, and generating of content based on the user query.

[0101] FIGs. 2A-2E depict exemplary architecture and flow diagrams, according to one or more embodiments. FIG. 2A depicts an exemplary agent based architecture, according to one or more embodiments. The agent architecture 200 may include one or more components including frontend applications component 201 , server component 203, and external data sources component 205. The frontend applications component 201 may include one or more sub-components including core web application sub-component 207 and custom applications subcomponent 209. The core web application sub-component 207 may define the interactions between applications, middleware systems, and databases to ensure multiple applications may work together. The core web application sub-component 207 may include the option to embed via inline frame (e.g., iFrame), a hypertext markup language (HTML) element that loads another HTML page within the document. The custom applications sub-component 209 may include features to provide follow-up questions to help clarify ambiguity of questions and scope before running analysis (and may provide the follow-up questions to the adaptive QA engine). The custom applications sub-component 209 may include guardrails for outputs (e.g., minimize hallucination) and drive trust to enable GenAI’s responses in the front-end to inform users of the questions it may not be able answer and / or has low confidence for results, as well as the reason behind.

[0102] The server component 203 may include one or more sub-components including external API sub-component 211 , administration and security subcomponent 213, conversational agent sub-component 215, adaptive question answering engine sub-component 217, algorithms sub-component 219, data hub sub-component 221 , and large language models (LLM) sub-component 223, for example customized by updating the prompt that is fed to the LLM with more context using prompt engineering before the system goes live. The external API subcomponent 211 may include one or more APIs and may enable the system to search, view, and / or fetch information regarding the user query. For example, the one or more APIs may fetch information from the administration and security subcomponent 213 with respect to the user submitting the query (e.g. does the user have sufficient access) and the query content (e.g., does the user have sufficient access to the view the requested content). The one or more APIs may facilitate theexchange of information between one or more applications and databases, enabling the system to seamlessly integrate functionalities securely and swiftly. For example, the one or more APIs may assist the conversational agent sub-component 215 to exchange information with the administration and security sub-component 213.

[0103] The administration and security sub-component 213 may include information pertaining to users (e.g., user profile), permissions (e.g., access restrictions or limitations), and rules. The conversational agent sub-component 215 may include one or more sets of rules and programs to determine the context of the user query (thus it may act as the semantic knowledge component). The conversational agent sub-component 215 may utilize one or more of NLP, LLM, machine learning models, or a combination thereof. The adaptive question answering engine sub-component 217 may include one or more sets of rules and instructions to determine a set of automated questions and answers to assist the user as previously described. For example, one or more prompts may be displayed to the user to further gather information regarding the user query to further define the information requested by the user. The adaptive question answering engine subcomponent 217 may be performed at any time during the process to further determine the content, context, output, etc. The algorithms sub-component 219 may perform the execution of the one or more sets of rules and instructions of the one or more sub-components. The algorithms sub-component 219 involves certain logical components meant to perform a specific task which is outside the purview of LLMs. This may be viewed, for example, as the response calculation happening through an algorithm but the response being presented using LLMs. The algorithms subcomponent 219 may include one or more sets of rules and instructions to assist the adaptive question answering engine sub-component 217 in performing the user prompts (and so act as the autonomous Al engine). It may be equivalent to the custom agents sub-component 251 in Fig. 2B described below.

[0104] The data hub sub-component 221 may include a central repository for storing and managing data from one or more data sources. The data hub subcomponent 221 may serve as a single point of access for the system data, allowing for easier component access, sharing, and analysis of the information from the one or more data sources. The LLM sub-component 223 may include one or more LLMs that may be incorporated into the system. The one or more LLMs may assist in the process of summarizing, translating, predicting, and generating of content based onthe user query. The external data sources component 205 may include one or more external data sources that may be accessed before, during, and after any of the above processes are performed.

[0105] FIG. 2B depicts an exemplary agent based architecture, according to one or more embodiments. For brevity, similar features as described with FIG. 2A will not be repeated. The agent architecture 230 may include one or more components including frontend applications component 231 , server component 233, agents component 235, LLM component 237, and external data sources component 239. The frontend applications component 231 may be similar to the frontend applications component 231 as described in FIG. 2A. The server component 233 may include one or more sub-components including external APIs 241 , memory subcomponent 243, question understanding (ASK) sub-component 245, and agent classifier sub-component 247. The external APIs sub-component 241 may be similar to that as described in FIG. 2A. The memory sub-component 243 may be similar to the data hub sub-component 221 as described in FIG. 2A. The question understand (ASK) sub-component 245 may be similar to the adaptive question answering engine sub-component 217 as described in FIG. 2A. The agent classifier sub-component 247 may be similar to the conversational agent sub-component 215 as described in FIG. 2A (and thus may act as the semantic knowledge component).

[0106] The agents component 235 may include one or more sub-components including out of the box sub-component 249 and custom agents sub-component 251 . The out of the box agents sub-component 249 may include one or more out of the box agents (e.g., DataSage, Diagnoz, HotSpot, and DocuSage). The custom agents sub-component 251 may include features to provide follow-up questions to help clarify ambiguity of questions and scope before running analysis.

[0107] The frontend applications component 231 may include a core web application sub-component 253 and a custom applications sub-component 255. The custom applications sub-component 255 may include guardrails for outputs (e.g., minimize hallucination) and drive trust to enable GenAI’s responses in the front-end to inform users of the questions it may not be able answer and / or has low confidence for results, as well as the reason behind.

[0108] FIG. 2C depicts an exemplary flow diagram using the exemplary agent architectures of FIGs. 2A-2B. A user 261 may interact with one or more prompts to input a query. The user 261 may receive a prompt displayed on a user device. Inresponse to the prompt, the user may then provide an input. Their integration in the system 260 together with LLMs 263 allows efficient LLM usage. The system 260 may use additional prompts to further identify or clarify the content and information requested. The system 260 then determines information (e.g., content, context, intent, etc.) about the query. In doing so, the system may identify relevant agents 265 to be used based on the determined information of the user query. One or more of the agents as described in FIGs. 2A-2B may be employed. The system 260 then compiles the summaries of content for output 267 to the user. The system 260 may further employ additional agents to guardrails outputs (e.g., minimize hallucination). Once complete, the system 260 will then present the output information 267 to the user as defined by the user query (e.g., text, video, audio, graphs, etc.). At this point, the system 260 may provide additional prompts to determine if the output information 267 answered the user query, if additional information is requested, if the user is satisfied with the output 267 displayed, or a combination thereof.

[0109] FIG. 2D depicts an exemplary execution flow using the system as described in FIGs. 2A-2B. The system 300 may present one or more prompts to the user to input a query 301 and / or additional information relating to a query 301 . The system 300 may utilize one or more LLMs 303 to determine one or more sets of information relating to the user query 301 . The one or more LLMs 303 may include an entity identification LLM 305, a conversation context LLM 307, and an agent classification LLM 309. The entity identification LLM 305 may include an ASK engine to determine one or more sets of prompts to gather more information from the user (which may thus act as the adaptive question and answer (QA) engine component). The one or more prompts may include automated questions determined by the system 300 to clarify the content identified in the user query 301 . The ASK engine may include sophisticated NLP algorithms and GenAI models to understand the question and identify relevant agents. The conversational context LLM 307 may determine the context of the user query 301 using NLP, keywords, etc. (and thus may act as the semantic knowledge component). The agent classification LLM 309 may determine one or more relevant agents 311 to assist in determining the question data (and thus act as the learning engine). This triggering of the right agent / algorithm based on the asked question is an important feature of the system, enabling the system to function efficiently and accurately.

[0110] Agent (or algorithm) classification may use a two-stage classification approach to identify the relevant agents 311 that can consider the user question. The first stage classifier may use a retrieval-based approach to classification which identifies a set of agents / algorithms based on the documents (questions) on which it is trained. The second stage classifier may be an LLM-based classifier that understands the capabilities of an agent / algorithm to reason out a query and that can identify the relevant agents / algorithms that can answer a question. To enable agent classification, each agent may go through an agent registration in which the agent provides necessary information to train the agent classifier for that agent.

[0111] Next, the system may then transmit the identified information and metadata to the respective agents 311 for further analysis. Each of the agents 311 may determine the relevant information from a data and knowledge base either incorporated with or connected to the system. Each agent 311 may also obtain further user assistance from one or more subject matter experts (SME) to better analyze each item identified if necessary, but equally the use of human SMEs may not be integrated in the system, so “Assistance” shown in Fig. 2D here is optional. The system may then, after the analysis is complete, transmit the output 313 information to the user. The user may be presented with the information in the form of text, video, audio, graphic, or a combination thereof.

[0112] FIG. 2E depicts an exemplary execution flow using the system described in FIGs. 2A-2B. The algorithms mentioned could be provided in agents. For brevity, some similarities as described in FIG. 2D will not be repeated. The system 320 may start by presenting one or more user interfaces (III) to the user. The user may be prompted as similarly described in FIG. 2D. The system may then determine the scope and complexity 323 of the user query 321 as described in FIG. 2D. The system may then pass the identified information and data 323 for analysis 325. The information and data 323 may be analyzed using a combination of information available from structured datasets, analysis packs, algorithms, and semantic knowledge sets. The sections below illustrate the concept of the agents / algorithms sub-component for analysis. Upon completion of the one or more analyses 325, the system 320 may then transmit the information for summarization 327. The information may be summarized to determine how the information and content is to be formatted for the user, which is a new technique when employed in conjunction with LLM usage. For example, if determined that the user queryrequested a forecast of the product over a specific time period, the system 320 may determine the analyzed information to be displayed in a graph, plot, text, or a combination thereof. Once determined, the information is output 329 to the user for display.

[0113] FIGs. 3A-3D depict exemplary architecture diagrams, according to one or more embodiments. FIG. 3A depicts a high-level architecture diagram 340. The architecture 340 may include data sources 341 , data ingestion 343, data harmonization 345, analytical data store 347, access layer 349, and consumption 351 . Data sets 353 are also shown. These data sets 353 are use-case specific and the data source section represents where they can be stored. Taking the first data set as an example, the method has been tested on Nielsen Syndicated sales data. The data sources 341 may include both internal (e.g., local documents and files) and external sources (e.g., Data Lake Gen2, SQL repositories, and one or more APIs (e.g., knowledge and / or data as described in FIG. 1 B)). The data ingestion 343 may include both extraction (e.g., RestAPI and Simpel) and data quality (e.g., Cyborg). Data harmonization 345 may include curated and common data layers. The analytical data store 347 may include semantic / feature store. The access layer 349 may include one or more SQL repositories (e.g., SQL SPOT, One Demand Portal). Consumption 351 may include one or more services to assist in the display of information to the user 355. The one or more services may include app services, React JS, Flask, SPOT and One Demand Portal.

[0114] FIG. 3B depicts an exemplary architecture diagram of the deployment architecture and shows a link to data on which the solution is tested. A proprietary data set is not a requirement for the method. The architecture 400 may include one or more resource groups including an LLM resource group 401 and an application resource group 403. The LLM resource group 401 may include an LLM subnet 405 including Azure OpenAI. Azure OpenAI provides access to language models (e.g., GPT-4) to easily adapt to specific tasks including content generation, summarization, image understanding, semantic research, and natural language to code translation. The application resource group 403 may include a Bastion subnet 407, Azure BLOB storage 409, services subnet 411 , Azure Database for SQL servers 413, and web applications 415. The Bastion subnet 407 may include Bastion, a service that provides secure and seamless remote desktop protocol (RDP) and secure shell protocol (SSH) access to virtual machines within exposure through public IPaddresses. The services subnet 411 may include an istio mesh 417, istio services 419 and Azure internal load balancer 421 . The istio mesh 417 may include a datahub 423, RabbitMQ 425, Cache 427, couchbase 429, and default 431. Datahub 423 may include one or more databases for access by the system. RabbitMQ 425 is a message-broker for efficiently managing information between applications within a system. Couchbase 429 is a source-available and distributed multi-model NoSQL document oriented database for interactive applications.

[0115] The Istio services 419 may simplify the traffic of data between applications and / or components of a system. The Azure internal load balancer 421 may balance the traffic inside a virtual network. The web applications 415 may be similar to those described in FIG. 2A.

[0116] FIG. 3C depicts an exemplary architecture diagram of the deployment architecture similar to FIG. 3B but does not show a link to data on which the solution is tested. For brevity, similarities as described in FIG. 3A, will not be repeated. The architecture 440 may include one or more resource groups including an LLM resource group 441 and an application resource group 443. The LLM resource group 441 may include an LLM subnet 445 including Azure OpenAI. The application resource group 443 may include a Bastion subnet 447, Azure BLOB storage 449, services subnet 451 , Azure Database for SQL servers 453, and web applications 455. The services subnet 451 may include an istio mesh 457, istio services 459 and Azure internal load balancer 461 . The istio mesh 457 may include a datahub 463, RabbitMQ 465, Cache 467, couchbase 469, and default 471 .

[0117] In addition to LLM resource group 441 , an additional data asset 473 may be included (e.g., MARS data assets) for additional content. The architecture 440 may include a log analytics workspace 475 to receive and log information from the Istio mesh components. The architecture 440 may include an application gateway subnet 477 including Azure web application firewall (WAF), which may be useful in some cases.

[0118] FIG. 3D depicts an exemplary flow diagram using one of the deployment architecture diagrams as described in FIGs. 3B-3C. The flow diagram 500 may allow users 501 to generate a wide variety of analyses and plots without requiring access to developers or data specialists. The flow diagram 500 may determine the intent behind a user query and executes relevant algorithms. The flow diagram 500 may use GenAI models to summarize the results to the user query. Theflow diagram 500 may detect anomalies and patterns in your enterprise data. The flow diagram 500 may start by receiving a user query. The user query may be parsed by a semantic knowledge component 503.

[0119] The semantic knowledge component 503 may include measure relations, synonyms, analysis packs, and business calendars, each of which assists in the determination of the intent, context, and content of the user query. One or more context items of the query may be determined using the semantic knowledge component. Next the flow diagram 500 may transmit the information to a learning engine 505. The learning engine 505 may send and receive information from one or more algorithms and models 507 and one or more databases 509. The one or more algorithms and models 507 may include time series models, key driver analysis models, LLMs (e.g., GPT-3.5, GPT-4), and transformer-based models (e.g., BERT). After analysis, the learning engine 505 may transmit the information to insights 511 . Insights 511 may determine the output to the user. The output may be in the form of text, audio, video, graphic, or a combination thereof. The learning engine may output a summary on the user device based on the output from the insights of the one or more responses. Once received by the user, the user may input additional information 513 to modify the initial query. The flow diagram may then repeat any one or more of the steps as necessary. Input from the user may be given at any time during the flow.

[0120] FIG. 4 depicts an exemplary architecture diagram, according to one or more embodiments. The architecture 520 may include one or more components that may send and receive information bidirectionally between components. Starting from the back-end perspective, the architecture 520 may include a documents component 521 . Starting from the front-end perspective, the architecture may include a front end component 523 which is accessed by users 525.

[0121] Looking at the back end, the documents component 521 may include one or more storage databases. The architecture 520 may then transfer information from the documents component to one or more LLM components. The one or more LLM components may include LLMs for indexing 524, prompt management 526, and deployment 527. The indexing LLM 524 may include preprocessing 529 and embedding 531. Preprocessing 529 may include the use of third party applications (e.g., LangChain and Weaviate) to simplify the creation of applications using LLMs, including document analysis, summarization, chatbots, and code analysis.Embedding 531 may include third party applications for masked and permuted pretraining for language understanding (e.g., MPNet). Prompt management 526 may include custom LLMs to determine one or more prompts for display to a user. For example, the prompts may be in response to a user query to assist in further defining or clarifying the user query. The prompt may include a series of one or more additional prompts to determine the user (e.g., user profile and / or access permissions) corresponding to the queried information. Deployment 527 and deployment 533 may include one or more private LLMs. The one or more LLM components may send and receive information from an LLM model selection component 535. The LLM model selection component 535 may assist in determining what LLMs to utilize during the processing of the user query and gathered documents.

[0122] The architecture 520 may transfer the information to a vector database 537 for storage and retrieval (e.g., Weaviate). The architecture 520 may then transfer the information to an application component 539. The application component 539 may include a backend component 541 and front end component 523. The backend component 541 may include agents component 543 and agent identification component 545. Agents component 543 may include structured and unstructured agents (e.g., Python, JavaScript, and Docker), connected with documents component 521 and structured data 547. Agent identification component 545 may include entity identification, conversation context, and agent classification components. Entity identification may include one or more LLMs (e.g., ASK engine) to determine one or more sets of prompts to gather more information from the user.

[0123] The one or more prompts may include automated questions determined by the system to clarify the content identified in the user query. For example, the ASK engine may include sophisticated NLP algorithms and GenAI models to understand the question and identify relevant agents. Conversation context may include one or more LLMs to determine the context of the of the user query using NLP, keywords, etc. Agent classification may include one or more LLMs to determine one or more relevant agents to assist in determining the question data. Front end component 523 may include the use of one or more applications (e.g., ReactJS, NodeJS, Docker) to assist in the building of user interfaces based on components. For example, ReactJS, may be used to generate user interfaces for rendering content found in libraries, while only re-rendering portions of the contentthat has changed since last rendered, avoiding unnecessary re-rendering of unchanged content elements. After the front end component 523 generates content for consumption, the content may be displayed to one or more users 525. The user 525 may also provide information for the system by giving additional feedback. The feedback may include user queries, responses to prompts, etc. The users may include SMEs that (if present) will provide additional information to the system based on the generated information, providing feedback for correcting, modifying, and updating the system and LLMs to generate finely tuned content for later users.

[0124] The architecture 520 may include the use of a DevOps component 549, an observability component 551 , a database component 553, and an authentication component 555. The DevOps component 549 may include a set of practices and tools integrating and automating software development and IT operations. For example, DevOps 549 may use version control tools to manage code to track changes made over time (e.g., Azure Repos). The observability component 551 may include an analytics service and log service. The analytics service may help understand the performance and usage of live web applications (e.g., Azure Insights). The log service may act as a repository of one or more services (e.g., Azure Insights). The observability component 551 may include one or more databases to send and receive information. The one or more databases 553 may include Azure Database or PostgreSQL Server. The authentication component 555 may include a service that allows for storage and authentication of user information (e.g., user profile). The service may act as an identity determination solution. For example, a user may input a query requesting certain business forecast data. The system may determine, using this service (e.g., Okta), to determine the identity of the user and any corresponding access permissions. Based on the determined user and access permissions, the system may determine the user 525 is qualified to view the generated content. If there is an ambiguity, the system may provide additional prompts (e.g., user login information) to determine the user identity.

[0125] After the application component 539 has determined the agents 543 to be used, the context and classification of the query, etc., the architecture 520 may transmit the identified information to a query acceleration component 557. The query acceleration component 557 may include an SQL query accelerator to determine a set of SQL queries based on the identified information. The determined SQL queries may be used to determine content from the structured data component 547. Theinformation determined from the SQL query may be transferred back through the architecture 520 for use in one or more components before being sent to the front end component 523 for rendering and display to the user.

[0126] FIG. 5 depicts interaction on a user interface. The original query 532 is shown together with the initials 534 of the user at the top of the screen. The additional information shown below the query 532 as “Value Sales”, “January 1 - December 31 , 2023” is derived by the system using the functionality described herein and provided as context items. The response to the user’s question is shown as the output 536 below the additional information, in this case as a number in the unit of pounds. Below the output 536 is a summary 538 provided automatically in the method. Both the output 536 and the summary 538 are displayed here, but one or the other may be displayed.

[0127] FIG. 6 again depicts interaction on a user interface. The original query 542 is shown together with the initials 544 of the user at the top of the screen. The context items shown as “Volume Sales”, “BRAND:SNICKERS”, “Customer: Asda, Tesco Mains” and “January 1 - December 31 , 2023” are derived by the system using the functionality described herein. The response to the user’s question is shown as the output 546 below the context items and in this case the system generates an X-Y graph plot 548 by analyzing the nature of the requested response from the question. Below the output 546 is a summary 550 provided automatically in the method. Both the output 546 and the summary 550 are displayed here, but one or the other may be displayed.

[0128] FIG. 7 depicts a description of how the output was calculated as viewed on the user interface. This description may be generated for every response but only visible when the user clicks a button on the platform to deep dive into the details. Alternatively, the description may be displayed automatically as part of the display shown in FIGs. 5 and 6 or the description may flash up separately but automatically.

[0129] FIG. 8 depicts assistance provided by the system while the user is entering a query, based on internal system knowledge and parameters. User prompting on the interface allows not only a better user experience but improves understandability of queries to the system by use of context. Here, the system offers “MAC aka margin after conversion”, “MARS WRIGLEY - in Manufacturer” and“MARS - in Brand” as alternative completion for the user’s part-finished entry of“What is mar.”

[0130] FIG. 9 depicts the user interface after a query has been entered and when assistance is required to refine the context of the query (the adaptive QA engine component). Here, drop down menus allow the user to pick the parameter that they wish to explore. The first choices for query words “base,” “sales,” and “mars” are offered as inputs and the user can activate the drop-down menu to check other possibilities. For example, the parameter selected for “base” is “Base Price Elasticity.” The parameter selected for “sales” is “Unit Sales.” The parameter selected for mars is “MARS WRIGLEY in Manufacturer.”

[0131] FIG. 10 again depicts interaction on a user interface. The original query 561 is shown together with the initials 563 of the user at the top of the screen. The context items shown as “TOP 5 Brand (On: Value Sales”)”, “Manufacturer: MARS WRIGLEY”, and “January 1 - December 31 , 2023” are derived by the system using the functionality described herein. The response to the user’s question is shown as the output 565 below the context items and in this case the system generates a bar graph 567 by analyzing the nature of the requested response from the question. Below the output 565 is a summary 569 provided automatically in the method. Both the output 565 and the summary 569 are displayed here, but one or the other may be displayed.

[0132] FIG. 1 1 depicts a user introduction screen 600, with a pick list of possibilities as a left-hand column 601. The user has selected “new conversation” and the system suggests some example questions 603 to stimulate the user’s curiosity and displays a question box 605 at the bottom of the screen. FIG. 12 depicts the same user screen with a query 607 entered as “What is mars sales ytd.” FIG. 13 shows the system requiring additional information to answer the query.

[0133] FIG. 14 shows the same user screen with a different query 621 and the context items 623 and the output 625, together with a summary 627, as in FIG. 5. The follow-up question 629 “How did we arrive at this” is highlighted and may be selected by the user to give a description of the system processing.

[0134] FIG. 15 shows the same user screen, again with a different query 641 , and the output 643 but without the context items and the summary. In this instance, the method uses the appropriate agent / algorithm to produce brand price incentivecurves for five different brands, across, small, standard, large and bulk / multi and displays them as five different plots.

[0135] FIG. 16 illustrates an implementation of a computer system that executes techniques presented herein. The computer system 800 includes a set of instructions that are executed to cause the computer system 800 to perform any one or more of the methods or computer based functions disclosed herein. The computer system 800 operates as a standalone device or is connected, e.g., using a network, to other computer systems or peripheral devices.

[0136] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification, discussions utilizing terms such as "processing," "computing," "calculating," “determining”, analyzing” or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0137] In a similar manner, the term "processor" refers to any device or portion of a device that processes electronic data, e.g., from registers and / or memory to transform that electronic data into other electronic data that, e.g., is stored in registers and / or memory. A “computer,” a “computing machine,” a "computing platform," a “computing device,” or a “server” includes one or more processors.

[0138] In a networked deployment, the computer system 800 operates in the capacity of a server or as a client user computer in a server-client user network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. The computer system 800 is also implemented as or incorporated into various devices, such as a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile device, a palmtop computer, a laptop computer, a desktop computer, a communications device, a wireless telephone, a land-line telephone, a control system, a camera, a scanner, a facsimile machine, a printer, a pager, a personal trusted device, a web appliance, a network router, switch or bridge, or any other machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. In a particular implementation, the computer system 800 is implemented using electronic devices that provide voice, video, or data communication. Further,while the computer system 800 is illustrated as a single system, the term “system” shall also be taken to include any collection of systems or sub-systems that individually or jointly execute a set, or multiple sets, of instructions to perform one or more computer functions.

[0139] As illustrated in FIG. 16, the computer system 800 includes a processor 802, e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 802 is a component in a variety of systems. For example, the processor 802 is part of a standard personal computer or a workstation. The processor 802 is one or more processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now known or later developed devices for analyzing and processing data. The processor 802 implements a software program, such as code generated manually (i.e. , programmed).

[0140] The computer system 800 includes a memory 804 that communicates via bus 808. Memory 804 is a main memory, a static memory, or a dynamic memory. Memory 804 includes, but is not limited to, computer-readable storage media such as various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media and the like. In one implementation, the memory 804 includes a cache or random-access memory for the processor 802. In alternative implementations, the memory 804 is separate from the processor 802, such as a cache memory of a processor, the system memory, or other memory. Memory 804 is an external storage device or database for storing data. Examples include a hard drive, compact disc (“CD”), digital video disc (“DVD”), memory card, memory stick, floppy disc, universal serial bus (“USB”) memory device, or any other device operative to store data. The memory 804 is operable to store instructions executable by the processor 802. The functions, acts, or tasks illustrated in the figures or described herein are performed by processor 802 executing the instructions stored in memory 804. The functions, acts, or tasks are independent of the particular type of instruction set, storage media, processor, or processing strategy and are performed by software, hardware, integrated circuits, firmware, micro-code, and the like, operating alone or in combination. Likewise,processing strategies include multiprocessing, multitasking, parallel processing, and the like.

[0141] As shown, the computer system 800 further includes a display 810, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, a cathode ray tube (CRT), a projector, a printer or other now known or later developed display device for outputting determined information. The display 810 acts as an interface for the user to see the functioning of the processor 802, or specifically as an interface with the software stored in the memory 804 or in the drive unit 806.

[0142] Additionally or alternatively, the computer system 800 includes an input / output device 812 configured to allow a user to interact with any of the components of the computer system 800. The input / output device 812 is a number pad, a keyboard, a cursor control device, such as a mouse, a joystick, touch screen display, remote control, or any other device operative to interact with the computer system 800.

[0143] The computer system 800 also includes the drive unit 806 implemented as a disk or optical drive. The drive unit 806 includes a computer-readable medium 822 in which one or more sets of instructions 824, e.g. software, is embedded. Further, the sets of instructions 824 embodies one or more of the methods or logic as described herein. Instructions 824 resides completely or partially within memory 804 and / or within processor 802 during execution by the computer system 800. The memory 804 and the processor 802 also include computer-readable media as discussed above.

[0144] In some systems, computer-readable medium 822 includes the set of instructions 824 or receives and executes the set of instructions 824 responsive to a propagated signal so that a device connected to network 830 communicates voice, video, audio, images, or any other data over network 830. Further, the sets of instructions 824 are transmitted or received over the network 830 via the communication port or interface 820, and / or using the bus 808. The communication port or interface 820 is a part of the processor 802 or is a separate component. The communication port or interface 820 is created in software or is a physical connection in hardware. The communication port or interface 820 is configured to connect with the network 830, external media, display 810, or any other components in the computer system 800, or combinations thereof. The connection with network830 is a physical connection, such as a wired Ethernet connection, or is established wirelessly as discussed below. Likewise, the additional connections with other components of the computer system 800 are physical connections or are established wirelessly. Network 830 alternatively be directly connected to the bus 808.

[0145] While the computer-readable medium 822 is shown to be a single medium, the term "computer-readable medium" includes a single medium or multiple media, such as a centralized or distributed database, and / or associated caches and servers that store one or more sets of instructions. The term "computer-readable medium" also includes any medium that is capable of storing, encoding, or carrying a set of instructions for execution by a processor or that causes a computer system to perform any one or more of the methods or operations disclosed herein. The computer-readable medium 822 is non-transitory, and may be tangible.

[0146] The computer-readable medium 822 includes a solid-state memory such as a memory card or other package that houses one or more non-volatile readonly memories. The computer-readable medium 822 is a random-access memory or other volatile re-writable memory. Additionally or alternatively, the computer-readable medium 822 includes a magneto-optical or optical medium, such as a disk or tapes or other storage device to capture carrier wave signals such as a signal communicated over a transmission medium. A digital file attachment to an e-mail or other self-contained information archive or set of archives is considered a distribution medium that is a tangible storage medium. Accordingly, the disclosure is considered to include any one or more of a computer-readable medium or a distribution medium and other equivalents and successor media, in which data or instructions are stored.

[0147] In an alternative implementation, dedicated hardware implementations, such as application specific integrated circuits, programmable logic arrays, and other hardware devices, is constructed to implement one or more of the methods described herein. Applications that include the apparatus and systems of various implementations broadly include a variety of electronic and computer systems. One or more implementations described herein implement functions using two or more specific interconnected hardware modules or devices with related control and data signals that are communicated between and through the modules, or as portions of an application-specific integrated circuit. Accordingly, the present system encompasses software, firmware, and hardware implementations.

[0148] Computer system 800 is connected to network 830. Network 830 defines one or more networks including wired or wireless networks. The wireless network is a cellular telephone network, an 802.10, 802.16, 802.20, or WiMAX network. Further, such networks include a public network, such as the Internet, a private network, such as an intranet, or combinations thereof, and utilizes a variety of networking protocols now available or later developed including, but not limited to TCP / IP based networking protocols. Network 830 includes wide area networks (WAN), such as the Internet, local area networks (LAN), campus area networks, metropolitan area networks, a direct connection such as through a Universal Serial Bus (USB) port, or any other networks that allows for data communication. Network 830 is configured to couple one computing device to another computing device to enable communication of data between the devices. Network 830 is generally enabled to employ any form of machine-readable media for communicating information from one device to another. Network 830 includes communication methods by which information travels between computing devices. Network 830 is divided into sub-networks. The sub-networks allow access to all of the other components connected thereto or the sub-networks restrict access between the components. Network 830 is regarded as a public or private network connection and includes, for example, a virtual private network or an encryption or other security mechanism employed over the public Internet, or the like.

[0149] In accordance with various implementations of the present disclosure, the methods described herein are implemented by software programs executable by a computer system. Further, in an example, non-limited implementation, implementations can include distributed processing, component / object distributed processing, and parallel processing. Alternatively, virtual computer system processing can be constructed to implement one or more of the methods or functionality as described herein.

[0150] Although the present specification describes components and functions that are implemented in particular implementations with reference to particular standards and protocols, the disclosure is not limited to such standards and protocols. For example, standards for Internet and other packet switched network transmission (e.g., TCP / IP, UDP / IP, HTML, HTTP) represent examples of the state of the art. Such standards are periodically superseded by faster or more efficient equivalents having essentially the same functions. Accordingly, replacementstandards and protocols having the same or similar functions as those disclosed herein are considered equivalents thereof.

[0151] It will be understood that the steps of methods discussed are performed in one embodiment by an appropriate processor (or processors) of a processing (i.e., computer) system executing instructions (computer-readable code) stored in storage. It will also be understood that the disclosure is not limited to any particular implementation or programming technique and that the disclosure is implemented using any appropriate techniques for implementing the functionality described herein. The disclosure is not limited to any particular programming language or operating system.

[0152] It should be appreciated that in the above description of example embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of the present disclosure, however, is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of the present disclosure.

[0153] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the present disclosure, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0154] Furthermore, some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a computer system or by other means of carrying out the function. Thus, a processor with the necessary instructions for carrying out such a method or element of a method forms a means for carrying out the method or element of a method. Furthermore, an element described herein of an apparatus embodiment isan example of a means for carrying out the function performed by the element for the purpose of carrying out the present disclosure.

[0155] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present disclosure are practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0156] Thus, while there has been described what are believed to be the preferred embodiments of the present disclosure, those skilled in the art will recognize that other and further modifications are made thereto without departing from the spirit of the present disclosure, and it is intended to claim all such changes and modifications as falling within the scope of the present disclosure. For example, any formulas given above are merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present disclosure.

[0157] The above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other implementations, which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description. While various implementations of the disclosure have been described, it will be apparent to those of ordinary skill in the art that many more implementations and implementations are possible within the scope of the disclosure. Accordingly, the disclosure is not to be restricted except in light of the attached claims and their equivalents.

Claims

What is claimed is:1 . A method comprising: displaying, on a user device, a prompt to a user; receiving, from the user device, an input from the user, wherein the input is a query; determining, using a semantic knowledge component, one or more context items of the query; determining, using a learning engine accessing one or more algorithms and one or more data sources, one or more responses; determining, using an insight, a format of one or more outputs from the one or more responses based on at least one of the one or more context items; and outputting, on the user device, the one or more outputs and a summary based on the one or more outputs from the one or more responses.

2. The method of claim 1 , wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, or a business calendar.

3. The method of any of the preceding claims, wherein the semantic knowledge component is configured to parse and analyze the query to determine intent, conversational context and content of the query.

4. The method of claim 3, where the intent of the query indicates a type of question within the query, and the conversational context incorporates any further information previously entered by the user to clarify the query.

5. The method of any of the preceding claims, wherein the one or more context items indicate information requested in the query in a form that is readable by the learning engine.

6. The method according to claim 5, wherein the one or more context items are displayed to the user along with the query, the one or more outputs, and the summary based on the one or more outputs from the one or more responses.

7. The method of any of the preceding claims, wherein one or more sets of information relating to the query are determined using one or more of an entity identification LLM determining prompts to gather more information from the user, a conversational context LLM determining the one or more context items of the query, or an agent classifier preferably including a retrieval-based agent classifier and an agent classification LLM and determining an agent to be used for the query.

8. The method of any of the preceding claims, wherein the learning engine is configured to continuously update a set of rules and instructions for processing and determining portions of a user query.

9. The method of any of the preceding claims, wherein the one or more algorithms are provided within software agents, including both out of box agents and custom agents.

10. The method of any of the preceding claims, wherein the one or more algorithms include at least one of a time series model, a key driver analysis model, a LLM, or a transformer based model.11 . The method of any of the preceding claims, further comprising generating a description of how the one or more outputs were arrived at, and displaying the description on user request.

12. The method of any of the preceding claims, wherein the user includes a user profile, wherein the user profile includes data access permissions.

13. The method of any of the preceding claims, further including an adaptive question and answer (QA) engine component determining one or more automated questions to assist the user, displaying the one or more automated questions on the user device, receiving from the user device an automated question response, and using the automated question response for processing and determining portions of a user query.

14. The method of claim 13, wherein an automated question is determined at one or more of: before analysis of the user query; or after outputting, on the user device, the one or more outputs and the summary based on the one or more outputs from the one or more responses.

15. The method of claim 13 or 14, wherein an autonomous Al engine includes one or more sets of rules and instructions and assists the adaptive QA engine component in providing the one or more automated questions.

16. A system comprising: a memory configured to store instructions; and one or more processors configured to execute the instructions to perform operations according to any of the preceding method claims.

17. A computer program or a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations according to any of claims 1-15.

18. The computer program or the non-transitory computer-readable medium of claim 17, wherein the semantic knowledge component includes at least one of a measure relation, one or more synonyms, one or more analysis packs, or a business calendar.

19. The computer program or the non-transitory computer-readable medium of claim 17, the operations further including displaying an additional prompt to the user after outputting, on the user device, the summary based on the one or more outputs from the one or more responses.

Images