Hierarchical language model system for optimized query response generation

Abstract

A hierarchical language model system for generating optimal responses to user queries is disclosed. The system includes multiple low-level large language models (LLLMs), which are domain-specific models trained on specific datasets to generate pseudo-responses. At least one LLLM is a fine-tuned model for a specific domain. The system also includes a high-level language model (HLLM) that analyzes the pseudo-responses from the LLLMs based on criteria such as relevance, accuracy, redundancy, and domain relevance. The HLLMs use algorithms to select the best response or synthesize the best response by combining elements of multiple pseudo-responses. This allows for customization of the response by identifying and integrating the most salient information from different LLLMs. An adaptive learning component allows the HLLM selection strategy to be refined over time based on user feedback. This hierarchical ensemble approach balances the breadth of a general model with the depth of a domain-specific understanding. By orchestrating specialized low-level models with a comprehensive high-level model, the system provides accurate, refined, and consistent responses across a wide range of user queries.

Description

Technical Field

[0001] This disclosure relates to language models. More specifically, but not exclusively, this disclosure relates to a hierarchical system for generating optimal responses to user queries using a combination of a low-level language model and at least one high-level language model. Background Technology

[0002] With the emergence and development of artificial intelligence, especially deep learning, the development and deployment of large-scale language models for various applications, including chatbots, assistants, and information retrieval systems, has increased significantly. Notably, models developed by OpenAI, such as GPT-2 and GPT-3, have demonstrated human-like text generation capabilities, making them particularly valuable for answering user queries across a wide range of domains [Radford, A. et al., “Language Models are Unsupervised Multitask Learners” OpenAI, 2019].

[0003] Despite these advances, current language models still have their limitations:

[0004] 1. Generalization ability: Although large-scale models like GPT-3 are designed to be general, their vast knowledge base often results in answers that may lack specificity or fine-grained understanding for domain-specific queries.

[0005] 2. Consistency and reliability: Due to the probabilistic nature of its underlying neural network, language models may sometimes produce inconsistent or varying answers to similar or slightly different queries.

[0006] 3. Lack of specialization: While general-purpose models can answer a wide range of questions, they often fail to match the accuracy of domain-specific fine-tuned models. However, exclusively using domain-specific models may limit the breadth of topics the system can handle.

[0007] For example, comparisons between general models and domain-specific fine-tuned models (such as BioGPT, which has been fine-tuned on a large-scale biomedical literature) have shown that, although fine-tuned models lack the versatility of general models, they can outperform general models on domain-specific queries [Renqian, L. et al. (2022). "BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining", Briefings in Bioinformatics, arXiv:2210.10341].

[0008] Other studies have shown that models such as GPT-3 may provide inconsistent responses. For example, when the same question is asked multiple times, the accuracy of the answers may vary significantly—see, for example, Sheng, E., Chang, KW., Natarajan, P. and Peng, N. (2019), “The Woman Worked as a Babysitter: OnBiases in Language Generation”, arXiv:1909.01326.

[0009] One object of this disclosure is to provide an improved system and method for optimizing response. Summary of the Invention

[0010] In a first aspect, the present invention provides a method for generating an optimal response to a query, the method comprising:

[0011] - Receive queries;

[0012] - Process the query across multiple low-level large language models (LLLMs);

[0013] - Generate a tentative response to the query from each of the LLMs in the LLM;

[0014] - Analyze the pseudo-response in a High-Level Large Language Model (HLLM) based on one or more criteria; and

[0015] - The HLLM generates the optimal response based on the analysis and the query.

[0016] In a second aspect, the present invention provides an apparatus for executing a method for generating an optimal response to a query according to any of the methods described herein.

[0017] In a third aspect, the present invention provides an apparatus for generating an optimal response to a query, the apparatus comprising:

[0018] - A receiving unit configured to receive queries;

[0019] - Multiple low-level language model (LLLM) modules, each configured to generate a pseudo-response to the query; and

[0020] - High-Level Language Model (HLLM) module, which is configured as follows:

[0021] o Analyze the proposed response based on one or more criteria; and

[0022] o Generate the optimal response based on the analysis and the query.

[0023] In a fourth aspect, the present invention provides an apparatus for generating an optimal response to a query, the apparatus comprising:

[0024] - A receiving unit configured to receive queries;

[0025] - A first processing module comprising multiple low-level language models (LLLMs), each of which is configured to generate a tentative response to the query;

[0026] - A second processing module, which includes a High-Level Language Model (HLLM) module, configured as follows:

[0027] o Analyze the proposed response based on one or more criteria; and

[0028] o Generate the optimal response based on the analysis and the query.

[0029] - A storage module configured to store the LLLM, HLLM, and data associated with the query and response.

[0030] In a fifth aspect, the present invention provides a non-transitory computer-readable database stored on a memory, wherein the database includes queries and optimal responses to those queries generated by HLLM according to any of the methods described herein.

[0031] Therefore, those skilled in the art will recognize that this invention introduces a hierarchical language model system comprising multiple Large Language Models (LLLMs) at lower hierarchical levels and a Large Language Model (HLLM) at higher hierarchical levels. Each LLLM generates a pseudo-response to a query for analysis and consideration by the HLLM. The HLLM then generates an optimal response based on its analysis and the query. In some cases, the HLLM may select one of the pseudo-responses as the optimal response. In other cases, the HLLM may synthesize the optimal response by combining two or more pseudo-responses or by combining fragments from one or more pseudo-responses. For example, the HLLM may identify useful fragments from at least two pseudo-responses and curate the optimal response based on the identified useful fragments. That is, the optimal response may not be derived entirely from any single pseudo-response.

[0032] Therefore, it should be understood that the steps to generate the optimal response may include selecting or synthesizing the optimal response from two or more pseudo-responses. In these ways, HLLM can filter pseudo-responses to output the best response, or combine the best parts of different pseudo-responses into a single answer.

[0033] Preferably, HLLM can be configured to compare the best pseudo-response with its synthesized responses to determine which of these responses to select as the optimal response.

[0034] One or more criteria used by HLLM to analyze the proposed response may include: relevance; clarity; completeness; accuracy; coherence relative to the query; and / or one or more characteristics of the proposed response. Characteristics of the proposed response generated by a particular LLLM may include:

[0035] - Is this LLLM an indication of the fine-tuned model?

[0036] - The size of the pre-training dataset for this LLLM;

[0037] - The size of the fine-tuning dataset for this LLLM;

[0038] - The context length and / or attention span of the LLLM;

[0039] - Whether the LLLM is multimodal;

[0040] - Did the LLLM retrieve external information (e.g., from the internet or external documents) to create an indication of the proposed response; and / or

[0041] - The cutoff date for the training data of this LLLM.

[0042] In some implementations, the optimal response may be selected based on weights assigned to each proposed response, the weights being determined by: historical accuracy; relevance to a specific domain of the query; and / or one or more characteristics of the proposed response.

[0043] In some implementations, HLLM can use clustering techniques to group similar pseudo-responses and select or synthesize the optimal response from the densest clusters.

[0044] In some implementations, HLLM may assign a ranking to each proposed response based on one or more of the criteria, and select the highest-ranked response or a synthesis of the highest-ranked responses.

[0045] In some implementations, HLLMs may employ Bayesian inference or another probabilistic method to select the optimal response, with each LLLM's pseudo-response accompanied by a probability score.

[0046] In some implementations, HLLM can assign a diversity score to each pseudo-response based on the difference between each pseudo-response and other pseudo-responses, and select or prioritize pseudo-responses that bring unique perspectives or information.

[0047] In some cases, user feedback on prior optimal responses can be incorporated to adjust the HLLM's selection mechanism over time. For example, HLLM can use feedback mechanisms to adaptively refine its selection or synthesis process based on previous interactions and / or user feedback.

[0048] HLLM can include the use of neural network architectures within HLLM to combine two or more pseudo-responses and / or fragments.

[0049] HLLM analysis can include sentiment analysis, allowing HLLM to select or carefully craft optimal responses that are consistent with desired sentiment.

[0050] The optimal response can be output to, for example, a user interface, a computer system, or stored in a database.

[0051] The optimal response may also include text generated by HLLM to, for example, expand on any key points, coherently and / or grammatically combine different pseudo-responses / fragments, and / or explain the reasoning used to select the optimal response. Specifically, HLLM can identify and remove redundant information in the selected or synthesized pseudo-responses to form a concise optimal response. Additionally or alternatively, HLLM can refine the optimal response in post-processing steps for grammar, coherence, and / or style.

[0052] Each LLLM can additionally output a confidence score in addition to its simulated response, and HLLMs can take these confidence scores into account when selecting or synthesizing the optimal response.

[0053] In some cases, an LLLM can also output one or more characteristics of its intended response. These characteristics can be provided directly to an HLLM or stored in memory for later retrieval by the HLLM. An HLLM can also be configured to query one or more LLLMs to provide them with one or more characteristics of its intended response.

[0054] Preferably, at least one of the LLLMs is a fine-tuned large language model (LLM) for a specific domain, task, and / or dataset. Additionally or alternatively, one or more LLLMs may be a general-purpose LLM (i.e., a pre-trained LLM that has not yet been fine-tuned). In some implementations, multiple LLLMs may include multiple instances of the same LLM. In other implementations, at least some of the LLLMs in the LLLM layers may be different.

[0055] Those skilled in the art will understand that a fine-tuned LLM refers to a pre-trained language model that has undergone additional training on a smaller, more specific dataset to adapt it to a specific task or domain. Fine-tuning involves slightly adjusting the model's parameters to improve performance on the desired task while retaining the general language understanding gained during pre-training. This process helps to tailor the model to better handle domain-specific language or tasks, making it more useful and accurate for specialized applications. It should be understood that the nature and accuracy of a given LLLM's response will be influenced by the specific dataset or domain on which it was already trained.

[0056] Examples of pre-trained LLMs include BERT, Claude 1.3, Claude 2, GPT-3, GPT-3.5, GPT-4.5, Cohere, Falcon 40B, Ernie, GPT-3.5, GPT-4, LaMDA, Llama-2, Palm, Falcon-40B, Mistral7B, and any variants or versions thereof. It should be understood that these pre-trained LLMs can be customized by those skilled in the art to create fine-tuned LLMs for specific tasks or domains. Examples of domains may include, but are not limited to, medicine, finance, technology, literature, law, entertainment, mathematics, engineering, chemistry, physics, biology, communication technologies, and cryptography.

[0057] In some cases, each LLLM can be implemented using a state-of-the-art neural network architecture, such as Transformer, BERT, or any of the GPT models. For example, a medical LLLM could utilize a GPT-3 architecture specifically fine-tuned on medical corpora such as PubMed, clinical trial data, and electronic health records. The LLLM can then output probability distributions for possible responses or fragments, enabling probabilistic analysis by higher-level models.

[0058] More specifically, LLLM can leverage Transformer-based architectures such as BERT, GPT-2, GPT-3, GPT-4, or T5, which have demonstrated state-of-the-art performance on domain-specific datasets. For example, medical LLLM can utilize BERT-based models pre-trained on large corpora such as PubMed and MIMIC-III. This domain-oriented pre-training deepens the contextual understanding of biomedical-specific vocabulary and concepts.

[0059] In some implementations, HLLM is a pre-trained LLM, ideally pre-trained on massive corpora and datasets to enable broad generalization capabilities. Preferably, HLLM is a complex model, possibly similar in scale and generalization ability to models such as GPT-3.5 or GPT-4. The primary role of HLLM is to take the responses generated by LLLM, analyze them, and then select or synthesize the most appropriate answer based on the query.

[0060] HLLM can employ Transformer or GPT architectures, pre-trained on massive corpora and datasets to achieve broad generalization capabilities. Specific analytics and selection techniques can be incorporated into attention layers and decoders for synthesis and optimization. For clustering analysis, algorithms such as K-means, hierarchical clustering, or density-based clustering, along with similarity metrics, can be used to compare answer distances. Ranking methods can apply techniques such as rank summation and dense vector scoring, using metrics for grammar, coherence, relevance, and accuracy. Sentence-level fusion for synthesizing responses can be implemented using pointer networks or similar architectures to smoothly combine important segments.

[0061] Additionally or alternatively, high-level general-purpose HLLMs can employ a scaled-up TransformerBERT architecture with billions of parameters, similar to GPT-3; or in fact, state-of-the-art LLMs such as GPT-4. The massive model size allows for the capture of diverse contexts across domains. Specifically, the decoder component of an HLLM can leverage algorithms such as beam search to generate and rank multiple candidate answers for selection.

[0062] To further enhance specialization, contextual embeddings for domain-salient terms can be injected into the HLLM, allowing for dynamic adjustment of lexical weights based on the domain of the question. This extends the capabilities beyond static pre-trained representations.

[0063] Advantageously, the claimed invention significantly reduces out-of-domain errors compared to a single general-purpose model with a comparable parameter size. Furthermore, the invention provides consistent, domain-aware, and high-quality responses, maximizing the benefits of both a broad-based language model and a fine-tuned language model.

[0064] Queries can be preprocessed before being processed by LLLM. Preprocessing steps may include categorizing the query into a domain. The categorized domain can affect the weights or relevance of the LLLM. For example, if the query involves medical questions, HLLM may consider a proposed response from a medical domain-specific LLLM to have a higher weight than a proposed response from a general LLLM or a tax domain-specific LLLM.

[0065] In some cases, the domain category of a query can be used to target the query to a subset of relevant LLLMs. For example, a query involving statistics on different medical trials could be specifically targeted to one or more LLLMs fine-tuned for the medical domain and / or one or more LLLMs fine-tuned for statistical calculations. Therefore, at least one LLLM can be dynamically selected based on the nature or domain of the user's query.

[0066] In some cases, HLLM can employ meta-learning methods, allowing it to adaptively adjust its selection or synthesis strategies based on trends and / or patterns across a series of queries.

[0067] Optionally, the pseudo-response from LLLM and the selection or synthesis by HLLM occur in real time, allowing for interactive and dynamic sessions.

[0068] The selection or synthesis process performed by HLLM can also be influenced by external data sources or real-time data feeds, allowing the optimal response to include the latest information.

[0069] HLLM can use attention-based neural networks to focus the analysis on the most relevant fragments of the proposed response in order to select or synthesize the optimal response.

[0070] Therefore, it can be seen that some implementation methods may further include:

[0071] - When a query is received, topic modeling is used to programmatically categorize the query into one of multiple domains;

[0072] - Assign weights to each of the multiple LLLMs and / or its intended response based on the correspondence between the domain of each LLLM and the domain of the query, such that the LLLM and / or query with a higher correspondence are assigned higher weights.

[0073] Preferably, HLLM utilizes attention-based neural networks to focus the analysis on the most relevant fragments of the proposed response in order to select or synthesize the optimal response.

[0074] The apparatus described herein can be implemented as a local device or a cloud-based system, capable of executing methods for generating optimal responses to queries, preferably through a structured workflow involving multiple layers of language models and analysis.

[0075] Those skilled in the art will recognize that various methods for selecting, synthesizing, and optimizing responses by HLLM, as detailed herein, can further enhance the system's adaptability and performance across diverse user queries.

[0076] By orchestrating the integration of low-level specialized models supervised by high-level general models, the claimed methods, apparatus, and systems can respond to domain-specific queries (e.g., specialized medical and legal inquiries) and general queries with greater accuracy and relevance. Furthermore, the claimed apparatus, systems, and methods overcome the consistency and specialization problems faced by individual state-of-the-art models.

[0077] It should be understood that any feature of any aspect, example, or implementation described herein may be applied, where appropriate, to any other aspect, example, or implementation described herein. In the case of references to different implementations or sets of implementations, it should be understood that these are not necessarily different but may overlap. Attached Figure Description

[0078] Some preferred embodiments of the invention will now be described by way of example only, with reference to the accompanying drawings, in which:

[0079] Figure 1 It is a high-level diagram showing the components of a hierarchical language model system, including the user interface, LLLM, HLLM, and data flow.

[0080] Figure 2 Examples of LLLM specialists in different fields are given, such as medicine, finance, and engineering.

[0081] Figure 3 The structure of neural network architectures and algorithms used in high-level general HLLM models is illustrated.

[0082] Figure 4 This illustrates a possible flow from receiving a user query to generating the final response.

[0083] Figure 5 The synthesis steps illustrate how fragments from different LLLM responses can be combined.

[0084] Figure 6 It describes how user ratings of responses are fed back to improve HLLM over time. Detailed Implementation

[0085] Figure 1 A high-level diagram of the overall architecture and components of a hierarchical language model system for generating optimal responses to user queries is shown. The system includes:

[0086] The user interface (100) includes a communication channel through which user queries (101) are received as input text, images, video, or audio. The interface (100) may include components such as a microphone, touchscreen, camera, keyboard, speaker, etc. It also transmits the final response (108) back to the user.

[0087] Multiple low-level language model (LLLM) modules (102) act as domain experts, each trained on data from a specific domain or possessing specific properties that make it more efficient for a particular task. Example domains include medicine, engineering, finance, etc. Examples of specific properties include the ability to process video or audio data, the ability to process complex mathematical formulas, etc. Four example LLLM modules (102A, 102B, 102C, and 102D) are shown, but any number of LLLM modules can exist in the system. User queries (101) are propagated to all relevant LLLM modules (102) substantially simultaneously. In some cases, user queries can be served to all LLLMs in the LLLMs, regardless of which LLLM modules are relevant to the topic / domain of the user query.

[0088] The High-Level Language Model (HLLM) module (103) operates at the overall system level to analyze, select, and synthesize the most appropriate response derived from the pseudo-response generated by the HLLM (102). The HLLM (103) implements the algorithms detailed in this disclosure for identifying the optimal response based on relevance, accuracy, redundancy, and / or other criteria.

[0089] The central controller / processor (not shown) processing modules (102, 103) coordinate the data flow. It sends user queries (101) to the LLLM (102), collects their responses (105), feeds them to the HLLM (103), obtains the final response (106), and propagates it back (108) via the user interface (100). In addition to sending queries to the LLLM, the central controller / processor can also send queries directly to the HLLM. Alternatively, queries can be sent to the HLLM by one or more LLLMs along with their responses.

[0090] The database (107) may optionally serve as a system storage component, storing corpora and datasets used to train the LLLM (102), queries (101), associated pseudo-responses (105A to 105D), associated optimal responses (106), interaction history for learning by feedback from the HLLM (103), and other system data. The optimal response may be transmitted directly from the HLLM (103) to the user interface (100), for example, for interactive user access, such as in a chat; or it may be stored in the database (107). In the latter case, the user can query the database (107) using the user interface (100). In both cases, the user interface (100) communicates the optimal response to the user (108).

[0091] Ultimately, feedback (such as assessments of the quality or relevance of the optimal response obtained from users or other quality assessment systems) can be sent to the HLLM (103) to improve the quality of future responses. This is in Figure 1 It is depicted as an optional feedback path (109).

[0092] In short, Figure 1 It provides a comprehensive overview of the key hardware and software modules in the layered language model system, as well as high-level workflows. The interfaces between components are designed to be adaptable to different specific implementations based on system requirements.

[0093] Operating procedures

[0094] refer to Figure 1 When a user query is received through the interface, the system simultaneously sends the query to all relevant LLLMs. LLLMs can be determined as relevant based on whether the domain or dataset associated with the LLLM corresponds to the domain or topic of the query.

[0095] Each relevant LLLM then generates a pseudo-response based on its training and expertise. The pseudo-responses can then be compiled into a set of candidate answers.

[0096] The HLLM receives the pseudo-response directly from the HLLM. Alternatively, the pseudo-response can be stored in memory (107), from which the HLLM can retrieve the pseudo-response. The HLLM then proceeds with a multi-step analysis:

[0097] 1. Relevance check: HLLM evaluates the relevance of each answer to the original query, and may assign scores based on how well each answer solves the user's question.

[0098] 2. Redundancy Removal: To ensure diverse perspectives, HLLM identifies and removes or downgrades redundant or highly similar answers.

[0099] 3. Response synthesis: Based on the analyzed response, HLLM can choose to directly select the best response, or it can choose to synthesize a new response by integrating elements from responses generated by multiple LLLMs (i.e., from one or more pseudo-responses).

[0100] 4. Optimization: Before finalizing the answer, HLLM can perform additional refinement to ensure that the response is coherent, grammatically correct, and stylistically appropriate for the target audience or application.

[0101] Finally, the selected or synthesized answer is sent back to the user through the interface.

[0102] During relevance analysis, HLLM can employ an attention mechanism to compare question and answer embeddings, using cosine similarity to assign relevance scores. Answers with low relevance scores are filtered out.

[0103] For redundancy removal, fuzzy string matching can be used to identify semantic overlap between answers. A threshold can determine when the overlap is too high. Clustering algorithms (such as K-means) can also group semantically similar answers / pseudo-responses.

[0104] When synthesizing optimal responses, HLLM can use a sequence-to-sequence model with pointer networks to fluently combine key segments from different answers / quasi-responses. Pointers will allow for the direct copying of text segments from the input.

[0105] Bundle search methods can be used to generate multiple candidate combinations. These candidates are then ranked using a scaled dot product attention mechanism, which learns the optimal way to fuse the answers. The highest-scoring combination can then be selected as the synthesized response.

[0106] During optimization, HLLM or a standalone language model fine-tuned in the target domain can be used to rewrite, correct grammar, and improve the overall coherence of the final response.

[0107] Synthesis and optimization significantly improve relevance compared to simply selecting among potential responses. The overall pipeline reduces repetitive and irrelevant responses compared to a single LLLM.

[0108] Figure 4 This illustrates the step-by-step workflow of a hierarchical language model system from receiving a user query to generating the final response. The process includes:

[0109] i. Query Input (400) — The user enters a query through the user interface, which is received by the central controller.

[0110] ii. Query Sending (401) - The central controller forwards the query to all relevant LLLM modules and optionally to HLLM.

[0111] iii. LLLM response generation (402) — Each LLLM independently analyzes the query and generates candidate / proto-response based on its domain-specific training.

[0112] iv. Compile the response — The controller compiles all the responses generated by LLLM into a collective set of candidate answers.

[0113] v. Send to HLLM (403) — The controller sends a set of candidate responses to the HLLM.

[0114] vi. HLLM analysis (404) — HLLM analyzes each candidate response based on criteria such as relevance, accuracy, and redundancy.

[0115] vii. Optimal Selection / Synthesis (405) — HLLM uses its algorithm to select the best response or synthesize a new response from components.

[0116] viii. Post-processing (406) - Optimized responses can undergo syntax / style refinement.

[0117] ix. Output response (407) — The final response is sent back to the user interface for immediate access by the user, and / or stored in the database (107) for later access by the user.

[0118] x. User feedback (408) - Users can optionally provide feedback ratings on the response, enabling HLLM to improve over time (409).

[0119] xi. Optionally, HLLM can be refined in real time to enable feedback-based improvements. HLLM will then repeat response analysis and selection and / or synthesis until the optimized response meets a specific quality metric. This in Figure 4 The middle line is shown as a dashed line (410).

[0120] In short, Figure 4 The workflow and key steps in generating the system's response to user queries are outlined. This process utilizes both low-level domain-specific models and high-level general models to produce optimized output.

[0121] Figure 2 This illustrates a domain-specific architecture for a low-level language model (LLLM) in a hierarchical system. Multiple LLLM modules exist, each trained on data from a specific domain or knowledge domain.

[0122] As depicted, example areas include:

[0123] • Medical LLLM (200A) – Trained on datasets such as medical textbooks, journals, and patient health records to generate responses to medical queries.

[0124] • Financial LLLM (200B) – Trained on earnings reports, financial news, and stock data to respond to finance-related questions.

[0125] • Engineering LLLM (200C) – Training in textbooks, publications, manuals, standards and other technical documents to provide answers focused on engineering.

[0126] • Law LLLM (200D) – Training in statute law, case law, law textbooks, and other legal content to handle law-related inquiries.

[0127] The number and specificity of LLLMs can be adaptively adjusted according to the breadth of the domains the system aims to cover. LLLMs can leverage their domain-specific lexicons, ontology, and linguistic conventions.

[0128] Their training utilizes state-of-the-art neural network architectures like Transformer, BERT, and GPT-3, tailored to each domain. This modular, domain-based architecture allows for a combination of specialized understanding and overall versatility.

[0129] In short, Figure 2 It provides an overview of the domain specialization employed in low-level language models to enable optimized responses to diverse queries. The multi-domain approach balances broad knowledge with targeted expertise.

[0130] Figure 3 This illustrates the internal architecture and algorithms implemented within a High-Level Language Model (HLLM) for analyzing candidate / quasi-responses and generating optimal output / response. Components include:

[0131] • Encoder (300) – Employs a self-attention layer to map the input query and candidate response to a dense vector representation. This helps in evaluating relevance.

[0132] • Candidate analysis (301) – Components such as clustering, scoring, or ranking are used to evaluate accuracy, redundancy, and other factors.

[0133] • Decoder (302) – Uses algorithms such as bundle search to generate and rank multiple optimized candidate responses for selection.

[0134] • Synthesis mechanism (303) – Pointer network, seq2seq model and attention layer combine relevant segments of candidate answers.

[0135] • Post-processing (304) – Weights, heuristics, and language models refine the output for coherence, syntax, and style.

[0136] • Policy update unit (305) – The reinforcement learning algorithm updates the model parameters based on user feedback to improve the response optimization policy.

[0137] • External knowledge sources (306) – API, database and corpus resources enhance the contextual knowledge of the model.

[0138] These components leverage state-of-the-art techniques in deep learning and natural language processing to balance generalization capabilities with custom optimization capabilities. The model is built on standard architectures like BERT and GPT-3.

[0139] In short, Figure 3 This provides an overview of the algorithms and architectural components within HLLM that analyze and generate high-quality responses tailored to input queries. These units are selected to be able to be trained and adaptively tuned using feedback data.

[0140] Figure 5 This provides a more detailed view of the response synthesis mechanism implemented in HLLM to combine relevant components from candidate answers into an optimized output. The steps include:

[0141] 1. Input Candidates (500) – Candidate responses generated by LLLM are fed into the HLLM synthesis algorithm.

[0142] 2. Segmentation (501) – Each candidate is segmented into blocks or phrases representing different ideas using boundary detection techniques.

[0143] 3. Content scoring (502) – The relevance of each block to the original query is quantified using similarity metrics and weights.

[0144] 4. Pointer network (503) – Key fragments are copied from candidates as pointers to form a sequence-to-sequence mapping.

[0145] 5. Combination (504) – Pointers are smoothly combined into a complete sequence using an encoder-decoder architecture.

[0146] 6. Candidate generation (505) – Bundle search generates multiple candidate combinations of synthesis for consideration.

[0147] 7. Candidate Scoring (506) – Candidates are evaluated for coherence, conciseness, grammar and other attributes.

[0148] 8. Selection (507) – The candidate synthesis with the highest score is selected as the optimal response.

[0149] Synthesis techniques utilize innovations like pointer networks to seamlessly stitch together relevant content from multiple sources. This allows for tailored responses to user queries by identifying and combining the most relevant information.

[0150] In short, Figure 5 The synthesis process was streamlined to create optimized and nuanced responses that integrate the strengths of multiple specialized LLLMs. These components were designed to be trained to continuously improve the quality of the synthesis.

[0151] Adaptive learning and feedback loop

[0152] An optional but valuable component of the system is the feedback loop. After the HLLM provides the final answer, the user can provide feedback, rating the quality, relevance, and accuracy of the response. This feedback can be used to fine-tune the selection or synthesis process of the HLLM, making the system more accurate and efficient over time.

[0153] The adaptive techniques of feedback loops can draw on reinforcement learning principles, constructing rewards from user ratings and preferences. Details such as reward architecture, the balance between exploration and exploitation, and experience replay can help optimize system performance across multiple query-response cycles. Feedback loops enable HLLM to refine its selection and synthesis capabilities over multiple query-response cycles. User ratings on dimensions such as relevance and coherence can be compiled into rewards.

[0154] Policy gradient reinforcement learning algorithms can use these reward signals to update parameters that control the ranking, combination, and optimization phases. Actions that generate higher rewards can be reinforced.

[0155] To balance exploration, epsilon-greedy selection can be used. HLLM can take advantage of learned high-reward strategies most of the time, but sometimes it can explore new selection mechanisms.

[0156] Synthesized responses, user ratings, and the HLLM's internal Q-value can be stored in a database. This experience replay allows for periodic retraining of the HLLM on past successful and unsuccessful responses to prevent overfitting.

[0157] It should be understood that feedback loops can improve the cumulative success rate of responses. HLLM can evolve its strategy based on Darwinian selection of high-performance actions.

[0158] Figure 6 An example of an adaptive feedback loop implemented in the system is shown to allow for continuous learning and improvement of the response optimization capabilities of the High-Level Language Model (HLLM). The process includes:

[0159] 1. Generate Response (600) — HLLM generates a response to the user query based on its current policy.

[0160] 2. Presented to the user (601) – The response is provided to the user through the interface.

[0161] 3. User ratings of the response (602) – Users rate the response on dimensions such as relevance, accuracy, and consistency.

[0162] 4. Compilation Feedback (603) — Scores and feedback are compiled into reward signals.

[0163] 5. Feedback to HLLM (604) – The reward is fed back to the HLLM's policy update unit.

[0164] 6. Policy Adaptation (605) – HLLM uses reinforcement learning to update its parameters to evolve its response optimization policy.

[0165] 7. Historical data storage (606) – Interaction data is stored for training and testing to prevent overfitting.

[0166] 8. Model retraining (607) - HLLM is retrained periodically on accumulated interactive data.

[0167] Feedback loops allow the selection and composition capabilities of HLLMs to adaptively adjust over time based on experiential user interactions. This improves performance, customization, and human-like conversations.

[0168] In short, Figure 6 The self-improvement mechanisms built into the system architecture are outlined. These components enable empirical learning by the model to optimize the session flow.

[0169] Specialized processing of mixed input modes

[0170] In another example of the invention, the system may include a hybrid of low-level language models (LLLMs) with different specializations in processing different types of input data. For example, a formula processing LLLM may excel at using, for example, LaTeX decoding, symbolic computation, and mathematically aware encoders to analyze queries containing mathematical expressions and formulas. A table processing LLLM may combine techniques from semantic parsing and data mining to interpret queries about statistical tables. A graphics LLLM may leverage computer vision and multimodal understanding to extract meaning from charts, diagrams, and other visual elements.

[0171] When responding to queries about technical documents (such as technical standards) that contain a mix of text, graphs, tables, and mathematical symbols, a High-Level Language Model (HLLM) can delegate different modalities to the specialized LLLM best suited for each modality. For text paragraphs, the HLLM can rely on an LLLM with strong natural language capabilities. To interpret mathematical formulas, it can invoke a formula processing LLLM. For tabular information, it can leverage a table processing LLLM. For diagrams, it can use a graph processing LLLM. The HLLM then assimilates complementary inputs, inferences, and conclusions from the diverse LLLMs. It can use meta-learning techniques to combine these fragmented insights across modalities into a unified response that synthesizes salient phrases. This will enable the system to process mixed-modal documents and leverage mixed inference across specialized models tailored to different input types. The HLLM orchestrates these models to produce coherent responses that integrate multimodal analysis.

[0172] In a system architecture with a low-level LLLM dedicated to a specific modality (e.g., text, formulas, tables, graphs), a user's original query can be directly routed to a high-level language model (HLLM) instead of being broadcast to all LLLMs simultaneously. The HLLM's encoder then parses and interprets the initial user query to determine which components involve text paragraphs, mathematical symbols, tabular data, visual illustrations, etc.

[0173] Based on this modulation analysis, the HLLM can then selectively route specific parts, subqueries, or extracted features to the specialized LLLM best suited to that modality. For example, detected formula fragments might be passed to a formula processing LLLM, tabular data might be delegated to a table processing LLLM, and graph / image features might be sent to a graph LLLM. This allows for efficient distributed processing by assigning each query component to the most relevant specialized LLLM based on the modality pre-detected by the HLLM.

[0174] HLLM will later assimilate the output from the modality-specific LLLM into a combined response. Direct routing via HLLM (instead of broadcasting the full query) thus enables the use of the most appropriate LLLM for different modalities.

[0175] Improved inter-model dialogue for responses

[0176] In some examples of the invention, the high-level language model (HLLM) may additionally engage in dialogue with one or more low-level language models (LLLM) to request clarification, additional information, or refinement to improve the quality of the proposed response.

[0177] For example, if the initial proposed response from Medical LLLM contains ambiguous abbreviations or terms, HLLM can query Medical LLLM to expand on unclear terms to make the answer more understandable.

[0178] As another example, HLLM can detect that the proposed response from engineering LLLM lacks sufficient detail or explanation of key principles. HLLM can then probe engineering LLLM with subsequent questions to fill in the gaps in the initial response.

[0179] HLLM can also request financial LLLM to review key numerical data in its proposed response, or request the latest values ​​from real-time market data feeds.

[0180] Additionally, if the initial proposed response appears incomplete, biased, or contradictory, HLLM can involve a single LLLM in the back-and-forth conversation to clarify ambiguous points, enhance objectivity, provide supporting references or document excerpts, or resolve conflicting statements.

[0181] Dialogue can employ conversational protocols and conventions common in the LLLM field to elicit an expansive, balanced, and comprehensive perspective.

[0182] The interaction between these models leverages the specialized knowledge within each LLLM to iteratively improve response quality in terms of completeness, accuracy, objectivity, and consistency.

[0183] HLLMs can use a hybrid approach to synthesize their own final response after the dialogue ends, rather than directly selecting the response of any single LLLM.

[0184] LLLM capability profile for optimized routing

[0185] Beyond modal analysis of queries, High-Level Language Models (HLLMs) can also profile the capabilities of each Low-Level Language Model (LLM) through dialogue. During initialization or periodically, HLLMs can probe LLLMs using domain-specific questions, boundary cases, and example inputs to benchmark their strengths and limitations. For instance, an HLLM can evaluate the mathematical reasoning capabilities of a formula processing LLLM by posing complex physics problems requiring calculus or linear algebra. A table processing LLLM can be tested on handling complex datasets and queries requiring multivariate analysis. HLLMs catalog the capabilities discovered by each LLLM into a knowledge base.

[0186] When routing query components, HLLM consults the knowledge base to optimally assign each component based on both modulation type and aligned LLLM competence. For example, a statistically dense table extracted from a query might be routed to a table processing LLLM specializing in econometrics, rather than a general table processing model.

[0187] Dynamic capability profiling via interrogative dialogue allows HLLMs to make evidence-based routing decisions, matching query facets with the most appropriate LLLM, rather than being limited to a single modality. This results in responses tailored to the specialized competencies integrated into the underlying LLLM.

[0188] Advantageously, the hierarchical language model system proposed in this paper provides a balanced blend of broad knowledge encapsulated in a general model and refined, domain-specific expertise in a specialized model. By orchestrating these components in a hierarchical manner, the system significantly improves consistency, reliability, and accuracy in generating responses to a wide variety of user queries.

Claims

1. A method for generating an optimal response to a query, the method comprising: a. Receive queries; b. Process the query in multiple low-level large language models (LLLMs), wherein at least one of the LLLMs is a fine-tuned large language model; c. Generate a tentative response to the query from each of the LLLMs. d. Analyze the proposed response in a High-Level Large Language Model (HLLM) based on one or more criteria; e. The HLLM generates the optimal response based on the analysis and the query.

2. The method of claim 1, wherein the criteria include relevance, accuracy, coherence relative to the query, and / or one or more characteristics of the intended response.

3. The method of claim 1, wherein the step of generating the optimal response comprises selecting or synthesizing an optimal response from two or more pseudo-responses.

4. The method of claim 1, further comprising a mechanism for adjusting the selection of the HLLM over time by incorporating user feedback on the prior optimal response.

5. The method of claim 1, wherein the HLLM identifies useful fragments from at least two of the simulated responses and selects the optimal response based on the useful fragments.

6. The method of claim 1, wherein the HLLM identifies and removes redundant information in the selected or synthesized optimal response.

7. The method of claim 1, wherein the HLLM assigns a diversity score to each pseudo-response based on the difference between each pseudo-response and other pseudo-responses, and selects or prioritizes pseudo-responses that bring unique perspectives or information.

8. The method of claim 1, further comprising using a neural network architecture within the HLLM to combine two or more pseudo-responses and / or fragments.

9. The method of claim 1, wherein the HLLM synthesizes portions of multiple pseudo-responses to generate an optimal response that is not entirely derived from any single LLLM.

10. The method of claim 1, further comprising a post-processing step, wherein the selected or synthesized optimal answer is refined for grammar, coherence, and / or style.

11. The method of claim 1, wherein the HLLM employs a meta-learning approach, allowing it to adaptively adjust its selection and / or synthesis strategies based on trends and / or patterns across a series of queries.

12. The method of claim 1, wherein the pseudo-response from the LLLM and the selection or synthesis performed by the HLLM occur in real time, allowing interactive and dynamic sessions.

13. An apparatus for generating an optimal response to a query, the apparatus comprising: a. A receiving unit configured to receive queries; b. Multiple low-level language model (LLLM) modules, each configured to generate a pseudo-response to the query; and c. A High-Level Language Model (HLLM) module, which is configured as follows: d. Analyze the proposed response based on one or more criteria; and e. Generate the optimal response based on the analysis and the query.

14. The apparatus of claim 13, wherein the criteria include relevance, accuracy, coherence relative to the query, and / or one or more characteristics of the intended response.

15. The apparatus of claim 13, wherein the HLLM is configured to select or synthesize the optimal response from two or more pseudo-responses.

16. The apparatus of claim 13, further comprising a feedback component coupled to the HLLM module, allowing the HLLM module to refine its selection or synthesis process based on previous interactions and responses.

17. The apparatus of claim 13, wherein the HLLM module is further configured to split the pseudo-response into fragments and combine the different fragments from one or more LLLM modules to form an optimal response.

18. The apparatus of claim 13, wherein the HLLM module is further configured to identify and remove redundant information in the selected or synthesized optimal response.

19. The apparatus of claim 13, wherein the HLLM module is further configured to: assign a diversity score to each pseudo-response based on the difference between each pseudo-response and other pseudo-responses; and select or prioritize pseudo-responses that bring unique perspectives or information.

20. The apparatus of claim 13, wherein the HLLM module includes a neural network architecture to combine two or more pseudo-responses and / or fragments.

21. The apparatus of claim 13, wherein the HLLM module is further configured to synthesize portions of a plurality of pseudo-responses to generate an optimal response that is not entirely derived from the answers of any single LLLM module.

22. The apparatus of claim 13, further comprising a post-processing module configured to refine the selected or synthesized optimal answer for grammar, coherence, and / or style.

23. The apparatus of claim 13, wherein the HLLM module is further configured to employ a meta-learning method that allows it to adaptively adjust its selection and / or synthesis strategy based on trends and / or patterns across a series of queries.

24. The apparatus of claim 13, wherein the communication interface is further configured to enable real-time sending of queries and responses, allowing interactive sessions.

25. A non-transitory computer-readable database stored in memory, wherein the database includes queries and optimal responses to the queries generated by HLLM according to any of the methods described herein.

Images