Hierarchical language model system for optimized query response generation

Abstract

A hierarchical language model system is disclosed for generating optimal responses to user queries. The system comprises multiple lower-level large language models (LLLMs) 102A-102D that may be domain-specific models trained on particular datasets to generate proposed responses 105A-105D. At least one LLLM is a fine-tuned model for a specialized domain. The system also includes a higher-level language model (HLLM) 103 that analyses the proposed responses from the LLLMs, preferably based on criteria such as relevance, accuracy, redundancy, and domain relevance. The HLLM can use algorithms to select the best response or synthesize an optimal response 106 by combining elements of multiple proposals, this allows customizing the response by identifying and integrating the most salient information from different LLLMs. An adaptive learning component may allow refining the HLLM’s selection strategy over time based on user feedback.

Description

FIELD OF THE INVENTION The present disclosure relates to language models. More particularly, but not exclusively, the present disclosure relates to a hierarchical system for generating optimal responses to user queries using a combination of lower-level language models and at least one higher-level language model. BACKGROUND OF THE INVENTION With the advent and growth of artificial intelligence, especially deep learning, there has been a significant surge in the development and deployment of large-scale language models for various applications, including chatbots, assistants, and information retrieval systems. Notably, models such as GPT-2 and GPT-3 developed by OpenAI have demonstrated human-like text generation capabilities, making them especially valuable for answering user queries in various domains [Radford, A., et al. "Language Models are Unsupervised Multitask Learners." OpenAI, 2019], Despite the advancements, current language models have their limitations: 1. Generalization: Although large-scale models like GPT-3 are designed to be generalists, their vast knowledge bases often lead to answers that might lack specificity or nuanced understanding for domain-specific queries. 2. Consistency and Reliability: Given the probabilistic nature of their underlying neural networks, language models can sometimes produce inconsistent or varying answers to similar or slightly altered queries. 3. Lack of Specialization: While general models can answer a broad range of questions, they often can't match the accuracy of models fine-tuned for specific domains. Yet, exclusively using domain-specific models may limit the breadth of topics the system can address. For example, comparisons between general models and models fine-tuned for certain domains, such as BioGPT which has been fine-tuned on large scale biomedical literature, has shown that while finetuned models lack the versatility of general models they can outperform general models on domainspecific queries [Renqian, L., et al.. (2022). "BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining", Briefings in Bioinformatics, arXiv:2210.10341], In other studies, it has been shown that the model such as GPT-3 can provide inconsistent responses. For example, when posed with the same question multiple times, the answers may vary significantly in accuracy - e.g., see, Sheng, E., Chang, K-W., Natarajan, P., &Peng, N. (2019), "The Woman Worked as a Babysitter: On Biases in Language Generation", arXiv:1909.01326. An object of the present disclosure is to provide an improved system and method for optimising responses. SUMMARY OF THE INVENTION From a first aspect, the invention provides a method for generating an optimal response to a query, comprising: receiving a query; processing the query in a plurality of lower-level large language models (LLLMs); generating, from each of the LLMs, a proposed response to the query analysing the proposed responses in a higher-layer large language model (HLLM) based on one or more criteria; and generating, by said HLLM, an optimal response based on said analysis and the query. From a second aspect, the invention provides an apparatus for executing a method for generating an optimal response to a query in accordance with any of the methods described herein. From a third aspect, the invention provides an apparatus for generating an optimal response to a query, comprising: a receiving unit configured to receive a query; a plurality of lower-level language model (LLLM) modules, each module being configured to generate a proposed response to the query; and a higher-level language model (HLLM) module configured to: o analyse the proposed responses based on one or more criteria; and o generate an optimal response based on said analysis and the query. From a fourth aspect, the invention provides an apparatus for generating an optimal response to a query, comprising: a receiving unit configured to receive a query; a first processing module comprising a plurality of lower-level language models (LLLMs), each configured to generate a proposed response to the query; a second processing module comprising a higher-level language model (HLLM) module configured to: o analyse the proposed responses based on one or more criteria; and o generate an optimal response based on said analysis and the query. a storage module configured to store the LLLMs, HLLM, and data related to the query and responses. From a fifth aspect, the invention provides a non-transitory computer-readable database stored on a memory, wherein the database comprises a query and an optimal response to the query generated by a HLLM in accordance with any of the methods described herein. Thus, it will be seen by those skilled in the art that the invention introduces a hierarchical language model system comprising multiple large language models in a lower hierarchical-layer (LLLMs) and a large language model in a higher hierarchical-layer (HLLM). The LLLMs each generate a proposed response to the query for the HLLM to analyse and consider. The HLLM then generates an optimal response based on its analysis and the query. In some cases, the HLLM may select one of the proposed responses as the optimal response. In other cases, the HLLM may synthesize an optimal response by combining two or more proposals together or by combining segments from one or more proposals together. For example, the HLLM may identify useful segments from at least two proposed responses and curate an optimal response based on the identified useful segments. That is, the optimal response may not be solely derived from any single proposed response. Thus, it will be understood that the step of generating the optimal response may comprise selecting or synthesizing an optimal response from two or more proposed responses. In these ways, the HLLM can filter the proposals to output the best response or incorporate the best parts of different proposals into a single answer. Preferably, the HLLM may be configured to compare the best proposed response with its synthesized response to decide which one of these responses to choose as the optimal response. The one or more criteria used by the HLLM to analyse the proposed responses may comprise: relevance; clarity; completeness; accuracy; coherence in relation to the query; and / or one or more characteristics of the proposed responses. The characteristics of a proposed response generated by a particular LLLM may include: an indication of whether the LLLM is a fine-tuned model; the size of the LLLMs pre-training dataset; the size of the LLLMs fine-tuning dataset; the LLLM's context length and / or attention span; an indication of whether the LLLM is multi-modal; an indication of whether LLLM retrieved external information (e.g. from the internet or external documents) to create the proposed response; and / or the cut-off date for the LLLM's training data. In some embodiments, the optimal response may be selected based on weights assigned to each proposed response, said weights determined by: historical accuracy; relevance to the specific domain of the query; and / or one or more characteristics of the proposed responses. In some embodiments, the HLLM may group similar proposed responses using clustering techniques and select or synthesize the optimal response from the most populated cluster. In some embodiments, the HLLM may assign ranks to each proposed response based on said one or more criteria and select the top-ranked response or a synthesis of the top-ranked responses. In some embodiments, the HLLM may employ Bayesian inference or another probabilistic approach to choose the optimal response, each LLLM's proposed response being accompanied by a probability score. In some embodiments, the HLLM may assign a diversity score to each proposed response based on its difference from other proposals and select or prioritize proposals that bring a unique perspective or information. In some cases, user feedback on prior optimal responses maybe incorporated to adjust the HLLM's selection mechanism over time. For example, the HLLM may use a feedback mechanism to adaptively refine its selection or synthesis process based on previous interactions and / or user feedback. The HLLM may comprise the use of a neural network architecture within the HLLM to combine two or more proposed responses and / or segments. The HLLM's analysis may include a sentiment analysis, allowing the HLLM to select or craft optimal responses that align with a desired sentiment. The optimal response may be outputted to, for example, a user interface, a computer system, or stored in database. The optimal response may also include text generated by the HLLM to, for example, expand on any points, coherently and / or grammatically combine different proposals / segments together, and / or to explain the reasoning for choosing the optimal response. In particular, the HLLM may identify and remove redundant information in the selected or synthesized proposal to form a concise optimal response. Additionally, or alternatively, the HLLM may refine the optimal response for grammar, coherence, and / or style in a post-processing step. Each LLLM may additionally output a confidence score alongside its proposed response, and the HLLM may take these confidence scores into account when selecting or synthesizing an optimal response. In some cases, the LLLMs may also output one or more characteristics of their proposed responses. These characteristics maybe provided to the HLLM directly or stored in memory for the HLLM to subsequently read. The HLLM may also be configured to query one or more LLLMs to provide it with one or more characteristics of their proposed responses. Preferably, at least one of the LLLMs is a fine-tuned large language model (LLM) for a specific domain, task and / or dataset. Additionally, or alternatively, one or more of the LLLMs may be a general LLM (i.e. a pre-trained LLM that has not be fine-tuned). In some embodiments, the plurality of LLLMs may comprise multiple instances of the same LLM. In other embodiments, at least some of the LLMs in the LLLM-layer may be different. Those skilled in the art will appreciate that a fine-tuned LLM refers to a pre-trained language model that has undergone additional training on a smaller, specific dataset to adapt it for a particular task or domain. Fine-tuning adjusts the model's parameters slightly to improve performance on the desired task while retaining the general language understanding acquired during pre-training. This process helps in customizing the model to better handle domain-specific language or tasks, making it more useful and accurate for specialized applications. It will be appreciated that the nature and accuracy of a given LLLM's response will be influenced by the specific dataset or domain on which it has been trained. Examples of pre-trained LLMs include BERT, Claude 1.3, Claude 2, GPT-3, GPT-3.5, GPT-4.5, Cohere, Falcon 40B, Ernie, GPT-3.5, GPT-4, LaMDA, Llama-2, Palm, Falcon-40B, Mistral 7B, and any variants or versions of these. It will be appreciated that these pre-trained LLMs may be customised by those skilled in the art to create a fine-tuned LLM for a particular task or domain. Examples of domains might include, but are not limited to, medicine, finance, technology, literature, law, entertainment, mathematics, engineering, chemistry, physics, biology, communication technology, and cryptography. In some cases, each LLLM could be implemented using state-of-the-art neural network architectures such as Transformers, BERT, or any one of the GPT models. For example, the medical LLLM could leverage a GPT-3 architecture specifically fine-tuned on medical corpora such as PubMed, clinical trial data, and electronic health records. The LLLMs may then output probability distributions over possible responses or snippets to enable probabilistic analysis by the higher-level model. More specifically, the LLLMs could leverage Transformer-based architectures such as BERT, GPT-2, GPT-3, GPT-4, orT5 which have shown state-of-the-art performance on domain-specific datasets. For example, the medical LLLM could utilize a BERT-based model pretrained on large corpora such as PubMed and M IM IC-Ill. This domain-focused pretraining deepens the contextual understanding of biomedicine-specific vocabulary and concepts. In some embodiments, the HLLM is a pre-trained LLM, ideally pre-trained on massive corpora and datasets to enable broad generalization capabilities. Preferably, the HLLM is a sophisticated model, potentially akin to models such as GPT-3.5 or GPT-4 in its scale and generalization capability. The HLLM's primary role is to take the responses generated by the LLLMs, analyze them, and then select or synthesize the most appropriate answer based on the query. The HLLM could employ a Transformer or GPT architecture pre-trained on massive corpora and datasets to enable broad generalization capabilities. Specific analysis and selection techniques may be incorporated into attention layers and a decoder to enable synthesis and optimization. For clustering analysis, algorithms such as K-means, hierarchical clustering, or density-based clustering could be used along with similarity metrics to compare answer distances. The ranking methodology could apply approaches such as sum-to-rank and dense vector scoring, using metrics for grammar, coherence, relevance, and accuracy. Sentential fusion for synthesizing responses can be achieved using pointer networks or similar architectures to fluidly combine important segments. Additionally, or alternatively, the higher-level generalist HLLM could employ a scaled-up Transformer BERT architecture with billions of parameters, similar to GPT-3; or indeed state of the art LLMs such as GPT-4. The sheer model size allows capturing diverse contexts across domains. Specifically, the HLLM's decoder component could utilize algorithms like beam search to generate and rank multiple candidate answers for selection. To further enhance specialization, contextual embeddings for domain-salient terms may be injected into the HLLM, allowing dynamic adjustment of vocabulary weights based on the question's domain. This extends capabilities beyond static pretrained representations. Advantageously, the claimed invention reduces out-of-domain errors significantly compared to single generalist models of equivalent parameter size. Moreover, the invention delivers consistent, domain-aware, and high-quality responses, maximizing the benefits of both broad-based and fine-tuned language models. The query may be pre-processed prior to being processed by the LLLMs. The pre-processing step may comprise categorising the query into a domain. The categorised domain may influence the weights or relevance of the LLLMs. For example, if the query relates to a medical question, the HLLM may consider the weight of a proposed response from an LLLM fine-tuned for the medical domain to be higher than the proposed response from a general LLLM or a fine-tuned LLLM for the tax domain. In some cases, the domain category of the query may be used to direct the query to a subset of relevant LLLMs only. For example, a query relating to the statistics of different medical trials may be exclusively directed to one or more LLLMs fine-tuned for the medical domain and / or one or more LLLMs fine-tuned for statistical calculations. Thus, at least one of the LLLMs may be dynamically chosen based on the nature or domain of the user query. In some cases, the HLLM may employ a meta-learning approach, allowing it to adapt its selection or synthesizing strategy based on trends and / or patterns across a series of queries. Optionally, the proposed responses from the LLLMs and the selection or synthesis by the HLLM occur in real-time, allowing for interactive and dynamic conversations. The selection or synthesis process by the HLLM may also be influenced by external data sources or real-time data feeds, allowing the optimal response to include up-to-date information. The HLLM may utilize attention-based neural networks to focus analysis on the most relevant segments of the proposed responses for selection or synthesis of the optimal response. Thus, it will be seen that some embodiments may also comprise: upon receiving the query, programmatically classifying the query into one of a plurality of domains using topic modelling; assigning weights to each of the plurality of LLLMs and / or their proposed responses based on the correspondence between their domain and the domain of the query, such that LLLMs and / or queries with a higher correspondence have are assigned higher weights. Preferably, the HLLM utilizes attention-based neural networks to focus analysis on the most relevant segments of the proposed responses for selection or synthesis of the optimal response. The apparatuses described herein may be implemented as a local device or a cloud-based system, capable of executing the method for generating an optimal response to queries, preferably through a structured workflow involving multiple levels of language models and analysis. It will be seen by those skilled in the art that the various methods for the selection, synthesis, and optimization of responses by the HLLM, as detailed herein, may further enhance a system's adaptability and performance across diverse user queries. By orchestrating an ensemble of lower-level specialized models overseen by a higher-level general model, the claimed methods, apparatuses and systems can respond to both domain-specific queries (e.g. specialist medical and legal questions) and general queries with greater accuracy and relevancy. Moreover, the claimed apparatuses, systems and methods overcome the consistency and specialization problems facing individual state-of-the-art models. It will be understood that features of any aspect, example or embodiment described herein may, wherever appropriate, be applied to any other aspect, example or embodiment described herein. Where reference is made to different embodiments or sets of embodiments, it should be understood that these are not necessarily distinct but may overlap. Detailed Description Certain preferred embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Fig. 1 is a high-level diagram showing the components of the hierarchical language model system, including user interface, LLLMs, HLLM, and data flows. Fig. 2 illustrates the different domain specialists as LLLMs with examples such as Medicine, Finance, Engineering. Fig. 3 illustrates the structure of the neural network architecture and algorithms used within the higher-level generalist HLLM model. Fig. 4 illustrates a possible flow from receiving user query to generating the final response. Fig. 5 illustrates a synthesis step showing how segments from different LLLM responses can be combined. Fig. 6 depicts how user ratings on responses are fed back to improve the HLLM over time. Fig. 1 shows a high-level diagram of the overall architecture and components of the hierarchical language model system for generating optimal responses to user queries. The system includes: A user interface (100) comprises the communication channel through which a user query (101) is received as input text, images, video or audio. This interface (100) could include components such as microphones, touchscreens, cameras, keyboards, speakers, etc. It also conveys the final response (108) back to the user. Multiple lower-level language model (LLLM) modules (102) act as domain specialists, each trained on data from a specific field or having specific attributes which make it more efficient for particular tasks. Example fields include medicine, engineering, finance, etc. Examples of specific attributes include the ability to process video or audio data, the ability to handle complex mathematical formulae etc. Four example LLLM modules are shown (102A, 102B, 102C and 102D), however any plural number of LLM modules could exist in the system. The user query (101) is substantially simultaneously propagated to all relevant LLLM modules (102). In some cases, the user query may be provided to all of the LLLMs, without considering which LLLM modules are relevant to subject-matter / domain of the user query. The higher-level language model (HLLM) module (103) operates at an overall system level to analyze, select and synthesize the most appropriate response from the proposed responses generated by the LLLMs (102). The HLLM (103) implements algorithms detailed in this disclosure for identifying optimal responses based on relevance, accuracy, redundancy and / or other criteria. A central controller / processor (not shown) handles coordination of the modules (102, 103) and data flows. It transmits the user query (101) to the LLLMs (102), collects their responses (105), feeds them to the HLLM (103), obtains the final response (106), and propagates it back (108) through the user interface (100). The central controller / processor may also transmit the query directly to the HLLM in addition to the LLLM's. Alternatively, the query may be transmitted to the HLLM by one or more of the LLLMs together with it / their responses. Database (107) may optionally act as a systems memory component, storing corpora and datasets for training the LLLMs (102), queries (101), the associated proposed responses (105A-105D), the associated optimal responses (106), interaction history for feedback learning by the HLLM (103), and other system data. Optimal responses may be sent directly from the HLLM (103) to the user interface (100) e.g. for interactive user access, such as in a chat; or may be stored in the database (107). In the latter case, the user may interrogate the database (107) by means of the user interface (100). In both cases, the user interface (100) communicates the optimal responses to the user (108). Finally, feedback such as the assessments of the quality or relevance of the optimum responses obtained from the user or other quality assessment systems could be transmitted to the HLLM (103) in order to improve the quality of future responses. This is depicted in Fig. 1 as an optional feedback path (109). In summary, Fig. 1 provides a broad overview of the key hardware and software modules, and the high-level workflow in the hierarchical language model system. The interfaces between components are intended to be adaptable to different implementations as per system requirements. Operational Flow With reference to Fig. 1, upon receiving a user query through an interface, the system simultaneously transmits the query to all relevant LLLMs. A LLLM may be determined as relevant based on whether the domain or dataset associated with the LLLM corresponds to the domain or subject-matter of the query. Each relevant LLLM, in turn, generates a proposed response based on its training and expertise. The proposed responses may then be compiled into a set of alternative answers. The HLLM receives the proposed responses directly from the HLLM. Alternatively, the proposed responses may be stored in the memory (107) and the HLLM may retrieve the proposed response from there. Thereafter, the HLLM embarks on a multi-step analysis: 1. Relevance Check: The HLLM evaluates each answer for its relevance to the original query, potentially assigning a score based on how well each answer addresses the user's question. 2. Redundancy Removal: To ensure diverse perspectives, the HLLM identifies and removes or de-prioritizes redundant or highly similar answers. 3. Answer Synthesis: Based on the analyzed responses, the HLLM may choose to either select the best response outright or synthesize a new answer by integrating elements from multiple LLLM-generated answers (i.e. from one or more proposed responses). 4. Optimization: Before finalizing the answer, the HLLM may perform additional refinements, ensuring the response is coherent, grammatically correct, and styled appropriately for the intended audience or application. Finally, the selected or synthesized answer is transmitted back to the user through the interface. During relevance analysis, the HLLM could employ attention mechanisms to compare question and answer embeddings, assigning relevance scores using cosine similarity. Answers with low relevance scores are filtered out. For redundancy removal, fuzzy string matching could be used to identify semantic overlaps between answers. A threshold could determine when overlap is too high. Clustering algorithms such as k-means could also group semantically similar answers / proposed responses. When synthesizing optimal responses, the HLLM may use a Sequence-to-Sequence model with pointer networks to fluidly combine important segments from different answers / proposed responses. The pointers would allow copying pieces of text directly from the inputs. A beam search method could be used to generate multiple candidate combinations. The candidates then being ranked using a scaled dot-product attention mechanism that learns optimal ways to fuse answers. The highest scoring combination could then be selected as the synthesized response. During optimization, the HLLM or a separate language model fine-tuned on the target domain may be used to paraphrase, correct grammar, and improve the overall coherence of the final response. Synthesis and optimization greatly improve relevance over merely selecting between proposed responses. The overall pipeline reduces repetitive and irrelevant responses compared to individual LLLMs. Fig. 4 illustrates the step-by-step operational workflow of the hierarchical language model system from receiving a user query to generating the final response. The process includes: i. Query Input (400) - The user enters a query through the user interface which is received by the central controller. ii. Query Transmission (401) - The central controller forwards the query to all relevant LLLMs modules and optionally to the HLLM. iii. LLLM Response Generation (402) - Each LLLM independently analyzes the query and generates a candidate / proposed response based on its domain-specific training. iv. Compile Responses - The controller compiles all LLLM generated responses into a collective set of candidate answers. v. Transmit to HLLM (403) - The controller sends the set of candidate responses to the HLLM. vi. HLLM Analysis (404) - The HLLM analyses each candidate response based on criteria such relevance, accuracy, redundancy. vii. Optimal Selection / Synthesis (405) - Using its algorithms, the HLLM selects the best response or synthesizes a new response from components. viii. Post-Processing (406) - The optimized response may undergo grammar / style refinement. ix. Output Response (407) - The final response is either sent back to the user interface to be accessed immediately by the user and / or stored in the database (107) for later access by the user. x. User Feedback (408) - The user can optionally provide feedback ratings on the response, enabling the HLLM to improve over time (409). xi. Optionally, the HLLM can be refined in real-time so as to implement improvements based on the feedback. The HLLM would then repeat the response analysis and the selection and / or synthesis until the optimized response(s) satisfied specific quality metrics. This is shown as dashed line (410) in fig. 4. In summary, Fig. 4 maps out the operational workflow and key steps in generating the system's responses to user queries. The process leverages both lower-level domain-specific models and higher-level general models to produce an optimized output. Fig. 2 illustrates the domain-specific architecture of the lower-level language models (LLLMs) in the hierarchical system. Multiple LLLM modules exist, each trained on data from a particular domain or field of knowledge. As depicted, example domains include: • Medicine LLLM (200A) - Trained on datasets like medical textbooks, journals, patient health records, etc. to generate responses to medical queries. • Finance LLLM (200B) - Trained on earnings reports, financial news, stock data to respond to finance-related questions. • Engineering LLLM (200C) - Trained on textbooks, publications, manuals, standards, and other technical documents to provide engineering-focused answers. • Legal LLLM (200D) - Trained on statutes, case law, legal textbooks, and other legal content to handle law-related queries. The number and specificity of LLLMs can be adapted based on the breadth of domains the system is meant to cover. LLLMs may utilize domain-specific vocabularies, ontologies, and language conventions particular to their field. Their training leverages state-of-the-art neural network architectures like Transformers, BERT, and GPT-3 customized for each domain. The modular domain-based architecture allows combining specialized understanding with overall versatility. In summary, Fig. 2 provides an overview of the domain specialization employed in the lower-level language models to enable optimized responses for diverse queries. The multi-domain approach balances broad knowledge with targeted expertise. Fig. 3 illustrates the internal architecture and algorithms implemented within the higher-level language model (HLLM) for analysing candidate / proposed responses and generating an optimal output / response. The components include: • Encoder (300) - Employs self-attention layers to map the input query and candidate responses to dense vector representations. Helps assess relevance. • Candidate Analysis (301) - Components such as clustering, scoring or ranking are used to evaluate accuracy, redundancy, and other factors. • Decoder (302) - Uses algorithms such as beam search to generate and rank multiple optimized candidate responses for selection. • Synthesis Mechanisms (303) - Pointer networks, seq2seq models, and attention layers combine relevant pieces of candidate answers. • Post-Processing (304) - Weights, heuristics, and language models refine output for coherence, grammar, and style. • Policy Update Unit (305) - Reinforcement learning algorithms update model parameters based on user feedback to improve response optimization policy. • External Knowledge Sources (306) - APIs, databases, and corpus resources augment the model's contextual knowledge. The components leverage the latest techniques in deep learning and natural language processing to balance generalizability with custom optimization abilities. The model builds on standard architectures like BERT and GPT-3. In summary, Fig. 3 provides an overview of the algorithms and architectural components within the HLLM that enable analysis and generation of high-quality responses tailored to the input query. The unit selections are designed to be trainable and adaptable using feedback data. Fig. 5 provides a more detailed look at the response synthesis mechanisms implemented in the HLLM to combine relevant components from candidate answers into an optimized output. The steps include: 1. Input Candidates (500) - The candidate responses generated by the LLLMs are fed into the HLLM synthesis algorithms. 2. Segmentation (501) - Each candidate is segmented into chunks or phrases representing distinct ideas using boundary detection techniques. 3. Content Scoring (502) - The relevance of each chunk to the original query is quantified using similarity metrics and weights. 4. Pointer Networks (503) - Key segments are copied from the candidates as pointers to form a sequence-to-sequence mapping. 5. Combination (504) - The pointers are fluidly combined into complete sequences using an encoder-decoder architecture. 6. Candidate Generation (505) - Beam search generates multiple synthesized candidate combinations for consideration. 7. Candidate Scoring (506) - The candidates are evaluated for coherence, conciseness, grammar, and other attributes. 8. Selection (507) - The highest scoring synthesized candidate is selected as the optimal response. The synthesis techniques leverage innovations like pointer networks to fluidly stitch relevant content from multiple sources. This allows tailoring responses to user queries by identifying and combining the most pertinent information. In summary, Fig. 5 maps out the synthesis process to create optimized and nuanced responses that integrate the strengths of multiple specialized LLLMs. The components are designed to be trainable to continually improve combination quality. Adaptive Learning and Feedback Loop An optional but valuable component of the system is a feedback loop. After the HLLM provides the final answer, users may offer feedback, rating the quality, relevance, and accuracy of the response. This feedback may be utilized to fine-tune the HLLM's selection or synthesis process, making the system more accurate and efficient over time. The feedback loop's adaptation techniques may draw from reinforcement learning principles, structuring user ratings and preferences as rewards. Details such as the reward architecture, balancing exploration vs exploitation, and experience replay may help optimize the system's performance over multiple query-response cycles. The feedback loop enables the HLLM to refine its selection and synthesis capabilities over multiple query-response cycles. User ratings on dimensions like relevance and coherence may be compiled as rewards. A policy gradient reinforcement learning algorithm may use these reward signals to update parameters that control the ranking, combination, and optimization stages. Actions that produce higher rewards may be reinforced. To balance exploration, epsilon-greedy selection may be used. The HLLM may exploit learned high-reward strategies most of the time HLLM but at times may explore new selection mechanisms. The synthesized responses, user ratings, and HLLM's internal Q-values may be stored in a database. This experience replay could allow periodic retraining of the HLLM on past successful and unsuccessful responses to prevent overfitting. It will be understood that the feedback loop may improve the cumulative success rate of responses. The HLLM may evolve its strategies based on a Darwinian selection of high-performing actions. Fig. 6 illustrates the adaptive feedback loop implemented in the system to allow continual learning and improvement of the higher-level language model's (HLLM) response optimization capabilities. The process includes: 1. Generate Response (600) - The HLLM produces a response to the user query based on its current policies. 2. Present to User (601) - The response is provided to the user through the interface. 3. User Rates Response (602) - The user rates the response on dimensions such as relevance, accuracy, coherence, etc. 4. Compile Feedback (603) - The ratings and feedback are compiled into reward signals. 5. Feedback to HLLM (604) - The rewards are fed back into the HLLM's policy update unit. 6. Policy Adaptation (605) - Using reinforcement learning, the HLLM updates its parameters to evolve its response optimization policy. 7. Historical Data Storage (606) - Interaction data is stored for training and testing to prevent overfitting. 8. Model Re-training (607) - The HLLM is periodically re-trained on accumulated interaction data. The feedback loop allows adapting the HLLM's selection and synthesis capabilities based on empirical user interactions over time. This bolsters performance, customization, and human-like conversation. In summary, Fig. 6 outlines the self-improvement mechanisms built into the system architecture. The components enable experiential learning by the models to optimize conversational flow. Specialized Handling of Mixed Input Modalities In a further example of the invention, the system may include a mix of lower-level language models (LLLMs) with distinct specializations in processing different types of input data. For instance, Formula Handling LLLMs may excel at analyzing queries containing mathematical expressions and formulas using e.g. LaTeX decoding, symbolic manipulation, and math-aware encoders. Table Processing LLLMs could incorporate techniques from semantic parsing and data mining to interpret queries about statistical tables. Graphics LLLMs may leverage computer vision and multimodal understanding to extract meaning from charts, diagrams, and other visuals. When responding to queries about technical documents (e.g. technical standards) containing a blend of text, figures, tables, and mathematical notation, the higher-level language model (HLLM) could delegate different modalities to the specialized LLLMs best suited for each. For text passages, the HLLM may rely on an LLLM with strong natural language capabilities. For interpreting a mathematical formula, it could call the Formula Handling LLLM. For tabular information, it could leverage the Table Processing LLLM. For for a diagram, it could use the Graphics LLLM. The HLLM would then assimilate the complementary inputs, inferences, and conclusions from the diverse LLLMs. Using meta-learning techniques, it could combine these fragmented insights across modalities into a unified response synthesizing the salient pieces. This would enable the system to handle mixed-modality documents and leverage hybrid reasoning across specialized models tailored for different input types. The HLLM orchestrates the models to produce cohesive responses integrating multimodal analysis. In the system architecture with lower-level LLLMs dedicated to particular modalities (e.g. text, formulas, tables, graphics), the user's original query could be routed directly to the higher-level language model (HLLM) rather than broadcasting to all LLLMs simultaneously. The HLLM's encoder would then parse and interpret the initial user query to determine which components involve textual passages, mathematical notation, tabular data, visual diagrams, etc. Based on this modulation analysis, the HLLM could then selectively route specific portions, subqueries, or extracted features to the specialized LLLM(s) optimally suited for that modality. For example, detected formula snippets might be passed to the Formula Handling LLLM, table data delegated to the Table Processing LLLM, and chart image features sent to the Graphics LLLM. This would allow allocating each query component to the most relevant specialized LLLM for efficient distributed processing based on modalities detected by the HLLM upfront. The HLLM would later assimilate the outputs from the modal-specific LLLMs into a combined response. The direct routing via the HLLM (rather than broadcasting the full query) would therefore enable leveraging the most apt LLLMs for different modalities. Inter-Model Dialogue for Response Improvement In some examples of the invention, the higher-level language model (HLLM) may additionally engage in a dialogue with one or more of the lower-level language models (LLLMs) to request clarification, additional information, or refinements to improve the quality of the proposed responses. For example, if the initial proposed response from the Medical LLLM contains ambiguous acronyms or terminology, the HLLM may query the Medical LLLM to expand on the unclear terms to make the answer more understandable. As another example, the HLLM may detect that the proposed response from the Engineering LLLM lacks sufficient detail or explanation of key principles. The HLLM can then probe the Engineering LLLM with follow-up questions to fill in gaps in the initial response. The HLLM may also request the Finance LLLM to double-check critical numerical data in its proposed response or ask for up-to-date figures from real-time market data feeds. Additionally, if the initial proposals appear incomplete, biased, or contradictory, the HLLM may engage individual LLLMs in a back-and-forth conversation to clarify ambiguous points, improve objectivity, provide supporting references or excerpts of documents or resolve conflicting statements. The dialogue exchange may employ conversational protocols and conventions common in the LLLMs' domains to elicit expanded, balanced, and well-rounded perspectives. This inter-model interaction enables tapping into the specialized knowledge within each LLLM to iteratively improve response quality in terms of completeness, accuracy, objectivity, and coherence. The HLLM may use a hybrid approach, synthesizing its own final response after concluding the dialogue exchanges rather than selecting any single LLLM's response outright. LLLM Capability Profiling for Optimized Routing In addition to modality analysis on the query, the higher-level language model (HLLM) can profile the capabilities of each lower-level LLM (LLLM) through dialogue exchanges. During initialization or periodically, the HLLM can probe the LLLMs with domain-specific questions, edge cases, and sample inputs to benchmark strengths and limitations. For instance, the HLLM may evaluate the Formula Handling LLLM on mathematical reasoning skills by posing esoteric physics problems requiring calculus or linear algebra. The Table Processing LLLM may be tested on handling complex datasets and queries requiring multivariate analysis. The HLLM catalogues the capabilities uncovered for each LLLM into a knowledge base. When routing query components, the HLLM consults this knowledge base to allocate portions optimally based on both modulation type and aligned LLLM competencies. For example, a statisticsheavy table extracted from the query may get routed to a Table Processing LLLM specialized in econometrics rather than a generalist table-handling model. Dynamic capability profiling via interrogative dialogues allows the HLLM to make informed routing decisions to match query facets with the most apt LLLMs beyond just modalities. This results in responses tailored to the specialized competencies of the underlying LLLM ensemble. Advantageously, the hierarchical language model system proposed herein offers a balanced blend of the vast knowledge encapsulated in generalist models and the nuanced, domain-specific expertise of specialized models. By orchestrating these components in a hierarchical manner, the system significantly improves consistency, reliability, and accuracy in generating responses to a wide array of user queries.

Claims

16 10241. A method for generating an optimal response to a query, comprising:a. receiving a query;b. processing the query in a plurality of lower-level large language models (LLLMs), wherein at least one of the LLLMs is a fine-tuned large language model;c. generating, from each of the LLLMs, a proposed response to the queryd. analysing the proposed responses in a higher-layer large language model (HLLM) based on one or more criteria;e. generating, by said HLLM, an optimal response based on said analysis and the query.

2. The method of Claim 1, wherein the criteria comprise relevance, accuracy, coherence in relation to the query, and / or one or more characteristics of the proposed responses.

3. The method of Claim 1, wherein the step of generating the optimal response comprises selecting or synthesizing an optimal response from two of more proposed responses.

4. The method of Claim 1, further comprising incorporating user feedback on prior optimal responses to adjust the HLLM's selection mechanism overtime.

5. The method of Claim 1, wherein the HLLM identifies useful segments from at least two of the proposed responses and curates an optimal response based on the useful segments.

6. The method of Claim 1, wherein the HLLM identifies and removes redundant information in the selected or synthesized optimal response.

7. The method of Claim 1, wherein the HLLM assigns a diversity score to each proposed response based on its difference from other proposals and selects or prioritizes proposals that bring a unique perspective or information.

8. The method of Claim 1, further comprising the use of a neural network architecture within the HLLM to combine two or more proposed responses and / or segments.

9. The method of Claim 1, wherein the HLLM synthesizes portions of multiple proposed responses to generate an optimal response that is not solely derived from any single LLLM's answer.

10. The method of Claim 1, further comprising a post-processing step wherein the selected or synthesized optimal answer is refined for grammar, coherence, and / or style.

11. The method of Claim 1, wherein the HLLM employs a meta-learning approach, allowing it to adapt its selection and / or synthesizing strategy based on trends and / or patterns across a series of queries.16 102412. The method of Claim 1, wherein the proposed responses from the LLLMs and the selection or synthesis by the HLLM occur in real-time, allowing for interactive and dynamic conversations.

13. An apparatus for generating an optimal response to a query, comprising:a. a receiving unit configured to receive a query;b. a plurality of lower-level language model (LLLM) modules, each module being configured to generate a proposed response to the query; andc. a higher-level language model (HLLM) module configured to:d. analyse the proposed responses based on one or more criteria; ande. generate an optimal response based on said analysis and the query.

14. The apparatus of Claim 13, wherein the criteria comprise relevance, accuracy, coherence in relation to the query, and / or one or more characteristics of the proposed responses.

15. The apparatus of Claim 13, wherein the HLLM is configured to select of synthesis the optimal response from two of more proposed responses.

16. The apparatus of Claim 13, further comprising feedback means coupled to the HLLM module, allowing the HLLM module to refine its selection or synthesis process based on previous interactions and responses.

17. The apparatus of Claim 13, wherein the HLLM module is further configured to break down the proposed responses into segments and combine different segments from one or more LLLM modules to form an optimal response.

18. The apparatus of Claim 13, wherein the HLLM module is further configured to identify and remove redundant information in the selected of synthesized optimal response.

19. The apparatus of Claim 13, wherein the HLLM module is further configured to: assign a diversity score to each proposed response based on its difference from other proposals; and select or prioritize proposals that bring a unique perspective or information.

20. The apparatus of Claim 13, wherein the HLLM module includes a neural network architecture to combine the two or more proposed responses and / or segments.

21. The apparatus of Claim 13, wherein the HLLM module is further configured to synthesize portions of multiple proposed responses to generate an optimal response that is not solely derived from any single LLLM module's answer.

22. The apparatus of Claim 13, further comprising a post-processing module configured to refine the selected or synthesized optimal answer for grammar, coherence, and / or style.

23. The apparatus of Claim 13, wherein the HLLM module is further configured to employ a meta-learning approach, allowing it to adapt its selection and / or synthetization strategy based on trends and / or patterns across a series of queries.

24. The apparatus of Claim 13, wherein the communication interface is further configured to enable real-time transmission of queries and responses, allowing for interactive conversations.

25. A non-transitory computer-readable database stored on a memory, wherein the database comprises a query and an optimal response to the query generated by a HLLM in accordance with any of the methods described herein.16 1024

Images