Hierarchical language model system for optimized query response generation

The hierarchical language modeling system addresses the limitations of current models by using a combination of low-level and high-level models to generate optimal responses, improving accuracy and consistency across diverse queries.

JP2026539122APending Publication Date: 2026-11-19アイピー マインド リミテッド
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2026549603
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-10-30
Filing Date
2024-10-20
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Current language models face challenges in providing specific, consistent, and reliable responses due to their generalist nature, leading to inconsistent outputs for domain-specific queries and limitations in addressing a wide range of topics.

Method used

A hierarchical language modeling system comprising multiple low-level language models (LLLMs) and a high-level language model (HLLM) that analyzes and synthesizes responses based on relevance, accuracy, and domain-specific criteria to generate optimal responses.

Benefits of technology

The system enhances response accuracy and consistency by leveraging both broad and specialized models, reducing out-of-domain errors and providing domain-aware, high-quality answers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026539122000001_ABST
    Figure 2026539122000001_ABST
Patent Text Reader

Abstract

A hierarchical language modeling system for generating optimal responses to user queries is disclosed. The system comprises multiple low-level large-scale language models (LLLMs), which are domain-specific models trained on specific datasets to generate proposed responses. At least one LLLM is a finely tuned model for a specialized domain. The system also includes high-level language models (HLLMs) that analyze the proposed responses from the LLLMs based on criteria such as relevance, precision, redundancy, and domain relevance. The HLLMs use algorithms to select the best response or synthesize an optimal response by combining elements of multiple proposals. This allows for customization of responses by identifying and integrating the most significant information from different LLLMs. Adaptive learning components allow for the improvement of the HLLM selection strategy over time based on user feedback. The hierarchical ensemble approach balances the breadth of general models with the depth of domain-specific understanding. By orchestrating specialized low-level models with comprehensive high-level models, the system provides accurate, nuanced, and consistent responses across a wide range of user queries.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to language models. More specifically, though not exclusively, the present disclosure relates to a hierarchical system for generating optimal responses to user queries using a combination of a low-level language model and at least one high-level language model.

Background Art

[0002] With the advent and growth of artificial intelligence, particularly deep learning, the development and deployment of large-scale language models for various applications including chatbots, assistants, and information retrieval systems have increased dramatically. In particular, models such as GPT-2 and GPT-3 developed by OpenAI have demonstrated human-like text generation capabilities, and are particularly valuable for answering user queries in various domains [Radford, A., et al. "Language Models are Unsupervised Multitask Learners." OpenAI, 2019].

[0003] Despite the advances, current language models have the following limitations. 1. Generalization: Large-scale models such as GPT-3 are designed to be generalist, but their vast knowledge bases often lead to responses that may lack specificity or nuanced understanding for domain-specific queries. 2. Consistency and reliability: Given the stochastic nature of these underlying neural networks, language models may generate inconsistent or varying responses to similar or slightly modified queries. 3. Lack of specialization: While general models can answer a wide range of questions, they often cannot match the accuracy of models fine-tuned for a specific domain. However, the exclusive use of domain-specific models may limit the breadth of topics that the system can address.

[0004] For example, a comparison between a general model and a model fine-tuned for a specific domain, such as BioGPT, which is fine-tuned on a large biomedical literature, shows that the fine-tuned model lacks the generality of the general model, but performs better than the general model for domain-specific queries [Renqian, L., et al. (2022). “BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining”, Briefings in Bioinformatics, arXiv:2210.10341].

[0005] Other studies have shown that models such as GPT-3 can provide inconsistent responses. For example, when the same question is presented multiple times, the accuracy of the response may vary significantly. See, for example, Sheng, E., Chang, KW., Natarajan, P., & Peng, N. (2019), “The Woman Worked as a Babysitter: On Biases in Language Generation”, arXiv:1909.01326.

[0006] The purpose of this disclosure is to provide improved systems and methods for optimizing responses. [Overview of the Initiative]

[0007] In a first aspect, the present invention relates to a method for generating an optimal response to a query, - Receiving queries and, - Processing queries in multiple low-level large-scale language models (LLLMs), -Generating proposed responses to queries from each LLM, - Analyze the proposed response in the Higher Layer Large-Scale Language Model (HLLM) based on one or more criteria, - The HLLM provides a method that includes generating an optimal response based on the analysis and query.

[0008] In a second aspect, the present invention provides an apparatus for performing a method for generating an optimal response to a query according to any of the methods described herein.

[0009] In a third aspect, the present invention relates to an apparatus for generating an optimal response to a query, - A receiving unit configured to receive queries, - Multiple Low-Level Language Model (LLLM) modules, each module configured to generate a proposed response to a query, - A high-level language model (HLLM) module, Based on one or more criteria, analyze the proposed response. The present invention provides an apparatus comprising an HLLM module configured to generate an optimal response based on the analysis and query.

[0010] From a fourth aspect, the present invention relates to an apparatus for generating an optimal response to a query, - A receiving unit configured to receive queries, - A first processing module comprising multiple low-level language models (LLLMs), each LLLM configured to generate a proposed response to a query, - A second processing module comprising a High-Level Language Model (HLLM) module, wherein the HLLM module is Based on one or more criteria, analyze the proposed response. A second processing module is configured to generate the optimal response based on the analysis and query. - Provides a device comprising a storage module configured to store LLLM, HLLM, and query and response-related data.

[0011] In a fifth aspect, the present invention provides a non-temporary computer-readable database stored in memory, the database comprising queries and optimal responses to queries generated by HLLM according to any of the methods described herein.

[0012] Therefore, those skilled in the art will see that the present invention introduces a hierarchical language modeling system comprising multiple large-scale language models in a lower-level hierarchical layer (LLLM) and a large-scale language model in a higher-level hierarchical layer (HLLM). Each LLLM generates proposed responses to queries for the HLLM to analyze and examine. The HLLM then generates an optimal response based on its analysis and queries. In some cases, the HLLM may select one of the proposed responses as the optimal response. In other cases, the HLLM may synthesize an optimal response by combining two or more proposals together, or by combining segments from one or more proposals together. For example, the HLLM may identify useful segments from at least two proposed responses and curate an optimal response based on the identified useful segments. That is, the optimal response may not be derived from any single proposed response alone.

[0013] Therefore, it will be understood that the step of generating the optimal response may include selecting or combining the best response from two or more proposed responses. In this way, HLLM can filter the proposals to output the best response, or combine the best parts of different proposals into a single answer.

[0014] Preferably, HLLM may be configured to compare the best proposed response with its synthesized response and determine which of these responses to select as the optimal response.

[0015] The one or more criteria used by the HLLM to analyze the proposed responses may include relevance, clarity, completeness, accuracy, consistency with respect to the query, and / or one or more characteristics of the proposed responses. The characteristics of the proposed response generated by a particular LLLM are, - an indicator of whether the LLLM is a fine-tuned model, - the size of the LLLM pre-training dataset, - the size of the LLLM fine-tuning dataset, - the context length and / or attention span of the LLLM, - an indicator of whether the LLLM is multimodal, - an indicator of whether the LLLM retrieved external information (e.g., from the Internet or external documents) to produce the proposed response, and / or - the cutoff date for the training data of the LLLM.

[0016] In some embodiments, the optimal response may be selected based on weights assigned to each proposed response, where the weights are determined by historical accuracy, relevance to the specific domain of the query, and / or one or more characteristics of the proposed responses.

[0017] In some embodiments, the HLLM may use clustering techniques to group similar proposed responses, and select or synthesize the optimal response from the most dense cluster.

[0018] In some embodiments, the HLLM may assign a rank to each proposed response based on the one or more criteria, and select the highest-ranked response or a synthesis of the highest-ranked responses.

[0019] In some embodiments, the HLLM may employ Bayesian estimation or another probabilistic approach to select the optimal response, where the proposed response of each LLLM is associated with a probability score.

[0020] In some embodiments, the HLLM may assign a diversity score to each proposed response based on differences from other proposals, and select or prioritize proposals that provide unique perspectives or information.

[0021] In some cases, user feedback regarding previous optimal responses may be incorporated to adjust the selection mechanism of the HLLM over time. For example, the HLLM may use a feedback mechanism to adaptively refine its selection or synthesis process based on previous interactions and / or user feedback.

[0022] The HLLM may include the use of a neural network architecture within the HLLM to combine two or more proposed responses and / or segments.

[0023] The analysis of the HLLM may include sentiment analysis, enabling the HLLM to select or generate an optimal response that matches a desired sentiment.

[0024] The optimal response may, for example, be output to a user interface, a computer system, or stored in a database.

[0025] The optimal response may also include text generated by the HLLM, for example, to expand any points, combine different proposals / segments consistently and / or grammatically, and / or explain the rationale for selecting the optimal response. In particular, the HLLM may identify and remove redundant information within the selected or synthesized proposals to form a concise optimal response. Additionally or alternatively, the HLLM may refine the optimal response with respect to grammar, consistency, and / or style in a post-processing step.

[0026] Each LLLM may further output a confidence score along with its proposed response, and the HLLM may take these confidence scores into account when selecting or synthesizing the optimal response.

[0027] In some cases, the LLLM may also output one or more characteristics of these proposed responses. These characteristics may be provided directly to the HLLM or stored in memory for later retrieval by the HLLM. The HLLM may be configured to query one or more LLLMs to provide one or more characteristics of these proposed responses.

[0028] Preferably, at least one of the LLLMs is a finely tuned Large-Scale Language Model (LLM) for a specific domain, task, and / or dataset. Additionally or alternatively, one or more of the LLLMs may be general-purpose LLMs (i.e., unfinely tuned, pre-trained LLMs). In some embodiments, multiple LLLMs may comprise multiple instances of the same LLM. In other embodiments, at least some of the LLMs in an LLLM layer may be different.

[0029] Those skilled in the art will understand that a finely tuned LLM refers to a pre-trained language model that has undergone additional training on a smaller, specific dataset to adapt it to a particular task or domain. Fine-tuning involves slightly adjusting the model's parameters to improve its performance on the desired task while retaining the general language understanding acquired during pre-training. This process helps to customize the model to better handle domain-specific languages ​​or tasks, making it more useful and accurate for specialized applications. It will be understood that the nature and accuracy of a given LLM's response are influenced by the specific dataset or domain on which it was trained.

[0030] Examples of pre-trained LLMs include BERT, Claude 1.3, Claude 2, GPT-3, GPT-3.5, GPT-4.5, Cohere, Falcon 40B, Ernie, GPT-3.5, GPT-4, LaMDA, Llama-2, Palm, Falcon 40B, Mistral 7B, and any variations or versions thereof. It will be understood that these pre-trained LLMs can be customized by those skilled in the art to create finely tuned LLMs for specific tasks or domains. Examples of domains may include, but are not limited to, medicine, finance, technology, literature, law, entertainment, mathematics, engineering, chemistry, physics, biology, communication technology, and cryptography.

[0031] In some cases, each LLLM may be implemented using a state-of-the-art neural network architecture, such as one of the Transformers, BERT, or GPT models. For example, a medical LLLM could leverage a GPT-3 architecture specifically fine-tuned for medical corpora such as PubMed, clinical trial data, and electronic health records. The LLLM could then output a probability distribution across possible responses or snippets, enabling probabilistic analysis by a high-level model.

[0032] More specifically, LLLM can leverage Transformer-based architectures such as BERT, GPT-2, GPT-3, GPT-4, or T5, which have demonstrated state-of-the-art performance on domain-specific datasets. For example, a medical LLLM could utilize a BERT-based model pre-trained on large corpora such as PubMed and MIMIC-III. This domain-focused pre-training would deepen the contextual understanding of biomedical-specific vocabulary and concepts.

[0033] In some embodiments, the HLLM is a pre-trained LLM, ideally pre-trained on a large corpus and dataset to enable broad generalization capabilities. Preferably, the HLLM is a sophisticated model, potentially similar in scale and generalization capabilities to models such as GPT-3.5 or GPT-4. The main role of the HLLM is to retrieve the responses generated by the LLLM, analyze them, and then, based on the query, select or synthesize the most appropriate answer.

[0034] HLLM can employ pre-trained Transformer or GPT architectures on large corpora and datasets to enable broad generalization capabilities. Specific analysis and selection techniques may be incorporated into the attention layer and decoder to enable synthesis and optimization. Clustering analysis may use algorithms such as K-means, hierarchical clustering, or density-based clustering along with similarity metrics to compare response distances. Ranking methods may apply approaches such as sum-to-rank and dense vector scoring using grammatical, coherence, relevance, and accuracy metrics. Sentence fusion for synthesizing responses can be achieved using pointer networks or similar architectures to fluidly combine important segments.

[0035] Additionally or alternatively, higher-level generalist HLLMs may employ scaled-up Transformer BERT architectures with billions of parameters, similar to state-of-the-art LLMs such as GPT-3, or even GPT-4. Larger model sizes allow for capturing diverse contexts across domains. Specifically, HLLM decoder components may utilize algorithms such as beam search to generate and rank multiple candidate answers for selection.

[0036] To further enhance specialization, contextual embeddings of domain-specific terms can be injected into HLLM, allowing for dynamic adjustment of lexical weights based on the domain of the question. This extends its capabilities beyond static, pre-trained representations.

[0037] Advantageously, the present invention significantly reduces out-of-domain errors compared to a single generalist model with equivalent parameter sizes. Furthermore, the present invention provides consistent, domain-aware, high-quality responses, maximizing the advantages of both broad and finely tuned language models.

[0038] Queries may be preprocessed before being processed by LLLM. The preprocessing step may include classifying the query into domains. The classified domains may influence the weighting or relevance of the LLLM. For example, if a query relates to a medical question, HLLM may consider the proposed response from an LLLM fine-tuned for the medical domain to have a higher weight than a proposed response from a general LLLM or an LLLM fine-tuned for the tax domain.

[0039] In some cases, the domain category of a query can be used to direct the query to only a subset of relevant LLLMs. For example, a query about statistics for different medical trials may be directed exclusively to one or more LLLMs fine-tuned for the medical domain and / or one or more LLLMs fine-tuned for statistical calculations. Thus, at least one LLLM may be dynamically selected based on the nature or domain of the user query.

[0040] In some cases, HLLM may employ a meta-learning approach to adapt its selection or synthesis strategy based on trends and / or patterns across a series of queries.

[0041] Optionally, the proposed responses from LLLM and the selection or synthesis by HLLM are performed in real time, enabling interactive and dynamic conversations.

[0042] The selection or synthesis process using HLLM can also be influenced by external data sources or real-time data feeds, ensuring that the optimal response includes the latest information.

[0043] HLLM utilizes attention-based neural networks to focus analysis on the most relevant segments of a proposed response for the selection or synthesis of the optimal response.

[0044] Therefore, some embodiments also, -When a query is received, topic modeling is used to programmatically classify the query into one of several domains, -It can be seen that this may include assigning weights to each of the multiple LLLMs and / or proposed responses such that higher weights are assigned to the LLLMs and / or queries that have a higher correspondence, based on the correspondence between these domains and the domain of the query.

[0045] Preferably, HLLM utilizes an attention-based neural network to focus the analysis on the most relevant segments of the proposed response for the selection or synthesis of the optimal response.

[0046] The apparatus described herein may preferably be implemented as a local device or a cloud-based system capable of performing a method for generating optimal responses to queries through a structured workflow that includes multiple levels of language modeling and analysis.

[0047] As detailed herein, those skilled in the art will understand that the various methods for selecting, composing, and optimizing responses using HLLM can further enhance the system's adaptability and performance across a wide range of user queries.

[0048] By orchestrating an ensemble of lower-level specialized models overseen by a high-level general model, the claimed methods, apparatus, and systems can respond with greater accuracy and relevance to both domain-specific queries (e.g., expert medical and legal questions) and general queries. Furthermore, the claimed apparatus, systems, and methods overcome the consistency and specialization issues faced by individual state-of-the-art models.

[0049] It will be understood that any aspect, example, or embodiment described herein may, at any time, be applied to any other aspect, example, or embodiment described herein, where appropriate. Where different embodiments or sets of embodiments are referenced, it should be understood that they may not necessarily be separate but may overlap. [Brief explanation of the drawing]

[0050] Specific preferred embodiments of the present invention are described below, merely as examples, with reference to the accompanying drawings. [Figure 1] This is a high-level diagram showing the components of a hierarchical language model system, including the user interface, LLLM, HLLM, and data flow. [Figure 2] Using examples from fields such as healthcare, finance, and engineering, the text illustrates different domain specialists as LLLMs (Limited Liability and Management). [Figure 3] This illustrates the structure of neural network architectures and algorithms used within high-level generalist HLLM models. [Figure 4] This illustrates possible flows from receiving user queries to generating the final response. [Figure 5] This illustrates a synthesis step that shows how segments from different LLLM responses can be combined. [Figure 6] This illustrates how user feedback on responses is incorporated to improve HLLM over time.

[0051] Figure 1 shows a high-level diagram of the overall architecture and components of a hierarchical language model system for generating optimal responses to user queries. The system includes: The user interface (100) includes a communication channel through which user queries (101) are received as input text, images, video, or audio. This interface (100) may include components such as a microphone, touchscreen, camera, keyboard, and speaker. It also returns a final response (108) to the user.

[0052] Multiple low-level language model (LLLM) modules (102) function as domain specialists, each trained with data from a specific domain or possessing specific attributes that make them more efficient for a particular task. Illustrative domains include medicine, engineering, and finance. Examples of specific attributes include the ability to process video or audio data, or the ability to process complex mathematical formulas. Four exemplary LLLM modules (102A, 102B, 102C, and 102D) are shown, but any number of LLM modules may exist in the system. User queries (101) are propagated substantially simultaneously to all relevant LLLM modules (102). In some cases, a user query may be provided to all LLLMs without considering which LLLM module is relevant to the subject / domain of the user query.

[0053] The High-Level Language Model (HLLM) module (103) operates at a system-wide level to analyze, select, and synthesize the most appropriate response from the proposed responses generated by LLLM(102). HLLM(103) implements algorithms detailed in this disclosure for identifying the optimal response based on relevance, accuracy, redundancy, and / or other criteria.

[0054] A central controller / processor (not shown) handles the coordination of modules (102, 103) and data flow. It sends user queries (101) to the LLLM (102), collects these responses (105), feeds them to the HLLM (103), obtains the final response (106), and propagates it back through the user interface (100) (108). The central controller / processor may also send queries directly to the HLLM in addition to the LLLM. Alternatively, queries may be sent to the HLLM by one or more LLLMs, along with their responses.

[0055] The database (107) may optionally function as a system memory component that stores a corpus and dataset for training LLLM (102), queries (101), related proposed responses (105A-105D), related optimal responses (106), dialogue history for feedback learning by HLLM (103), and other system data. The optimal response can be sent directly from HLLM (103) to the user interface (100) for interactive user access, such as chat, or it can be stored in the database (107). In the latter case, the user can query the database (107) through the user interface (100). In both cases, the user interface (100) communicates the optimal response to the user (108).

[0056] Finally, feedback such as evaluations of the quality or relevance of the best response obtained from the user or other quality evaluation system may be sent to HLLM(103) to improve the quality of future responses. This is illustrated in Figure 1 as an optional feedback path(109).

[0057] In summary, Figure 1 provides a comprehensive overview of the key hardware and software modules, as well as the high-level workflow, within the hierarchical language model system. The interfaces between the components are intended to be adaptable to different implementations depending on the system requirements.

[0058] Operation flow Referring to Figure 1, when the system receives a user query through the interface, it simultaneously sends the query to all relevant LLLMs. An LLLM may be determined to be relevant based on whether the domain or dataset associated with the LLLM corresponds to the domain or subject of the query.

[0059] Next, each relevant LLLM generates a proposed response based on its training and expertise. The proposed responses can then be compiled into a set of alternative answers.

[0060] The HLLM receives the proposed response directly from the HLLM. Alternatively, the proposed response may be stored in memory (107), from which the HLLM can retrieve it. The HLLM then proceeds to multi-step analysis. 1. Relevance Check: HLLM evaluates each answer for its relevance to the original query and may assign a score based on how well each answer addresses the user's question. 2. Redundancy Removal: To ensure diverse perspectives, HLLM identifies and removes or lowers the priority of redundant or very similar answers. 3. Response Synthesis: Based on the analyzed responses, the HLLM may choose to either completely select the best response or synthesize a new response by combining elements from responses generated by multiple LLLMs (i.e., from one or more proposed responses). 4. Optimization: Before finalizing the answer, HLLM may perform additional refinements to ensure that the response is consistent, grammatically correct, and appropriately styled for its intended audience or application.

[0061] Finally, the selected or combined answer is sent back to the user through the interface.

[0062] During relevance analysis, HLLM employs an attention mechanism to compare question and answer embeddings and assigns relevance scores using cosine similarity. Responses with low relevance scores are removed.

[0063] To remove redundancy, fuzzy string matching can be used to identify semantic overlap between responses. A threshold can be set to determine when the overlap is too high. Clustering algorithms such as k-means can also group semantically similar responses / suggested responses.

[0064] When synthesizing the optimal response, HLLM can fluidly combine key segments from different answers / suggested responses using a sequence-to-sequence model with a pointer network. The pointers allow for the direct copying of text portions from the input.

[0065] A beam search method can be used to generate combinations of multiple candidates. The candidates are then ranked using a scaled dot product attention mechanism that learns the optimal way to fuse the answers. The highest-scoring combination can then be selected as the synthesized response.

[0066] During optimization, HLLM or a separate language model fine-tuned for the target domain may be used to rephrase, correct grammar, and improve the overall consistency of the final response.

[0067] Synthesis and optimization significantly improve relevance compared to simply selecting from proposed responses. The overall pipeline reduces repetitive and irrelevant responses compared to individual LLLMs.

[0068] Figure 4 illustrates the stepwise operational workflow of a hierarchical language model system from receiving a user query to generating the final response. The process includes the following: i. Query Input (400) - The user enters a query through the user interface, which is received by the central controller. ii. Query Submission (401) - The central controller forwards the query to all relevant LLLM modules, and optionally to HLLM. iii. LLLM Response Generation (402) - Each LLLM independently analyzes the query and generates a candidate / proposed response based on its domain-specific training. iv. Compiling the response - The controller compiles all LLLM-generated responses into a set of candidate answers. v. Sending to HLLM (403) - The controller sends a set of candidate responses to the HLLM. vi.HLLM Analysis (404) - HLLM analyzes each candidate response based on criteria such as relevance, precision, and redundancy. vii. Optimal Selection / Synthesis (405) - Using its algorithm, HLLM selects the best response or synthesizes a new response from its constituent elements. viii. Post-processing (406) - Optimized responses may be subject to grammar / style improvements. ix. Output response (407) - The final response is sent back to the user interface for immediate access by the user, and / or stored in the database (107) for later access by the user. x. User Feedback (408) - Users may optionally provide feedback evaluations regarding responses, allowing HLLM to improve over time (409). xi. Optionally, HLLM can be refined in real time to implement improvements based on feedback. HLLM then repeats response analysis and selection and / or synthesis until the optimized response meets a specific quality metric. This is shown as the dashed line (410) in Figure 4.

[0069] In summary, Figure 4 illustrates the operational workflow and key steps involved in generating the system's response to a user query. The process leverages both low-level domain-specific models and high-level general models to produce optimized output.

[0070] Figure 2 illustrates a domain-specific architecture of a low-level language model (LLLM) in a hierarchical system. Multiple LLLM modules exist, each trained with data from a specific domain or field of knowledge.

[0071] As illustrated, the exemplary domains include: • Medical LLLM(200A) - Trained on datasets such as medical textbooks, journals, and patient health records to generate responses to medical queries. • Financial LLLM (200B) - Trained in earnings reporting, financial news, and stock data to answer financial-related questions. • Engineering LLLM (200C) - Trained in textbooks, publications, manuals, standards, and other technical documents to provide answers focused on engineering. • Legal LLLM (200D) - Trained in statutes, case law, legal textbooks, and other legal content for handling legal queries.

[0072] The number and specificity of LLLMs can be adapted based on the breadth of the domains the system intends to cover. LLLMs can utilize domain-specific vocabulary, ontology, and linguistic conventions specific to these fields.

[0073] These trainings leverage state-of-the-art neural network architectures such as Transformers, BERT, and GPT-3, customized for each domain. The modular, domain-based architecture allows for the combination of specialized understanding with overall versatility.

[0074] In summary, Figure 2 provides an overview of domain specialization employed in low-level language models to enable optimized responses to diverse queries. The multi-domain approach balances broad knowledge with the expertise to be addressed.

[0075] Figure 3 illustrates the internal architecture and algorithms implemented within a high-level language model (HLLM) for analyzing candidate / proposed responses and generating the optimal output / response. The components include: • Encoder (300) - Employs a self-attention layer to map input queries and candidate responses to dense vector representations. This helps in evaluating relevance. • Candidate analysis (301) - Components such as clustering, scoring, or ranking are used to evaluate accuracy, redundancy, and other factors. • Decoder (302) - Uses algorithms such as beam search to generate and rank multiple optimized candidate responses for selection. • Synthesis mechanism (303) - Pointer network, seq2seq model, and attention layer combine relevant parts of candidate answers. • Post-processing (304) - Weights, heuristics, and language models are used to refine the output for coherence, grammar, and style. • Policy update unit (305) - The reinforcement learning algorithm updates model parameters based on user feedback to improve the response optimization policy. • External knowledge sources (306) - APIs, databases, and corpus resources enhance the contextual knowledge of the model.

[0076] The components leverage the latest technologies in deep learning and natural language processing to balance generalizability with custom optimization capabilities. The models are built on standard architectures such as BERT and GPT-3.

[0077] In summary, Figure 3 provides an overview of the algorithmic and architectural components within HLLM that enable the analysis and generation of high-quality responses tailored to input queries. Unit selection is designed to be trainable and adaptable using feedback data.

[0078] Figure 5 provides a more detailed diagram of the response synthesis mechanism implemented in HLLM to combine relevant components from candidate answers to produce an optimized output. The steps include: 1. The candidate responses generated by the input candidate(500)-LLLM are supplied to the HLLM synthesis algorithm. 2. Segmentation (501) - Each candidate is segmented into chunks or phrases representing distinct concepts using boundary detection techniques. 3. Content Scoring (502) - The relevance of each chunk to the original query is quantified using similarity metrics and weights. 4. Pointer Network (503) - Key segments are copied from candidates as pointers to form sequence-to-sequence mappings. 5. Combination (504) - Pointers are fluidly combined into a complete sequence using an encoder-decoder architecture. 6. Candidate Generation (505) - Beam search generates combinations of multiple synthesized candidates for consideration. 7. Candidate Scoring (506) - Candidates are evaluated for consistency, conciseness, grammar, and other attributes. 8. Selection (507) - The composite candidate with the highest score is selected as the optimal response.

[0079] Synthesis techniques leverage innovations such as pointer networks to fluidly stitch together relevant content from multiple sources. This allows for tailoring responses to user queries by identifying and combining the most relevant information.

[0080] In summary, Figure 5 illustrates a synthesis process for generating an optimized, nuanced response that integrates the strengths of multiple specialized LLLMs. The components are designed to be trainable to continuously improve the combined quality.

[0081] Adaptive learning and feedback loops A valuable, albeit optional, component of the system is the feedback loop. After the HLLM provides its final answer, the user can provide feedback evaluating the quality, relevance, and accuracy of the response. This feedback can be used to fine-tune the HLLM's selection or synthesis process, making the system more accurate and efficient over time.

[0082] The adaptive technique of feedback loops can be derived from reinforcement learning principles, structuring user evaluations and preferences as rewards. Details such as reward architecture, the balance between exploration and exploitation, and experience retrieval can help optimize system performance across multiple query-response cycles. Feedback loops allow HLLMs to improve their selection and synthesis capabilities across multiple query-response cycles. User evaluations on dimensions such as relevance and consistency can be compiled as rewards.

[0083] Policy gradient reinforcement learning algorithms can use these reward signals to update parameters that control the ranking, combination, and optimization stages. Actions that generate higher rewards can be reinforced.

[0084] Epsilon-greedy selection may be used to balance the search. While HLLMs can most often utilize learned high-reward strategies, they may sometimes explore novel selection mechanisms.

[0085] Synthesized responses, user evaluations, and HLLM's internal Q-values ​​can be stored in a database. This experience retrieval can allow for periodic retraining of HLLM against past successful and unsuccessful responses to prevent overfitting.

[0086] It will be understood that feedback loops can improve the cumulative success rate of responses. HLLM can evolve its strategy based on the Darwinian selection of high-performance actions.

[0087] Figure 6 illustrates an adaptive feedback loop implemented in a system to enable continuous learning and improvement of the response optimization capabilities of a high-level language model (HLLM). The process includes the following: 1. Response Generation (600) - HLLM generates a response to a user query based on its current policy. 2. Presentation to the user (601) - The response is provided to the user through the interface. 3. Users evaluate responses (602) - Users evaluate responses in dimensions such as relevance, accuracy, and consistency. 4. Compiling Feedback (603) - Evaluation and feedback are compiled into a reward signal. 5. Feedback to HLLM (604) - Rewards are fed back to the HLLM policy update unit. 6. Policy Adaptation (605) - Using reinforcement learning, HLLM evolves its response optimization policy by updating its parameters. 7. Historical data storage (606) - Dialogue data is stored for training and testing to prevent overfitting. 8. Model Retraining (607) - HLLM is periodically retrained on accumulated dialogue data.

[0088] The feedback loop allows for the adaptation of HLLM's selection and synthesis capabilities based on experiential user interactions over time. This enhances performance, customization, and human-like conversation.

[0089] In summary, Figure 6 outlines the self-improvement mechanism incorporated into the system architecture. The components enable empirical learning by a model to optimize the flow of conversation.

[0090] Specialized processing of mixed input modalities In yet another example of the present invention, the system may include a mixture of low-level language models (LLLMs) having distinct specializations when processing different types of input data. For example, an expression processing LLLM may excel at analyzing queries containing mathematical expressions and formulas, using, for example, LaTeX decoding, symbol manipulation, and mathematical recognition encoders. A table processing LLLM may incorporate techniques from semantic analysis and data mining to interpret queries on statistical tables. A graphics LLLM may extract meaning from charts, figures, and other visuals by leveraging computer vision and multimodal understanding.

[0091] When responding to queries concerning technical documents (e.g., technical standards) containing a mixture of text, figures, tables, and mathematical notation, a high-level language model (HLLM) can delegate different modalities to specialized LLLMs best suited to each. For text passages, the HLLM may rely on an LLLM with strong natural language capabilities. To interpret mathematical formulas, a symbolic processing LLLM can be invoked. For tabular information, a table processing LLLM can be utilized. For figures, a graphics LLLM can be used. The HLLM then assimilates complementary inputs, inferences, and conclusions from the various LLLMs. Using meta-learning techniques, these fragmented insights across modalities can be combined into a unified response that synthesizes the prominent parts. This allows the system to process mixed-modality documents and leverage hybrid inference across specialized models tailored to different input types. The HLLM orchestrates the models to produce a cohesive response that integrates the multimodal analysis.

[0092] In a system architecture with low-level LLLMs dedicated to specific modalities (e.g., text, expressions, tables, graphics), the user's original query may be routed directly to a high-level language model (HLLM) rather than being broadcast to all LLLMs simultaneously. The HLLM encoder then analyzes and interprets the initial user query to determine which components include text passages, mathematical notation, tabular data, visual diagrams, etc.

[0093] Based on this modality analysis, the HLLM can selectively route specific parts, subqueries, or extracted features to the specialized LLLM best suited to that modality. For example, detected expression snippets can be passed to an expression processing LLLM, tabular data to a table processing LLLM, and chart image features to a graphics LLLM. This allows the HLLM to assign each query component to the most relevant specialized LLLM for efficient distributed processing, based on the modality it has previously detected.

[0094] The HLLM then assimilates the output from the modality-specific LLLM into the synthesized response. Therefore, direct routing via the HLLM (rather than broadcasting the entire query) allows for leveraging the most appropriate LLLM for different modalities.

[0095] Inter-model dialogue for improved response In some examples of the present invention, a high-level language model (HLLM) may further engage in dialogue with one or more low-level language models (LLLMs) to request clarification, additional information, or improvements in order to improve the quality of the proposed response.

[0096] For example, if the initial proposed response from the medical LLLM contains ambiguous acronyms or terms, the HLLM may query the medical LLLM to expand on the unclear terminology and make the response easier to understand.

[0097] As another example, HLLM may detect that the proposed response from Engineering LLLM lacks sufficient detail or explanation of key principles. HLLM can then probe Engineering LLLM with follow-up questions to fill in the gaps in the initial response.

[0098] HLLM may also require Finance LLLM to double-check key numerical data within its proposed response, or to request the most up-to-date figures from real-time market data feeds.

[0099] Furthermore, if the initial proposal appears incomplete, biased, or contradictory, HLLMs may involve individual LLLMs in the context of the conversation to clarify ambiguities, improve objectivity, provide supporting references or excerpts to the document, or resolve contradictory statements.

[0100] Dialogue exchange, by adopting conversational protocols and rules common to the LLLM domain, can elicit an expanded, balanced, and well-rounded perspective.

[0101] This inter-model dialogue allows for the iterative improvement of response quality in terms of completeness, accuracy, objectivity, and consistency by leveraging the expertise within each LLLM.

[0102] HLLM may employ a hybrid approach that synthesizes its own final response after the dialogue exchange is complete, rather than completely selecting the response of any single LLLM.

[0103] LLLM capability profiling for optimized routing In addition to modality analysis of queries, high-level language models (HLLMs) can profile the capabilities of each low-level language model (LLLM) through dialogue exchange. During initialization or periodically, HLLMs can probe LLLMs with domain-specific questions, edge cases, and sample inputs to benchmark their strengths and limitations. For example, an HLLM may evaluate an expression processing LLLM in terms of mathematical reasoning skills by posing an obscure physics problem requiring calculus or linear algebra. A table processing LLLM can be tested when handling complex datasets and queries requiring multivariate analysis. The HLLM catalogs the capabilities discovered for each LLLM in a knowledge base.

[0104] When routing query components, HLLM consults this knowledge base and optimally allocates parts based on both the modality type and the capabilities of the corresponding LLLM. For example, a statistically heavy table extracted from a query may be routed to a table processing LLLM specialized for econometrics rather than a general table processing model.

[0105] Dynamic capability profiling via query dialogs enables HLLM to make informed routing decisions to match aspects of a query not only with modalities but also with the most appropriate LLLM. This results in responses tailored to the specialized capabilities of the underlying LLLM ensemble.

[0106] Advantageously, the hierarchical language modeling system proposed herein provides a balanced blend of vast knowledge encapsulated in generalist models and domain-specific expertise that takes into account the nuances of specialized models. By hierarchically orchestrating these components, the system significantly improves consistency, reliability, and accuracy in generating responses to diverse user queries.

Claims

1. A method for generating the optimal response to a query, a. Receiving queries and b. Processing the query in multiple low-level large-scale language models (LLLMs), wherein at least one of the LLLMs is a finely tuned large-scale language model. c. To generate a proposed response to the query from each of the LLLMs, d. Analyzing the proposed response in a high-level large-scale language model (HLLM) based on one or more criteria, e. A method comprising generating an optimal response based on the analysis and the query using the HLLM.

2. The method according to claim 1, wherein the criteria include relevance, accuracy, consistency, and / or one or more characteristics of the proposed response to the query.

3. The method according to claim 1, wherein the step of generating the optimal response includes selecting or combining the optimal response from two of the more proposed responses.

4. The method according to claim 1, further comprising incorporating user feedback regarding a previous optimal response in order to adjust the HLLM selection mechanism over time.

5. The method according to claim 1, wherein the HLLM identifies useful segments from at least two of the proposed responses and curates an optimal response based on the useful segments.

6. The method according to claim 1, wherein the HLLM identifies and removes redundant information in the selected or synthesized optimal response.

7. The method according to claim 1, wherein the HLLM assigns a diversity score to each proposed response based on its differences from other proposals, and selects or prioritizes proposals that bring a unique perspective or information.

8. The method according to claim 1, further comprising the use of a neural network architecture within the HLLM for combining two or more proposed responses and / or segments.

9. The method according to claim 1, wherein the HLLM synthesizes portions of multiple proposed responses to generate an optimal response that cannot be derived from any single LLLM response alone.

10. The method according to claim 1, further comprising a post-processing step in which the selected or synthesized best answer is improved with respect to grammar, consistency, and / or style.

11. The method according to claim 1, wherein the HLLM employs a meta-learning approach and allows it to adapt its selection and / or synthesis strategy based on trends and / or patterns across a series of queries.

12. The method according to claim 1, wherein the proposed response from the LLLM and the selection or synthesis by the HLLM are performed in real time, enabling interactive and dynamic conversation.

13. A device for generating the optimal response to a query, a. A receiving unit configured to receive queries, b. Multiple Low-Level Language Model (LLLM) modules, each module configured to generate a proposed response to the query, c. A high-level language model (HLLM) module, d. Analyze the proposed response based on one or more criteria, e. An apparatus comprising an HLLM module configured to generate an optimal response based on the analysis and the query.

14. The apparatus according to claim 13, wherein the criteria include relevance, accuracy, consistency, and / or one or more characteristics of the proposed response to the query.

15. The apparatus according to claim 13, wherein the HLLM is configured to select or synthesize the optimal response from two of the more proposed responses.

16. The apparatus according to claim 13, further comprising a feedback means coupled to the HLLM module, which enables the HLLM module to improve its selection or synthesis process based on previous interactions and responses.

17. The apparatus according to claim 13, wherein the HLLM module is further configured to decompose the proposed response into segments and combine different segments from one or more LLLM modules to form an optimal response.

18. The apparatus according to claim 13, wherein the HLLM module is further configured to identify and remove redundant information in the selected or synthesized optimal response.

19. The apparatus according to claim 13, wherein the HLLM module is further configured to assign a diversity score to each proposed response based on its difference from other proposals, and to select or prioritize proposals that bring a unique perspective or information.

20. The apparatus according to claim 13, wherein the HLLM module includes a neural network architecture for combining the two or more proposed responses and / or segments.

21. The apparatus according to claim 13, wherein the HLLM module is further configured to synthesize a plurality of proposed response portions to generate an optimal response that cannot be derived from the response of any single LLLM module alone.

22. The apparatus according to claim 13, further comprising a post-processing module configured to improve the selected or synthesized optimal answer with respect to grammar, consistency, and / or style.

23. The apparatus according to claim 13, wherein the HLLM module is further configured to employ a meta-learning approach, enabling it to adapt its selection and / or synthesis strategy based on trends and / or patterns across a series of queries.

24. The apparatus according to claim 13, wherein the communication interface is further configured to enable real-time transmission of queries and responses and to enable interactive conversation.

25. A non-temporary computer-readable database stored in memory, the database comprising queries and optimal responses to the queries generated by HLLM in accordance with any of the methods described herein.