System for generating adaptive language model using user-affected data

Abstract

A method for adapting a language model to represent a user's intent is provided. The method includes receiving a user input including user data for a request for a language model from a user, analyzing an intent of the user based on the user input and the user data, generating user-affected data reflecting the analyzed intent of the user, adapting the language model by at least one adaptation method based on the user-affected data, and outputting the adapted language model. And generating an output from the adapted language model.

Description

TECHNICAL FIELD

[0001] Embodiments disclosed herein relate to the field of language models (LMs), and in particular to systems and methods for augmenting language models using generated, user-influenced data. BACKGROUND

[0002] Pre-trained LMs, such as large language models (LLMs), like generative pre-trained transformers (GPTs), suffer from issues related to training data dependency. LMs can generate incoherent content (e.g., off-topic content, content that contradicts itself, etc.), can produce false content (e.g., content that is unrealistic, not true, etc.), can lack common sense, and can be misaligned with users and / or user intent. In short, LMs are not trustworthy. Training data often has unfair or unrepresentative biases or pervasive misconceptions. Actual data for accurate training can not be available, sensitive, or private, which limits the application of LMs in many domains. The generalization capability of LMs is limited to the training data, such that their performance on specific or esoteric topics can be affected due to the sparsity of the training data. Currently, there is not enough data to support the broad generative capability of LMs.

[0003] Fundamentally, these are data problems: the training data accumulated in the past cannot support the new knowledge that individuals are trying to create with these systems in the present moment. LMs lack the data to effectively cater to individuals and their subjective interests, unique perspectives, and personal knowledge. Individuals are constantly expanding the accelerating boundaries of new knowledge in every moment of every interaction, and even the largest LMs cannot keep pace.

[0004] One class of solutions to this problem currently is to extend LMs to enable real-time interaction with external knowledge bases. Pre-trained LMs can retrieve additional data through search engines or interaction with specialized applications, databases, and reasoners. While these improvements increase the quantity of available data, the same problems persist because these solutions are based on existing knowledge bases. Illusions or lack of common sense problems can not be solved simply by increasing the magnitude of data. These additional sources of knowledge cannot satisfy the limitless potential of generative systems — new knowledge that has not yet existed in the form of training data that people want to create. The full potential of LMs lies in knowledge creation, not knowledge retrieval. LMs are misunderstood as improved search engines, and their full potential cannot be realized through better search.

[0005] Accordingly, there is a need for systems and methods that enable LMs to create knowledge in real-time that is both accurate and relevant to users. SUMMARY

[0006] A method for adapting a LM to represent user intent is provided. The method includes receiving, from a user, a user input comprising user data of a request to the LM, analyzing an intent of the user based on the user input and the user data; generating user-affected data reflecting the analyzed intent of the user; adapting the LM by at least one adaptation method based on the user-affected data, and generating an output from the adapted LM.

[0007] In one embodiment, analyzing the intent of the user includes analysis of at least one of a context, a user prompt, a user preference, and a user action.

[0008] In one embodiment, generating the user-affected data includes receiving, at an input translator, the user-affected data, translating, by the input translator, the user-affected data into a form that is usable by a semantic engine, receiving, at the semantic engine, the translated user-affected data, generating, by the semantic engine, a user-affected knowledge representation (KR) representing the translated user-affected data, and adapting the LM based on the user-affected KR.

[0009] In one embodiment, the at least one adaptation method includes generating a prompt at a prompt selector from the knowledge representation; and augmenting the prompt at a prompt injector.

[0010] In one embodiment, the at least one adaptation method includes fine-tuning a machine learning model using training data derived from the user-affected knowledge representation.

[0011] In one embodiment, the at least one adaptation method includes deriving a conditional variable from the user-affected knowledge representation, and augmenting the prompt at the prompt injector by applying the conditional variable to the prompt.

[0012] In one embodiment, the at least one adaptation method includes deriving a logical constraint from the user-affected knowledge representation, and applying the logical constraint to a loss function optimizer of the machine learning model.

[0013] In one embodiment, the method further includes evaluating, using a semantic evaluator, semantic coherence between the output and the analyzed intent of the user.

[0014] In one embodiment, the semantic evaluator is an atomic knowledge representation model (AKRM).

[0015] An apparatus for adapting a language model to represent a user intent is provided. The apparatus includes a network interface, a processor, and a non-transitory memory having stored thereon computer executable instructions executable by the processor. The apparatus is configured to receive a user input comprising user data of a request for a language model. The apparatus is further configured to analyze an intent of the user based on the user input and the user data. The apparatus is further configured to generate user-affected data reflecting the analyzed intent of the user. The apparatus is further configured to adapt the language model by at least one adaptation method based on the user-affected data. The apparatus is further configured to generate an output from the adapted language model.

[0016] In one embodiment, analyzing the intent of the user includes analysis of at least one of a context, a user cue, a user preference, and a user action.

[0017] In one embodiment, generating the user-affected data includes receiving the user-affected data at an input translator, translating the user-affected data by the input translator into a form that is usable by a semantic engine, receiving the translated user-affected data at the semantic engine, generating a user-affected knowledge representation (KR) representing the translated user-affected data by the semantic engine, and adapting the LM based on the user-affected knowledge representation.

[0018] In one embodiment, the at least one adaptation method includes generating a cue at a cue selector from the knowledge representation; and augmenting the cue at a cue injector.

[0019] In one embodiment, the at least one adaptation method includes fine-tuning a machine learning model using training data derived from the user-affected knowledge representation.

[0020] In one embodiment, the at least one adaptation method includes deriving a conditional variable from the user-affected knowledge representation and augmenting a cue at a cue injector by applying the conditional variable to the cue.

[0021] In one embodiment, the at least one adaptation method includes deriving a logical constraint from the user-affected knowledge representation and applying the logical constraint to a loss function optimizer of a machine learning model.

[0022] In one embodiment, the method further includes evaluating semantic coherence between the output and the analyzed intent of the user using a semantic evaluator.

[0023] In one embodiment, the semantic evaluator is an atomic knowledge representation model (AKRM).

[0024] A system for adapting a language model to represent a user intent is provided. The system includes at least one processor and at least one non-transitory memory storing instructions for execution by the at least one processor to implement a user input device, and a computing engine. The user input device is configured to provide user input to the computing engine. The computing engine is configured to receive the user input comprising user data of a request for a language model. The computing engine is further configured to analyze an intent of the user based on the user input and the user data. The computing engine is further configured to generate user-impacted data reflecting the analyzed intent of the user. The computing engine is further configured to adapt the language model by at least one adaptation method based on the user-impacted data. The computing engine is further configured to generate an output from the adapted language model.

[0025] In one embodiment, analyzing the intent of the user includes analysis of at least one of a context, a user prompt, a user preference, and a user action.

[0026] In one embodiment, generating the user-impacted data includes receiving the user-impacted data at an input translator, translating the user-impacted data by the input translator into a form that is usable by a semantic engine, receiving the translated user-impacted data at the semantic engine, generating a user-impacted knowledge representation (KR) representing the translated user-impacted data by the semantic engine, and adapting the LM based on the user-impacted knowledge representation.

[0027] In one embodiment, the at least one adaptation method includes generating a prompt at a prompt selector from the knowledge representation; and augmenting the prompt at a prompt injector.

[0028] In one embodiment, the at least one adaptation method includes fine-tuning a machine learning model using training data derived from the user-impacted knowledge representation.

[0029] In one embodiment, the at least one adaptation method includes deriving a conditional variable from the user-impacted knowledge representation, and augmenting a prompt at a prompt injector by applying the conditional variable to the prompt.

[0030] In one embodiment, the at least one adaptation method includes deriving a logical constraint from the user-impacted knowledge representation, and applying the logical constraint to a loss function optimizer of a machine learning model.

[0031] In one embodiment, the method further includes evaluating semantic coherence between the output and the analyzed intent of the user using a semantic evaluator.

[0032] In one embodiment, the semantic evaluator is an atomic knowledge representation model (AKRM).

[0033] Other aspects and features of the present application will become apparent to those regularly skilled in the art upon reading the following description of some example embodiments with reference to the accompanying figures. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings are used to illustrate various examples of items, methods, and apparatuses in the present specification. In the drawings: Figure 1 is a block diagram of various adaptation methods that can be used to adapt a pre-trained LM; Figure 2A is an overview of a system for enhancing a LM according to an embodiment; Figure 2B is a flowchart of a method of enhancing a LM according to an embodiment; Figure 3 is a block diagram of various user interfaces for receiving and adapting user input that are suitable for use with the systems and methods described herein; Figure 4 is an overview of an input translation system for use during LM enhancement according to an embodiment; Figure 5 is an overview of a prompt engineering system for use during LM enhancement according to an embodiment; Figure 6 is an overview of a data engineering system for use during LM enhancement according to an embodiment; Figure 7 is an overview of a data generation method for use in LM enhancement according to an embodiment; Figure 8 is an overview of a data generation method for use in LM enhancement according to an embodiment; Figure 9 is an overview of a method of enhancing a LM using analyzed user intent according to an embodiment; Figure 10A is a block diagram of a system for enhancing a LM using user-influenced KR enhanced prompts according to an embodiment; Figure 10B is a block diagram of a system for enhancing a LM using user-influenced KR enhanced prompts according to an embodiment, the system including a mediator; Figure 11A is a block diagram of a system for using user-influenced KR and LM to label input data for a machine learning (ML) classifier according to an embodiment; Figure 11B is a block diagram of a system for using user-influenced KR and LM to label input data for a ML classifier according to an embodiment, the system including a mediator; Figure 12 is a block diagram of a system for using user-influenced KR to develop a semantic evaluator for a LM according to an embodiment; Figure 13 This is a block diagram of a system for evaluating candidate outputs of an LM using an atomic knowledge representation model (AKRM) according to an embodiment; Figure 14 This is a flowchart of a method for using AKRM as a source of logical constraints when training a probe for evaluating semantic consistency latent knowledge of LM, according to an embodiment. Figure 15 This is a block diagram of a computing device used to enhance systems and methods for LM; Figure 16 This is an overview of a system for managing condition variables derived from prompt words during LM enhancement, according to an embodiment; Figure 17 These are schematic diagrams of AKRM for coherent semantics according to various embodiments; and Figure 18 This is an overview of a system for selecting semantically coherent concepts to self-train a LM, based on an embodiment. Detailed Implementation

[0035] Various devices or processes will be described below to provide examples of embodiments of each claim. No embodiment described below is intended to limit the scope of any claimed embodiment, and any claimed embodiment may include processes or devices different from those described below. Claimed embodiments are not limited to devices or processes having all the features of any one of the devices or processes described below, nor are they limited to features common to multiple or all of the devices described below.

[0036] It should be understood that the one or more systems described herein are implemented in a computer program running on a programmable computer, each programmable computer comprising at least one processor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, but not limited to, a programmable computer can be a programmable logic unit, a mainframe computer, a server, a personal computer, a cloud-based program or system, a laptop computer, a personal data assist device, a mobile phone, a smartphone, or a tablet computer.

[0037] Each program is preferably implemented using a high-level procedural or object-oriented programming language and / or scripting language to communicate with the computer system. However, if necessary, the program may also be implemented using assembly language or machine language. In either case, the language may be a compiled language or an interpreted language. Each such computer program is preferably stored on a general-purpose or special-purpose programmable computer-readable storage medium or device so that the computer can be configured and operated when it reads the storage medium or device to execute the program described herein.

[0038] The description of embodiments containing several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present application.

[0039] Furthermore, although process steps, method steps, algorithms or the like can be described in a sequential order, such processes, methods and algorithms can be configured to work in other orders. In other words, any sequence or order of steps that can be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein can be performed in any practical order. Further, some steps can be performed simultaneously.

[0040] When only a single device or article is described herein, it will be readily apparent that multiple devices / artifacts (whether or not they cooperate) can be used in place of a single device / article. Similarly, where a device or article is described as being capable of performing a number of functions, it will be understood that multiple devices or articles can be configured to perform only one or a subset of the functions.

[0041] The embodiments and implementations described below provide at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to enhance an LM based on input from a user received from the same processor or a different processor and user data representative of the user’s intent.

[0042] Various components of the systems described herein can be encoded as non-transitory computer-readable data structures, such as KR.

[0043] As described above, LMs are not able to reliably provide coherent, non-illusory, and user-relevant content. While current solutions that have LMs search and acquire supplemental knowledge from other knowledge bases can seem to improve content output, the aforementioned problems often cannot be solved due to the lack of historical data. These solutions merely allow for stronger knowledge retrieval capabilities, not knowledge creation.

[0044] The solution to the problem is not more data, as data simply does not exist and is only created when users interact with LMs. LMs are given instructions on how to create data in order to work collaboratively and innovatively with individuals.

[0045] The solution provided herein is able to assess and identify whether the results generated by the LM are good and accurate according to the context of each user.

[0046] The solutions provided herein enhance the generation capabilities of LMs, making them less likely to produce outputs that are misaligned with user needs, based on analysis of user data and / or user intent obtained directly or indirectly.

[0047] There are many ways to improve the performance of LMs, such as changing loss functions, regularization, hyperparameter tuning, and various training strategies, including reinforcement learning, supervised learning, curriculum learning, and so on.

[0048] Figure 1 is a block diagram showing examples of several different types of adaptations that can be implemented by adaptation method 110 for adapting or enhancing LMs. Figure 1 The types of adaptations that can be applied to LMs in include fine-tuning 121, secondary models 122, additional training data 123, configuration parameter adjustment 124, and configuration changes 125. This list is not exhaustive, and many other adaptations of LMs are possible.

[0049] Herein, these various adaptation types are deployed according to adaptation methods intended to enhance LMs based on analysis of user intent. By measuring the coherence of the adapted LM with user intent, the degree of success of the adaptation in achieving intended outputs can be determined. By adapting LMs to be more coherent with analyzed user intent, LMs are not only more aligned with users, but can also reduce incoherent, hallucinated, and nonsensical outputs, as compared to merely increasing the amount of data used to train LMs.

[0050] Fine-tuning 121, LMs can involve data engineering to supply additional data specific to a desired task for further training.

[0051] Secondary models 122 can involve using secondary models in conjunction with LMs to perform task-specific evaluations.

[0052] Additional training data 123 can involve generating or otherwise obtaining more training data to train LMs.

[0053] Prompt engineering 124 can involve manipulating the parameters or content of inputs.

[0054] Configuration changes 125 can involve manipulating the configuration of LMs by modifying parameters such as temperature or token constraints.

[0055] Other methods modify their internal mechanisms, such as incorporating new constraints into loss function optimization.

[0056] To illustrate the flexibility that LMs provide, the functional components that have been briefly discussed above, and in more detail below in FIGS. 2-9, will be deployed and redeployed in various ways in subsequent specific embodiments.

[0057] This document discusses a measure of the "coherence" or "fit" between various aspects and other aspects. Both terms refer to the degree of similarity between two things, and can be used interchangeably or separately depending on the particular embodiment being discussed. Generally, these terms measure the degree to which the system output (which can be an intermediate output or a final output) aligns with the user intent determined through analysis of the user-derived data.

[0058] As shown in FIGS. Figure 2A and 2B depicted are block diagrams of a system (2A) and method (2B) for enhancing a LM to align it with a user, according to embodiments.

[0059] Figure 2A The functional elements of the system are an analysis of user intent 202a, user-affected data generated 204a, at least one adaptation method 206a, the adapted LM and its output 208a, and a semantic evaluator 210a.

[0060] Figure 2B The functional elements include an analysis of user intent 202b, user-affected data generated via a plurality of interfaces and methods 204b, adaptation of a LM using the user-affected data 206b, generation of an output from the adapted LM 208b, and evaluation of the adapted LM and / or the output of the LM according to the user intent by a semantic evaluator 210b.

[0061] The method, wherein the use of the functional elements begins with an analysis of the user's intent 202a of the LM 208a.

[0062] In some embodiments, the analysis 202a can be initiated after the user actively inputs data into an application or computing engine. That is, the user can have a specific query or desired output that is being actively sought, and the user's intent can be analyzed in the context of that specific query or desired output.

[0063] In other embodiments, the analysis of intent can be initiated by implicit user input obtained by monitoring the user's operations and preferences on one or more devices and in one or more applications. That is, the user can not specifically request an output, but the output can be generated and displayed or otherwise provided to the user based on the user's activity. In still other embodiments, the user can actively request a specific output, but the output can be generated from historical inputs that demonstrate the user's intent.

[0064] Herein, the input from the user to initiate the analysis of intent, whether explicit / active or implicit / passive, is referred to as a "user request." The request can take any form capable of triggering an output from the system.

[0065] In all embodiments, the system leverages knowledge of the user's intent to augment the user's stated request or query. That is, the system uses user-derived contextual information to better understand the intended output of the user's provided request by analyzing the user's intent. The term "understanding" is intended to convey that the system's ability to process and interpret information from and about the user is improved.

[0066] User intent analysis 202a can be performed after the user request, before the user request, or the user can trigger the analysis after determining that the output they received for the request was not sufficient. User intent analysis 202a can be performed continuously as new user information is acquired, or initiated before the user enters the request and stored for future requests where the user intent analysis is applied as the request occurs. User intent analysis 202a can be initiated before the request, but further expanded based on the context of the current request. User intent analysis 202a can also begin only upon receiving the request, in which case the analysis can be a broad analysis or one specific to the context of the request. Initiating the analysis before the request or performing a broad analysis can save time, but initiating the analysis specific to the context of the request can ensure more accurate results. User intent analysis 202a can be an automatically running background process or a process initiated by the user.

[0067] Data for the user intent analysis can come from user behavior data, search query data, form input and other textual data, collaboration data, user profile data, user roles, user context data, and the like. In some embodiments, data from these different sources can be collected and integrated into a user model.

[0068] Regardless of the input method, the user-influenced data can be generated by analyzing the user's intent. The user's request can be augmented with the user-influenced data, providing input for the adaptation method. This input includes the generated user-influenced data 204a. The user intent analysis 202a outputs an analysis of the goal the user actually wants to accomplish based on the user's request (active or passive) and their intent. The user-influenced data 204a represents this user intent. The generated user-influenced data can be semantized, encoding the meaning of the intent, and can be in any format or vocabulary form that can be used to represent semantic data.

[0069] In some embodiments, this user-influenced data can be generated by the LM, as described below. In other embodiments, the data can be generated elsewhere.

[0070] As discussed herein in various embodiments, user-impacted data is applied to the LM 208a through an adaptation method 206a to change the output of the LM. A semantic evaluator 210a is then used to evaluate whether the output from the LM aligns with the initial analyzed user intent Figure 1 . If the LM does not align with the analyzed user intent, the method can return to the user intent analysis and prompt the user for more data about their intent, or the application can acquire more extensive user data.

[0071] The adaptation method 206a can use any of the adaptation types discussed in Figure 1 or any other implementable and suitable adaptation types to enhance the LM. The selection of adaptation type can be done automatically or manually based on the user intent analysis 202a. The adaptation method 206a applies the user-impacted data to the LM to adapt appropriate aspects of the LM to provide output that aligns with the user intent.

[0072] In other embodiments of the system, the user-impacted data can be both an input to the adaptation method and a product of the adaptation method. That is, the adaptation method itself can receive user-impacted data as input, for example, in the form of a prompt word or query, which is adapted to conform to the user intent, and the adaptation method can also generate more user-impacted data, for example, KRs, on the fly. In the simplest embodiment, the user-impacted data includes data that is applied directly to the LM, the LM outputs a result, which is then compared to the user intent to verify its coherence. In more complex embodiments, the user-impacted data can include any or all data generated between the analysis of the user intent and the semantic evaluation of the output, in addition to the output, where the generation of the data is impacted by the user-derived data. In embodiments where the adaptation method generates user-impacted data, the generation of the user-impacted data and the adaptation method can occur at least partially simultaneously.

[0073] Subsequent embodiments implement several methods of enhancing LMs (e.g. Figure 1 as shown) to illustrate the flexibility of the architecture. These embodiments include other such variant interfaces, implementations, and integrations, as well as the opportunity to combine these various embodiments in new ways.

[0074] Language Modeling (LM) is used to predict sequences of words and symbols. Many architectures exist for LM, including transformers, convolutional neural networks (CNNs), long short-term memory (LSTM) models, recurrent neural networks (RNNs), and numerous probabilistic, statistical, and rule-based models. In this paper, a "language model" or "LM" is a black box that takes text as input and predicts new text sequences. The input can be text or other media, such as images or videos. Any form that can ultimately be used as text input is considered input to this paper. Specific interfaces related to LM, such as training data, cue words, and loss function optimizers, will be discussed in more detail within the context of each particular interface and embodiment.

[0075] like Figure 3 As shown in the figure, a block diagram of various user interfaces that can receive user input is presented.

[0076] Figure 3 The main user interface is shown. People who use LM and other computing engines are referred to as users. These people hold a variety of different roles, including but not limited to data scientists, machine learning engineers, cue word engineers, application developers, academics, knowledge workers, and content creators, producers, and consumers.

[0077] As described in subsequent embodiments, users can interact with the system by inputting user information into the application, or by inputting user information through prompt word engineering or data engineering methods, to identify objects of interest (such as documents, images, or videos). Users can interact with a single interface or multiple interfaces. That is, for example, a user can select only the object of interest or only create prompt words, or they can select both the objects of interest and create prompt words.

[0078] Figure 3 Showing Figure 2B The first two steps of the method involve analyzing user intent and generating user-influenced data, identifying potential interfaces and paths. The outputs of Object of Interest 312, Cue Word 322, and Data 332 are examples of user-influenced data and are not intended to cover all situations.

[0079] Application 310 may include any program or application that allows a user to select an object of interest 312 via user input 302. These objects of interest are then used, as described above, to analyze user intent. User input 302 may be input actively requested by the user, or passive input derived from the user through indirect methods, such as user browsing activity or other contextual or environmental indicators.

[0080] The cue word project 320 is a method for generating one or more appropriate cue words, i.e., inputs to the model, to achieve the model's expected output. The cue word project 312 allows the user to create cue words 322 through user input 302.

[0081] Data engineering 330 is a methodology for collecting, storing, processing, and analyzing data. Here, data engineering provides a foundation for assessing coherence. Figure 4 It is a flowchart of the translation of various possible user inputs. Figure 3 It demonstrates various ways users can interact with the interface, including inputting data to express their intent or requesting output from the LM. Figure 4 The process of input is demonstrated to enable interoperability between various inputs and downstream computing engines.

[0082] Any computational engine, including inferencers, search engines, semantic engines, LMs, or other ML models, can interoperate via natural language input. These inputs can be coupled, allowing a single user input to be sent to multiple computational engines simultaneously. However, the underlying user intent will vary depending on the computational engine used, and the form of each input will also differ. Search engines are query-based; semantic engines are context-sensitive; LMs are cue-based; and so on. When a single input is distributed across multiple tools, it may be necessary to translate the user input to align it with each downstream tool. This functionality is referred to here as an "input translator," and more generally falls under the category of data transformation tools, although the function can also be described as extraction or translation.

[0083] Figure 4 The input 410 includes context 412, prompts 414, and queries 416, which are then translated by an input translator 420, which transforms the input into a form that is aligned with and usable by various computing engines 430, including semantic engine 432, LM 434, and search engine 436.

[0084] Figure 4 Showing Figure 2B The second and third steps of the method. Input 410 includes examples of data influenced by the user. Input translator 420 and computation engine 430 can be components of various adaptation methods used in the systems and methods described herein.

[0085] For example, user input 410 can be in the form of a query 416, which can be used for a search engine but not for a semantic engine. Thus, query 416 can be translated into a contextual form so that semantic engine 432 can also use it. In some embodiments, a single input 410 can be translated into multiple forms so as to be used with multiple computing engines. In some embodiments, multiple forms of input can be aggregated and translated into a form that can be used by various computing engines. In some embodiments, the form of input 410 can not be contextual 412, prompt 414, or query 416, and can be translated into one of these forms.

[0086] As mentioned above, the input provided by a user to a LM is referred to as a prompt. In general, the activity of enhancing the performance of a LM through prompting is referred to as prompt engineering (as described below, and demonstrated in Figure 5 Evaluation of the coherence of a prompt with user needs can drive the development of a prompt engineering project.

[0087] A prompt is composed of text, usually strung together in a sequence. A prompt provides background knowledge, constraints, and context for a LM. A prompt encapsulates an intended direction or intent for inference by a LM, usually in the form of an instruction, sometimes accompanied by an example. A prompt can include metadata or labels associated with an object of interest (e.g., a document, image, or video), or an expression of an abstract concept (e.g., a semantic, sentiment, tone, or style).

[0088] Figure 5 is a block diagram overview of a method for prompt engineering to enhance a LM.

[0089] As mentioned above, prompt engineering is a method of generating a suitable single or multiple prompts to achieve a model’s intended output. As shown in Figure 3 Prompt engineering is one of the possible inputs that a user can input through a prompt engineering interface for input to generate output from a LM. As shown in Figure 4 Other forms of input information can be translated into prompts that can be used by a LM through an input translator.

[0090] As shown in Figure 5 Prompt engineering 510 is performed to create a prompt 520 (or multiple prompts) for a LM 530. Prompt engineering can be direct prompt engineering by a user, or indirect prompt engineering through a translator. Regardless of the origin of the prompt, analysis of the prompt and coherence with user needs can determine whether the prompt is adequate and can be used directly to prompt the LM 530.

[0091] When the analysis of the coherence of the prompt 520 to the user need indicates that the prompt 520 is inadequate, the prompt can be augmented or expanded. The augmentation or expansion of the prompt can be via secondary processes, often through data transformations. These processes are referred to as prompt augmentation or prompt injection.

[0092] In Figure 5 the illustrated embodiment, the object of interest 522 collected from the user is subjected to data transformation 524 to prompt augmentation 526 of the prompt to make it adequate for the user. After prompt augmentation, the prompt 520 can be subjected to further coherence analysis and multiple rounds of augmentation as needed.

[0093] Similarly, as Figure 6 illustrated, data can be used to train, validate, evaluate, and interpret the LM. The process of adapting the LM through data is referred to as data engineering, as described above for Figure 3 .

[0094] Semantic data is a category of data. Semantic representations, i.e., data that expresses the meaning of words and sentences, provide the basis for evaluating the coherence of the LM to the user intent. Semantic representations can be constructed as graph data structures, where nodes represent concepts and edges represent relationships between concepts, and both nodes and edges are associated with various attributes to enhance their expressiveness. There are many other methods of semantic representation, including so-called black-box models derived from ML. For the purposes of the disclosure herein, semantic representations provide a computer with a model of the underlying meaning of natural language.

[0095] One common type of training data is question and answer. Other types of data can be rephrased (or transformed) into question and answer (QA) pairs. Semantic representations are transformed into QA pairs. For example, the statement “<car> is a <vehicle>” can be rephrased into a QA pair through a simple data transformation: is (car, vehicle) = {true / false}. In this way, the transformed QA pair can be used as a semantic evaluator with a binary ML classifier.

[0096] Figure 6 is a block diagram overview of a data engineering method for augmenting a LM.

[0097] In Figure 6 , data engineering 510 is performed to generate data 620 that can be used by the LM 630 as well as other ML models 640. Data 620 used as input to the LM 630 or ML models 640 can be entered directly from data engineering into the model, or can first be subjected to data transformation 624 to transform from semantic representations 622 to question and answer pairs 626.

[0098] In other embodiments, data 620 from data engineering 610 can be subjected to other forms of data transformation and entered into the model in forms other than question-answer pairs.

[0099] Figure 7 and Figure 8 is an overview of a method of generating data for adapting a LM using a semantic engine according to embodiments. As mentioned above, user-affected data can be generated from the LM that is being adapted.

[0100] Data generation refers to an automated process of generating data through data induction, data aggregation or integration, or extracting data from KR, etc. In this document, various semantic data generators are collectively referred to as “semantic engines”.

[0101] Figure 7 and Figure 8 Data generation discussed in Figure 7 is not the same as data engineering discussed above. Figure 8 and Figure 4 The semantic engine discussed in

[0102] The LM itself is an important data generation engine, including the semantic engine. In the data generation process of the LM, the LM generates candidate training data, selects (filters) the candidate training data using evaluation criteria, and uses the filtered training data to train a secondary model (e.g., a ML model, see FIG. 11 below) or retrain the LM.

[0103] In Figure 7 , semantic training data 720 is derived from LM 710 by a semantic engine that has received input from a user representing the user’s intent (as discussed above in Figure 4 The semantic engine uses the input from the user to evaluate and filter the semantic training data, generates user-affected enhanced training data, and uses it as input for adaptive retraining of the LM.

[0104] Figure 8 is a block diagram overview of how a semantic evaluator is used to determine the coherence of the adapted LM in the systems and methods described herein to the user’s intent.

[0105] A semantic coherence LM is considered coherent if its output directly or indirectly aligns with the user's intent or the expected semantic representation of that intent. Various methods can be used to determine the level of coherence, such as measurements of semantic similarity. This problem can be evaluated directly based on user input or indirectly through proxy models, agents, applications, or data that act on behalf of the user. In this paper, the evaluation tool is referred to as an evaluator, and the semantic coherence evaluator is referred to as a semantic evaluator.

[0106] like Figure 8 As shown, semantic engines can also be based on other models. Figure 8 A method for generating data in the form of semantic representations is demonstrated. The AKRM 820 generates a semantic representation 830 from a given user context derived from user input 810.

[0107] The AKRM discussed herein may be similar to the AKRM described in U.S. Patent No. 9,177,248 and U.S. Patent No. 17 / 713,096, both of which are incorporated herein by reference.

[0108] AKRM, based on semantic compositionality, imposes constraints on how concepts can be combined and organized within a relational hierarchy. In this embodiment, AKRM 820 extracts context from user-provided input 810, which may include prompts, queries, or any other object of interest that can be described in text, such as web pages, images, or videos. User input 810 can be translated by an input translator into context available to AKRM 820. AKRM takes context as input and returns a semantic representation expressing the meaning of that context.

[0109] Semantic representation 830 can be used as Figure 6 The input data, where semantic representation 622 is transformed into question-answer pairs for use in LM and ML models.

[0110] Figure 9 This includes analyzed user intents 910, which are used to generate training data 920 or objects of interest (cues, queries, etc.), which serve as input to adaptively train the LM based on the user intents. After adapting the LM based on the inputs, at least one evaluator 940 is used to evaluate the coherence of any output of the adapted LM with the analyzed user intents 910. The evaluator may include an ML model 942, an AKRM 944, and / or a KR 946.

[0111] ML models can be deployed as evaluators to adapt LMs. These models can be called probes. For example, ML model 944 can be used to evaluate the validity or coherence of the adapted LM output relative to user input.

[0112] Classification can be used to evaluate question-answer (QA) pairs derived from semantic representations. Here, the ML model is referred to as a classifier. The model itself can be implemented in different ways, e.g., using a regression technique for classification, and the data provided can be used for various training, evaluation, or explanation purposes during the development process. These approaches can be used to train any ML model for any task by using KR generated from the common context of these objects to annotate task-specific data. Examples of these objects include question-answer pairs or descriptions of images and videos. KR can be used as metadata for data augmentation.

[0113] One way to directly evaluate semantic coherence is to use AKRM 944. Other KR that indirectly express user intent can also be used for semantic evaluation. In Figure 8 AKRM 944 or KR 946 can be generated based on the output of the LM and compared to the analyzed user intent 910 to evaluate coherence.

[0114] For example, KR, such as KR 946, derived by a browser for a web page can represent the underlying intent of the browsing activity. KR derived from the user’s past interactions with an application or content can similarly imply their current intent. Authoritative sources, such as rules that govern professional practices or laws, or the consensus of a group of experts expressed in academic literature, can imply the user’s intent in these communities. Many such KR can be generated or prepared to represent the user’s intent.

[0115] The semantic evaluations discussed herein can be applied as pre-training to annotate training data; the semantic evaluations can be used as post-training reinforcement learning targets; the training data can be prioritized based on these semantic evaluations as a form of curriculum learning; the hint words provided by the user can be restructured as pre-training under supervised learning; and so on. The embodiments contained herein can be applied to the above-mentioned multiple techniques, aiming to improve the alignment or coherence of the LM with its user.

[0116] The functional components described above will be deployed and redeployed in various ways in subsequent specific implementations.

[0117] Enhancing prompt words with user-affected knowledge representation (KR) to improve language models (LM) In the implementation of this adaptation method, the hint word is augmented by KR to improve the LM. The user input representing the user’s intent is received through the hint word interface, the semantic engine creates a knowledge representation using the user input, and then augments the hint word using the knowledge representation and uses it as input to the LM, thereby adapting the LM to the user input.

[0118] The hint word provides instructions to the LM, indicating what kind of output it should generate. The evaluation of the output depends on the accuracy of the input. Only when the evaluation basis derived from the hint word (or alternative input) supports the evaluation, will the output be evaluated as coherent with the user’s intent.

[0119] As shown in Figure 10A and 10B , a user first provides a user input 1002 to a prompt interface 1004. The user input 1002 and prompt interface 1004 can be the same as the prompt engineering system shown in Figure 3 . The prompt interface can also optionally receive additional contextual information (e.g., from an existing user profile, user model, history of previous interactions) as part of the user input, or provided by other components in the larger system.

[0120] The user can be, for example, a software developer, data scientist, or prompt engineer, among others. The user agent can also be implemented as an automated step as part of a larger system.

[0121] In certain variations of this implementation, the prompt interface can facilitate the aggregation of a prompt chain, such that the KR (below) can be injected into one or more prompts in the chain. The prompt chain uses the output from one prompt as input to create a subsequent prompt, resulting in a related chain of prompts. In other variations, the KR can be injected into one or more independent prompts (a sequence or set of prompts).

[0122] In Figure 10A , the prompt augmentation mediator 1010 follows the mediator pattern in software system design to coordinate the various components of the system.

[0123] The mediator pattern defines an object that encapsulates how a collection of objects interact. Since the mediator pattern can alter the run-time behavior of a program, it can be considered a behavioral pattern. However, any design pattern that allows a sequence of operations to be performed is applicable, such as the pipeline design pattern common in data engineering.

[0124] Figure 10B An embodiment without a mediator is shown, however, coordination between system components can exist in other forms. Figure 10A The discussion below and below follows the mediator design pattern.

[0125] The prompt augmentation mediator 1010 is an example of a component that coordinates the actions of other components within the system. In other embodiments, this role can be fulfilled by any component that is able to coordinate other components, such as an “orchestrator” that is similar to a mediator but can not follow the mediator pattern, or an agent. In still other embodiments, components can interact directly without an intermediary such as a mediator.

[0126] The prompt word augmentation mediator 1010 receives the user input 1002 through the prompt word interface 1004. The mediator 1010 then provides the user input 1004 to the semantic engine 1020. In some variations, the topic translator 1022 can translate the user input 1002 into objects of interest, from which the semantic engine generates a KR 1024. The KR 1024 can be a taxonomy of concepts in graphical form, where nodes represent concepts and edges represent relationships between concepts. These relationships can be hierarchical (broader or narrower) or non-hierarchical (related). Each node has attributes called labels, which represent preferred and alternative (synonymous) ways to describe the concept in natural language.

[0127] A set of evaluation criteria 1026 for the KR determines which concepts should be included in the KR 1024. In this embodiment, the KR 1024 contains narrower concepts than the initial concepts in the user input 1002 or synonyms of the initial concepts in the user input 1002. The KR should limit broader concepts or related (non-hierarchical) concepts to the scope of the initial concepts in the user input 1002. All concepts in the KR should be semantically coherent with each other, including the concepts that explicitly represent the user input. The KR can be graph-based (e.g., based on an ontology model or a knowledge graph), and / or based on the distribution of labels in a text corpus, which is determined by querying a content index or inputting prompts to the LM.

[0128] Next, the mediator 1010 provides the user input and, optionally, the generated KR 1024 to the prompt word generator 1030. The prompt word generator 1030 uses this information to select an appropriate prompt word 1040. The prompt word can be stored in a prompt word store 1032 for reuse and easier management. The selection of the prompt word 1040 can be user-driven (selecting an explicit prompt to be included alongside the user input) or automatic. If automatic, the selection of the prompt word can be determined from a configuration file. Another variation is that the automatic selection of the prompt word template can be based on a set of heuristic rules. Another variation is that the selection can be based on the prediction of a machine learning (ML) classifier that is trained to recognize the most suitable prompt word for the current user input. For either the heuristic rules or the ML, the KR can be a useful source of information or features that influence the selection of the prompt word.

[0129] As mentioned above, the prompt word contains instructions to the LM. The prompt word can be a prompt word template, where the injection sites are explicitly indicated by placeholder text strings (e.g., “[INJECTION SITE]”). Alternatively, the prompt word can also not indicate explicit injection sites. In either case, the selected prompt word is returned to the mediator 1010 for further processing.

[0130] Next, the mediator 1010 provides the selected prompt 1040 and the KR 1024 to the prompt injector. If the prompt does not have explicit injection sites marked, some algorithm (e.g., a set of heuristics or an ML model) can be used to identify the best injection locations in the prompt. For example, a simple heuristic can insert an injection placeholder near the end of the prompt, before the common text "answer:". The prompt template can be expressed as an API, schema, or the like.

[0131] Once the prompt has injection sites (whether the prompt itself has them or they are inserted), the prompt injector 1050 injects the KR 1024 into the prompt template, replacing the placeholders. In one variant, the KR 1024 can be expressed in a conventional way (e.g., using the text syntax Turtle for Resource Description Framework (RDF) or the triple syntax schema for SPARQL), but this assumes that the LM has been trained with text containing the common format. When the conventional way is a graph, one possible prolegomenon can be "When these instructions are executed, the output should conform to the graph [to which it is constrained]." Alternatively, the concept relationships in the KR 1024 can be expressed as natural language assertions (e.g., "Canada refers to the Canadian Women's National Soccer Team," "football is equivalent to soccer"). A suitable prolegomenon can be added before the set of triples or assertions, e.g., for a natural language variant: "When these instructions are executed, assume the following assertions are true:"

[0132] After the knowledge identification 1024 is injected into the prompt 1040, the mediator obtains an enhanced prompt 1045. Next, the mediator 1010 provides this enhanced prompt 1045 as input to the LM 1060 and receives the generated response (answer) from the LM as output. The generated response is in textual form.

[0133] By including semantically coherent, more narrowly scoped concepts (and optionally synonyms), the enhanced prompt 1045 is more specific than the original prompt 1040. As a result, the generated answer is more likely to be assessed as aligned with the user's intent.

[0134] For example, assume the user input is “Name a soccer player who scored for Canada in the World Cup.” Typically, including synonyms in the KR helps disambiguate between concepts that have the same label but different meanings. In the current example, the concept of “soccer” can appropriately refer to “soccer” primarily because it is coherent with “World Cup.” At the same time, a more narrow term can also disambiguate the concept (e.g., “Canadian National Soccer Team” can disambiguate “Canada”). In some variations of this implementation, additional context from the user model can further influence the generated KR such that “Canada” refers to women’s soccer instead of men’s soccer (following the user’s preference for women’s sports). In other cases, a sufficiently narrow set of concepts will disambiguate the concept (e.g., “goalkeeper” and “forward” instead of “soccer player”). Note that incorporating broader concepts (e.g., “sports”) or related concepts (e.g., “football”) can risk expanding the LM’s possible inferences space, and thus should be excluded from the KR.

[0135] In this case, the user’s intent for “women’s soccer” is analyzed from the user’s preference for women’s sports, and this intent can provide the correct answer; without this analysis, the provided answer can be based on the most recent World Cup, or the majority of people’s preference for men’s soccer over women’s soccer.

[0136] The mediator 1010 can return the enhanced answer to the user via the prompt word interface.

[0137] Note that the system described in this implementation does not change the state of the LM. Instead, the prompt word augmentation system takes a stateless approach by augmenting the LM’s prompt word input in real-time to adapt its behavior.

[0138] Optionally, the LM can also generate a response based on the un-augmented prompt word and display it to the user, or generate multiple responses based on multiple augmented responses and display them to the user (e.g., in a research or development scenario).

[0139] Using user-affected knowledge representation (KR) and language models (LM) to label input data for machine learning (ML) classifiers Figure 11A A second implementation is to adapt the LM by generating training data for a secondary ML model. This implementation is illustrated in FIG. 11.

[0140] In some cases, ML models are more useful than LMs. LMs can be very large and cannot fit into devices with insufficient memory. LMs can be hosted in data centers accessed over the internet, which can be costly in terms of hosting and operation. The results of LMs are not deterministic (the same input can produce different outputs). This makes LMs potentially difficult to troubleshoot (lack of predictability and reproducibility), and unreliable (which can erode user trust and be prone to catastrophic errors in some applications). LMs present privacy issues because they are typically trained using large amounts of data, including public data that contains personally identifiable information (PII).

[0141] However, ML models suffer from performance issues when there is a lack of training data related to a particular user domain to support a task. For example, a data scientist can lack the data needed to research a certain esoteric problem. To achieve the desired level of performance, a supervised learning approach is often employed, in which the user manually labels training data consistent with the task. These supervised approaches can be prohibitively expensive to produce and can introduce errors. While training data can be efficiently derived from an LM or any unsupervised data generation process, this data can not be sufficient for the intended use. That is, the topic of the generated data can deviate from the user’s intended intent, and the generative nature of the LM exacerbates this issue.

[0142] To address these issues, the embodiment shown in FIG. 11 combines automatic (unsupervised) data generation using an LM, and automatic data labeling based on a KR system, in place of a manual labeling approach. This two-pronged approach maintains the efficiency of a fully automated system, while keeping the output aligned with the user’s intent. The user configures a process to derive QA pairs from the LM. This process can use a prompt word template, prompt word parameters, and / or existing data sources in the user’s domain to generate inputs to the LM, but there are many other variations of this approach. An input translator transforms the QA pairs output from the LM into inputs for the semantic engine. As with the previous implementation of FIG. 10, the semantic engine is configured to maintain the user’s standards for the context of influence, semantic coherence with the context, and specificity through narrower concept expression. Each QA pair in the initial training set for the LM is annotated with KR specific to each QA, and the automatically labeled training data is used to train a secondary ML model. In this way, the LM is adapted to produce training data that is more aligned with the user’s intent.

[0143] Figure 10A and 11BEmbodiments of the system include: a prompt interface 1104 for receiving user input 1102; a mediator 1110; a LM 1120; an input translator 1130; a semantic engine 1132; a knowledge base 1140; a question-answer pair 1150; a semantic annotator 1160; an ML pipeline 1170; and an ML classifier 1172.

[0144] Similar to the embodiments of the system, Figure 11A the embodiments of the system, Figure 10A the embodiments of the system begin with receiving user input 1102 through a prompt interface 1104 and sending to a mediator 1110.

[0145] Similar to the embodiments of the system, Figure 3 and 10B the user input and prompt interface can be the same as the prompt engineering system shown in Figure 10A . Likewise, the prompt interface can selectively receive additional context (e.g., from an existing user profile, user model, history of previous interactions) as part of the user input or provided by other components in the larger system.

[0146] Similar to the embodiments of the system, Figure 11A in Figure 11B , the mediator 1110 is an example of a component that coordinates the actions of other components within the system. In other embodiments, this role can be fulfilled by any component capable of coordinating other components, such as an “orchestrator” similar to a mediator but possibly not following the mediator pattern, or a proxy. But in other embodiments, components can interact directly without an intermediary such as a mediator.

[0147] Figure 10A An embodiment is shown that does not have a mediator, however, coordination between system components can exist in other forms. Similar to the embodiments of the system, Figure 11A the discussion below follows the Mediator design pattern. Developing semantic evaluators for language models (LM) using user-affected knowledge representation (KR)

[0148] The mediator 1110 sends the user input 1104 to the LM 1120, which in this embodiment outputs a LM answer 1122 to a question or request derived from the user input 1102.

[0149] ​The LM answer is sent to the mediator 1110, which then sends it to the input translator 1130. The input translator 1130 extracts the QA pairs from the LM answer 1122 and inputs them into the semantic engine 1132. The semantic engine 1132 uses the KR evaluation criteria 1134 to generate a KR for each QA and outputs the KR(s) 1140 to the mediator. The KR evaluation criteria 1134 contains criteria for maintaining the coherence and specificity of the KR to the user impact context derived from the user input and other user-related data.

[0150] The mediator sends the KR(s) 1140 and the associated QA(s) 1150 to the semantic annotator 1160. The semantic annotator 1160 automatically annotates the QA pairs to generate annotated QA pairs 1155 and sends them back to the mediator 1110.

[0151] The mediator 1110 outputs the annotated QA pairs to the prompt word interface 1104 for review by the user or user agent. The annotated QA pairs 1155 are then used as user input to the ML pipeline 1170 as training data for the ML classifier 1172.

[0152] One variant of this implementation is to annotate (label) the question-answer pairs, where the LM takes the question as the prompt word and returns the answer. This variant reinforces the user’s intent in asking the question and answering it.

[0153] In some variants of this implementation, the input to the LM can be a question, while in other variants, the input can be an instruction. Similarly, the output of the LM can be an answer or any response generated by the LM, such as the result of executing an instruction.

[0154] Another variant includes annotating the answer from the LM with the KR-answer intersection. In this case, the question is discarded, and the ML classifier is trained based on the annotated LM output (e.g., the answer to the question or the result of executing an instruction).

[0155] Another variant is to take the answer (rather than the question) as input to the LM and annotate the generated question with the KR-question intersection. In this case, the answer is discarded, and the ML classifier is trained based on the annotated question. Alternatively, the entire answer-question (or result-instruction) pair can be annotated. In this case, the input and output order of the LM is swapped, but the resulting pair is still fully involved in the annotation (labeling) process.

[0156] We describe the input data to the ML classifier in terms of training, but this input data can also be used for other purposes, such as validation, evaluation, and explanation (which is common when developing an ML model). Similarly, when describing the ML training pipeline, this description can also be extended to include, for example, an evaluation pipeline.

[0157] Figure 12 The LM uses the KR to generate a semantic evaluator for the LM to enhance its functionality. This enhancement employs a similar configuration as the previous implementation, including the LM, user input and user intent, training data of question-answer pairs, a semantic engine, and an input translator. As described above, the LM is notorious for its ease of outputting hallucinations that lack evidence of existing knowledge. This implementation addresses this issue by exploiting the independence between two separate generative systems: the LM and the semantic engine. If these two independent systems generate a coherent result using different processes, this coherence enhances the confidence in the validity of the result. Here, rather than using the output from the semantic engine to annotate the output of the LM, the system is configured to compare the two results. The semantic engine is configured to apply the analysis of the user intent and generate a more descriptive and specific semantic representation than the user context using the same criteria as described above. As described above, in the use case of question-answering, the input translator is used to derive the corresponding input for the semantic engine and the LM. For each input, the semantic engine generates a unique KR representing the user context, while the LM generates an answer representing the user context (i.e., the question). In other embodiments where the input-output is not question-answering, the input can be a prompt word or an instruction, resulting in the appropriate output or completion of the process.

[0158] Each KR and its associated answer are evaluated for their coherence using semantic similarity (or relevance) as the evaluation criteria. Coherence here refers to the evaluation of the degree of alignment between the KR and its associated answer, which ultimately reflects the evaluation of the coherence with the initial user intent. Coherence here can be evaluated on a continuous spectrum.

[0159] In some embodiments, a binary classifier can be used to classify each result as coherent or incoherent based on the semantic similarity measure. The two classes of coherent or incoherent can still be measured on a continuous spectrum, where values above a certain threshold result in a classification of coherent and values below the threshold result in a classification of incoherent. In this way, each result can be classified as a "hallucination" (incoherent) or "real" (coherent) based on the user's intent.

[0160] Techniques for assessing semantic similarity can include word embeddings, latent semantic analysis, formal concept analysis, distributed semantics, knowledge base methods (incorporating dictionaries, glossaries, and other knowledge representations), and search-based assessments discussed in the background. Notably, search-based factuality assessments (or fact checking) or "leveraging availability" are among the most common approaches currently, where users manually assess LM outputs through search engines. However, as noted in the background, search-based solutions often use the same underlying data as large LMs, raising concerns about interdependence between data sources. This implementation restores independence of the overall system by introducing an additional, fully decoupled generative data source.

[0161] Figure 12 is a block diagram of the above implementation. Figure 12 includes user input 1202, prompt word interface 1204, input translator 1220, LM or multiple LMs 1230, LM configuration parameters 1232, semantic engine 1240, KR assessment criteria 1242, user-specific KR 1250, semantic evaluator 1260, and fit output 1270.

[0162] Similar to the embodiments of FIGS. 10 and 11, Figure 3 the embodiment first receives user input 1202 through prompt word interface 1204. Similar to FIG. 10, the user input and prompt word interface can be the same as the system for prompt engineering in Evaluating candidate outputs from language models (LM) using atomic knowledge representation models (AKRM) Likewise, the prompt word interface can optionally receive additional context (e.g., from an existing user profile, user model, history of previous interactions) as part of the user input or provided by other components in the larger system. In certain variations of this implementation, a different interface than the prompt word interface can be used.

[0163] Although not shown, the prompt word interface can interact with the input translator 1220 through an intermediary or orchestrator.

[0164] The user input 1202 includes a user request and data about the user's intent.

[0165] The prompt word interface 1204 provides the user-affected data to the input translator 1220.

[0166] The input translator 1220 translates the output from the prompt word interface into a form that can be input to the LM 1230.

[0167] LM 1230 generates an output based on the user input and sends the output to semantic evaluator 1260. The output can be a question-answer pair. The LM is trained using training data that is independent of the training data used by semantic engine 1240. In some embodiments, multiple LMs can be used, each trained using different training data and / or configured using different configuration parameters, and each generating an output.

[0168] Input translator 1220 translates the output from the prompt word interface into a form that can be input to semantic engine 1240, such as a contextual form.

[0169] Semantic engine 1240 generates user-affected KR 1250 using KR evaluation criteria 1242. User-affected KR 1250 is sent to semantic evaluator 1260. The semantic engine can be built from a knowledge domain that is independent of the knowledge domain used to train the LM. The semantic engine can employ AKRM.

[0170] Semantic evaluator 1260 compares the output from LM 1230 to user-specific KR 1250 to judge its fit and, where possible, outputs the fittest output 1270, which is the output from LM 1230 that is the most "fit." In embodiments where multiple LMs exist, the output from each LM is compared to KR 1250.

[0171] A binary classifier can also be used to classify each output as coherent or incoherent. While fit and coherence can be evaluated using the same methods and values, coherence can be evaluated in a binary system where there can be a threshold above which outputs are labeled as coherent and below which outputs are labeled as incoherent; while fit is evaluated on a continuous spectrum such that the "most" fit output has the highest score or rank. These labels can be interchanged or other terminology can be used. These classifications and labels can have relevance when using the outputs as training data.

[0172] The fit output 1270 can then be used to further train the specific LM 1230 to adapt to the user, while any unfit outputs can also be used to train the LMs to create a user-adapted LM.

[0173] Figure 12 The LMs are augmented by the user-specific KR to evaluate and select from possible output results. This implementation is an extension of the previous implementation and also includes AKRM.

[0174] Existing evaluation methods are typically applied prior to generating LM output. For example, evaluation criteria can be chosen to align with an existing body of knowledge or the intended task of the LM. As noted above, LMs are used to generate new knowledge rather than retrieve existing knowledge, which presents a challenge to any method that sets selection criteria prior to output generation. Accordingly, this implementation generates a user-specific KR representing the meaning of the user prompt or user intent while generating LM output, and derives evaluation criteria from the KR. It first configures one or more LMs to generate multiple outputs for a given prompt. There can be multiple ways to cause the LM to generate multiple outputs, such as adjusting sampling parameters of the LM to increase uncertainty in its outputs, in combination with adjusting the number of completions attempted by the LM for a given prompt. Sampling parameters include temperature, initialization randomness seed, topP, and other conventional settings of the LM.

[0175] Alternatively, or in addition, a set of different LMs can be used. Differences between these LMs can be due to their having been trained and / or fine-tuned on different data sets, although variations in configuration parameters can also cause such differences.

[0176] At the same time, this implementation also includes a process for semantic data generation that uses the same criteria for the AKRM as the previous implementations, in particular Figure 12 discussed above, to generate a KR for the corresponding LM prompt. Depending on the task and domain, the LM prompt can be extended into a chain of prompts to create a more comprehensive KR. Here, the components used to implement the AKRM are derived from the LM, although other tools can also provide the required functionality, including the possibility of using different LMs so that the AKRM and the LM being adapted are independent of each other. In this implementation, the LM is used to disambiguate and link the tokens in the user prompt (as entities) to internal representations that are relatively simple (or atomic) concepts and concept relationships. For each prompt, the semantic engine generates a KR, which is typically implemented as a graph data structure with concept nodes (representing entities) and relationship edges (representing connections between entities). Thus, each prompt is represented in the form of a graph data structure rather than unstructured or semi-structured text. Alternative data structures can also be used to represent the KR.

[0177] The LM is then again used to produce natural language labels for each node in the KR. The set of labels is recombined into a natural language summary of the KR. A semantic evaluator is used to compare the generated KR to the set of candidate outputs of the LM. This semantic evaluator can be similar to the semantic evaluators discussed in other implementations and generated in the same way, for example Figure 12the semantic evaluator 1260. Similar to previous implementations, this evaluation can be based on an evaluation of semantic similarity between the natural language summary of the KR and the candidate results generated by the LM. These embodiments also include other alternative evaluation methods, including search techniques.

[0178] Similar to the embodiments shown and discussed above, Figure 12 Similar to the embodiments shown and discussed above, the most fitting output, determined by evaluating the semantic similarity between the KR and the output of the LM(s), will be selected as the output that is most aligned with the user's intent. As discussed above, Figure 13 the outputs can be rated as coherent or incoherent based on semantic similarity using a binary classifier. Regardless of the fit or coherence of each output, it can be used as training data for the LM in order to adapt the LM based on the user-related information obtained from the user. Fitting or coherent outputs can be used for regular training of the LM, while non-fitting or incoherent outputs can be used for contrastive training of the LM.

[0179] Figure 12 is a block diagram of the interaction of an example system for the above implementation, up to the generation of the KR 1324. At this point, Figure 13 depicts the remaining steps, starting with the KR 1236, through the semantic evaluator 1260, the LM 1220, and resulting in the fitting output 1270.

[0180] Figure 13 includes the user input 1302, the cue interface 1304, the input translator 1310, the label-to-concept translator (LCT) 1320, the synthesizer (or generator) 1322, the KR 1324, the at least one LM 1330, the seed data 1335, the vocabulary extractor 1340, the analyzer 1342, the atomic knowledge 1350, and the AKRM 1360.

[0181] The LCT 1320 and the synthesizer 1322 constitute a semantic (KR synthesis) engine (shown in dashed box). The functions of the semantic engine can be performed online (in real-time), including receiving the input and outputting the KR. In some variations of this implementation, an offline generation process can be used as a batch process.

[0182] The LCT 1320 is an example of a component of the adaptive approach that itself generates user impact data, for example, in the form of the KR 1324.

[0183] The vocabulary extractor 1340 and the analyzer 1342 constitute an analysis engine (shown in dashed box). The functions of the analysis engine are typically performed offline and include developing and building the model of the meta-knowledge 1350, including the KR, from the input seed data 1335. In some variations of this implementation, the analysis engine can be run online as a service that enables users to build new models or update existing models.

[0184] As Figure 10A-13 shown, user input 1302 is received through a prompter interface 1304, similar to other embodiments in Figure 3 . The user input and prompter interface can be the same as the system for prompter engineering shown in Figure 13 . Again, the prompter interface can optionally receive additional context (e.g., from an existing user profile, user model, history of previous interactions) as part of the user input, or provided by other components in the larger system. In certain variations of this implementation, a different prompter interface can be used, such as a word processor.

[0185] Although not shown, the prompter interface can interact with the input translator 1310 through an intermediary or orchestrator.

[0186] Managing conditional variables using user-affected knowledge representation (KR) and language models (LM) The tasks that the system shown can accomplish will be discussed below.

[0187] As an explanatory model, the AKRM 1360 describes how to build a set of meta (atomic) knowledge data 1350 by using knowledge processing rules. The implementation of the AKRM often involves tasks similar to natural language processing, but in this embodiment one or more of the tasks can be accomplished by one or more LMs 1330.

[0188] As described in U.S. Patent No. 11,474,979, as cited herein, the analysis process is typically performed offline, first by a vocabulary extractor 1340 analyzing a collection of unstructured, structured, or semi-structured knowledge sources. In this implementation, the vocabulary extractor can prompt the LM to extract or summarize complex concepts from the text of a given knowledge source (e.g., an ontology model or other complex vocabulary). This way of using the LM essentially implements information extraction (including entity extraction and relation extraction) in place of traditional NLP methods for this task.

[0189] In certain variations of this implementation, the vocabulary extractor can prompt the LM to provide supplemental meaning for the extracted concepts by generating textual descriptions or additional complex concepts and relations to be included in the vocabulary.

[0190] The output of the vocabulary extractor 1340 is a set of input KRs for the analyzer 1322. The analyzer then decomposes the input KRs into smaller, atomic or elementary concepts and relations 1350. In this implementation, the analyzer does not rely on traditional text processing techniques, but can instead prompt the LM 1330 to identify smaller, independent concepts (atomic concepts) from the more complex concepts obtained from the vocabulary extractor 1340. In other variations, the LM 1330 can also be prompted to evaluate possible relations between atomic concepts as a form of relation discovery. As in the case of the AKRM, the set of atomic concepts and relations generated by the analysis engine can be described as a large meta-knowledge graph 1350.

[0191] As an explanation model, the AKRM 1360 also describes how to use the meta-knowledge 1350 to synthesize or generate KRs 1324. This is typically an online (real-time) process that begins with the user providing the synthesis engine with an object of interest, although it can also be provided by a user agent in a larger automated system.

[0192] In the synthesis engine, the role of the LCT 1320 is to translate the object of interest in the input translator 1310 into a corresponding set of concepts in the atomic knowledge graph, in order to disambiguate word senses and link entities. In this implementation, the LCT 1320 can explicitly prompt the LM 1330 to propose atomic concepts corresponding to the user input, given a list of candidate concepts.

[0193] From these atomic concepts, the synthesizer 1322 traverses the meta-knowledge graph 1350 to find candidate objects that can be coherent with the user input in 1310 to generate new concepts and relations. The knowledge processing rules in the AKRM 1360 dictate how the synthesizer combines the visited concepts and relations into the graph, and the LM 1330 can be prompted to validate these choices.

[0194] The synthesizer 1322 also includes a scoring mechanism to assess the coherence of the synthesized concepts with the input concepts corresponding to the user object of interest. The most coherent concepts are included in the generated KR 1324. In this implementation, the LM 1330 can be prompted to use as a ranking function, whose results can be interpreted as one of a number of scoring functions that assess the semantic coherence of the synthesized concepts in the KR.

[0195] The notion of composition is inherently complex and requires new tags to describe its semantics in natural language for human users to understand. While the rules in AKRM 1360 can be used to combine the tags of the parts of a complex notion, a rule-based approach tends to result in suboptimal, and even grammatically incorrect, output. In this implementation, the compositor 1322 can prompt the LM to generate a natural, fluent surface tag for a composed notion in KR 1324.

[0196] The compositor 1322 can also prompt the LM 1330 to summarize the entire KR 1324 in natural language. As previously described, this KR summary can be compared to the candidate output of the LM being evaluated as a basis for determining the coherence or fit of the LM output to the user input.

[0197] Figure 16 The LM is augmented with user-specific KRs to select and manage conditional variables derived from user input or user information, such as a user instruction set or a user profile. This implementation is an extension of the previous implementation and also includes a mechanism to inject the user-specific KRs representing the selected and managed conditional variables into the LM prompts.

[0198] The management of conditional variables is a vital task in many fields, including ML and statistics. Typically, conditional variables are used to provide context or control confounding factors (i.e., factors or variables that can introduce bias). In ML and statistics, conditional variables are also known as covariates or control variables. In LMs, conditional variables are used to provide context for predictions. In simple applications, conditional variables can be used directly as context, such as prompts provided to the LM. In more complex applications, conditional variables can be derived from sources external to the model, such as a knowledge base, a user profile, or an activity log.

[0199] In managing conditional variables, the primary goal is to isolate the effect of a target variable by reducing or eliminating the influence of other variables. There are many methods that can be used to manage confounding variables, such as stratification, multivariate analysis, or randomization, and the application of these methods can vary depending on the field of application. Typically, causal relationships are implicit or explicit and play an important role in many methods. Knowledge of how the underlying system works is crucial to the identification and management of conditional variables. Causal relationships between variables reveal how variables affect outcomes, which in turn can prompt the identification and management of suitable methods for confounding variables.

[0200] However, in black-box machine learning, especially in the LM domain, the causal mechanisms are not always clear, and often completely absent. As a result, the design and use of these black-box systems often rely on trial and error. Different combinations of conditioning variables need to be explored post-hoc. While there are tools that can speed up this process, e.g., interfaces that provide quick selection of conditioning variables, the whole process is still tedious and prone to unexpected results.

[0201] At least two opposing forces are at play in the dynamic process of conditioning variables and their management.

[0202] On one hand, as the understanding of the underlying system mechanisms deepens, the set of possible conditioning variables can become more constrained. For each discrete behavior or task, the conditioning variables can converge to a discrete set, representing an optimal solution. Certain, discrete sets of conditioning variables can emerge, optimizing the model’s ability in a targeted, controllable way. For example, personas can be defined for desired behaviors, such as historical figures, or roles can be defined for anticipated tasks or functions, such as accountants or lawyers.

[0203] On the other hand, the set of possible conditioning variables and their value spaces can expand as the number of model parameters (internal weights and biases) increases. The model is able to express higher levels of complex output, integrating more nuanced, more comprehensive instructions or contexts. For example, a relatively small model can be able to distinguish aspects of tone (formal, informal, etc.), but not aspects of style (descriptive, explanatory, narrative, etc.). As the model scales up, new aspects will emerge as viable conditioning variables. Continuing the previous example, a larger model will incorporate aspects of tone, emotion, and even intent. The largest models begin to simulate personalities as individuals similar to their users.

[0204] There are multiple high-level organizational bases for conditioning variables. The set of conditioning variables can be constructed based on the intended behavior of the model or the intended tasks / functions. The optimized set of conditioning variables can be organized using taxonomies or other organizational schemes to facilitate management. As intended behaviors are discovered or tasks are resolved, the decisive conditioning variables are retained, while non-intended or counterproductive variables are discarded. Taxonomies are often used to make these variable sets accessible and modifiable in the future. Aspects such as desired topics or subject matter ranges, desired tone or style, etc., constitute top-level categories. Various strategies are used to identify these optimized conditioning variables, including analyzing prompt words used by experts or documents representing specialized domains.

[0205] Similar to previous embodiments, the problem is that existing management strategies are regressive or retrospective; these solutions encode existing knowledge obtained through trial and error. The advantage of these generative systems is not to retrieve existing knowledge, but to create new knowledge. There is an infinite potential space of behaviors and tasks that can be discovered in collaboration with the user; there is also an infinite potential set of discrete condition variables that can emerge to be managed.

[0206] Thus, similar to previous implementations, this implementation generates user-specific KR to represent the meaning of the user input (or user intent). This serves as a preparatory stage to the main LM generation process. This approach is useful when condition variables cannot be managed in the traditional sense of a model training process. This alternative approach uses semantic data generation techniques, such as the AKRM standard, to generate user-specific KR, similar to previous embodiments.

[0207] Figure 16 is a block diagram overview of how user-influenced KR is utilized to derive and select dynamic condition variables to enhance the LM in the systems and methods described herein. Those skilled in the art will appreciate that there are multiple design patterns that can support the flow of functional components, such as shown here, including the Mediator pattern and the Pipeline pattern discussed previously.

[0208] As shown in Figure 10A and explained in the previous Figure 3 and 10B , the user first provides user input 1602 to the prompt interface 1604. The user input 1602 and prompt interface 1604 can be the same as the system for prompt engineering shown in Using atomic knowledge representation models (AKRM) as a source of logical constraints when training probes to assess semantic coherence of language models (LM) . As previously discussed, the prompt interface can facilitate a prompt chain to support variations discussed below where condition variables are injected into multiple prompts.

[0209] To manage condition variables, the aspect analyzer 1606 is used to analyze the terms extracted from the user input to determine an aspect or high-level category. There are many techniques and approaches that can be used for this analysis, including unsupervised clustering or using a LM. For example, the following instruction can be passed to the LM: “Identify a set of key words or key phrases associated with the prompt (or user input) and generate a high-level category (or aspect) to represent the meaning of the set of key words.” One or more knowledge base synthesis methods are applied to these extracted keywords to expand the main aspects to new concepts and concept relationships. The user-specific KR 1612 is a data structure as previously described that can again take the form of a hierarchical graph of concepts (nodes) and semantic relationships (edges). The semantic engine 1608 is configured with evaluation criteria 1610 similar to the previous embodiments - to enforce criteria such as user influence context (from prompt words or user input), semantic coherence with the context, and specificity expressed through main narrower concepts - that can be used to perform the synthesis of this KR.

[0210] Prior to main LM inference generation 1616, a user influence knowledge representation selection condition variable 1614 is used. In a simple case, the selection of condition variables is achieved by traversing the user-influenced KR using the aspects and terms (aspect values) associated with the user input. For example, if the user input is associated with the aspect "topic" and the terms {aircraft, war, tactics}, then the user-influenced KR can yield an additional set of condition variables, such as {air combat, interceptor, strategic bombing, electronic warfare}, that are not present in the user input.

[0211] The selected condition variables are injected into the user prompt via the prompt interface, as previously discussed, using the prompt injection method. It is noted that a multi-step prompt injection method can also be employed, such as in a dialog with a dialog LM. Typically, such a method first identifies a persona, such as "you are a senior financial analyst." Here, a large set of condition variables are combined as the user-influenced KR and injected into the dialog over multiple interactions. This method helps simulate the gradual revelation of the "personality" that the LM is expected or intended to have, or, the gradual understanding of the subject matter.

[0212] Based on user feedback 1618, the user-influenced KR can be stored in a KR repository 1620, such as a database, to influence future interactions with the user, or to reflect the "personality" of an organization or collective application. This storage is also useful when there are few user inputs or prompts, such as for new users of the LM; one will recognize this as a cold start problem. In addition to or in place of the traditional practice of prompt guidelines, the stored user-influenced KR can be shared to reflect the desired "personality" or style. One skilled in the art will recognize that many data analysis methods can be used to derive novel inputs or new sets of condition variables from this new data source, as well as use the stored KR as input to the semantic engine 1608.

[0213] Those skilled in the art will also recognize other ways of using conditional variables to influence the output generated by the LM. For example, in addition to indirectly influencing the output prior to generation, the output can be directly manipulated after generation by text editing or formatting operations (e.g., deleting, adding, or replacing keywords and phrases). Alternatively, the conditional variables can be evaluated in terms of various reasoning parameters, such as adjusting temperature settings to more closely align with the intended meaning of the output suggested by the conditional variables.

[0214] Using knowledge generation rules to train language models (LM) by selecting semantically coherent concepts Figure 8 An LM can be augmented by incorporating logical constraints derived from the evaluation of semantic coherence of the LM. The implementation includes an LM, a semantic engine, a semantic evaluator, an input translator, and a loss function optimizer. As described in the background, an LM can not understand the meaning of all sequences of predicted words. The implementation generates an ML model as a probe to evaluate the performance of the LM in terms of semantic consistency based on an interpretation of the expected coherent semantic information structure. The interpretation of semantic coherence is expressed in the form of logical constraints for an augmented loss function optimizer. The semantic coherence of the LM is evaluated by using a semantic evaluator configured as an AKRM. The AKRM imposes consistency constraints based on the combinatorial nature of concepts and concept relations. Under the AKRM, each constituent concept contains relations between the simpler concepts that combine it. Based on this insight, the loss function optimizer can be augmented to enforce this semantic consistency. The KR as input to the optimizer is analyzed based on the constraints of the AKRM to derive a set of underlying (or relatively simple) concepts and concept relations. The KR can be used that has a high degree of authority and semantic succinctness, such as a dictionary, a controlled vocabulary, or an upper ontology model. For each set of concept relations, an internal representation is extracted from the LM. A mapping is established between the set of semantically derived QA pairs and the corresponding internal representations of the LM. An unsupervised loss function of the ML is optimized to maximize the consistency of the semantically derived logical constraints. In this implementation, the starting point is the AKRM, which establishes constraints on the structure of semantic data. That is, for the output of the LM to be considered semantically coherent, the structure of this data should be consistent with the data generated under the AKRM. This implementation renormalizes these semantic constraints into a set of logical negated consistency constraints. These embodiments include alternative approaches to applying consistency constraints to adapt the LM.

[0215] Here, the AKRM analyzes a set of reliable knowledge sources for a particular or a variety of domains. Possible knowledge sources can be unstructured (e.g., emails, corporate documents, product descriptions, job descriptions), structured (e.g., ontology models, taxonomies, and other controlled vocabularies), or semi-structured (i.e., a mix of structured and unstructured data).

[0216] AKRM is used to generate a set of sentences (or assertions) that are consistent with its encoded knowledge about the input source. For example, given that there is a hierarchical relationship between "medical doctor" and "professional degree," AKRM can have discovered that there is a hierarchical relationship between "doctor" and "professional." The generated sentence will be "doctor is a professional" or some well-formed and accurate variant of it. In some embodiments, the variant can be a structured representation, such as a logical form (e.g., is (doctor, professional)). The choice of representation can depend on the data used to train the LM under test. If the LM was not trained using sufficient data containing structured representations, then a natural language formulation is preferred.

[0217] Each sentence is rephrased as a pair of yes / no questions. This rephrasing process generates a contrast pair for each question. Using the same example, a contrast pair can be "Is doctor a professional? Yes." and "Is doctor a professional? No." Alternatively, it can be "is (doctor, professional) = yes" and "is (doctor, professional) = no."

[0218] For each contrast pair, the internal representation of the pre-trained LM under test is extracted and normalized. These extracted representations will be used as features to train a secondary ML model. Normalization eliminates the influence of phrasing each sentence as a yes / no question.

[0219] Using conventional ML techniques, the probe model is trained as a binary classifier that can predict whether the LM's answer to a given question is true or false. This outcome reflects the LM's latent or parametric knowledge about the given question. The binary classifier can be linear or non-linear, although a simple linear model can be sufficient.

[0220] The probe model has an unsupervised loss function that the training process optimizes to learn the parameters of the probe model. Specifically, the loss function contains a consistency constraint that, for a given contrast pair, aims to minimize the difference between the probability of the true proposition (e.g., p ("Is doctor a professional? Yes.") or p (is (doctor, professional) = true)) and the complement of the probability of the false proposition (i.e., 1 - p ("Is doctor a professional? No.") or 1 - p (is (doctor, professional))). If the LM is perfectly consistent in its latent knowledge for a given contrast pair, then the two calculations will be identical, and the consistency loss will be zero.

[0221] A variant of this implementation can include an additional constraint in the loss function of the probe model to avoid degenerate solutions. In some variants, this can be a confidence constraint that computes the minimum of the true proposition probability (e.g., p(“2+2=4? Yes”) can be 0.9999) and the corresponding false proposition probability (e.g., p(“2+2=4? No”) can be 0.0001) for a given alignment. When the LM’s latent knowledge is “confident,” the minimum of these two probabilities will be zero or close to zero.

[0222] After the probe training is complete, it can be used to assess the LM’s latent knowledge for a specific set of “yes / no” questions with expected answers. The assessment of the LM’s latent knowledge can be used to identify its limitations, as well as opportunities for improvement. For example, knowledge areas where the LM gives incorrect answers can indicate specific gaps in the LM’s training data. Identifying these gaps can motivate us to retrain the LM, or fine-tune the LM or its tokenizer with more comprehensive data. In some applications of this implementation, the identified gaps in the LM’s knowledge can be filled by prompt engineering, e.g., injecting KR into prompts (as described earlier in this application).

[0223] Figure 14 Self-training is a form of semi-supervised learning in which a model, e.g., a LM, uses its own predictions to generate new training data. These techniques can include data augmentation methods such as back-translation (reverse translation), text augmentation, question-answer synthesis, programmatic annotation, and co-training. The purpose of data augmentation is to enrich the samples in a given training dataset, or to create entirely new data.

[0224] Another main approach to improve LMs is to use reinforcement learning with human feedback (RLHF). Variants of this approach, called reinforcement learning from AI feedback (RLAIF) and large language models as judges (LLM-as-a-Judge), use LMs to provide evaluations, feedback, and responses. Data augmentation, reinforcement learning, and instruction methods can be combined into different solutions. Models built based on these data can use techniques such as uncertainty estimation, regularization (e.g., dropout), and graph-based confidence assessment to assess their robustness. There are other validation methods that rely on external feedback and human annotators, such as theorem verification, unit testing, adversarial testing, sensitivity analysis, and transfer learning evaluation (for different tasks or domains). These methods are usually targeted at specific knowledge areas and inherit the same limitations of domain-specific data, i.e., lack of generalization.

[0225] In the context of the present invention, self-training is used to address the problem of data sparsity, i.e., the lack of sufficient training or evaluation data, especially in terms of user-specific knowledge and interests. In these cases, existing self-training techniques can fail to effectively augment the existing training or evaluation data, resulting in the production of noisy, inaccurate, or misleading examples. In other cases, if the new knowledge is esoteric or non-derivative, there can be no existing training or evaluation examples to draw from. In these cases, self-training techniques can result in the production of noisy, inaccurate, or misleading data.

[0226] One of the capabilities required of a self-training system is a deep understanding of semantics and common sense that make up coherent and accurate models. For example, semantic entailment relationships can be used to generate more reliable training or evaluation data. This technique employs a prompt word strategy to formulate NLU tasks as contextual entailment problems. Typically, given a premise and a hypothesis, the task is to determine whether the hypothesis logically follows (entails) from the premise. The limitation of such an approach is the need for high-quality labeled entailment data, which is frustrated by the sparsity of data in creative and esoteric knowledge environments.

[0227] The solution to these challenges disclosed herein is based on an explanation of how coherent semantic data can be generated and evaluated. This explanation, called AKRM, has been discussed above Figure 17 and Figure 17 . AKRM will be described in more detail here and applied to a self-training method that generates data for an ML pipeline. Those skilled in the art will recognize that this general explanation of coherent semantics and ML can be applied to other self-training methods.

[0228] The goal of the following self-training is to have the LM recognize and apply the internal structure of coherent semantic data. That is, the structure of coherent semantics is explicitly expressed and presented in the form of data that in turn can be used to recognize the structure of coherent semantics when presented with esoteric contexts or novel inputs.

[0229] Figure 18 is a brief illustration of the coherent semantic structure imposed on the LM by this method. The rules are symmetrical, with the analysis rules used to decompose complex concepts into simpler ones, and the corresponding synthesis rules used to combine simpler concepts into complex ones. For simplicity, the example here is limited to just two basic rules: a) concept combination or decomposition; b) hierarchical relationship combination or decomposition. These two rules, in combination, can deconstruct complex knowledge representations through iterative operations on seed data and synthesized data, and symmetrically, build complex knowledge representations from simple seed data and analyzed data. This combinatorial approach enables AKRM-based systems to manage large language models, creating progressively complex concepts from simpler premises. In Figure 15In this case, concept C1 is composed of concepts C2 and C3; alternatively, concept C1 can be decomposed into concepts C2 and C3. Similarly, concept C4 is composed of concepts C5 and C6. If there is a hierarchical relationship between concepts C1 and C4, then it can be inferred that there is also a hierarchical relationship between C2 and C5, and between C3 and C6. That is, the complex relationship between C1 and C4 can be decomposed into simpler relationships between C2 and C5, and between C3 and C6; alternatively, the complex relationship between C1 and C4 can be composed using the relationships between C2 and C5, and between C3 and C6.

[0230] Figure 15 An overview of system 1800 is depicted, which uses knowledge generation rules to select semantically coherent concepts for self-training a LM.

[0231] In the analysis phase, the seed data 1802 is analyzed to discover combinations of coherent concepts and relationships between concepts. This approach starts with seed data, which is a collection of reliable knowledge sources that encompass simple, basic, or general knowledge. Examples include dictionaries, upper ontology models, or high-quality encyclopedic knowledge, whose data is typically organized in structured formats designed to improve semantic clarity and interoperability, such as RDF (Resource Description Framework), OWL (Web Ontology Language), XML (Extensible Markup Language), or others. These data sources can be extended to domain-specific scopes, such as medical legal vocabulary. The goal is to populate the system with relatively simple and high-quality data (referred to as seed data) to enable the start of the data synthesis process.

[0232] First, the seed data 1802 is transformed into natural language using an input translator 1804 that can optionally interact with a pre-trained LM 1806. For example, the dictionary entry "is (an orchard, a garden)" can be translated into the sentence "an orchard is a garden." Those skilled in the art will recognize that this translation step can be implemented using a pre-trained LM or other NLP techniques capable of generating natural language from structured or machine-readable representations of text. The natural language sentence will then be included in the subsequent instruction set, regardless of whether the original structured representation is included. If the input translator has input length limitations, then subsets of the seed data can be processed iteratively. These modifications for interfacing with various input translators (e.g., LMs) are well known.

[0233] The AKRM 1808 applies generative rules to the analysis and synthesis of complex semantic concepts.

[0234] The following pseudo-code shows a method for generating atomic concepts and relationships through the analysis phase, or for generating complex concepts and relationships through the synthesis phase (depending on an input parameter that indicates which phase to run): Generating knowledge Input: Context: User and / or application context (e.g., request, active concepts, domain restrictions) KR in : Knowledge representation (e.g., taxonomy) Rules: Coherent concepts, hierarchical relationships Phase: One of {analysis, synthesis} (Composite) Output: Complex concepts and relationships in the output semantic representation (Side-effects of analysis, and optionally synthesis): Instantiation of new atomic concepts and relationships in the atomic knowledge representation model The following example formalizes the above rules for analyzing and synthesizing coherent concepts and hierarchical concept relationships.

[0235] Input / Output: A set of concepts C A set of acyclic, hierarchical relationships Rule 1 (Coherent Concept Synthesis): Create a new concept c = {c i , c j}. c is said to be composed of c i and c j if and only if c i and c j . Two concepts c i, , c j are coherent if a certain distance metric M, M(c i c j ) < T, where T is a pre-set threshold. Possible metric examples include: the co-occurrence frequency of the two concepts in the input corpus, or a tree distance function applied to the taxonomy hierarchy. Rule 2 (Hierarchical Relationship Synthesis): Let c1 = {c 11 , c 12 , …, c 1n} be a concept composed of n concepts c 11 to c 1n . Similarly, let c2 = {c 21 , c 22 ,... c 2m} be a concept composed of m concepts c 21 to c 2m . c1 is said to be a parent of c2 if and only if for every c 1i , there exists a concept c2j , a new hierarchical relationship h(c1, c2) is created. The hierarchical relationship h(c 1i , c 2i ) or equivalence relationship e(c 1i , c 2i ).

[0237] Again, for simplicity, only one simple hierarchical relationship rule is shown here, which can represent, for example, a hypernym or hyponym relationship, a superior or inferior relationship, etc. Those skilled in the art will appreciate that more complex rules can be constructed to interpret the underlying basis of more complex conceptual semantics.

[0238] In the synthesis phase, the seed data is combined with the analyzed data to assemble new concepts and concept relationships.

[0239] Continuing the example above, suppose the analyzed data also contains the hierarchical relationship "h(-ist, -er)", where the suffix "-ist" indicates a professional knowledge, more specific than the suffix "-er", which indicates a participant in an activity. Depending on the context, "fruit", and the instructions shown in the pseudocode, the LM can synthesize the complex concepts (orchard, -ist) and (fruit, garden, -er), and the complex relationship h((orchard, -ist), (fruit, garden, -er)). It is important to note that the synthesized data can contain valid data and invalid data (which the system tolerates as acceptable noise), as well as real data and imagined data (e.g., neologisms). In this example, "orchardist" is an existing English word, but it is rare, and due to its relationship to the complex concept (fruit, garden, -er), it can not be in the original lexicon. Similarly, for the example h(-ist, -er), it can be recognized as coherent data that satisfies the AKRM constraints, but is not available through regular sources (e.g., the seed data discussed earlier), nor discovered through human analysis.

[0240] The aforementioned analysis and synthesis processes generate data, and assemble these data into one or more output semantic representations. This structure can be used to validate or refute synthesized concepts by applying constructive rules to the overall semantic representation. For example, a rule can allow (or disallow) multiple hierarchies, or a rule can limit the minimum or maximum number of instances in a class. Supplementary validation systems, such as discussed in other embodiments, can also be applied.

[0241] After validation, the synthesized data can be used in some adaptation method discussed in the aforementioned embodiments. For example, the data can be input into the ML pipeline 1816 to create a secondary model. This secondary model can then be used in subsequent self-learning rounds, or for any other adaptation method discussed above. For example, the secondary model can be utilized when the primary LM encounters a context in which real data is sparse.

[0242] These rules constitute the analysis instructions 1810 and the synthesis instructions 1812 that will interact with the LM to produce the new semantic representation 1814 for fine-tuning. The instruction templates derived using the pseudo-code above can be used to interact with the LM.

[0243] In one variation of this embodiment, the LM can attempt to execute all instructions, including the pseudo-code. In another variation of this embodiment, specific control flows (e.g., conditional logic, iterating over a list) can be handled by a mediator processor in the surrounding system that will prompt the LM according to each rule. In a third variation of this embodiment, the LM will generate executable code (using a suitable programming language) from the pseudo-code. The mediator will then hand the code to other components in the system that are capable of executing the code, thereby managing the execution of the code. In a fourth variation of this embodiment, the LM can generate more specific instructions (or subsets of instructions) from a larger initial set of instructions for other LM agents to follow. Those skilled in the art will recognize that this approach also requires a mediator, for example, using a multi-agent LLM framework.

[0244] It is to be understood that any of the implementations discussed herein can work in conjunction with other implementations in an integrated or recursive manner, where appropriate. For example, when generating annotated or annotated data from a LM, prompt word augmentation can also be used to ensure that the output of the LM aligns with the user’s intent.

[0245] The various aspects described herein can be implemented on one or more computers and / or devices, each having one or more processors that can be programmed to perform any of the operations described above to enhance a LM. For example, both server and client computing systems can be implemented as one or more computers, as described above.

[0246] ​ A simplified block diagram of components of a device 1500, such as a mobile device or a portable electronic device, is shown, on which various aspects disclosed herein can be implemented. The device 1500 includes a number of components, such as a processor 1502 (or processing unit) that controls the functioning of the device 1500. Communication functions, including data and voice communications, can be performed through a communication subsystem 1504. Data received by the device 1500 can be decompressed and decrypted by a decompression and decryption circuit 1506.

[0247] The communication subsystem 1504 can receive messages from and send messages to a wireless network 1550.

[0248] The wireless network 1550 can be any type of wireless network, including, but not limited to, a data-centric wireless network, a voice-centric wireless network, and a dual-mode wireless network that supports both voice and data communications.

[0249] Device 1500 may be a battery-powered device, as shown, which includes a battery interface 1542 for receiving one or more rechargeable batteries 1544.

[0250] The processor 1502 also interacts with other subsystems, such as random access memory (RAM) 1508, flash memory 1510, display 1512 (e.g., connected to electronic controller 1516 with touch-sensitive overlay 1514, together forming touch-sensitive display 1518), actuator assembly 1520, one or more optional force sensors 1522, auxiliary input / output (I / O) subsystem 1524, data port 1526, speaker 1528, microphone 1530, short-range communication system 1532, and other device subsystems 1534.

[0251] In some embodiments, user interaction with the graphical user interface can be performed via the touch-sensitive overlay 1514. The processor 1502 can interact with the touch-sensitive overlay 1514 via the electronic controller 1516. Information generated by the processor 1502, such as text, characters, symbols, images, icons, and other items that can be displayed or presented on a portable electronic device, can be displayed on the touch-sensitive display 1518.

[0252] like ​ As shown, the processor 1502 can also interact with the accelerometer 1536. The accelerometer 1536 can be used to detect the direction of gravity or the reaction force caused by gravity.

[0253] According to this embodiment, to identify subscribers accessing the network, device 1500 can use a Subscriber Identity Module (SIM / RUIM) card 1538 inserted into SIM / RUIM interface 1540 to communicate with a network (e.g., wireless network 1550). Alternatively, the subscriber identification information can be programmed into flash memory 1510 or executed using other technologies.

[0254] Device 1500 also includes an operating system 1546 and software components 1548 executed by processor 1502, which can be stored in a persistent data storage device such as flash memory 1510. Other applications can be loaded onto device 1500 via wireless network 1550, auxiliary I / O subsystem 1524, data port 1526, short-range communication subsystem 1532, or any other suitable device subsystem 1534.

[0255] For example, during use, a received signal such as a text message, a voicemail, a web page download, or other data, can be processed by the communication subsystem 1504 and input to the processor 1502. The processor 1502 can then process the received signal for output to the display 1512 or alternatively to an auxiliary I / O device 1524. A subscriber can also compose data items, such as e-mail messages, using the processor 1502 and the keyboard 1520. These data items can then be transmitted over the communication subsystem 1504 through the wireless network 1550.

[0256] For voice communications, overall operation of the portable electronic device 1500 can be similar. The speaker 1528 can output audible information converted from electrical signals, and the microphone 1530 can transform audible information into electrical signals for processing.

[0257] In addition to memory, the computer 1500 can also include storage devices (e.g., one or more disk drives).

[0258] The memory and / or storage devices can store one or more sets of computer-executable instructions for programming the processor 1502 to perform any of the functions described herein. The storage devices can also selectively store one or more data sets as desired. For example, in some embodiments, the computer can store an AKRM data set in the storage device.

[0259] A computer referred to herein can include any device having a program processor, including a rack-mounted computer, a desktop computer, a notebook computer, a tablet computer, or many devices that are not typically considered computers but contain program processors (e.g., PDAs, MP3 players, mobile phones, wireless headsets, etc.).

[0260] Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of data, and speakers or other sound generating devices for audible presentation of data. Examples of input devices that can be used for a user interface include keyboard and pointing devices, such as mice, touch pads, and digitizing tablets. Additionally, a computer can receive input information through speech recognition or other audio format.

[0261] Examples of networks include local area networks or wide area networks, such as an enterprise network or the Internet. Such networks can be based on any suitable technology and can operate according to any suitable protocol and can include wireless networks, wired networks or fiber optic networks.

[0262] Having thus described several aspects of at least one embodiment of the present systems and methods, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the present systems and methods described herein. Accordingly, the foregoing description and drawings are by way of example only.

[0263] The above-described embodiments can be implemented in a number of ways. For example, the embodiments can be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Such processors can be implemented as integrated circuits, with one or more processors in an integrated circuit component. Of course, in the alternative, a processor can be implemented using circuitry that is located on a circuit board, as desired.

[0264] The various methods or processes outlined herein can be written as software code, and can be deployed in many ways, such as on one computer or across multiple computers or other processing devices. In addition, such software can be written for use with any operating system or platform, and in a variety of programming languages. Furthermore, the software can be deployed in a variety of manners, such as on a stand-alone device, or across several devices. It will be appreciated that each of the various methods or processes outlined herein can be implemented by one or more computing devices, such as with a general purpose computer or using application specific computing devices.

[0265] In this regard, various aspects described herein, for example, can be embodied as a machine- or computer-readable medium (or more than one medium) having encoded thereon computer readable instructions or a program, wherein the instructions, when executed by one or more computers or processors, can implement the various embodiments described herein. The computer-readable medium or media can be transport or storage media, such as, but not limited to, portable or fixed storage devices, optical storage devices, magnetic tape, semiconductor memories, programmable logic devices, arrays of logic elements coupled to

[0266] As used herein, the terms "program" or "software" are used in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to implement various aspects as described herein. Additionally, it should be appreciated that according to one aspect of this embodiment, one or more computer programs that when executed perform methods described herein need not reside on a single computer or processor, but can be distributed in a modular fashion, as desired, over a number of different computers or processors to implement various aspects.

[0267] Computer-executable instructions can be in many forms, such as program modules, executed by one or more computers or other machines. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules can be combined or distributed as desired in various embodiments.

[0268] Also, data structures can be stored in computer-readable media in any suitable form. For simplicity of illustration, data structures are sometimes shown as having fields that are related through location in the data structure. Such relationships can likewise be achieved by using location references, tags or other mechanisms that create relationships between data elements. However, any suitable mechanism can be used to establish a relationship between information elements, including the use of pointers, tags, or other information locating or referencing mechanisms.

[0269] Various aspects of the embodiments described above can be used alone, in combination, or in various arrangements not specifically described herein, and therefore the application should not be limited to the details and arrangement of components depicted in the foregoing description or illustrated in the accompanying drawings. For example, aspects described in one embodiment can be combined in any manner with aspects described in other embodiments.

[0270] While the above description provides examples of one or more devices, methods or systems, it will be appreciated within the scope of the claims that other devices, methods or systems can be employed.

Claims

1. A method for adapting a language model to represent a user's intent, the method comprising: Receive user input from the user, including user data requesting a language model; Analyze the user's intent based on the user input and user data; Generate user-influenced data that reflects the intent of the analyzed user; Based on the user-influenced data, the language model is adapted using at least one adaptation method; and Output is generated from the adapted language model.

2. The method according to claim 1, wherein, Analyzing the user's intent includes analyzing at least one of the following: context, user prompts, user preferences, and user actions.

3. The method according to claim 1, wherein, The data generated that is affected by users includes: Receive the user-influenced data at the input translator; The input translator translates the user-influenced data into a form that can be utilized by the semantic engine. The translated, user-influenced data is received at the semantic engine; The semantic engine generates a user-influenced knowledge representation representing the translated user-influenced data; and The language model is adapted based on the knowledge representation influenced by the user.

4. The method according to claim 1, wherein, The at least one adaptation method includes: Based on the aforementioned knowledge, a prompt word is generated at the prompt word selector; and Enhance the prompt at the prompt injector.

5. The method according to claim 1, wherein, The at least one adaptation method includes fine-tuning a machine learning model using training data derived from the user-influenced knowledge representation.

6. The method according to claim 1, wherein, The at least one adaptation method includes: deriving condition variables from the user-influenced knowledge representation; and The prompt word is enhanced at the prompt word injector by applying the condition variable to the prompt word.

7. The method according to claim 1, wherein, The at least one adaptation method includes: deriving logical constraints from the user-influenced knowledge representation; and The logical constraints are applied to the loss function optimizer of the machine learning model.

8. The method of claim 1, further comprising using a semantic evaluator to evaluate the semantic coherence between the output and the analyzed intent of the user.

9. The method according to claim 8, wherein, The semantic evaluator is the Atomic Knowledge Representation Model (AKRM).

10. An apparatus for adapting a language model to represent a user's intent, the apparatus comprising: Network interface; processor; A non-transitory memory storing computer-executable instructions, which, when executed by a processor, configures the device to: Receive user input, including user data requesting a language model; analyze the user's intent based on the user input and user data; Generate user-influenced data that reflects the intent of the analyzed user; Based on the user-influenced data, the language model is adapted using at least one adaptation method; and output is generated from the adapted language model.

11. The apparatus according to claim 10, wherein, Analyzing the user's intent includes analyzing at least one of the following: context, user prompts, user preferences, and user actions.

12. The apparatus according to claim 10, wherein, Generating user-influenced data includes: receiving the user-influenced data at the input translator; The input translator translates the user-influenced data into a form that can be used by the semantic engine; the translated user-influenced data is received at the semantic engine; The semantic engine generates a user-influenced knowledge representation representing the translated user-influenced data; and the language model is adapted based on the user-influenced knowledge representation.

13. The apparatus according to claim 10, wherein, The at least one adaptation method includes: generating a prompt word at a prompt word selector based on the knowledge representation; and Enhance the prompt at the prompt injector.

14. The apparatus according to claim 10, wherein, The at least one adaptation method includes fine-tuning a machine learning model using training data derived from the user-influenced knowledge representation.

15. The apparatus according to claim 10, wherein, The at least one adaptation method includes: deriving condition variables from the user-influenced knowledge representation; and The prompt word is enhanced at the prompt word injector by applying the condition variable to the prompt word.

16. The apparatus according to claim 10, wherein, The at least one adaptation method includes: deriving logical constraints from the user-influenced knowledge representation; and The logical constraints are applied to the loss function optimizer of the machine learning model.

17. The apparatus of claim 10, further comprising using a semantic evaluator to evaluate the semantic coherence between the output and the analyzed intent of the user.

18. The apparatus according to claim 17, wherein, The semantic evaluator is the Atomic Knowledge Representation Model (AKRM).

19. A system for adapting a language model to represent a user's intent, the system comprising: At least one processor and at least one non-transitory memory for storing instructions, said instructions being executed by said at least one processor to implement: User input device, configured to provide user input to the computing engine; The computing engine is configured to: Receive user input, including user data requesting a language model; Analyze the user's intent based on the user input and user data; Generate user-influenced data that reflects the intent of the analyzed user; Based on the user-influenced data, the language model is adapted using at least one adaptation method; and Output is generated from the adapted language model.

20. The system according to claim 10, wherein, Analyzing the user's intent includes analyzing at least one of the following: context, user prompts, user preferences, and user actions.

21. The system according to claim 10, wherein, Generating user-influenced data includes: receiving the user-influenced data at the input translator; The input translator translates the user-influenced data into a form that can be used by the semantic engine; the translated user-influenced data is received at the semantic engine; The semantic engine generates a user-influenced knowledge representation representing the translated user-influenced data; and the language model is adapted based on the user-influenced knowledge representation.

22. The system according to claim 12, wherein, The at least one adaptation method includes: Based on the knowledge representation, a prompt word is generated at the prompt word selector; and the prompt word is enhanced at the prompt word injector.

23. The system according to claim 12, wherein, The at least one adaptation method includes fine-tuning a machine learning model using training data derived from the user-influenced knowledge representation.

24. The system according to claim 12, wherein, The at least one adaptation method includes: deriving condition variables from the user-influenced knowledge representation; and The prompt word is enhanced at the prompt word injector by applying the condition variable to the prompt word.

25. The system according to claim 12, wherein, The at least one adaptation method includes: deriving logical constraints from the user-influenced knowledge representation; and The logical constraints are applied to the loss function optimizer of the machine learning model.

26. The system of claim 10 further includes using a semantic evaluator to evaluate the semantic coherence between the output and the analyzed intent of the user.

27. The system of claim 15, wherein the semantic evaluator is an atomic knowledge representation model (AKRM).

Images