Managing the recall of long-term memory for large-scale language models via self-reflection protocols

Abstract

This paper discloses a method for developing and managing long-term memory solutions for large-scale language model (LLM) agents in the context of providing LLM agents as a service. [Solution] Following task-related communication between the LLM agent and the service user, domain knowledge, user preferences, and information regarding the success or failure of the requested task are extracted into a data sample by the service's reflection agent. The data sample is then stored in a long-term memory database that can be accessed in the future by the LLM agent, so that the agent can recall information from previous interactions to perform new tasks for the user more efficiently.

Description

Technical Field

[0003] , , , , , ,

[0001] The present disclosure relates to enabling long-term memory solutions for large language models.

Background Art

[0002] Background Large language models (LLMs) have demonstrated powerful performance for a variety of tasks, leading to an expansion of deployment to large-scale systems. For example, large language models are often used to provide supervision (monitoring, management) in many cases or as a tool in the decision-making process. Large open-source datasets applied as training datasets for LLMs make it possible to apply LLMs to generalized tasks. However, there has not yet been a successful prior implementation of LLMs in certain enterprise environments that require additional specific context. Furthermore, prior implementations often require white-box access to such models that continuously analyze and adjust weights, hidden states, or other internal parameters.

Summary of the Invention

Means for Solving the Problems

[0003] Summary This disclosure relates to the development and management of a long-term memory database for an LLM service, which allows the LLM agent of the service to access the database and apply previously learned knowledge to new tasks that are related or in some way similar. A reflection agent analyzes the logs of communications between the user and the LLM agent, and the reflection agent executes a reflection protocol that distills the information in the messages between the user and the LLM agent into compact data samples, which are then stored in the long-term memory database. At a later point in time, the LLM agent can recall some of these data samples to perform new tasks for the user more efficiently without having to be re-prompted using predetermined domain knowledge or user preferences. Furthermore, the long-term memory database allows the LLM service to function as a shared knowledge base within an enterprise setting. [Brief explanation of the drawing]

[0004] [Figure 1] This figure shows a system for training and utilizing machine learning models, such as large-scale language models, according to several embodiments. [Figure 2] This figure shows computer-implemented methods for training and utilizing machine learning models, such as large-scale language models, according to several embodiments. [Figure 3] This figure shows a service provider network configured to implement LLM services and manage LLM long-term memory storage in several embodiments. [Figure 4] This flowchart shows a process in which, according to several embodiments, a task is performed for a user of the LLM service, and then a reflection protocol is executed that allows the LLM service to store the results in a long-term memory database and later recall the results of the task during the execution of future tasks. [Figure 5]This flowchart shows a first subprocess that executes the reflection protocol described in Figure 4, relating to a domain knowledge category, according to several embodiments. [Figure 6] This flowchart shows a second subprocess that executes the reflection protocol described in Figure 4, relating to user preferences, according to several embodiments. [Figure 7] This flowchart shows a third subprocess that executes the reflection protocol introduced in Figure 4, relating to the success of the LLM agent in task execution in several embodiments. [Figure 8] This flowchart shows a fourth subprocess that executes the reflection protocol described in Figure 4, relating to LLM agent failures in task execution in several embodiments. [Figure 9A] This figure shows examples of user interfaces in several embodiments that allow users of an LLM service to chat with the LLM agent of the service in order to perform tasks. [Figure 9B] This figure shows another part of a user interface that, according to several embodiments, can provide data samples generated during the reflection protocol for viewing and editing by users of the LLM service. [Figure 9C] This figure shows an example, in several embodiments, where a user can add additional data samples to data samples generated during the reflection protocol via a user interface. [Figure 10A] This figure shows another part of the user interface that allows users of an LLM service to browse and explore data samples already stored in a long-term memory database, according to several embodiments. [Figure 10B] This figure shows examples of data samples that a user can view using a user interface in long-term memory storage, according to several embodiments. [Figure 10C]This figure shows another example of a data sample that a user is viewing using a user interface in long-term memory storage, according to several embodiments. [Modes for carrying out the invention]

[0005] Detailed explanation While embodiments of this disclosure are described herein, it should be understood that these embodiments are merely examples and that various alternative forms can be adopted as other embodiments. The figures are not necessarily drawn to scale, and some features are exaggerated or reduced to illustrate details of certain elements. Therefore, certain structural and functional details disclosed herein should not be construed as limitations, but rather as representative grounds for teaching those skilled in the art various ways of utilizing each embodiment. As those skilled in the art will understand, various features illustrated and described with reference to any one of the figures can be combined with features shown in one or more other figures to form embodiments not expressly illustrated or described. Combinations of illustrated features provide representative embodiments for typical uses. Various combinations and modifications of features consistent with the teachings of this disclosure may be desirable for a particular use or implementation.

[0006] In this specification, “a,” “an,” and “the” refer to both singular and plural unless the context clearly indicates otherwise. For example, “a processor” programmed to perform various functions refers to a single processor programmed to perform each function one at a time, or it refers collectively to two or more processors programmed to perform each of the various functions.

[0007] LLM has demonstrated remarkable capabilities in key generation and reasoning, and these generation and reasoning capabilities have been extended to the use of decision-making instances and tools in LLM applications. Here, companies, corporations, and other organizations are interested in integrating LLM agents into the execution of routine tasks to improve employee productivity and streamline workplace processes. However, previous implementations of LLM integrated into the workplace have yielded limited results because the LLMs have been trained on generalized training datasets lacking company-specific knowledge, context, or other internal procedures that cannot be learned from generalized open-source training datasets unless otherwise provided.

[0008] Therefore, attempts have been made to inject domain knowledge and skills into LLM training in order to implement LLMs into various workplace settings. In previous implementations, in particular, attempts have been made to create a more robust "memory" for LLMs through supervised and unsupervised reflection generation. However, these earlier implementations require a huge amount of training data or cause inconsistencies due to unsupervised or unverified unsupervised reflection generation that is not monitored or verified before the model is implemented in the workplace.

[0009] To address these challenges, this disclosure describes how to develop and manage LLM's long-term memory so that the LLM agent can “recall” information learned by the model over time, and further edit or otherwise update reflections that change over time as the enterprise deployment progresses. When a user chats with an LLM agent, messages in the conversation are provided to a reflection agent, which then uses the communication log to execute a reflection protocol. The reflection protocol is used to generate data samples from conversations related to different domain knowledge categories, which are then used for user preferences that can be used to build individual user profiles of employees within the enterprise, and for the success or failure of the LLM agent in completing various tasks for the user. The data samples are then stored in a long-term memory database so that the LLM agent can refer to them at a later point in time when performing the relevant tasks. This eliminates the need for users to explain enterprise-specific procedures, preferences, or other protocols to the LLM agent each time, and allows LLM to develop its own long-term memory for implementations that integrate LLM into specific work environments.

[0010] The following description follows a general introduction to machine learning techniques related to methods for developing and managing long-term memory databases for LLMs as described herein. Next, various embodiments of architectures and processes enabling such long-term memory for recall by LLMs are discussed. Subsequently, this disclosure demonstrates the versatility of the methods and systems described herein by illustrating various application embodiments of the methods and systems described herein.

[0011] Figure 1 shows a system 100 that utilizes a Large-Scale Language Model (LLM). An LLM can describe a machine learning model configured to learn complex patterns and representations based on training and / or validation datasets used as input to the LLM. For the purposes of this disclosure, LLM means an LLM that has been at least partially trained using one or more additional training datasets. For example, an open-source dataset may have been applied prior to be a training dataset so that the LLM learns general knowledge about interactions with people, such as chat-based interactions. However, the LLM does not possess specific knowledge about company-specific information, such as the preferences of company employees.

[0012] Additional embodiments relating to the LLM and agents are described herein with respect to LLM agents 310, 312 and 314, reflection agent 316, and blocks 404, 406, 410, 412, 502-508, 602-608, 702-708 and 802-808.

[0013] To further illustrate the various states of the untrained and trained versions of the LLM, the following paragraphs detail the process of training the LLM, referring to the diagram shown in Figure 1.

[0014] In some embodiments, the system 100 may include an input interface for accessing training data 102 for the LLM. For example, as shown in Figure 1, the input interface can be configured by a data storage interface 104 that can access the training data 102 from data storage 106. For example, the data storage interface 104 may be a memory interface or persistent storage interface, such as a hard disk or SSD interface, but it may also be a personal network interface such as Bluetooth, ZigBee or Wi-Fi interface or Ethernet or fiber optic interface, a local area network interface or a wide area network interface. The data storage 106 may be not only internal data storage of the system 100, such as a hard drive or SSD, but also external data storage, such as network-accessible data storage.

[0015] In some embodiments, the data storage 106 may further include a data representation 108 of an untrained version of the model (e.g., an untrained version of a machine learning model) accessible from the data storage 106 by the system 100. However, it is understood that the training data 102 and the data representation 108 of the untrained LLM are each accessible from different data storages, for example, via different subsystems of the data storage interface 104. Each subsystem may be of the type described above with respect to the data storage interface 104. In other embodiments, the data representation 108 of the untrained LLM can be generated internally by the system 100 based on the design parameters of the LLM and therefore does not need to be explicitly stored in the data storage 106. The system 100 may further include a processor subsystem 110, which can be configured to provide a recursive function as a substitute for the layer stack of the LLM to be trained during the operation of the system 100. Here, each layer of the layer stack being replaced may have weights shared with each other and may receive the output of the previous layer as input, or, in the case of the first layer of the layer stack, may receive the initial activation and a portion of the input of the layer stack. The processor subsystem 110 can further be configured to iteratively train the LLM using the training data 102 (thus generating, for example, an updated version of the machine learning model relative to the initial "untrained" version of the model). Here, the iterations of training by the processor subsystem 110 may include forward propagation and backward propagation portions.The processor subsystem 110 can be configured to execute the forward propagation portion by determining, among other operations that define an executable forward propagation portion, an equilibrium point of a recursive function that converges to a fixed point, where determining the equilibrium point includes using a numerical root-finding algorithm that finds what is obtained by subtracting the input from the root solution of the recursive function by providing an equilibrium point as a replacement for the output of the layer stack in the LLM.

[0016] The system 100 may further include an output interface for outputting the data representation 112 of the trained LLM, and the data may also be referred to as the trained model data 112. For example, as also shown in FIG. 1, the output interface can be configured by the data storage interface 104, and the above-described interface is, in the embodiments herein, an input / output (“IO”) interface, and through this input / output (“IO”) interface, the trained model data 112 can be stored in the data storage 106. For example, the data representation 108 that defines an “untrained” LLM can be at least partially replaced by the data representation 112 of the trained LLM such that during or after training, the parameters of the LLM, such as weights, hyperparameters, and other types of parameters of the LLM, can be adjusted to reflect the training on the training data 102. This is also shown in FIG. 1 by the reference numerals 108 and 112 that refer to the same data record on the data storage 106. In other embodiments, the data representation 112 can be stored separately from the data representation 108 that defines an “untrained” LLM. In some embodiments, the output interface can be separate from the data storage interface 104, but generally, it may be of the type described above for the data storage interface 104.

[0017] Figure 2 shows computer-implemented methods for training and utilizing an LLM according to several embodiments. System 200 may include at least one computing system 202. Computing system 202 may include at least one processor 204 operably connected to a memory unit 208. Processor 204 may include one or more integrated circuits that implement the functions of a central processing unit (CPU) 206, and in some embodiments may include a graphics processing unit (GPU). CPU 206 may be a commercially available processing unit that executes one of the instruction set families, e.g., x86, ARM, Power, or MIPS. During operation, CPU 206 can execute program instructions stored in and retrieved from the memory unit 208. Stored program instructions may include software for controlling the operation of CPU 206 to perform the operations described herein. In some examples, processor 204 may be a system-on-a-chip (SoC) that integrates the functions of CPU 206, memory unit 208, network interface, and input / output interface into a single integrated device. The computing system 202 can implement an operating system for managing various modes of operation.

[0018] Memory unit 208 may include volatile and non-volatile memories for storing instructions and data. The non-volatile memory may include solid-state memory, such as NAND flash memory, magnetic storage media, and optical storage media, or any other suitable data storage device that retains data when computing system 202 is in an inactive state or when power is lost. The volatile memory may include static random access memory and dynamic random access memory (RAM) for storing program instructions and data. For example, memory unit 208 may store machine learning model 210 or algorithm, training and / or fine-tuning data set 212 for machine learning model 210, raw data set 214, and the like.

[0019] Non-volatile storage may include one or more persistent data storage devices, such as hard drives, optical drives, tape drives, non-volatile solid-state devices, cloud storage, or any other devices capable of persistently storing information. Processor 204 may include one or more devices selected from a high-performance computing (HPC) system, including high-performance cores, microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field-programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other devices that manipulate signals (analog or digital) based on computer-executable instructions present in memory unit 208. Memory 208 may include one or more memory devices, and may include, but is not limited to, random-access memory (RAM), volatile memory, non-volatile memory, static random-access memory (SRAM), dynamic random-access memory (DRAM), flash memory, cache memory, or any other devices capable of storing information. Furthermore, the processor 204 and memory 208 can be configured to provide the collected data to one or more other computing devices configured to perform LLM services within the scope of the various embodiments presented herein.

[0020] The processor 204 can be configured to read into memory 208 and execute computer executable instructions for implementing one or more machine learning algorithms and / or methodologies of one or more embodiments that reside in non-volatile storage. The non-volatile storage may include one or more operating systems and applications. The non-volatile storage may store compiled and / or interpreted computer programs written using a variety of programming languages ​​and / or technologies, including, but not limited to, Java, C, C++, C#, Objective-C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL, either alone or in combination.

[0021] Program code for implementing the algorithms and / or methodologies described herein may be distributed individually or collectively as various different forms of program products. The program code may be distributed using a computer-readable storage medium that stores computer-readable program instructions for causing a processor to execute one or more embodiments of the program code. A computer-readable storage medium that is essentially non-temporary may include volatile and non-volatile, as well as removable and non-removable tangible media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. A computer-readable storage medium may further include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disk read-only memory (CD-ROM), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other media that can be used to store the desired information. Computer-readable program instructions can also be downloaded from a computer-readable storage medium to a computer, other types of programmable data processing devices, or other devices, or via a network to an external computer or external storage device.

[0022] Computer-readable program instructions stored on a computer-readable medium can be used to instruct a computer, other types of programmable data processing devices, or other devices to function in a particular way, thereby enabling the production of a product containing instructions for realizing functions, actions, and / or operations specified in a flowchart or diagram. In a given alternative embodiment, the functions, actions, and / or operations specified in the flowchart and diagram are reorderable, sequential, and / or concurrent to match one or more embodiments. Furthermore, neither the flowchart nor the diagram may contain more or fewer nodes or blocks than those illustrated to match one or more embodiments.

[0023] The process, method, or algorithm may be implemented as a whole or in part using appropriate hardware components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or combinations of hardware, software, and firmware components.

[0024] The computing system 202 may include a network interface device 220 configured to provide communication with external systems and devices. For example, the network interface device 220 may include wired and / or wireless Ethernet interfaces as defined by the IEEE (Institute of Electrical and Electronics Engineers) 802.11 standard family. The network interface device 220 may also include a cellular communication interface for communicating with cellular networks (e.g., 3G, 4G, 5G). The network interface device 220 can further be configured to provide a communication interface to an external network 222 or the cloud.

[0025] The external network 222 may be referred to as the World Wide Web or the Internet. The external network 222 can establish standard communication protocols between computing devices. The external network 222 can facilitate the exchange of information and data between computing devices and the network. One or more servers 224 can communicate with the external network 222.

[0026] The computing system 202 may include an input / output (I / O) interface 218 configurable to provide digital and / or analog inputs and outputs. The I / O interface 218 may have an additional serial interface for communicating with external devices (e.g., a Universal Serial Bus (USB) interface).

[0027] The computing system 202 may include a human-machine interface (HMI) device 216, which may include any device that enables system 200 to receive control inputs. Examples of input devices may include human interface input units such as keyboards, mice, touchscreens, voice input devices, and other similar devices. The computing system 202 may include a display device 226. The computing system 202 may include hardware and software for outputting graphics information and text information to the display device 226. The display device 226 may include an electronic display screen, a projector, a printer, or other suitable device for displaying information to a user or operator. The computing system 202 can further be configured to interact with remote HMIs and remote display devices via a network interface device 220.

[0028] System 200 can be implemented using one or more computing systems. While this example shows a single computing system 202 that implements all the described features, it is intended that it be separable and implementable by multiple computing units with various features and functions communicating with each other. The specific system architecture chosen may depend on various factors.

[0029] System 200 can implement a machine learning algorithm 210 configured to analyze a raw-source dataset 214. The raw-source dataset 214 may include raw or unprocessed sensor data that can represent an input dataset for the machine learning system. In some examples, the machine learning algorithm 210 may be an LLM designed to perform a predetermined function. For example, the LLM algorithm can be configured to chat with employees of the aforementioned company, perform tasks for them, and otherwise assist these employees in their daily corporate settings, in the context of learning company-specific procedures.

[0030] The computer system 200 can store a training dataset 212 for the machine learning algorithm 210. The training dataset 212 may represent a dataset that has been constructed in advance to train the machine learning algorithm 210. The training dataset 212 is available to the machine learning algorithm 210 for learning the weight coefficients associated with the LLM algorithm. The training dataset 212 may include a set of source data having corresponding outcomes or results that the machine learning algorithm 210 attempts to replicate through the learning process.

[0031] The machine learning algorithm 210 can operate in training mode using the training dataset 212 as input. The machine learning algorithm 210 can run for a predetermined number of iterations using data from the training dataset 212. With each iteration, the machine learning algorithm 210 can update its internal weighting coefficients based on the results achieved. For example, the machine learning algorithm 210 can compare its output results (e.g., annotations) with those contained in the training dataset 212. Since the training dataset 212 contains the predicted results, the machine learning algorithm 210 can determine when its performance is acceptable. After achieving a predetermined performance level (e.g., 100% agreement with the results associated with the training dataset 212), the machine learning algorithm 210 can run with data not present in the training dataset 212. The trained machine learning algorithm 210 can be applied to new datasets to generate annotated data.

[0032] The machine learning algorithm 210 can be configured to identify specific features within the raw source data 214. The raw source data 214 may include multiple instances or input datasets for which annotation results are desired. The machine learning algorithm 210 can be programmed to process the raw source data 214 and identify the presence of specific features. The machine learning algorithm 210 can be configured to identify one feature within the raw source data 214 as a predetermined feature. The raw source data 214 can be derived from various sources. For example, the raw source data 214 may be actual input data collected by a machine learning system. The raw source data 214 may also be machine-generated for testing the system.

[0033] In this embodiment, the machine learning algorithm 210 can then process the raw source data 214 and output results to complete a task for an employee. The machine learning algorithm 210 can generate a confidence level or coefficient for each output produced. For example, a confidence value exceeding a predetermined high confidence threshold can indicate that the machine learning algorithm 210 is confident that the identified feature corresponds to a particular feature. A confidence value lower than a low confidence threshold can indicate that the machine learning algorithm 210 has some uncertainty about the existence of a particular feature.

[0034] Since the context for training and using LLMs was provided in the descriptions of Figures 1 and 2, from here on, in Figures 3 through 10C and the relevant descriptions in this specification, we will focus on developing and managing the long-term memory of such LLMs to perform tasks more efficiently within a corporate environment. Furthermore, at the points mentioned above related to LLM training, the modification, updating, or expansion of model weights and / or other parameters over time is described as part of the learning process. The following discussion concerns the use of a reflection agent to analyze the logs of communication between LLM users and various LLM agents. Therefore, it should be understood that the development and management of long-term memory does not belong to the fine-tuning of the model itself, which refers to further refining, updating, and / or changing the weights and parameters of the LLM by other methods during the fine-tuning and / or retraining process. Rather, the development and management of long-term memory concerns storing note or importance data samples in the long-term memory database so that LLM agents can refer to the stored data samples when performing other tasks for corporate employees in the future. This may also be referred to as the recall process in this specification.

[0035] Figure 3 shows several embodiments of a service provider network configured to implement LLM services and manage the long-term memory of LLMs.

[0036] As described above, the use of LLM can be provided as a service by a service provider. For example, LLM service 308 can be configured to allow company employees access to LLM and its corresponding agent. LLM service 308 can then be configured to allow employees, hereafter referred to as "users" of the service, to chat with an LLM agent to request the LLM agent to complete a given routine task. Since LLM in LLM service 308 is intended to remain accessible to the company over a long period, it is advantageous that the service develops and manages LLM's long-term memory so that LLM can recall relevant and useful conversations previously taken by LLM and its users when new tasks are filled in.

[0037] Because there are certain enterprise-specific protocols, procedures, and / or reliability information that LLM may be given access to in execution to perform tasks for the service's users, it is appropriate for LLM to recall data samples related to these protocols, procedures, and reliability information without having to re-explain these tasks to LLM each time there is a subsequent request from LLM agents to perform tasks. However, rather than retraining the model with enterprise-specific preferences or procedures that may not have enough data points to generate a complete training dataset or that may evolve over time, the service stores data samples in a long-term memory database corresponding to instances in which LLM has recognized enterprise-specific preferences or procedures.

[0038] For example, if a user of the LLM service 308 wants to prepare a series of quarterly plots relating to sales and costs for the final quarter and to provide to the company's investors, this user can request the LLM agent to prepare the series of plots. During the first instance of requesting these plots, the user must specify to the LLM agent that these plots should be bar graphs and not line graphs, and that the x-axis of these plots should originate from column 2 of the dataset supplied to the LLM agent, not column 3. However, after this conversation with the LLM agent, the reflection agent is configured to execute a reflection protocol based on the log of communication between the LLM agent and the user. The reflection protocol will be explained in more detail with reference to Figures 4 to 8. The LLM agent can recall this data sample, for example, by actively forming a bar graph with column 2 as the x-axis, between the next request to prepare the series of quarterly plots, after extracting a data sample with details such as (1) the series of plots should be generated quarterly, (2) the plots should be bar graphs, and (3) the x-axis of the plots should refer to column 2 of the supplied dataset. This recall process allows LLM agents to efficiently and proactively meet the needs of service users and ensures that they do not have to be re-taught specific company-specific protocols, procedures, and / or reliability information each time they chat with a user to perform other tasks.

[0039] Referring to Figure 3, the LLM service 308 is configured so that an LLM agent develops and manages the long-term memory of the LLM service while chatting with users. Users 302, 304, and 306 refer to computing devices used by company employees to chat with the LLM service 308. As indicated by the ellipsis, the LLM service 308 is configured to interact with any number of users. Furthermore, even if users 302, 304, and 306 are located in different physical locations or facilities, communication with the LLM service 308, for example, communication over network 222, is secure communication, as indicated by the logical designation of the service provider network 300. Furthermore, the service provider can be configured to use multiple services to provide the LLM service 308 to users. For example, storage service 322 may be similar to a data center, which may be yet another physical location different from where the computing devices configured to implement the LLM service 308 reside. Similarly, as indicated by the logical designation of the service provider network 300, communication between users, the LLM service, and the storage service are secure communication.

[0040] In some embodiments, the computing device implementing the LLM service 308 may include one or more agents of LLM, such as LLM agents 310, 312, and 314, as well as a reflection agent 316. Each of the LLM agents 310, 312, and 314 can represent a user-facing implementation of LLM, allowing multiple users to interact with or "chat" LLM simultaneously. The reflection agent 316 is yet another implementation of LLM and is configured to analyze communication between the LLM agent and the user when executing the reflection protocol 318. Thus, the reflection agent 316 can be further configured to provide data samples to the storage service 322 so that the data samples can be stored in long-term memory.

[0041] Furthermore, as shown in Figure 3, the LLM service 308 may also include a user interface 320. As further described below, the user interface 320 can be configured to present various data samples to the user of the LLM service 308 during "knowledge exploration," and can also be configured to present the user with the results of the recent reflection protocol 318 for exploration.

[0042] As described above, the storage service 322 is configured to accommodate data samples generated during the reflection protocol 318. For example, data samples 324, 326, and 328 may be similar to data samples relating to domain knowledge or user preferences learned through the analysis of logs of communication between the LLM agent and the user when the user requests the LLM service to perform a task. Data samples 324, 326, and 328 are data samples stored in long-term memory and accessible by the LLM service during future recall. Furthermore, data samples 324, 326, and 328 are also similar to formatted vectors relating to title, content, and generalized task.

[0043] In some embodiments, the storage service 322 may also store user permissions 330 that are accessible to the LLM service 308. For example, as indicated by the arrow “Knowledge Exploration” in Figure 3, user 306 (following the example above) may request via the user interface 320 to view any data sample related to a quarterly plot previously generated by the LLM service 308. The LLM service 308 may then request access to user permissions 330 to perform an initial verification that user 306 has sufficient security clearance to view the information before providing the user with the data sample.

[0044] As applied herein, the “long-term memory” of an LLM-based agent can be defined as follows, where analogies from human memory structures such as sensory memory, short-term memory, and long-term memory can be similarly applied to the memory structure of an LLM-based agent. For example, the short-term memory of an LLM agent refers to information obtainable within the context window, and the long-term memory refers to an external database, such as the database of storage service 322, from which the LLM agent can retrieve or “recall” data samples. As the log of conversations between the LLM agent and the user analyzed by the reflection agent increases over time, the long-term memory of the LLM service can be generated, developed, and subsequently managed using the database of storage service 322. Thus, in some embodiments, long-term memory can be classified into “explicit” memory or conscious memory based on events and facts, and “implicit” memory or unconscious memory based on sounds and sensations. Explicit memory can be further classified into episodic memory and semantic memory. In this case, episodic memory refers to memories corresponding to everyday events, while semantic memory refers to general knowledge of the world that humans naturally accumulate throughout their lives, such as thoughts, concepts, and facts.

[0045] By applying this recall method via long-term memory storage to LLM service 308, the LLM agent of the service can perform specific and enterprise-specific tasks more efficiently and effectively than when a simply and generically trained LLM is used in a similar enterprise context.

[0046] As further indicated by the arrows “Task Request,” “Recall,” “Remember,” “Knowledge Retrieval,” and “Security Check” in Figure 3, the various components of the systems within the boundary represented by the service provider network 300 are configured to interact with each other at various points in time. From here, these interactions will be explained in accordance with the entirety of the following description, with reference to Figures 4 through 8.

[0047] Figure 4 is a flowchart illustrating the process in which, in several embodiments, a task is performed for a user of the LLM service, followed by the execution of a reflection protocol, which allows the LLM service to store the results in a long-term memory database so that the results of that task can be recalled during the execution of a future task.

[0048] Process 400 corresponds to a computer-implemented method that can be executed by a computing system, for example, computing system 202, according to some embodiments. In block 402, a user of the LLM service initiates a chat with the LLM agent of the service via a user interface and requests the LLM agent to perform a task. For example, the user can provide the LLM agent with a spreadsheet containing product sales for the previous month, which includes columns such as sales date, price of the product sold, number of products sold, model number of the product sold, etc., and rows such as individual sales over the course of the previous month. The user can then request the LLM agent to sort the spreadsheet by the model number of the product sold, calculate the total sales for each product model number, and then plot the sales. The steps shown in block 402 are also shown in Figure 3 by arrows indicating “task requests” when user 302 initiates a chat with LLM agent 310 and requests LLM agent 310 to perform the respective tasks.

[0049] Next, in block 404 following the example above, the LLM agent responds to the user and attempts to perform the requested task. In some embodiments, the LLM agent can sort, calculate, and plot sales according to the user's instructions, and then output the results of the requested task to the user via the user interface. In other embodiments, the LLM agent can first prompt the user for more detailed information, for example, by asking whether the user wants to receive the plot directly, or by querying the programming code used for calculating and plotting the sales. Thus, a back-and-forth conversation can take place between the user and the LLM agent until the task is completed. Furthermore, the LLM agent can provide an initial result of the requested task and then receive indicators from the user, such as that this was not the requested result, or that this result is incomplete, or that the user prefers a different plotting format or programming language used to generate the plot. In this embodiment, the conversation between the LLM agent and the user can continue when the LLM agent retries to complete the task. Furthermore, the LLM agent infers that a task has been completed when the user explicitly states that the task is complete, or when the user leaves the chat interface.

[0050] In block 406, the LLM service makes the logs of communications from recent task executions by the LLM agent available to the LLM service's reflection agent. The reflection agent then executes a reflection protocol to analyze the communications logs of information used to generate data samples to be extracted and stored in long-term memory. The reflection protocol is executed in such a way that various types of data samples can be extracted, such as data samples labeled as belonging to a given domain knowledge category, data samples labeled as belonging to known user preferences of users who have recently engaged in chats with the LLM agent, data samples labeled as successfully completing a task by the LLM agent (e.g., "recipes" for success), and data samples labeled as incomplete or unsuccessful task completion by the LLM agent (e.g., "lessons learned" scenarios).

[0051] Furthermore, generating data samples refers to the conversion of information from communication logs into vectors.

[0052] As will be further explained below with reference to Figures 5 through 8, which show the individual parts of the overall process flow of the reflection protocol, the reflection protocol is similar to the internal monologue applied in prompt engineering to improve the reasoning capabilities of the LLM agent over time. The process of executing the reflection protocol is structured as an internal monologue framework, in which the reflection agent responds to a series of predetermined questions such as "Does this conversation query the LLM agent for assistance with a task?", "Does this message contain information about a given topic?", "What is the concept of this information?", and "Immediately extract the content of each concept." These predetermined questions are available to the reflection agent as a "self-reflection" of the LLM service's ability to accurately perform tasks for the service's users, and can each be queryed to extract data samples regarding domain knowledge categories (see also process 500 in Figure 5), user preferences (see also process 600 in Figure 6), successful task completion (see also process 700 in Figure 7), and task failure (see also process 800 in Figure 8).

[0053] Processes 500, 600, 700, and 800 further describe the parts of the reflection protocol performed by the LLM service's reflection agent. However, returning first to the overall process 400 shown in Figure 4, block 408 indicates the point at which the reflection agent extracts useful information or “reflections” from the communication log and generates corresponding data samples that should be kept accessible when the LLM agent needs to recall that information in the execution of a future task. Thus, block 408 indicates that the reflection agent supplies the data samples to the storage service for storage in long-term memory. This step is also indicated in Figure 3 by an arrow indicating “storage” when the reflection agent 316 supplies the generated data samples 328 for storage in the database within the storage service 322.

[0054] Furthermore, the generated data samples, which at this point resemble vectors, are stored in a long-term memory database in a format similar to a title, content, and generalized task.

[0055] Next, block 410 depicts a later point in time when the same user of the LLM service, or another user, requests the LLM agent to perform a new task on their behalf. Continuing with the example described above, the new task, in this case as well, involves the LLM agent sorting a spreadsheet by the product model number sold, calculating the total sales for each product model number, and then plotting the sales. This is done with respect to a different spreadsheet than the previous one, or with respect to the current spreadsheet, which includes sales from both May and June, unlike the previous spreadsheet which only included May sales. The LLM agent is then configured to first perform a "recall" using data samples already stored in long-term memory to determine whether any of these data samples are relevant to the current task. For example, the LLM agent can search for data samples that are specifically labeled to correspond to the user in question to determine whether it is recorded that the particular user has a first or second operating system, whether a previous user requested Python or Julia for calculating total sales and plotting sales, or whether any data samples have been previously stored and are labeled to correspond to the domain knowledge category of the company's product sales.

[0056] As described above, the data samples are stored in vector format, and therefore, "recall" refers to the LLM agent first establishing the intended artifacts within the user's request. The LLM agent then forms a prompt for a vector search on the long-term memory database, in which case the top K results considered relevant are applied when attempting to complete this new task.

[0057] Next, in block 412, the LLM agent applies knowledge from a data sample retrieved from long-term memory when performing the current task.

[0058] The process shown in Figure 4 is repeatable any number of times and over extended periods so that the LLM service becomes more efficient in performing tasks for the user over time. By configuring the reflection agent to store relevant and useful information from communication with the user, and then configuring the LLM agent to "recall" that information when performing the next task, the LLM service develops and maintains long-term memory for the LLM without the need to retrain the model or update the model's weights or internal parameters.

[0059] The following description of subprocesses 500, 600, 700, and 800, which execute the reflection protocol by the reflection agent, can be made in parallel with each other, sequentially with each other, or by any other combination that ensures that predetermined questions supplied by prompt engineering are self-reflected by the reflection agent with respect to domain knowledge categories, user preferences, task completion success, and task completion failure.

[0060] Figure 5 is a flowchart showing a first subprocess that executes the reflection protocol introduced in Figure 4, relating to a domain knowledge category, in several embodiments.

[0061] In block 502, the communication logs are analyzed to determine whether any of the messages sent by the user correspond to a given domain knowledge category. According to some embodiments, the domain knowledge category can span multiple dimensions, such as the professional's skills in the domain, facts about the company using the LLM service, and prior experience. The domain knowledge can also correspond to new facts (e.g., updated company location after the company has changed its location), new ways of completing a task (e.g., first obtaining a spreadsheet containing individualized sales, summing these up, and then plotting them), or new concepts (e.g., procedures for employee year-end performance reviews).

[0062] Following the self-reflection structure described above, and further following the example above, the reflection agent can respond to predetermined questions such as, "Does this communication log ask the LLM agent for assistance with a task?" "Yes, the task is to calculate monthly sales for each product model and plot the results." "Does this message contain domain knowledge?" "Yes." "What is the domain knowledge about?" "This domain knowledge concerns how to plot task completion and monthly sales reports in the sales department."

[0063] Next, in block 504, the reflection agent generates a text-based data sample from communications corresponding to responses to predetermined questions. For example, if the reflection agent determines that a particular task concerns how to plot monthly sales reports, the corresponding text-based data sample may include information such as the phrasing the user requested to be plotted and the resulting plot generated by the LLM agent during task execution.

[0064] When generating data samples corresponding to identified domain knowledge categories, the reflection agent first extracts high-level concepts from the relevant portion of the communication log, then proceeds to extract the specific context of the domain knowledge, and can summarize the knowledge into a limited number of words. The reflection agent can then generalize the task based on the current task and the extracted domain knowledge.

[0065] In block 506, the reflection agent may additionally search the long-term memory database for any data samples that have labels similar to or exactly matching "monthly sales report plot" in order to link related tasks together over time within the long-term memory database. With respect to the self-reflection structure, the specific predetermined question to the reflection agent may be similar to a question such as "Are there any other tasks where this domain knowledge could be useful?" This then forms a prompt to search for relevant data samples in the long-term memory data storage so that the reflection agent can appropriately label the data sample it is currently generating and form a connection to previously stored data samples.

[0066] In block 508, the generated data sample, a label corresponding to a specific domain knowledge category, and links to other related data samples already stored in long-term memory are supplied to the storage service.

[0067] Figure 6 is a flowchart showing a second subprocess that executes the reflection protocol introduced in Figure 4, relating to user preferences, in several embodiments.

[0068] In block 602, the communication log is analyzed to determine whether any of the messages sent by the user correspond to a user preference metric. For example, a user preference metric may include a signature indicating that a particular user uses the Linux or Mac operating system, or that a particular user prefers to write and manage programming code written in C++ while other users prefer to write and manage programming code written in Python. User preferences may also refer to a user's job title. For example, the metric that a given user works in the marketing department of a company can function as a user preference metric and, if linked to other data samples in the long-term storage database, can form a prompt to the LLM agent to format the plot in a predetermined manner, or to use colors corresponding to the company's logo colors when generating a plot for that user in the marketing department. Thus, according to some embodiments, there may be an overlap between user preferences and domain knowledge in a given environment.

[0069] Following the self-reflection structure described above, and further following the example above, the reflection agent may respond to predetermined questions such as, "Does this communication log ask the LLM agent for assistance with a task?" "Yes, the task is to write an outline of the company newsletter for user 302 in the marketing department." "Does this message include user preferences?" "Yes." "What are the user preferences?" "The user preferences are regarding preparing an outline of the company newsletter about Project XYZ and using the TimesNewRoman font."

[0070] Next, in block 604, the reflection agent generates text-based data samples from communications corresponding to responses to predetermined questions. For example, since the reflection agent has identified a specific task related to preparing an outline of the company newsletter, the corresponding text-based data sample may include information that the TimesNewRoman font is used when preparing the outline of the company newsletter.

[0071] In block 606, the data samples based on the generated text can be labeled with a username or some other unique identifier associated with that user. With respect to the self-reflection structure, the specific predetermined question to the reflection agent may be similar to a question such as "Are there any other tasks associated with this particular user?" This prompts the reflection agent to search for relevant data samples in long-term memory data storage related to tasks performed for that particular user in order to build a user profile. A user profile can refer to a logical term that labels any data samples corresponding to completed tasks for a given user as such.

[0072] In block 608, generated data samples and their labels corresponding to a specific user are supplied to the storage service, making them available for storage in the long-term memory database.

[0073] Figure 7 is a flowchart showing a third subprocess that executes the reflection protocol introduced in Figure 4, relating to the success of the LLM agent in task execution in several embodiments.

[0074] With respect to subprocess 700 and the subsequent subprocess 800, the following exemplary implementation of the reflection protocol is used to facilitate the discussion in this specification. Specifically, a user of the LLM service initiates a chat with the LLM agent of the service, provides a spreadsheet containing product sales from the previous month, and requests the LLM agent to "disclose how many products were returned." In the subsequent conversation with the LLM agent, the results are first output for returned products with manufacturing defects, returned products in good condition, and returned products sold more than two months ago. In this case, the user adds, "No, I only mean returned products with manufacturing defects." The LLM agent then outputs the results for returned products with manufacturing defects and can remove the returned products that were in good condition. The user then replies, "I want the total amount of loss incurred by the returns, not the number of returns we received." The LLM agent then sums up the totals using additional information from the original spreadsheet, and the user then ends the chat with the LLM agent.

[0075] Returning to subprocess 700, block 702 points to the point in time when the communication log is analyzed to determine whether any of the messages sent by the user correspond to an indicator that the LLM agent has successfully completed the task. In the example above, the “successful” completion of the task could refer to the third point in time when the LLM agent outputs the results to the user, where the results include the total amount of losses incurred due to returned products with manufacturing defects. Furthermore, an indicator of successful task completion by the LLM agent can be recognized by explicit communication from the user affirming that the output results correspond to the deliverables of the requested task. For example, after the LLM agent made a third attempt to perform the requested task, the user ended the chat with the LLM agent. Another example of explicit communication indicating successful task completion is when the user writes something like, “Yes, that’s right. I now have everything I needed.”

[0076] Following the self-reflection structure described above, and further following the example above, the reflection agent can respond to predetermined questions such as, "Does this communication log ask the LLM agent for assistance with a task?" "Yes, the task is to search a spreadsheet for products returned due to manufacturing defects and sum up the total amount of losses incurred from the return of these products." "Did the LLM agent successfully assist the user in completing the task?" "Yes." "What is the deliverable?" "The deliverable is the total amount of losses incurred from the return of products with manufacturing defects."

[0077] Next, in block 704, the reflection agent generates a text-based data sample from communications corresponding to responses to predetermined questions. In some embodiments, the data sample may be similar to a “recipe,” which is a step-by-step guide and specific artifacts, such as email text or code snippets for data analysis. In this case, these artifacts can be used as models for subsequent tasks performed by the LLM agent. Continuing the example described above, the “recipe” may not include the first and second attempts by the LLM agent to perform the task, because these did not indicate the “successful” completion of the task. Rather, the generated text-based data sample forming the recipe may include, for example, a user request and the LLM agent’s response indicating that the LLM agent’s third attempt was successful.

[0078] In block 706, the generated text-based data samples are labelable so that it is understood that a particular data sample represents a recipe for success. In block 708, the generated data samples and their labels, corresponding to a specific user, are supplied to a storage service so that they can be stored in a long-term memory database.

[0079] Figure 8 is a flowchart showing a fourth subprocess that executes the reflection protocol introduced in Figure 4, relating to LLM agent failures in task execution in several embodiments.

[0080] In block 802, the communication log is analyzed to determine whether any of the messages sent by the user correspond to an indicator that the LLM agent failed to complete the task. In the example above, task completion "failure" could refer to both the first and second times the LLM agent outputted results to the user, where the results included a list of inaccurate returns, and then a list or return value instead of the total amount of loss incurred due to each return. Furthermore, an indicator of task completion failure by the LLM agent may or may not be recognizable by explicit communication from the user denying that the output results correspond to the requested task deliverables. For example, in the example above, the user explicitly wrote "No, I only intend to return products with manufacturing defects" during the conversation with the LLM agent, indicating that the requested deliverables were not provided to the user in a particular output result. However, other responses from the user, such as "I want the total amount of loss incurred due to the return," could still result in an output result that does not resemble task completion and therefore still fall into the category of failure by the LLM agent.

[0081] Following the self-reflection structure described above, and further following the example above, the reflection agent can respond to predetermined questions such as, "Does this communication log query the LLM agent for assistance with a task?" "Yes, the task is to search a spreadsheet for products returned due to manufacturing defects and sum up the total amount of losses incurred from the return of these products." "Did the LLM agent fail to assist the user in completing the task?" "Yes." "What was the nature of the failure?" The reflection agent then reflects on the failed first and second trials in order to provide the user with the correct result, and can further form a link to the successfully completed third trial to draw conclusions about an improved "recipe" that the LLM agent can use instead when performing a similar task next time.

[0082] Next, in block 804, the reflection agent generates text-based data samples from communications corresponding to responses to predetermined questions. In block 806, the generated text-based data samples are labelable so that specific data samples are understood by the LLM agent as representing failure recipes that will not be directly repeated the next time the task is performed. In block 708, the generated data samples and their labels corresponding to a specific user are supplied to the storage service so that they can be stored in the long-term memory database.

[0083] Figure 9A shows examples of user interfaces in several embodiments that allow users of an LLM service to chat with the LLM agent of the service in order to perform tasks.

[0084] As shown in chat 900, the user is currently chatting with the LLM agent and the task is already in progress. Messages 902 and 906 are similar to messages sent by the LLM agent, and message 904 is similar to a message sent by the user. Messages 902, 904, and 906 become part of the communication log supplied to the reflection agent upon completion of the given task.

[0085] As illustrated in the specific example shown in Figure 9A, the user may be similar to a data scientist at a company using the LLM service, whose daily tasks include analyzing manufacturing data to discover patterns and potential problems.

[0086] Message 904 contains enterprise-specific and / or internal information, such as data specifications, conceptual definitions, and user programming preferences. During execution of the reflection protocol by the reflection agent, this information in message 904 is distilled into a data sample regarding domain knowledge, a data sample regarding the user's user preferences which are added to the user profile, and a data sample regarding the success or failure of the LLM agent in completing the final task.

[0087] Figure 9B shows another part of a user interface, according to several embodiments, that can provide data samples generated during the reflection protocol for viewing and editing by users of the LLM service.

[0088] In some embodiments, after a reflection agent executes a reflection protocol to extract information and generates a data sample corresponding to the relevant information, the user can request to view and / or edit the data sample. For example, if a user reads a data sample and realizes that a recent request in which the LLM agent performed a task has been incorrectly labeled as belonging to the domain knowledge category of internal newsletter summary rather than investor pitch, the user can prompt the LLM service via the user interface to correct the particular labeling before storing the data sample in long-term memory storage. In another example, if a user reads their current user profile and realizes that the label on a data sample does not indicate a recent job title change from site manager to district-wide manager, they can similarly prompt the LLM service to correct the data sample.

[0089] As shown in the user interface section labeled Reflection 920, two different data samples labeled "Domain Knowledge" are presented. For example, a parameter called "MaterialPressure," used within the company, is stored in long-term memory as something that significantly contributes to the results of leak tests in the manufacturing of home appliances. The second data sample shown in Figure 9B describes the semantic meaning of the measurement column, so that when the LLM agent performs a task related to "The Stage*.Output.Measurement*.U.Actual" in the future, the LLM agent will recall references to specific columns.

[0090] Figure 9C shows examples of several embodiments in which a user can add additional data samples to data samples generated during the reflection protocol via a user interface.

[0091] In addition to the various subprocesses 500, 600, 700, and 800 described above, users of the LLM service can add their own "user-defined" reflections to the data samples stored in the long-term memory database. For example, if a user is viewing reflections generated regarding a task that requires the context of a company's organizational chart for efficient completion, the user can proactively supply the company's organizational chart as an additional data sample to be stored in long-term memory for future recall by the LLM agent.

[0092] In the user interface shown in Figure 9C, block 940 refers to a user who provides details of additional reflections that are distilled into a data sample. The user prompts the LLM service with "Which machines are in the upper area? Which machines are in the lower area?", to which the LLM service responds "Machines 1 and 2 are in the upper area, and machines 3, 4, and 5 are in the lower area." Next, logs of these additional communications are distilled into a data sample that is stored in a long-term memory database.

[0093] Figure 10A shows another part of a user interface in several embodiments that allows users of an LLM service to browse and explore data samples already stored in a long-term memory database.

[0094] As mentioned above with respect to Figure 3, users of the LLM service can also perform knowledge searches to view data samples stored in long-term memory. For example, if a user is a new employee of a company, they can perform various searches through the user interface to learn the context of their new project and parts of future projects. In another example, if an employee is tasked with creating an outline for the company newsletter for the first time, but knows that other employees have previously created similar outlines, they can perform a knowledge search to use one of those outlines as a template.

[0095] As shown in the user interface 1000 section of Figure 10A, a search bar is provided for the user to explore specific domain knowledge stored in the LLM service. Furthermore, each bubble on the screen represents a given data sample, and the distance between bubbles indicates similarity in the vector space.

[0096] Figure 10B shows examples of data samples that a user can view using a user interface in long-term memory storage, according to several embodiments.

[0097] As shown in block 1020, the given data sample is organized as a recipe for the LLM agent to successfully complete the task. The data sample includes the requested artifact, "Perform data analysis and create a visualization," along with a code snippet.

[0098] Figure 10C shows other examples of data samples viewed by a user using a user interface in long-term memory storage, according to several embodiments.

[0099] As further shown in block 1040, a given data sample within the domain knowledge category “semantic meanings of Measurement columns” further directs the user to other data samples that the LLM service has labeled as related, relevant, or similar to the given data sample.

[0100] While exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms that are covered by the claims. The terms used herein are for illustrative purposes only, not limitation, and it should be understood that various modifications are possible without departing from the spirit and scope of this disclosure. As noted above, by combining features of various embodiments, further embodiments of the invention not expressly described or illustrated can be formed. While various embodiments have been described as offering advantages over or being more preferable to other embodiments or prior art implementations with respect to one or more desired characteristics, it should be recognized by those skilled in the art that one or more features or characteristics may be compromised to obtain desirable system attributes as a whole, depending on the particular application and implementation. These attributes may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, and ease of assembly. Therefore, even where any embodiment is described as undesirable with respect to one or more characteristics compared to other embodiments or prior art implementations, this does not mean that these embodiments are outside the scope of this disclosure, and they may be desirable for a particular application.

Claims

1. A computer-implemented method for managing the long-term memory of a large-scale language model (LLM), Receiving a first request from a user of the LLM service to the LLM agent of the LLM service to perform a first task, The LLM agent executes the first task and outputs the result of the first task to the user. Based on the analysis of the communication between the user and the LLM agent during the execution of the first task by the reflection agent of the LLM service, a reflection protocol is executed to generate data samples, The data samples are stored in the long-term memory database of the LLM service, In response to receiving a second request to the LLM agent to perform a second task, the LLM agent performs a recall using the long-term memory database and determines one or more data samples related to the performance of the second task. The LLM agent performs the second task based on the recall of one or more data samples stored in the long-term memory database, Computerized methods, including those mentioned above.

2. Executing the aforementioned reflection protocol means The communication is analyzed to determine which domain knowledge category the first task corresponds to, To generate text-based data samples from the domain knowledge category and the communications corresponding to the output results of the first task, Based on the aforementioned domain knowledge categories, the relevant tasks are searched for in the aforementioned long-term memory database. To supply the long-term memory database with data samples based on the aforementioned text and links to related tasks to be memorized, The computer-implemented method according to claim 1, including the method described in claim 1.

3. The computer-implemented method according to claim 2, wherein the domain knowledge categories include at least enterprise-specific procedures and professional workplace skill sets.

4. Executing the aforementioned reflection protocol means The communication is analyzed over the entire duration of the communication to determine the user preference indicators of the user, To generate text-based data samples from communications corresponding to the user preferences of the aforementioned user, To label the data sample based on the aforementioned text as corresponding to the aforementioned user, The data sample based on the aforementioned text and the label to be stored are supplied to the long-term memory database. The computer-implemented method according to claim 1, including the method described in claim 1.

5. The computer-implemented method according to claim 4, wherein the user preference indicators include at least an indicator of a computer operating system for a particular user and an indicator of a preferred programming language for a particular user.

6. Executing the aforementioned reflection protocol means The communication is analyzed to determine an indicator that the LLM agent has successfully completed the first task, To generate text-based data samples from communication corresponding to the deliverables in the first task requested by the user and the output results of the first task by the LLM agent, The data sample based on the aforementioned text is labeled as having been successfully completed by the LLM agent, The data sample based on the aforementioned text and the label to be stored are supplied to the long-term memory database. The computer-implemented method according to claim 1, including the method described in claim 1.

7. The computer-implemented method according to claim 6, wherein the indicator of successful completion includes explicit communication from the user confirming that the output result corresponds to the deliverable of the requested first task.

8. Executing the aforementioned reflection protocol means The communication is analyzed to determine an indicator that the LLM agent failed to execute the first task, To generate text-based data samples from communications corresponding to the deliverables in the first task requested by the user and the results of the failure of the first task by the LLM agent, The data sample based on the aforementioned text is labeled as a failure of the LLM agent, The data sample based on the aforementioned text and the label to be stored are supplied to the long-term memory database. The computer-implemented method according to claim 1, including the method described in claim 1.

9. The computer-implemented method according to claim 8, wherein the indicator of failure includes explicit communication from the user that the output result does not correspond to the deliverable of the first task for which it was requested.

10. The computer-implemented method further, The reflection agent provides the user with data samples generated during the execution of the reflection protocol via the user interface of the LLM service. The user provides an indicator for editing one of the aforementioned data samples or an indicator for adding additional data samples. The edited data sample or the added data sample is stored in the long-term memory database of the LLM service, The computer-implemented method according to claim 1, including the method described in claim 1.

11. A computer-implemented method for managing the long-term memory of a large-scale language model (LLM), The LLM service's reflection agent receives logs of communications between the LLM agent and the user of the LLM service, i.e., logs including requests from the user to perform a task and responses from the LLM agent performing the task. The reflection agent generates data samples by executing a reflection protocol based on an analysis of requests and responses in the communication logs, wherein the reflection protocol extracts the data samples based on domain knowledge categories, user preferences, and indicators of success or failure of the task execution by the LLM agent. The LLM agent stores the data samples in the long-term memory database of the LLM service for future recalls when it performs other tasks for other users of the LLM service. Computerized methods, including those mentioned above.

12. The computer-implemented method further, The process involves generating a reflection protocol for the LLM service, wherein the self-reflection problem to be performed by the reflection agent is generated using a prompt engineering method. The computer-implemented method according to claim 11, including the method described in claim 11.

13. The computer-implemented method further, The LLM agent receives other requests to perform other tasks, Using the long-term memory database, perform a recall and determine one or more data samples from among the data samples related to the execution of the other tasks, The LLM agent performs the other tasks based on the recall of one or more data samples stored in the long-term memory database. The computer-implemented method according to claim 11, including the method described in claim 11.

14. A database configured to store multiple data samples that are made accessible to the LLM agent and reflection agent of a Large-Scale Language Model (LLM) service, A computing device configured to implement the aforementioned LLM service, A system equipped with, The aforementioned LLM service is Upon receiving a first request from a user of the LLM service to the LLM agent to perform a first task, The LLM agent executes the first task and outputs the result of the first task to the user. The reflection agent executes a reflection protocol to generate additional data samples based on an analysis of the communication between the user and the LLM agent during the execution of the first task. The aforementioned additional data samples are supplied to the long-term storage within the database. In response to receiving a second request to the LLM agent to perform a second task, the LLM agent accesses the long-term storage in the database and determines one or more of the data samples related to the execution of the second task. The LLM agent performs the second task based on the one or more data samples. A system that is configured in such a way.

15. The computing device further, The user interface for the aforementioned LLM service is implemented, In response to the execution of the reflection protocol, the additional data samples are supplied to the user via the user interface. The user provides an index for editing one of the aforementioned additional data samples or an index for adding other data samples. One or more edited data samples from the aforementioned additional data samples are supplied to the long-term storage in the database. The system according to claim 14, configured as follows.

16. The aforementioned database further, It stores user permissions corresponding to access to the aforementioned multiple data samples. It is configured in such a way, The computing device further, In response to receiving another request from the user to access one or more of the data samples, the database verifies that the user has permission based on the user permissions stored in the database. The user is supplied with one or more of the data samples from the plurality of data samples via the user interface. The system according to claim 15, configured as described above.

17. In order to execute the reflection protocol, the computing device further: The communication is analyzed to determine which domain knowledge category the first task corresponds to. A text-based data sample is generated from the communication corresponding to the domain knowledge category and the output result of the first task. Based on the domain knowledge categories, the relevant tasks are searched for in the long-term storage within the database. The database provides data samples based on the aforementioned text and links to related tasks to be stored in long-term memory storage. The system according to claim 14, configured as follows.

18. In order to execute the reflection protocol, the computing device further: The communication is analyzed over the entire duration of the communication to determine the user preference indicators of the user. A text-based data sample is generated from the communication corresponding to the user preferences of the aforementioned user. The data sample based on the aforementioned text is labeled as corresponding to the user, The data sample based on the aforementioned text and the label to be stored are supplied to the long-term storage in the database. The system according to claim 14, configured as follows.

19. In order to execute the reflection protocol, the computing device further: The communication is analyzed to determine an indicator that the LLM agent has successfully completed the first task. A text-based data sample is generated from the communication corresponding to the deliverables in the first task requested by the user and the output results of the first task by the LLM agent. The data sample based on the aforementioned text is labeled as having been successfully completed by the LLM agent. The data sample based on the aforementioned text and the label to be stored are supplied to the long-term storage in the database. The system according to claim 14, configured as follows.

20. In order to execute the reflection protocol, the computing device further: By analyzing the aforementioned communications, an indicator is determined that the LLM agent failed to execute the first task. A text-based data sample is generated from the communication corresponding to the deliverables in the first task requested by the user and the result of the failure of the first task by the LLM agent. The data sample based on the aforementioned text is labeled as a failure by the LLM agent. The data sample based on the aforementioned text and the label to be stored are supplied to the long-term storage in the database. The system according to claim 14, configured as follows.

Images