Method and system for synchronizing context in large language model data processing session

Abstract

A computer-implemented method of synchronizing context in a large language model (LLM) data processing session. The method includes receiving, at an LLM context management data processing system, a set of one or more context related parameters corresponding to the LLM session. The method further includes converting the received set of one or more context related parameters into a unified LLM compatible message format and collecting and monitoring the results of the converting step for context changes. The method further includes updating a context used by the LLM session with results of the monitoring step upon detecting one or more pre-defined conditions.

Description

[0001] METHOD AND SYSTEM FOR SYNCHRONIZING CONTEXT E LARGE LANGUAGE MODEL DATA PROCESSING SESSION

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to the field of large language models; and more specifically, to a method and a system of synchronizing context in a Large Language Model (LLM) data processing session to ensure an accurate and up-to-date context information.

[0004] BACKGROUND

[0005] Advancements in the field of Natural Language Processing (NLP) have gained popularity over the years due to a plethora of applications, such as machine translation, sentiment analysis, and chatbots. The NLP is a subfield of Artificial Intelligence (Al) that focuses on the interaction between computers and human language. The NLP involves the development of algorithms and models that enable computers to understand, interpret, and generate human language in a way that is meaningful and contextually relevant. The NLP techniques have revolutionized various industries, including healthcare, finance, customer service, and information retrieval, by enabling efficient and accurate processing of large volumes of textual data. In the general domain of NLP, there are several challenges and problems that researchers and developers have been addressing. One of the key issues is the effective utilization of contextual information during output generation. In recent Large Language Model (LLM) solutions, users are often responsible for synchronizing session context with dynamically changing environmental conditions and intents. However, the manual synchronization process can be cumbersome and prone to errors. When changes occur rapidly or too frequently, users may fail to keep the context updated, leading to confusing, misleading, or irrelevant outputs. This can result in wrong decisions and operational errors made by the users.

[0006] Currently, certain techniques have been developed for maintaining runtime context in an LLM session, and the developed techniques mainly rely on information collected during the session or pre-defined context templates prepared by the users. Session chunking is one such technique that breaks down large sessions into smaller fragments to minimize the use of context across a long session period. While effective for short interactions, however, this technique often fails to maintain coherence in longer and more complex conversations. Other examples include summarization of session histories, external storage for context retrieval, using LLM originated feedback requests, or LLM sessions with pre-defined context templates, and the like. However, these existing approaches have limitations in effectively synchronizing the context with dynamically changing external conditions. In long-lasting LLM sessions, conditions, intents, and policies are likely to change, causing the session context to become inaccurate or irrelevant. Thus, there exists a technical problem of inefficient context synchronization in continuous LLM sessions leading to a degraded and irrelevant output.

[0007] Therefore, in light of the foregoing discussion, there exists a need to overcome the aforementioned drawbacks associated with the conventional methods of context management in the continuous LLM sessions.

[0008] SUMMARY

[0009] The present disclosure provides a method and a system of synchronizing context in a Large Language Model (LLM) data processing session. The present disclosure provides a solution to the existing problem of inefficient context synchronization in continuous LLM sessions leading to a degraded and irrelevant output. An aim of the present disclosure is to provide a solution that overcomes at least partially the problems encountered in the prior art, and provide an improved method, and system of synchronizing context in a Large Language Model (LLM) data processing session.

[0010] The object of the present disclosure is achieved by the solutions provided in the enclosed independent claims. Advantageous implementations of the present disclosure are further defined in the dependent claims.

[0011] In one aspect, the present disclosure provides a computer-implemented method of synchronising context in a Large Language Model (LLM) data processing session. The method comprising steps of receiving, at an LLM context management data processing system, a set of one or more context related parameters corresponding to the LLM session, converting the received set of one or more context related parameters into a unified LLM compatible message format, collecting and monitoring the results of the converting step for context changes and updating a context used by the LLM session with the results of the monitoring step upon detecting one or more pre-defined conditions.

[0012] The disclosed method is used to efficiently synchronize the context with dynamically changing external conditions in continuous LLM sessions leading to enhanced user experience by providing an accurate and reliable output. The method ensures the consistency and continuity of the LLM session by updating the LLM context based on monitoring the user’s activity or inactivity. The method is used to collect and consolidate the contextual messages and provide a log of events that can be used to track and tune contextual activities of the user. Consequently, an improved management and control of the contextual updates can be obtained. By converting the one or more context related parameters into the unified LLM message format, the method can lead to efficient processing and monitoring of the context. The collected and monitored results are then used to update the context, ensuring that the context remains up-to-date and aligned with the evolving requirements of the LLM session. The dynamic synchronization of context enhances the overall performance and effectiveness of the LLM, leading to improved user experiences and outcomes.

[0013] In an implementation form, the one or more pre-defined conditions include an LLM session being resumed after a timeout.

[0014] In a further implementation form, the one or more pre-defined conditions include a specific change in context.

[0015] In a further implementation form, the one or more context related parameters are received from a plurality of context providers, each of which registers with the system for publishing specific events of a pre-defined type which are related to the context related parameters.

[0016] By receiving the one or more context-related parameters from the plurality of context providers and registering them for specific event publishing, the contextual activities can be effectively tracked and managed.

[0017] In a further implementation form, the results of the converting and monitoring step are aggregated into a consolidated group and applied to the LLM session in the updating step.

[0018] By consolidating the results of the converting and monitoring step, the method can be used to provide a more accurate and comprehensive context for the LLM prompt.

[0019] In a further implementation form, the method further includes monitoring user activity of a user of the LLM session, and performing the updating step when user activity is detected.

[0020] In a further implementation form, the updating step involve a plurality of contextual events being translated into prompts and sent to the LLM session. The translation of the plurality of contextual events into prompts and sending them to the LLM session results in an improved ability of the method to handle and respond to typical LLM prompts.

[0021] In a further implementation form, the method further includes monitoring user activity of a user of the LLM session and when the monitoring of the user activity detects that a user is inactive in the LLM session, context caching is activated whereby context information is stored in a cache.

[0022] The context cache is activated to maintain a log of the contextual events which is used to fine-tune and track contextual activities.

[0023] In a further implementation form, when context caching is activated, contextual events are aggregated.

[0024] The use of context cache is advantageous in efficiently managing and updating the internal context of the LLM context management data processing system, and ensuring that the LLM context remains up-to-date and aligned with the user's requirements.

[0025] In a further implementation form, when user activity is detected to be resumed, aggregated events are retrieved from the cache and forwarded to the LLM session.

[0026] In a further implementation form, the set of one or more context related parameters includes information regarding an application a user is using while participating in an LLM session.

[0027] By incorporating information about the user's application into the context-related parameters, the LLM context management data processing system can better understand the user's context and provide more contextually relevant prompts.

[0028] In a further implementation form, the set of one or more context related parameters includes information regarding a location of a user participating in an LLM session.

[0029] This is advantageous to use the user's location as the context-related parameter to tailor the LLM data processing session accordingly and to deliver more accurate and relevant outputs to the user.

[0030] In a further implementation form, the set of one or more context related parameters includes information regarding an activity of a user participating in an LLM session.

[0031] By tracking and analyzing the activity of the user, the method can be used to improve the accuracy of the LLM, further leading to more reliable predictions.

[0032] In another aspect, the present disclosure provides a system comprising means adapted for carrying out all the steps of the method.

[0033] The system achieves all the advantages and technical effects of the method after execution of the method.

[0034] In a yet another aspect, the present disclosure provides a computer program comprising instructions for carrying out all the steps of the method, when said computer program is executed on a computer system.

[0035] The computer program achieves all the advantages and effects of the method after execution of the method.

[0036] It is to be appreciated that all the aforementioned implementation forms can be combined. It has to be noted that all devices, elements, circuitry, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof. It will be appreciated that features of the present disclosure are susceptible to being combined in various combinations without departing from the scope of the present disclosure as defined by the appended claims.

[0037] Additional aspects, advantages, features and objects of the present disclosure would be made apparent from the drawings and the detailed description of the illustrative implementations construed in conjunction with the appended claims that follow.

[0038] BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the present disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those skilled in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers.

[0040] Embodiments of the present disclosure will now be described, by way of example only, with reference to the following diagrams wherein:

[0041] FIG. 1 is a diagram illustrating a system for synchronizing context in a Large Language Model (LLM) data processing session, in accordance with an embodiment of the present disclosure;

[0042] FIG. 2 is a flowchart of a method of synchronizing context in an LLM data processing session, in accordance with an embodiment of the present disclosure;

[0043] FIG. 3A is an exemplary scenario of context auto-update in an LLM session, in accordance with an embodiment of the present disclosure;

[0044] FIG. 3B is another exemplary scenario of context auto-update in an LLM session, in accordance with another embodiment of the present disclosure;

[0045] FIG. 3C is yet another exemplary scenario of context auto-update in an LLM session, in accordance with yet another embodiment of the present disclosure;

[0046] FIG. 3D is yet another exemplary scenario of context auto-update in an LLM session, in accordance with yet another embodiment of the present disclosure;

[0047] FIG. 4 illustrates an exemplary scenario of accumulating changes in context and synchronizing the context according to accumulated context changes in an LLM session, in accordance with an embodiment of the present disclosure; FIG. 5 illustrates an apparatus and interfaces used for synchronizing context in an LLM session, in accordance with an embodiment of the present disclosure; and

[0048] FIG. 6 is an operational diagram of an LLM query processing, in accordance with an embodiment of the present disclosure.

[0049] In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.

[0050] DETAILED DESCRIPTION OF EMBODIMENTS

[0051] The following detailed description illustrates embodiments of the present disclosure and ways in which they can be implemented. Although some modes of carrying out the present disclosure have been disclosed, those skilled in the art would recognize that other embodiments for carrying out or practicing the present disclosure are also possible.

[0052] FIG. 1 is a diagram illustrating a system for synchronizing context in a large language model (LLM) data processing session, in accordance with an embodiment of the present disclosure. With reference to FIG. 1, there is shown a system 100 for synchronizing context in an LLM data processing session. The system 100 includes an LLM context management data processing system 102 that includes an LLM agent 104, a context controller 106, a context cache 108 and a session manager 110. Each of the LLM agent 104, the context controller 106, the context cache 108 and the session manager 110 is communicatively coupled to each other and work together in the LLM context management data processing system 102. There is further shown a plurality of context providers 112, a user 114, an LLM 116 and a series of operations 118 to 128. The system 100 and the LLM context management data processing system 102 are represented by dashed boxes, which are used for illustration purpose only.

[0053] The LLM context management data processing system 102 may be referred to as a system configured to manage and process data related to the context synchronization in continuous LLM sessions. The LLM context management data processing system 102 may be configured to synchronize the context with dynamically changing external conditions in continuous language model sessions to enhance the accuracy and relevance of language model outputs.

[0054] The LLM agent 104 may be referred to as a component or a software service that act as an intermediary between the user 114, the LLM 116, and the LLM context management data processing system 102. The LLM agent 104 may be configured to process user inputs and forward the user inputs to the LLM 116, receive responses from the LLM 116 and relay the received responses back to the user 114. The LLM agent 104 may be configured to work with the context controller 106 to incorporate relevant contextual information into user queries or LLM prompts.

[0055] The context controller 106 may be configured to manage the contextual information. The context controller 106 may also be configured to manage the context cache 108, store and retrieve contextual data. Examples of the context controller 106 may include, but are not limited to, a rule-based context controller, machine learning based context controller, time-aware context controller, multi-source context aggregator, adaptive context controller, real-time context controller, and the like. The context controller 106 may also be referred to as a context manager.

[0056] The context cache 108 may be referred to as a temporary storage component that rapidly stores and retrieves contextual information relevant to ongoing LLM sessions or interactions. The context cache 108 may be configured to store various types of contexts, such as user preferences, recent conversation history, environmental factors, or application states. The context cache 108 may be configured to operate in conjunction with the context controller 106 to ensure that cached data remains up- to-date and consistent with the current user context. Examples of the context cache 108 may include but are not limited to, a Central Processing Unit (CPU) cache, Random Access Memory (RAM) cache, disk cache, a web browser cache, an applicationlevel cache, a Content Delivery Network (CDN) cache, and the like.

[0057] The session manager 110 may be configured to manage overall user session, including authentication and authorization. The session manager 110 may also be configured to track session duration and handle timeouts, and manage multiple aspects of the user state, not just context. The session manager 110 often handles session persistence across multiple requests.

[0058] Each of the plurality of context providers 112 may be referred to as a component or service that supplies relevant contextual information to enhance the accuracy and relevance of LLM interactions. Each of the plurality of context providers 112 may be configured to supply various types of contexts, such as user location, device information, application state, time, user preferences, or environmental conditions to the LLM context management data processing system 102 to tailor the LLM 116 output more accurately to the user's specific circumstances.

[0059] The user 114 may be referred to as an individual or entity that interacts with the system 100 to perform specific actions, access information, or utilize functionalities provided by the system 100.

[0060] The LLM 116 is a type of artificial intelligence model designed to understand, generate, and manipulate human-like text across a wide range of topics and tasks. Examples of the LLM 116 may include but are not limited to, Generative Pre-trained Transformer (GPT) series, Bidirectional Encoder Representations from Transformers (BERT) and their variants, and the like.

[0061] There is provided the system 100 for synchronizing context in an LLM data processing session. The system 100 is configured to improve the synchronization of context in the LLM data processing session by receiving and converting context-related parameters into a unified LLM compatible message format, monitoring for context changes, and updating the LLM session's context based on predefined conditions. The system 100 may be configured to automatically update the LLM session context on change of specific conditions to enhance the accuracy and relevance of language model outputs. The system 100 augments existing LLM context synchronization techniques by providing the LLM contextual details that the user can't or does not care to provide himself.

[0062] The context controller 106 at the LLM context management data processing system 102 is configured to receive a set of one or more context related parameters corresponding to the LLM session. The one or more context related parameters may be referred to as parameters that are associated with the context of a given task or session in a continuous large language model. The LLM session refers to a session or instance of interaction with a continuous large language model, where the model is utilized to generate responses or perform tasks based on the provided context and input.

[0063] In accordance with an embodiment, the one or more context related parameters are received from the plurality of context providers 112, each of which registers with the system for publishing specific events of a pre-defined type which are related to the context related parameters. The term "publishing specific events" refers to the act of making particular occurrences or incidents known or accessible to other components or systems within a network or environment. The term "pre-defined type" refers to a predetermined or established category or classification that is predefined or specified in advance. The one or more context related parameters can be tracked using Operating system (OS), network, browser, application, and the like. Each context provider is responsible for tracking the pre-defined list of context-related parameters and generating events related to these parameters. Furthermore, at operation 118, each of the plurality of context providers 112 can be dynamically registered with the LLM context management data processing system 102. The registration and event publishing process allows for the collection of contextual messages from multiple sources. The Context controller 106 then consolidates, summarizes, and purifies the aggregated updates to ensure the relevance and accuracy of contextual updates. By receiving the one or more context-related parameters from the plurality of context providers 112 and registering them for specific event publishing, the system 100 can effectively track and manage contextual activities.

[0064] In accordance with an embodiment, the set of one or more context related parameters includes information regarding an application a user is using while participating in a LLM session. The one or more context related parameters includes information about the application in use, browser page, opened document, running session, and the like. By incorporating information about the user's application into the context-related parameters, the LLM context management data processing system 102 can better understand the user's context and provide more contextually relevant prompts. This improves the overall user experience and increases the efficiency of the language learning process in the LLM session.

[0065] In accordance with an embodiment, the set of one or more context related parameters includes information regarding a location of a user participating in an LLM session. The one or more context related parameters includes information about the location of the user 114 whether the user 114 is at home, or at work or in the car, etc. This is advantageous to use the user's location as the context-related parameter to enable the LLM system (i.e.,the system 100) to deliver location-specific information and tailor the LLM session accordingly.

[0066] In accordance with an embodiment, the set of one or more context related parameters includes information regarding an activity of a user participating in an LLM session. The one or more context related parameters includes information regarding the activity of the user 114, such as whether the user 114 is using public network or a home network or an enterprise network. In addition to the network attachment, the one or more context related parameters may include engagement status of the user 114, intent of the user 114, topic of interest of the user 114, any scheduled event of the user 114, information access to the user 114, and the like. The engagement status of the user 114 highlights whether the user 114 is busy or loaded or relaxed, etc. The intent of the user 114 indicates whether the user is solving a problem, or education or entertaining, and the like. The topic of interest of the user 114 may include art, science, production, health, etc. The scheduled event of the user 114 may include information about outlook, calendar, timer, date and time, etc. The information access to the user 114 may include whether the information is restricted to the user 114, or accessible to the user 114 or publicly available, etc. By tracking and analyzing the one or more context related parameters, the system 100 improves the accuracy of the LLM 116, leading to more reliable predictions.

[0067] The context controller 106 is further configured to convert the received set of one or more context related parameters into a unified LLM compatible message format. The unified LLM compatible message format refers to a standardized structure and syntax for messages that can be seamlessly processed and understood by continuous large language models, ensuring compatibility and interoperability across different systems and platforms. At operation 120, the context controller 106 is configured to receive the one or more context related parameters and convert the context related parameters into the unified LLM message format. By converting the one or more context related parameters into the unified LLM compatible message format, the system 100 enables the LLM 116 to effectively process and understand the context provided by the user 114. This improves the accuracy and relevance of the LLMs responses, leading to a more efficient and interactive user experience.

[0068] The context controller 106 is further configured to collect and monitor the results of the converting step for context changes. At operation 122, the context controller 106 is configured to monitor for the context changes during conversion of the context related parameters into the unified LLM message format. By collecting and monitoring the context changes, the system 100 ensures that the language model (i.e., the LLM 116) remains up-to-date and responsive to evolving contextual information. This optimization enhances the accuracy and relevance of the LLMs output, improving the overall performance and user satisfaction. Additionally, the tracking of the one or more context related parameters allows for efficient prompt processing and adaptation to user-specific requirements. The LLM agent 104 is configured to update a context used by the LLM session with the results of the monitoring step upon detecting one or more pre-defined conditions. The term "pre-defined conditions" refers to predetermined criteria or specifications that are established in advance and serve as a basis for evaluating or determining certain aspects or behaviors within a given context. When either a specific change occurs or the LLM session resumes after a timeout, the LLM agent 104 updates the LLM 116 with consolidated updates to fine-tune the internal context. This ensures that the LLM session always has an adjusted and relevant context before interacting with the user 114. By updating the context used by the LLM session with the results of the monitoring step upon detecting the one or more predefined conditions, the system 100 ensures that the LLM session always has an accurate and relevant context. This improves the accuracy and effectiveness of the LLM 116 in processing user inputs and generating appropriate responses. Additionally, the system 100 optimizes the context by consolidating, summarizing, and purifying contextual updates, leading to improved session consistency and continuity.

[0069] In accordance with an embodiment, the one or more pre-defined conditions include an LLM session being resumed after a timeout. In an implementation scenario, the LLM session is resumed after the timeout. In such scenario, the LLM agent 104 updates the LLM 116 with consolidated updates to fine-tune the internal context. This ensures that the LLM session always has an adjusted and relevant context before interacting with the user 114. By managing local pre-ready contexts at the LLM agent 104 side and injecting the most relevant ones during session initiation, known optimization techniques can be applied to enhance the user experience.

[0070] In accordance with an embodiment, the one or more pre-defined conditions include a specific change in context. In an implementation scenario, the specific change may occur in the context. In such scenario, the LLM agent 104 updates the LLM 116 with consolidated updates to fine-tune the internal context. This ensures that the LLM session always has an adjusted and relevant context before interacting with the user 114. Thus, the system 100 can enhance the effectiveness and accuracy of the LLM session by considering specific changes in context. By optimizing and purifying the context, the system 100 ensures that the LLM 116 maintains session consistency and continuity, leading to an improved user experience and more accurate responses.

[0071] In accordance with an embodiment, the results of the converting and monitoring step are aggregated into a consolidated group and applied to the LLM session in the updating step. By consolidating the results of the converting and monitoring step, the system 100 aims to provide a more accurate and comprehensive context for the LLM prompt. Alternatively, may be stated as, the aggregation of results of the converting and monitoring steps is done to optimize the context for the LLM session. This optimization enhances output of the LLM 116 by ensuring that the updated context is more relevant and aligned with the user's requirements.

[0072] In accordance with an embodiment, the session manager 110 of the LLM context management data processing system 102 is configured to monitor user activity of a user of the LLM session and the LLM agent 104 is configured to update the context when user activity is detected. At operation 124, the session manager 110 is configured to monitor the user activity (e.g., changes in state of the user 114) of the user 114 of the LLM session. Upon detection of the user activity, the LLM agent 104 is configured to update the context so that the LLM 116 can dynamically adapt and provide a more personalized and relevant experience. The system 100 may also include simulating test scenarios to inspect changes in the LLM reactions caused by various factors, such as switching between public and enterprise networks or scheduled events.

[0073] In accordance with an embodiment, updating of the LLM context involve a plurality of contextual events being translated into prompts and sent to the LLM session. The term "plurality of contextual events" refers to a collection or set of multiple events that occur within a specific context, where each event is characterized by its own unique attributes, circumstances, or conditions. At operation 126, the LLM agent 104 is configured to update the LLM context by translating the plurality of contextual events into prompts and send the prompts to the LLM 116 at operation 128. The translation of the plurality of contextual events into prompts and sending them to the LLM session results in an improved ability of the system 100 to handle and respond to typical LLM prompts. The prompts utilized in the prompt processing pipeline are provided by the user 114 during the LLM session, including historical prompts. The prompt engineering and tuning guidelines recommend specifying comprehensive and accurate contextual details at the beginning of the session and ensuring the freshness, relevance, and consistency of the accumulated context throughout the session period.

[0074] In accordance with an embodiment, the session manager 110 is configured to monitor user activity of a user of the LLM session and when the monitoring of the user activity detects that a user is inactive in the LLM session, context caching is activated whereby context information is stored in a cache. The context caching may be referred to as the process of storing and retrieving context information from the context cache 108 in order to enhance the efficiency and performance of the system 100 by reducing the requirement for repeated computations or data retrieval. The context information may be referred to as the relevant data or state that is associated with a particular task or operation, which can include variables, parameters, settings, or any other information that is required for the proper execution or understanding of the task. In an implementation scenario, when the LLM session is suspended due to inactivity of the user 114, the context controller 106 is configured to activate the context cache 108 and store the context information in to the context cache 108. The context controller 106 may be rule-driven and configured to maintain a log of the contextual events in the context cache 108 to fine-tune and track contextual activities.

[0075] In accordance with an embodiment, when context caching is activated, contextual events are aggregated. On detection of the user’s inactivity, the context cache 108 is activated to store the plurality of contextual events. The use of context cache 108 is advantageous in efficiently managing and updating the internal context of the LLM context management data processing system 102, and ensuring that the LLM context remains up-to-date and aligned with the user's requirements.

[0076] In accordance with an embodiment, when user activity is detected to be resumed, aggregated events are retrieved from the cache and forwarded to the LLM session. When the user 114 resumes the LLM session, the context controller 106 is configured to receive the aggregated contextual events from the context cache 108 and perform several operations on the aggregated contextual events. These operations include consolidating, summarizing, and purifying the aggregated updates to ensure that the contextual updates remain relevant and accurate. The context controller 106 may utilize rules to drive aforementioned operations. The context controller 106 also manages a log of events used for tuning and tracking contextual activities. The processed aggregated events are forwarded to the LLM session along with the user’s input (i.e., a kind of prompt engineering technique making the LLM 116 to refine accumulated context accordingly).

[0077] Additionally, the context controller 106 is configured to support pull and push updates of the LLM 116 on session resume or incoming update correspondingly, with the option of initiating a pull to update the context. The context controller 106 is configured to generate LLM adapted events using application-specific information, which can be unified or standardized in terms of event types, details, and output format.

[0078] Thus, the system 100 efficiently synchronizes the context with dynamically changing external conditions in continuous LLM sessions leading to enhanced user experience by providing an accurate and reliable output. The system 100 ensures the consistency and continuity of the LLM session by updating the LLM context based on monitoring the user’s activity or inactivity. The context controller 106 is configured to collect and consolidate the contextual messages and provide a log of events that can be used to track and tune contextual activities of the user 114. Consequently, an improved management and control of the contextual updates can be obtained. By converting the one or more context related parameters into the unified LLM message format, the system 100 can lead to efficient processing and monitoring of the context. The collected and monitored results are then used to update the context, ensuring that the context remains up-to-date and aligned with the evolving requirements of the LLM session. The dynamic synchronization of context enhances the overall performance and effectiveness of the LLM 116, leading to improved user experiences and outcomes. In contrast to conventional context management systems, the system 100 manifests minimized operational errors and manual operations. Conventionally, the operational errors are caused by misleading outputs by virtue of using inaccurate context during the LLM session. Such operational errors are reduced in the system 100 by automatically synchronizing the context with the dynamically changing external conditions including silent changes as well, about which the user 114 is not aware of. Moreover, the system 100 optimizes the output relevance and robustness in long-lasting LLM conversations. The system 100 minimizes the number of interactions with the LLM 116 and reduces number of potential operational mistakes. Thus, the system 100 manifests an improved context management and synchronization by using the LLM agent 104, the context controller 106, the context cache 108 and the session manager 110 in a synergistic manner.

[0079] FIG. 2 is a flowchart of a method of synchronizing context in a LLM data processing session, in accordance with an embodiment of the present disclosure. FIG. 2 is described in conjunction with elements from FIG. 1. With reference to FIG. 2, there is shown a method 200 of synchronizing context in an LLM data processing session. The method 200 includes steps 202 to 208. The system 100 (of FIG. 1) is configured to execute the method 200.

[0080] There is provided the method 200 of synchronizing context in an LLM data processing session. The method 200 is augmenting existing LLM context synchronization techniques by providing the LLM contextual details that user can't or does not care to provide himself. The method 200 may be used to automatically synchronize the LLM session context with the dynamically changing external conditions in continuous LLM sessions to enhance the accuracy and relevance of language model outputs. By applying context synchronization on change of specific conditions and updating the internal context of the language model based on consolidated and relevant updates, the method 200 ensures that the language model (i.e., the LLM 116) always operates with the most up-to-date and appropriate context. This results in improved accuracy, relevance, and consistency in the language model's responses, enhancing the overall performance and user experience.

[0081] At step 202, the method 200 comprises receiving, at an LLM context management data processing system, a set of one or more context related parameters corresponding to the LLM session. The context controller 106 of the LLM context management data processing system 102 (of FIG. 1) is configured to receive the set of one or more context related parameters corresponding to the LLM session.

[0082] In accordance with an embodiment, the one or more context related parameters are received from the plurality of context providers 112, each of which registers with the system for publishing specific events of a pre-defined type which are related to the context related parameters. Each of the plurality of context providers 112 can be dynamically registered with the LLM context management data processing system 102. Each context provider is responsible for tracking the pre-defined list of context-related parameters and generating events related to these parameters.

[0083] In accordance with an embodiment, the set of one or more context related parameters includes information regarding an application a user is using while participating in an LLM session. The set of one or more context related parameters includes information about the application in use, browser page, opened document, running session, and the like.

[0084] In accordance with an embodiment, the set of one or more context related parameters includes information regarding a location of a user participating in an LLM session. The one or more context related parameters includes information about the location of the user 114 whether the user 114 is at home, or at work or in the car, etc.

[0085] In accordance with an embodiment, the set of one or more context related parameters includes information regarding an activity of a user participating in an LLM session. The one or more context related parameters includes information regarding the activity of the user 114, such as whether the user 114 is using public network or a home network or an enterprise network. At step 204, the method 200 further comprises converting the received set of one or more context related parameters into a unified LLM compatible message format. By converting the set of one or more context related parameters into the unified LLM compatible message format, the method 200 ensures that the language models (i.e., the LLM 116) effectively process and understand the context provided by the user 114.

[0086] At step 206, the method 200 further comprises collecting and monitoring the results of the converting step for context changes. By collecting and monitoring the context changes, the method 200 ensures that the language model (i.e., the LLM 116) remains up-to-date and responsive to evolving contextual information.

[0087] At step 208, the method 200 further comprises updating a context used by the LLM session with the results of the monitoring step upon detecting one or more pre-defined conditions. When either a specific change occurs or the LLM session resumes after a timeout, the method 200 is used to update the context used by the LLM session with the results of the monitoring step (i.e., the step 206) upon detecting the one or more predefined conditions.

[0088] In accordance with an embodiment, the one or more pre-defined conditions include an LLM session being resumed after a timeout. In an implementation scenario, the LLM session is resumed after the timeout. In such scenario, the method 200 is used to update the language model (i.e., the LLM 116) with consolidated updates to fine-tune the internal context.

[0089] In accordance with an embodiment, the one or more pre-defined conditions include a specific change in context. In an implementation scenario, the specific change may occur in the context. In such scenario, the method 200 is used to update the language model (i.e., the LLM 116) with consolidated updates to fine-tune the internal context.

[0090] In accordance with an embodiment, the results of the converting and monitoring step are aggregated into a consolidated group and applied to the LLM session in the updating step. By consolidating the results of the converting and monitoring step, the method 200 provides a more accurate and comprehensive context for the LLM prompt which is further provided to the LLM 116.

[0091] In accordance with an embodiment, the method 200 further includes monitoring user activity of a user of the LLM session, and performing the updating step when user activity is detected. Upon detection of the user activity, the method 200 updates the LLM context so that the LLM 116 can dynamically adapt and provide a more personalized and relevant experience.

[0092] In accordance with an embodiment, the updating step involve a plurality of contextual events being translated into prompts and sent to the LLM session. The translation of the plurality of contextual events into prompts and sending them to the LLM session results in an improved output of the language model (i.e., the LLM 116).

[0093] In accordance with an embodiment, the method 200 further includes monitoring user activity of a user of the LLM session and when the monitoring of the user activity detects that a user is inactive in the LLM session, context caching is activated whereby context information is stored in a cache. In an implementation scenario, when the LLM session is suspended due to inactivity of the user 114 then, the context cache 108 is activated and the context information is stored in to the context cache 108.

[0094] In accordance with an embodiment, when context caching is activated, contextual events are aggregated. On detection of the user’s inactivity, the context cache 108 is activated to store the plurality of contextual events which are further aggregated and used to update the LLM context.

[0095] In accordance with an embodiment, when user activity is detected to be resumed, aggregated events are retrieved from the cache and forwarded to the LLM session. When the user 114 resumes the LLM session, the method 200 is used to receive the aggregated contextual events from the context cache 108 and process the aggregated contextual events to remove irrelevant data and consolidate the aggregated updates to ensure that the contextual updates remain relevant and accurate.

[0096] The steps 202 to 208 are only illustrative, and other alternatives can also be provided where one or more steps are added, or one or more steps are provided in a different sequence without departing from the scope of the claims herein.

[0097] There is provided a computer program comprising instructions for carrying out all the steps of the method 200. The computer program is executed on a computer system. The computer program is implemented as an algorithm, embedded in a software stored in the non-transitory computer-readable storage medium having program instructions stored thereon, the program instructions being executable by the one or more processors in the computer system to execute the method 200. The non- transitory computer-readable storage means may include, but are not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples of implementation of computer-readable storage medium, but are not limited to, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Random Access Memory (RAM), a Read Only Memory (ROM), a Elard Disk Drive (1TDD), a Flash memory, a Secure Digital (SD) card, a Solid-State Drive (SSD), a computer-readable storage medium, and / or a CPU cache memory.

[0098] FIG. 3A is an exemplary scenario of context auto-update in an LLM session, in accordance with an embodiment of the present disclosure. FIG. 3A is described in conjunction with elements from FIGs 1 and 2. With reference to FIG. 3A, there is shown that a context related parameter includes information regarding a location of a user participating in an LLM session. There is further shown that a user 302 is interacting with the chatbot 304 powered by the LLM 116 (of FIG. 1). Initially the user 302 is present in Tel Aviv (a city in Israel) and thereafter, the user 302 boarded in a Delta flight (D361) to Los Angeles (a city in the United States). The change in context is the change in location of the user 302 that is automatically updated in the LLM 116 and hence, in the chatbot 304, as shown at operation 306. The user 302 asks the chatbot 304 about a present for his grandmother (a specific change in context occurs), as shown at operation 308. The chatbot 304 automatically switches to duty-free options. The chatbot 304 suggests the user 302 for a perfume from onboard duty-free kiosk, as shown at operation 310. The chatbot 304 is automatically updated in such a way that the chatbot 304 considers only options available onboard in the transatlantic Delta Flights to US.

[0099] FIG. 3B is another exemplary scenario of context auto-update in an LLM session, in accordance with another embodiment of the present disclosure. FIG. 3B is described in conjunction with elements from FIGs 1 and 2. With reference to FIG. 3B, there is shown that a context related parameter includes information regarding network connectivity of a user participating in an LLM session. There is further shown that the user 302 is interacting with the chatbot 304 powered by the LLM 116 (of FIG. 1). Initially, the user 302 is connected to home network and then, the user 302 reaches office and gets connected to enterprise Virtual Private Network (VPN). The change in context is the change in network connectivity of the user 302 that is automatically updated in the LLM 116 and hence, in the chatbot 304, as shown at operation 312. The user 302 asks the chatbot 304 about a new product launch in 2024, as shown at operation 314. The chatbot 304 automatically uses the internal information of the enterprise and suggests the user 302 about “X” product launch in Q4 of “Y” organization. The chatbot 304 is automatically updated in such a way that the chatbot 304 uses the most recent information available on web portal of “Y” organization (including yet not disclosed publicly) while answering to the user 302.

[0100] FIG. 3C is yet another exemplary scenario of context auto-update in an LLM session, in accordance with yet another embodiment of the present disclosure. FIG. 3C is described in conjunction with elements from FIGs 1 and 2. With reference to FIG. 3C, there is shown that a context related parameter includes information regarding intent of a user participating in an LLM session. There is further shown that the user 302 is interacting with the chatbot 304 powered by the LLM 116 (of FIG. 1). Initially, the user 302 is using “X” website (e.g., sciencedirect.com) for accessing scientific research papers and then, switched to “Y” website (e.g., researchgate.com) for accessing scientific research papers. The change in context is the change in intent of the user 302 (i.e., using different website) that is automatically updated in the LLM 116 and hence, in the chatbot 304, as shown at operation 318. The user 302 asks the chatbot 304 about LLM context management techniques, as shown at operation 320. The chatbot 304 automatically switch to adjust mode (t = 0.1 ) and suggests the user 302 about the reference chaining and recursive summarization as LLM context management techniques, as shown at operation 322.

[0101] FIG. 3D is yet another exemplary scenario of context auto-update in an LLM session, in accordance with yet another embodiment of the present disclosure. FIG. 3D is described in conjunction with elements from FIGs 1 and 2. With reference to FIG. 3D, there is shown that a context related parameter includes information regarding status of a user participating in an LLM session. There is further shown that the user 302 is interacting with the chatbot 304 powered by the LLM 116 (of FIG. 1). Initially, the user 302 is relaxed and then, switched to a business meeting. The change in context is the change in status of the user 302 (i.e., business meeting) that is automatically updated in the LLM 116 and hence, in the chatbot 304, as shown at operation 324. The user 302 asks the chatbot 304 about best known performance of AGI models in 2023, as shown at operation 326. The chatbot 304 automatically switch to adjust mode (top-p»0) and suggests the user 302 about 102Mturings. By virtue of the automatic context update in the LLM 116, the chatbot 304 suggests the short answers to the user 302.

[0102] FIG. 5 illustrates an apparatus and interfaces used for synchronizing context in an LLM session, in accordance with an embodiment of the present disclosure. FIG. 5 is described in conjunction with elements from FIGs 1, 2, 3A-3D and 4. With reference to FIG. 5, there is shown an apparatus 500 for synchronizing context in an LLM session. The apparatus 500 comprises an environment mediator 502, the context controller 106 and the LLM synchronizer 504. The environment mediator 502 is communicatively coupled to a number of interfaces, such as a web application adapter 506, a network stack adapter 508, a browser adapter 510, the context provider 408 and a registry 512. The context controller 106 is communicatively coupled to an update generator 514, a database 516 and the context cache 108. The LLM synchronizer 504 is communicatively coupled to a notifier 518 and a retriever 520.

[0103] The environment mediator 502 is configured to register the context provider 408 for publishing specific events of the predefined type(s). The context provider 408, an event adapter that receives natively formatted event from the source and creates a unified LLM compatible message. The context controller 106 (i.e., the context manager) is configured to collect received contextual messages, consolidate the collected contextual messages, summarize and purify the aggregated updates to keep contextual updates relevant and accurate. The context controller 106 may be rules driven and the rules are stored in the database 516. The context controller 106 may be configured to manage the log of events in the database 516 to tune and track the contextual activities. The context cache 108 is used to store the context update records. The LLM synchronizer 504 is configured to support pull and push updates of the LLM 116 on session resume or incoming specific update correspondingly. In an implementation, notifying an initiated pull is one of the alternative context updates. The retriever 520 is used to provide context update prompt(s) to the LLM. The adapters, for example, the web application adapter 506, the network stack adapter 508, and the browser adapter 510 or a VPN adapter, may be configured to generate LLM adapted event using application specific information. The supported event types, details and format of the output might be unified or standardized.

[0104] Additionally, there is provided exemplary interfaces and data structures used in the LLM session.

[0105] Context Update Record {

[0106] [1] Reference ID: UID

[0107] Event Type : Change Report / Change Notification / Retrieve Response

[0108] Ctx. Subject : Location, Intent, Network Access, Status, Loading, etc.

[0109] Date & Time : e.g. 12 / 05 / 2024 15: 14

[0110] Prompt : e.g. I’m currently in Aroma Cafe etc.

[0111] }

[0112] [2] Context Provider Manifest}

[0113] Source ID : UID

[0114] Provider Type : Access Controller, Scheduler, WEB Browser, Application, etc.

[0115] Ctx. Subject : Location, Intent, Network Access, Status, Loading, etc.

[0116] Message Map : [ {EventID = 0x23456, Input = “In the office“} , { ... ,... }, . . . ] etc.

[0117] }

[0118] [3] Typical LLM prompt including summarized update, such as “I’m located at Aroma and running business meeting, i.e. busy and need short answers”

[0119] [4] Request Augmented Generation (RAG) Interface, where LLM (e.g., the LLM 116 of FIG. 1) is programmed explicitly to use external data in preparing outputs. In the operational mode LLM always append to the originally provided prompt and the contextual data stored at the end point. In this case, the context controller 106 prepares context on each update and let the LLM 116 to retrieve the update when required.

[0120] FIG. 6 is an operational diagram of an LLM query processing, in accordance with an embodiment of the present disclosure. FIG. 6 is described in conjunction with elements from FIGs 1, 2, 3A-3D, 4, and 5. With reference to FIG. 6, there is shown a flowchart 600 depicting a series of operations 610 to 646 in processing an LLM query. A context related parameter can be tracked using on Operating System (OS) 602. There is further shown that the LLM 116 is connected to the context controller 106 and an input processor 604. The flowchart 600 includes two major flows that is a first flow 606 and a second flow 608. The first flow 606 is related to automatic update of the context when an LLM session resumes and the second flow 608 depicts how to handle “silent” changes about which the LLM is not aware of and update of the context. The LLM agent 104 is configured to monitor user’s activities and manage context states (e.g., active or inactive states). At operation 610, the user 302 is active and provides inputs within pre-defined time window then, prompts are forwarded to the LLM 116.

[0121] At operation 612, the input processor 604 is configured to forward the user inputs to the context controller 106.

[0122] At operation 614, the context controller 106 is configured to consolidate, summarize, clean and optimize the collected contextual messages to keep the contextual updates relevant and accurate. The operations 612 and 614 are repeated up to N times.

[0123] At operation 616, the input processor 604 detects that time has expired and accordingly, instruct the context controller 106 to freeze the context.

[0124] At operation 618, the context controller 106 is configured to freeze the context.

[0125] The selected number of contextual events (related to registered and active context providers) are translated to prompts and sent “silently” to the LLM 116 on behalf of the user 302.

[0126] At operation 620, the context provider 408 is configured to provide a plurality of contextual events to the environment mediator 502. Meanwhile, at operation 622, the context controller 106 determines that the user is inactive for a time period and activate the context cache 108, at operation 624. Alternatively, may be stated, when session is suspended due to user’s inactivity, the context controller 106 is configured to activate the context cache 108. During the user’s inactivity period, the contextual events are aggregated in to the context cache 108.

[0127] At operation 626, the environment mediator 502 is configured to validate and normalize the plurality of contextual events and transform the plurality of contextual events into a unified LLM compatible message format. The unified LLM compatible message format is forwarded to the context controller 106.

[0128] At operation 628, the context controller 106 is configured to translate the unified LLM compatible message format into a prompt, which is forwarded to the context cache 108 for storage.

[0129] At operation 630, the context cache 108 is configured to aggregate such prompts.

[0130] At operation 632, the user 302 resumes the LLM session and provides an input.

[0131] At operation 634, the environment mediator 502 receives the user’s input and updates the context cache 108 about the session resumption.

[0132] At operation 636, the context controller 106 is configured to retrieve the aggregated events from the context cache 108.

[0133] At operation 638, the context controller 106 is configured to process the aggregated events to remove irrelevant data and provide a consolidated information and forwards to summarized data to the environment mediator 502.

[0134] At operation 640, the environment mediator 502 is configured to create a prompt using the summarized data and forward the created prompt along with the user’s input to the LLM 116. This is a kind of prompt engineering technique making the LLM 116 capable to refine the accumulated context accordingly. The operations 610 to 640 corresponds to the first flow 606 related to automatic update of the context when the LLM session resumes.

[0135] The operations 642 to 646 corresponds to the second flow 608 that depicts how to handle “silent” changes about which the LLM is not aware of and automatic update of the context.

[0136] At operation 642, the context provider 408 is configured to provide a plurality of contextual events to the environment mediator 502.

[0137] At operation 644, the environment mediator 502 is configured to validate and normalize the plurality of contextual events and transform the plurality of contextual events into a unified LLM compatible message format. The unified LLM compatible message format is forwarded to the context controller 106.

[0138] At operation 646, the context controller 106 is configured to translate the unified LLM compatible message format into a prompt, which is forwarded to the LLM 116.

[0139] Modifications to embodiments of the present disclosure described in the foregoing are possible without departing from the scope of the present disclosure as defined by the accompanying claims. Expressions such as "including", "comprising", "incorporating", "have", "is" used to describe and claim the present disclosure are intended to be construed in a non-exclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural. The word "exemplary" is used herein to mean "serving as an example, instance or illustration". Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features from other embodiments. The word "optionally" is used herein to mean "is provided in some embodiments and not provided in other embodiments". It is appreciated that certain features of the present disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the present disclosure, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable combination or as suitable in any other described embodiment of the disclosure.

Claims

CLAIMS1. A computer-implemented method (200) of synchronizing context in a large language model, LLM, data processing session, comprising steps of: receiving, at an LLM context management data processing system (102), a set of one or more context related parameters corresponding to the LLM session; converting the received set of one or more context related parameters into a unified LLM compatible message format; collecting and monitoring the results of the converting step for context changes; and updating a context used by the LLM session with the results of the monitoring step upon detecting one or more predefined conditions.

2. The method (200) of claim 1 , wherein the one or more pre-defmed conditions include an LLM session being resumed after a timeout.

3. The method (200) of claim 1 , wherein the one or more pre-defined conditions include a specific change in context.

4. The method (200) of claim 1, wherein the one or more context related parameters are received from a plurality of context providers (112), each of which registers with the system for publishing specific events of a pre-defined type which are related to the context related parameters.

5. The method (200) of claim 1, wherein the results of the converting and monitoring step are aggregated into a consolidated group and applied to the LLM session in the updating step.

6. The method (200) of claim 1 , wherein the method (200) further includes monitoring user activity of a user (114) of the LLM session, and performing the updating step when user activity is detected.

7. The method (200) of claim 1, wherein the updating step involve a plurality of contextual events being translated into prompts and sent to the LLM session.

8. The method (200) of claim 1, wherein the method (200) further includes monitoring user activity of a user (114) of the LLM session and when the monitoring of the user activity detects that a user (114) is inactive in the LLM session, context caching is activated whereby context information is stored in a cache.

9. The method (200) of claim 8, wherein when context caching is activated, contextual events are aggregated.

10. The method (200) of claim 9, wherein when user activity is detected to be resumed, aggregated events are retrieved from the cache and forwarded to the LLM session.

11. The method (200) of claim 1, wherein the set of one or more context related parameters includes information regarding an application a user (114) is using while participating in an LLM session.

12. The method (200) of claim 1, wherein the set of one or more context related parameters includes information regarding a location of a user (114) participating in an LLM session.

13. The method (200) of claim 1 , wherein the set of one or more context related parameters includes information regarding an activity of a user (114) participating in an LLM session.

14. A system (100) comprising means adapted for carrying out all the steps of the method (200) according to any preceding method claim.

15. A computer program comprising instructions for carrying out all the steps of the method (200) according to any preceding method claim, when said computer program is executed on a computer system.

Images