Computer implementation methods, computer programs, and computer systems (compression of tool prompts using relative information entropy)

Abstract

This invention provides a method and program for compressing tool prompts in a data processing system. [Solution] The method receives an original tool prompt, segments the original tool prompt into multiple text chunks, generates at least one semantic vector representation of the multiple text chunks, generates a first semantic distribution of the original tool prompt based on the at least one semantic vector representation, generates a perturbed semantic vector representation based on a subset of the multiple text chunks, generates a second semantic distribution based on the perturbed semantic vector representation, performs a comparison of the first and second semantic distributions to generate at least one similarity metric, and generates a compressed tool prompt based on a subset of the multiple text chunks in response to at least one similarity metric exceeding a threshold similarity value.

Description

Technical Field

[0001] This application generally relates to machine learning models, generative machine learning models, large language models (LLMs), artificial intelligence agents (AI agents) that can be automated to perform actions on their own environment based on observations, and interactions with LLMs by agents.

Summary of the Invention

Problems to be Solved by the Invention

[0002] LLMs are trained on vast amounts of text data and use deep learning techniques to perform various natural language processing tasks, such as text generation, translation, and summarization.

Means for Solving the Problems

[0003] This "Summary of the Invention" is provided to introduce a selection of concepts in a simplified form and is further described in the "Detailed Description of the Invention" of this specification. This "Summary of the Invention" is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used in limiting the scope of the claimed subject matter.

[0004] In one exemplary embodiment, a method is provided for a data processing system comprising the steps of receiving an original tool prompt and segmenting the original tool prompt into a plurality of text chunks. The method further comprises the step of generating at least one semantic vector representation of the plurality of text chunks. The method also comprises the step of generating a first semantic distribution of the original tool prompt based on the at least one semantic vector representation. Furthermore, the method comprises the steps of generating a perturbed semantic vector representation based on a subset of the plurality of text chunks, the subset being generated by excluding at least one text chunk from the plurality of text chunks, and generating a second semantic distribution based on the perturbed semantic vector representation. The method further comprises the step of performing a comparison of the first semantic distribution and the second semantic distribution to generate at least one similarity metric. Furthermore, the method comprises the step of generating a compressed tool prompt based on the subset of the plurality of text chunks in response to the at least one similarity metric exceeding a threshold similarity value.

[0005] In other exemplary embodiments, a computer-readable medium containing a computer-readable program or a computer program product comprising a computer-readable medium is provided. When the computer-readable program is executed on a computing device, it causes the computing device to perform various operations and combinations thereof that are outlined above with respect to exemplary embodiments of the method.

[0006] In yet another exemplary embodiment, a system / device is provided. This system / device may comprise one or more processors and memory coupled to one or more processors. This memory may have instructions, which, when executed by one or more processors, cause one or more processors to perform various operations and combinations thereof of those outlined above with respect to exemplary embodiments of the method.

[0007] These and other features and advantages of the present invention will be described or will become apparent to those skilled in the art in consideration of the following detailed description of exemplary embodiments of the present invention. [Brief explanation of the drawing]

[0008] The present invention, preferred modes of use, and further purposes and advantages will be best understood by referring to the following detailed description of exemplary embodiments in conjunction with the accompanying drawings.

[0009] [Figure 1A] This is an exemplary diagram of the process of a Large-Scale Language Model (LLM) agent in one exemplary embodiment.

[0010] [Figure 1B] This is an illustrative diagram of an LLM task prompt containing multiple agent tool prompts.

[0011] [Figure 2] Figure 1B shows an exemplary diagram illustrating one of the LLM agent tool prompts shown in Figure 1B, and a schematic diagram of the compression process of the present invention performed according to one exemplary embodiment.

[0012] [Figure 3] This is an illustrative diagram of a distributed data processing system environment in which an exemplary embodiment may be implemented and at least a portion of the computer code involved in performing the prompt compression method of the present invention may be executed.

[0013] [Figure 4] An exemplary block diagram illustrating another distributed computing environment in which the compression method of the present invention is performed according to one exemplary embodiment, and which includes a primary operating component of an LLM agent tool prompt compressor.

[0014] [Figure 5] This is an exemplary diagram illustrating the operation of performing vector space modeling using a tool prompt, according to one or more exemplary embodiments.

[0015] [Figure 6] This is an illustrative diagram illustrating the operation for performing a text chunk importance assessment using one exemplary embodiment.

[0016] [Figure 7] This flowchart outlines exemplary behavior for compressing an LLM agent tool prompt in one exemplary embodiment. [Modes for carrying out the invention]

[0017] Large-scale language models (LLMs) are a type of artificial intelligence (AI) model designed to understand and generate human language. LLMs are trained on massive amounts of text data and use deep learning techniques to perform various natural language processing tasks, such as text generation, translation, and summarization. Examples of LLMs include IBM's Granite, OpenAI's GPT model, Google's Gemini, and Meta's Large Language Model Meta AI (LLaMA).

[0018] An LLM agent is an AI computing system built around an LLM that functions as its core computing engine. An LLM agent can extend the capabilities of the LLM beyond text generation, providing additional logic and tools to enable it to perform other tasks, perform reasoning, and provide autonomous capabilities. An LLM agent uses prompts—text that can be processed and interpreted by the LLM, specifying the LLM's persona; instructions to the LLM regarding the functions it should perform; and other information specifying how the LLM agent should behave, what actions it should take, and the type of response the LLM should provide. An LLM agent may be equipped with various tools, such as calculators, application programming interfaces (APIs), search engines, and similar tools, which are accessible to the LLM via one or more tool prompts, allowing the LLM to use them to gather information, perform calculations and actions to complete tasks, and similarly.

[0019] Exemplary embodiments provide computing tools and computing tool behaviors / functions for compressing LLM agent tool prompts based on relative information entropy. The following description provides examples of embodiments of the present disclosure, which may be modified and substituted in other embodiments. Herein, several examples are provided to further clarify the various aspects of the present disclosure.

[0020] Example 1; A computer-implemented method is provided that includes receiving an original tool prompt and segmenting the original tool prompt into a plurality of text chunks. The method further includes generating at least one semantic vector representation of the plurality of text chunks and generating a first semantic distribution of the original tool prompt based on the at least one semantic vector representation. The method also includes generating a perturbed semantic vector representation based on a subset of the plurality of text chunks, the subset being generated by excluding at least one text chunk from the plurality of text chunks, and generating a second semantic distribution based on the perturbed semantic vector representation. Additionally, the method includes performing a comparison of the first semantic distribution and the second semantic distribution to generate at least one similarity metric. Further, the method includes generating a compressed tool prompt based on the subset of the plurality of text chunks.

[0021] Advantageously, the above limitation enables compression of the tool prompt by removing unnecessary or redundant information present within a portion of the original tool prompt. By reducing the amount of unnecessary or redundant text within the tool prompt, or the amount of context tokens, fewer resources are required to process the compressed tool prompt, and the space available within the limits of the generative machine learning computer model becomes available to provide more tool descriptions, enabling the definition and utilization of more complex agents for such generative machine learning computer models.

[0022] Example 2; The method further comprises storing the compressed tool prompt in a data storage accessible to an artificial intelligence agent communicating with the generative machine learning model, the method of any of Examples 1 and 3-13. Advantageously, the above limitation permits the AI agent to utilize and reuse such compressed tool prompts with the corresponding generative machine learning model and / or various generative machine learning models.

[0023] Example 3; The method further comprises adding the compressed tool prompt to a task prompt, inputting the task prompt into a generative machine learning model, and receiving a task output from the generative machine learning model in response to the inputting step, the method of any of Examples 1-2 and 4-13. Advantageously, the above limitation enables the compressed tool prompt to be used in a generative machine learning model in a manner that reduces the amount of resources and processing time for processing the tool prompt, yet provides sufficient context to permit accurate operation of the generative machine learning model.

[0024] Example 4; The original tool prompt serves to define a function tool and comprises one or more defining elements selected from the group consisting of: a function declaration identifying the identifier of the function tool, a function description describing what the function tool does, a parameter description describing the parameters used by the function tool, and a return description describing the type of output provided by the function tool in response to the function tool being invoked, the method of any of Examples 1-3 and 5-13. Advantageously, the above limitation permits the identification of the necessary components of a tool prompt for executing a function to initiate the operation of a generative machine learning model to achieve a task.

[0025] Example 5; the original tool prompt has a first number of the defining elements, and the compressed tool prompt has a second number of the defining elements, the second number being smaller than the first number, in any of the methods in Examples 1-4 and 6-13. Advantageously, the above limitations allow for compression of the tool prompt to a smaller size, which results in less resources and processing time required for the generative machine learning model to process the tool prompt.

[0026] Example 6; any limitation of Examples 1-5 and 7-13, further comprising the step of generating an associated tree data structure based on the plurality of text chunks and at least one similarity metric. The at least one similarity metric has a plurality of similarity metrics, and the connections between nodes in the associated tree data structure have corresponding similarity metrics. These similarity metrics identify similarities between nodes connected by corresponding connections. Advantageously, the above limitation allows for the identification of unnecessary or redundant parts of a tool prompt where nodes use a tree data structure corresponding to text chunks in the original tool prompt, and the elimination of nodes by tree pruning operations allows for the removal of such unnecessary or redundant parts.

[0027] Example 7; any limitation of Examples 1-6 and 8-13, wherein the step of generating the compressed tool prompt comprises pruning the association tree data structure by removing only nodes and paths whose corresponding similarity metrics satisfy a predetermined criterion, thereby generating a pruned association tree data structure. Advantageously, the above limitation allows for a reduction in the text content of the tool prompt by removing nodes and paths with low semantic importance in the tool prompt, as determined by the predetermined criterion.

[0028] Example 8; any limitation of Examples 1-7 and 9-13, wherein the step of generating the compressed tool prompt comprises traversing the pruned association tree data structure to reconstruct a tool prompt containing less text content than the original tool prompt. Advantageously, the above limitation allows for the generation of a tool prompt from a pruned association tree data structure that preserves the meaning of the original tool prompt but minimizes the redundancy and unnecessary content of the original tool prompt. This improves the performance of the generating machine learning model agent by reducing the amount of tool prompt content that needs to be processed.

[0029] Example 9; The at least one similarity metric is generated by performing at least one of a first algorithm that measures the maximum difference between the first and second semantic distributions, and a second algorithm that measures how similar or different the first and second semantic distributions are. Advantageously, the above limitations allow for the generation of a comprehensive dissimilarity index that quantifies the difference between the two distributions, thereby enabling the identification and pruning of unnecessary or redundant nodes in the related tree data structure, which in turn enables the generation of tool prompts.

[0030] Example 10; any limitation of Examples 1-9 and 11-13, wherein the first algorithm is the KS test algorithm and the second algorithm is the Jensen-Shannon divergence algorithm. Advantageously, the above limitation allows the use of the KS test algorithm to measure the maximum difference between two paths (distributions), while using the Jensen-Shannon divergence to measure how much the two maps (distributions) match or differ. Thus, a comprehensive identification of dissimilarity between the two distributions is produced, which leads to a more accurate identification of which parts of the tool prompt are unnecessary or redundant with respect to the meaning of the tool prompt.

[0031] Example 11; any limitation of Examples 1-10 and 12-13, wherein the step of segmenting the original tool prompt into multiple text chunks comprises the steps of parsing the original tool prompt and generating text chunks based on the identification of at least one of the tags, keywords, expressions, or structural elements specific to the description of the functional tools within the tool prompt. Advantageously, the above limitation allows for the automatic parsing and segmentation of the tool prompt in a manner customized for the specific content and structure of the tool prompt. Thus, the resulting segments will represent specific tool prompt segments or text chunks that have semantic meaning.

[0032] Example 12; any limitation of Examples 1-11 and 13, wherein the step of generating the first semantic distribution comprises processing the at least one semantic vector representation by a Gaussian mixture model (GMM), and the step of generating the second semantic distribution comprises processing the perturbed semantic vector representation by the GMM. Advantageously, the above limitation allows for leveraging the functionality of the GMM to produce a semantic distribution for identifying which parts or chunks of a tool prompt may contain unnecessary or redundant semantic information that can be removed in order to compress the tool prompt.

[0033] Example 13; any limitation of Examples 1 to 12, wherein the step of generating at least one semantic vector representation of the plurality of text chunks comprises a step of generating a separate semantic vector representation for each text chunk in the plurality of text chunks, and the step of generating the perturbed semantic vector representation comprises a step of generating a separate perturbed semantic vector representation for each text chunk in the plurality of text chunks, excluding the at least one excluded text chunk. Advantageously, the above limitation allows for the identification of unnecessary or redundant text chunks by perturbing the semantic vector representation and, together with similarity metric generation, identifying which perturbations cause significant differences and which do not. Those that cause significant differences are considered to have high self-information, while those that do not are considered to have low self-information and can therefore be excluded without significantly affecting the meaning of the original tool prompt.

[0034] Example 14; A system comprising one or more processors and one or more computer-readable storage media, the one or more computer-readable storage media collectively storing program instructions, which, when executed by the one or more processors, are configured to cause the one or more processors to execute the method of any one of Examples 1 to 13. Advantageously, the above limitation enables a system comprising one or more processors to perform and realize the advantages described with respect to Examples 1 to 13.

[0035] Example 15; A computer program product comprising one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media, wherein the program instructions include instructions configured to cause one or more processors to perform the method according to any one of Examples 1 to 13. Advantageously, the above limitation enables a computer program product having program instructions configured to cause one or more processors to perform and realize the advantages described with respect to Examples 1 to 13.

[0036] As mentioned above, the exemplary embodiment provides a computing mechanism for compressing LLM agent tool prompts based on relative information entropy. The LLM agent leverages various function tools to enhance the capabilities of LLM. Function tools are input into LLM in a specific prompt format, referred to as tool prompts. Often, a tool prompt will include some or all of the following: a function declaration that identifies the function identifier, a function description that describes what the function tool does, a parameter description that describes the parameters used by the function tool, and a return description that describes the type of output the function tool will provide when invoked by the LLM agent.

[0037] Figure 1A is an exemplary diagram illustrating the operation of an LLM agent in one exemplary embodiment. As shown in Figure 1A, LLM agent 110 operates in conjunction with LLM 120. The only input that LLM 120 accepts is prompt input. Thus, LLM agent 110 invokes LLM 120 with prompts that tell LLM 120 what the task is and what kind of tools LLM 120 has to accomplish that task, causing LLM 120 to decide which of these tools to use and what the relevant parameters are for invoking those tools. LLM 120 may generate a decision on which tools to use, but it is LLM agent 110 that executes the tools based on this decision by LLM 120. LLM 120 can only generate text and cannot directly interact with the tools.

[0038] LLM agent 110 runs the tool, retrieves the tool results, adds the results to the prompt, and calls LLM120 again, i.e., lets LLM make a decision, and iteratively invokes any additional tools that may be needed until LLM120's decision becomes FINAL_ANSWER, which means the process can be terminated.

[0039] As stated above, the LLM agent 110 needs to prepare a prompt before calling the LLM 120. According to an exemplary embodiment, the LLM agent 110 is an artificial intelligence (AI) agent, which means a system or program capable of autonomously performing tasks on behalf of a user or another system by designing its own workflow and utilizing available tools. An AI agent may encompass a wide range of functionalities beyond natural language processing, including decision-making, problem-solving, interacting with the external environment, and performing actions. These AI agents can be deployed in a variety of applications to solve complex tasks in various enterprise contexts, from software design and IT automation to code generation tools and conversational aids. The AI ​​agent uses advanced natural language processing techniques of a Large-Scale Language Model (LLM) to step-by-step understand and respond to user input and determine when to call external tools.

[0040] AI agents may have memory that enables them to remember past interactions and decisions. They use tool calls to a backend to retrieve the latest information, optimize workflows, and autonomously create subtasks to achieve complex goals.

[0041] In this process, the autonomous AI agent, i.e., LLM agent 110, learns to adapt to the user's expectations over time. The LLM agent's ability to remember past interactions in memory and plan future actions facilitates personalized experiences and comprehensive responses. This tool can be invoked without human intervention, expanding the possibilities for real-world applications of these AI systems. The approach taken by the AI ​​agent when achieving user-set goals consists of these three stages: (1) goal initialization and planning, (2) reasoning using available tools, and (3) learning and reflection. While AI agents are autonomous in their own decision-making processes, they require human-defined goals and environments.

[0042] AI agents improve the accuracy of their responses by using feedback mechanisms such as human-in-the-loop (HITL) and other AI agents. AI agents are sometimes referred to as LLM agents, such as LLM agent 110, and in some cases implement a Reasoning and Action (ReAct) architecture as a form of thought chain prompting. AI agents include goal-based agents, utility-based agents, and learning agents.

[0043] LLM agent 110 uses tool-related prompts to prompt LLM 120 to execute its decision and ultimately obtain the final result. Tool-related prompts provide a description of a list of available tools. For each tool, the tool-related prompt has a similar structure, including the tool name, a description of the tool, the required parameters of the tool, and descriptions of each parameter of the tool. Such tool-related prompts may list only a few available tools, or they may list hundreds or thousands of tools that LLM 120 can utilize to perform a given task; therefore, it should be understood that the specifications of the available tools within a tool-related prompt may constitute the majority of the prompt.

[0044] An exemplary embodiment provides a mechanism for compressing the specifications of tools within such tool-related prompts. This compression may be performed when a tool is created or updated so that the LLM agent 110 can generate tool-related prompts for LLM 120 that use the compressed representation of the tool within the tool-related prompts. That is, the source of the tool-related prompts is the description of each tool and the integration of this description into a template for tool-related prompts generated by the LLM agent 110. In most cases, these descriptions are loaded whenever the agent loads tool descriptions from the tool pool 130, and as a result, they only need to be compressed when tools 132-138 are created or updated within the tool pool 130. However, in some cases, tool prompts may be compressed at a stage when the LLM agent 110 calls LLM 120, for example, when it is not possible to perform updates or access to source tools within the tool pool 130.

[0045] Furthermore, these tool-related prompts are simply referred to herein as “tool prompts.” An example of a tool prompt is shown in Figure 1B. As shown in Figure 1B, three different tool prompts are identified, which are part of the overall task prompts entered into the LLM. The three tool prompts include the “create_ppt_from_content” tool, the “create_file_on_box” tool, and the “create_box_collaboration” tool, which correspond to function descriptions, parameter descriptions, and return descriptions. For example, with respect to the “create_ppt_from_content” tool 140, the function tool declaration 150 includes the function tool name and parameters passed to the function tool. The function tool description 160 specifies that the function should be used to dynamically create a PPT from given content, and if the creation is successful, the file should be uploaded to Box and a Box link should be returned, where “Box” refers to the IBM Box service, a secure cloud-based file sharing service for managing, operating, and collaborating on content. The parameter description 160 provides a list of parameters used by the function tool, specifying what these parameters are and other properties of the parameters, such as whether the parameters are optional. The return description 170 specifies what the function tool returns as a result of its operation.

[0046] To boost development efficiency, some LLM frameworks offer the ability to automatically generate function tool prompts based on the function definitions within the function tool's code. Whether automatically generated by the LLM or manually created by a human, function tool prompts often contain a significant amount of redundant information, as can be seen from the simplified example in Figure 1B. This redundancy grows as the number of function tools specified within the function tool prompt increases. This becomes a significant problem for LLM agents as their functionality becomes more complex and as more function tools are included within the LLM agent, resulting in larger tool prompts. The larger the tool prompt, i.e., the more prompt tokens used within it, the greater the computational cost for the LLM that must process the tool prompt. Furthermore, LLMs have context token limitations, which are consumed by redundant information, making it impossible to add additional function tools to the tool prompt.

[0047] General compression techniques for compressing text cannot be applied to the descriptions of functional tools in the tool prompts of LLM agents. Functional tools in LLM agents are often custom functions with a relatively large amount of self-information, and as a result, general text compression effects are insufficient, i.e., not much of the text is compressed. Self-information is used to evaluate the amount of information contained in a portion of text. For example, in the text content "I am from Beijing, the capital of China," the phrase "I am from Beijing" has high self-information, while "capital of China" has low self-information because it is known to LLM from its training that Beijing is the capital of China. In the case of tool prompts, tool definitions have high self-information within the context of LLM because they use several abbreviations, jargon, and customized or self-defined words. However, some parts of the tool prompt are expansions of other parts of the tool prompt so that they have low self-information relative to other parts of the tool prompt. Therefore, self-information is a measure of how much new information is present in corresponding parts of a text. High self-information indicates that the text contains significant new information, while low self-information indicates that there is little to no significant new information in the text.

[0048] Furthermore, the description of a functional tool in the tool prompt adheres to specific specifications, such as the requirements for function identification, function description, parameter description, and return description mentioned above. Interrelationships exist between these parts 140-170 of the functional tool description 100, and as a result, common text compression methods do not work well because they ignore the internal relationships between these parts within the functional tool description in the tool prompt.

[0049] Therefore, it would be beneficial if the descriptions of functional tools, for example, portions 140-170 in Figure 1B concerning each functional tool within the tool prompt, could be efficiently compressed by automatically identifying and reducing redundant or unnecessary information. This would allow more meaningful information to be communicated to the LLM in a compact manner, taking into account the LLM's content token limits. Furthermore, this would reduce the computational cost of the LLM agent for LLM calls using the tool prompt. Moreover, this would allow for the development of more complex LLM agents by providing additional space within the tool prompt for specifications of additional functional tools that can be utilized by the LLM agent to increase functionality without exceeding the LLM's token limits.

[0050] Exemplary embodiments provide computing tools and computing tool behaviors / functions for identifying unnecessary or redundant information in the description of functional tools in an LLM agent's tool prompt. Exemplary embodiments implement vector space modeling of an LLM agent tool prompt, which includes dividing the prompt into semantically independent text chunks and encoding each chunk into a corresponding vector representation by an LLM encoder. Exemplary embodiments further implement a significance assessor for the text chunks, which evaluates the significance of the text chunks by differential analysis of the data distribution based on vector encoding. Furthermore, exemplary embodiments further implement a compressed tool prompt generator, which constructs an association tree based on the results of the significance assessment from the significance assessor and then operates to eliminate paths with relatively low semantic information while retaining paths with high semantic information nodes. This results in a “pruned” association tree in which some nodes of the tree are excluded or no longer considered. The compressed tool prompt generator then traverses the resulting pruned association tree, reorganizes the pruned association tree, and generates a new compressed tool prompt.

[0051] Figure 2 is an exemplary diagram illustrating one of the LLM agent tool prompts shown in Figure 1B, and a schematic diagram of the compression process performed according to one exemplary embodiment. As shown in Figure 2, the exemplary LLM agent tool prompt 100 is the same as in Figure 1B with respect to the "create_ppt_from_content" function tool. As shown in Figure 2, the LLM agent tool prompt 100 is separated into text chunks corresponding to semantically significant segments or blocks, e.g., lines of function descriptions and parts of parameter definitions, to generate an association tree 210, where the nodes of the association tree 210 correspond to these various text chunks. The nodes of the paths in the association tree 210 are evaluated or scored based on one or more algorithms that determine similarity / dissimilarity between nodes to identify which nodes and corresponding paths have low self-information and which have high self-information.

[0052] Nodes and paths determined to have low self-information are excluded or removed from further consideration, which may result in a pruned association tree 220 from which these nodes and paths have been removed. The pruned association tree 220 can then be traversed to generate a new compressed functional tool description 230, in which the low self-information portions are not present. The new compressed LLM agent tool prompt 230 has fewer context tokens than the original LLM agent tool prompt 100. Therefore, the compressed prompt 230 has fewer context tokens that need to be processed by the LLM and does not consume the same amount of the LLM's context token limit.

[0053] Before continuing our discussion of exemplary embodiments and various aspects of the improved computer operation performed by these exemplary embodiments, it should first be understood that throughout this specification, the term “mechanism” will be used to refer to elements of the invention that perform various operations, functions, and similar ones. Where the term is used herein, a “mechanism” may be an implementation of a function or aspect of an exemplary embodiment in the form of an apparatus, procedure, or computer program product. In the case of a procedure, the procedure is implemented by one or more devices, apparatus, computers, data processing systems, or the like. In the case of a computer program product, logic represented by computer code or instructions embodied within or on the computer program product is executed by one or more hardware devices in order to implement a function associated with a particular “mechanism” or to perform an operation associated with a particular “mechanism.” Accordingly, the mechanisms described herein can be implemented as special hardware, software that runs on the hardware and thereby configures the hardware to implement special functions of the present invention that the hardware cannot perform in any other way, software instructions stored on a medium that are easily executable by the hardware and thereby configure the hardware to perform the functions and specific computer operations described herein, or any combination of the above.

[0054] In this specification and the claims, the terms “one (a),” “at least one of,” and “one or more of” may be used with respect to certain features and elements of exemplary embodiments. It should be understood that these terms and phrases are intended to state that at least one of certain features or elements present in a particular exemplary embodiment is present, but more than one may be present. That is, these terms / phrases are not intended to limit this specification or the claims to the presence of a single feature / element, or to require the presence of multiple such features / elements. On the contrary, these terms / phrases require only at least a single feature / element, and multiple such features / elements may be present within the scope of this specification and the claims.

[0055] Furthermore, the use of the term “engine” as used herein in relation to the description of embodiments and features of the present invention is not intended to limit any particular technical implementation for achieving and / or performing actions, steps, processes, etc. that may be caused by and / or performed by an engine, but it should be understood that “engine” is limited in that it is implemented in computer technology and its actions, steps, processes, etc. are not performed as mental processes or through manual effort, even if the engine may operate in conjunction with manual input or provide output intended for manual or mental consumption. An engine is implemented as one or more of software, dedicated hardware, and / or firmware, or any combination thereof, running on hardware, specifically configured to perform a specified function. Hardware may include, but is not limited to, the use of a processor loaded or stored in machine-readable memory and executed by a processor, thereby combining the processor with appropriate software that specifically configures the processor for a particular purpose, including one or more of the functions of one or more embodiments of the present invention. Furthermore, any names associated with a particular engine are for convenience of reference only, unless otherwise specified, and are not intended to limit any particular implementation. In addition, any function belonging to a particular engine can be performed equally by multiple engines, incorporated into and / or combined with the functions of another engine of the same or different type, or distributed across one or more engines in various configurations.

[0056] In addition, it should be understood that the following description uses several different examples of various elements of the exemplary embodiment to further illustrate exemplary implementations of the exemplary embodiment and to aid in understanding the mechanism of the exemplary embodiment. These examples are intended to be non-limiting and do not exhaust all possible implementations of the mechanism of the exemplary embodiment. In consideration of this specification, it will be apparent to those skilled in the art that with respect to these various elements, there are many other alternative implementations that can be used in addition to or instead of the examples provided herein without departing from the spirit and scope of the invention.

[0057] Various aspects of this disclosure are described by explanatory text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of computer program products (CPPs). With respect to any flowchart, depending on the technology involved, operations may be performed in a different order than those shown in a given flowchart. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated stage, simultaneously, or with at least partial time overlap.

[0058] Embodiments of a computer program product ("CPP Embodiment" or "CPP") are terms used in this disclosure to describe any set of one or more storage media ("mediums") that are collectively included in one or more sets of storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device capable of holding and storing instructions for use by a computer processor. Computer-readable storage media may be, but are not limited to, electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any preferred combination of those described above. Some known types of storage devices, including these media, include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices (such as pits / lands formed on the main surface of punch cards or disks), or any preferred combination of those described above. Computer-readable storage media, as used in this disclosure, shall not be construed as storage in the form of transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses passing through optical fiber cables, electrical signals communicated through wires, and / or other transmission media. As will be understood by those skilled in the art, data is moved at several intermittent points in the normal operation of a storage device, typically during access, defragmentation, or garbage collection; however, data is not transient while it is stored, so the foregoing does not make a storage device transient.

[0059] It should be understood that certain features of the invention described in the context of separate embodiments for clarity may be provided in combination in a single embodiment. Conversely, various features of the invention described in the context of a single embodiment for brevity may be provided separately or in any preferred secondary combination.

[0060] The present invention may be a specially configured computing system, a method implemented by the specially configured computing system, and / or a computer program product comprising hardware and / or software specifically configured to implement the specific mechanisms and functions described herein, the hardware and / or software specifically configured to implement the specific mechanisms and functions described herein. Notwithstanding the description of the computer program product as a system or method, it should be understood that the exemplary embodiments described herein particularly target improved computing tools and methods implemented by such improved computing tools. In particular, the improved computing tool of the exemplary embodiment specifically provides an LLM agent tool prompt compressor. The improved computing tool implements mechanisms and functionalities such as a vector space modeler, a significance assessor, and a compressed tool prompt generator of the LLM agent tool prompt compressor, which cannot actually be performed by a human without or with the help of a technical environment such as a mental process or similar. The improved computing tool offers a practical application of the method, at least in that it can compress the LLM agent's tool prompt by automatically identifying unnecessary or redundant parts of the tool prompt and removing those parts, thereby compressing the description of the functional tools within the tool prompt.

[0061] Figure 3 is an exemplary diagram of a distributed data processing system environment in which an exemplary embodiment of the present invention may be implemented and in which at least a portion of the computer code involved in performing the method of the present invention may be executed. That is, the computing environment 300 includes an example of an environment for executing at least a portion of the computer code involved in performing the method of the invention, such as the LLM agent tool prompt compressor 400. In addition to the LLM agent tool prompt compressor 400, the computing environment 300 includes, for example, a computer 301, a wide area network (WAN) 302, an end user device (EUD) 303, a remote server 304, a public cloud 305, and a private cloud 306. In this embodiment, the computer 301 includes a processor set 310 (including a processing circuit configuration 320 and a cache 321), a communication fabric 311, volatile memory 312, persistent storage 313 (including an operating system 322 and the LLM agent tool prompt compressor 400 identified above), a peripheral device set 314 (including a user interface (UI), a device set 323, storage 324, and an Internet of Things (IoT) sensor set 325), and a network module 315. The remote server 304 includes a remote database 330. The public cloud 305 includes a gateway 340, a cloud orchestration module 341, a host physical machine set 342, a virtual machine set 343, and a container set 344.

[0062] Computer 301 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device currently known or to be developed in the future that is capable of running programs, accessing networks, or querying databases such as remote database 330. As is well understood in the field of computer technology, and depending on the technology, the execution of a computer implementation method may be distributed among multiple computers and / or across multiple locations. On the other hand, in this presentation of the computing environment 300, in order to keep the presentation as concise as possible, the detailed considerations focus on a single computer, specifically computer 301. Although computer 301 is not shown in the cloud in Figure 3, it may be located in the cloud. On the other hand, it is not necessary for computer 301 to be located in the cloud, except to any extent that can be definitively shown.

[0063] The processor set 310 includes one or more computer processors of any type currently known or to be developed in the future. The processing circuit configuration 320 may be distributed across multiple packages, for example, multiple interconnected integrated circuit chips. The processing circuit configuration 320 may implement multiple processor threads and / or multiple processor cores. The cache 321 is memory located within the processor chip package and is typically used for data or code that should be available for high-speed access by threads or cores running on the processor set 310. The cache memory is typically organized into multiple levels, depending on its relative proximity to the processing circuit configuration. Alternatively, some or all of the cache for the processor set may be located "off-chip". In some computing environments, the processor set 310 may operate with qubits and be designed to perform quantum computing.

[0064] Computer-readable program instructions are typically loaded onto computer 301, causing the processor set 310 of computer 301 to execute a series of operational steps, thereby realizing a computer implementation method. As a result, the instructions thus executed instantiate the methods specified in the flowcharts and / or descriptions of the computer implementation methods contained herein (collectively referred to as the "Methods of the Invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as the cache 321 and other storage media considered below. The program instructions and associated data are accessed by the processor set 310 to control and direct the execution of the Methods of the Invention. In the computing environment 300, at least some of the instructions for executing the Methods of the Invention may be stored in the LLM agent tool prompt compressor 400 in persistent storage 313.

[0065] The communication fabric 311 is a signal conduction path that enables various components of the computer 301 to communicate with one another. Typically, this fabric is made up of switches and conductive paths such as buses, bridges, physical input / output ports and similar components. Other types of signal communication paths, such as optical fiber communication paths and / or wireless communication paths, may be used.

[0066] The volatile memory 312 is any type of volatile memory currently known or to be developed in the future. Examples include dynamic random-access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, but this is not required unless explicitly stated. In computer 301, the volatile memory 312 is located in a single package and resides inside computer 301, but alternatively or in addition, the volatile memory can be distributed across multiple packages and / or located externally to computer 301.

[0067] The persistent storage 313 is any form of non-volatile storage for a computer, currently known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether the computer 301 and / or the persistent storage 313 are directly powered. The persistent storage 313 may be read-only memory (ROM), but typically at least a portion of the persistent storage allows for writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 322 may take multiple forms, such as various known proprietary operating systems or open-source Portable Operating System Interface (CSI) type operating systems employing a kernel. The code contained within the LLM agent tool prompt compressor 400 typically includes at least a portion of computer code involved in performing the method of the invention.

[0068] The peripheral device set 314 includes a set of peripheral devices of the computer 301. Data communication connections between the peripheral devices and other components of the computer 301 can be implemented in various ways, including Bluetooth® connections, Near-Field Communication (NFC) connections, connections made by cables (such as Universal Serial Bus (USB) type cables), insertable connections (e.g., Secure Digital (SD) cards), connections made through local area communication networks, and even connections made through wide area networks such as the Internet. In various embodiments, the UI device set 323 may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smartwatches), keyboard, mouse, printer, touchpad, game controller, and haptic devices. Storage 324 is external storage such as an external hard drive, or insertable storage such as an SD card. Storage 324 may be persistent and / or volatile. In some embodiments, storage 324 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 301 is required to have a large amount of storage (for example, computer 301 locally stores and manages a large database), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 325 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.

[0069] The network module 315 is a collection of computer software, hardware, and firmware that enables computer 301 to communicate with other computers via the WAN 302. The network module 315 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for transmission over a communication network, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of the network module 315 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of the network module 315 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the methods of the present invention can typically be downloaded to computer 301 from an external computer or external storage device through a network adapter card or network interface included in the network module 315.

[0070] WAN302 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any currently known or future-developed technology for transmitting computer data. In some embodiments, the WAN may be replaced and / or complemented by a local area network (LAN), such as a Wi-Fi network, designed to transmit data between devices located in a local area. The WAN and / or LAN typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0071] The end-user device (EUD) 303 is any computer system used and controlled by an end-user (e.g., a customer of the company operating computer 301), and may take any of the forms considered above in relation to computer 301. Typically, EUD 303 receives useful and valuable data from the operation of computer 301. For example, in a hypothetical case where computer 301 is designed to provide recommendations to the end-user, these recommendations would typically be communicated from the network module 315 of computer 301 to EUD 303 via WAN 302. In this way, EUD 303 can display or otherwise present the recommendations to the end-user. In some embodiments, EUD 303 may be a client device such as a thin client, heavy client, mainframe computer, or desktop computer.

[0072] The remote server 304 is any computer system that provides at least some data and / or functionality to computer 301. The remote server 304 may be controlled and used by the same entity that operates computer 301. The remote server 304 represents a machine that collects and stores useful and valuable data for use by other computers, such as computer 301. For example, in a hypothetical case where computer 301 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 301 from the remote database 330 of the remote server 304.

[0073] Public Cloud 305 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of the computing resources of Public Cloud 305 is performed by the computer hardware and / or software of the Cloud Orchestration Module 341. The computing resources provided by Public Cloud 305 are typically implemented by virtual computing environments running on various computers that make up the computers of the host physical machine set 342, which is the universe of physical computers in and / or available to Public Cloud 305. Virtual computing environments (VCEs) typically take the form of virtual machines from the virtual machine set 343 and / or containers from the container set 344. These VCEs may be stored as images and are understood to be transferable either as images or after instantiation of VCEs, among and between various physical machine hosts. The cloud orchestration module 341 manages the transfer and storage of images, deploys new VCE instances, and manages active instances of VCE deployments. The gateway 340 is a collection of computer software, hardware, and firmware that enables the public cloud 305 to communicate over the WAN 302.

[0074] Here, we will provide some further explanation about virtualized computing environments (VCEs). A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user-space instances, called containers. These isolated user-space instances typically behave like actual computers in terms of the programs running within them. Computer programs running on a normal operating system can utilize all of that computer's resources, including connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and the devices allocated to the container, which is a feature known as containerization.

[0075] Private Cloud 306 is similar to Public Cloud 305, except that its computing resources are available only for use by a single enterprise. While Private Cloud 306 is shown as communicating with WAN 302, in other embodiments, the private cloud may be completely isolated from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a separate discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple configured clouds. In this embodiment, both Public Cloud 305 and Private Cloud 306 are part of a larger hybrid cloud.

[0076] As shown in Figure 3, one or more computing devices, for example, computer 301 or remote server 304, may be specifically configured to implement the LLM agent tool prompt compressor 400. The configuration of the computing devices may include providing application-specific hardware, firmware, or similar to facilitate the performance of the operations and generation of outputs described herein with respect to exemplary embodiments. The configuration of the computing devices may also, or alternatively, include providing a software application stored in one or more storage devices and loaded into the memory of a computing device such as computer 301 or remote server 304, for causing one or more hardware processors in the computing devices to run a software application that configures the processor to perform the operations and generate outputs described herein with respect to exemplary embodiments. Furthermore, any combination of application-specific hardware, firmware, software applications running on hardware, or similar may be used without departing from the spirit and scope of the exemplary embodiments.

[0077] When a computing device is configured in one of these ways, it should be understood that the computing device becomes a specialized computing device specifically configured to implement the mechanism of the exemplary embodiment, and not a general-purpose computing device. Furthermore, as will be described below, the implementation of the mechanism of the exemplary embodiment provides useful and clear results by improving the functionality of the computing device and promoting the compression of the LLM agent tool prompt by minimizing the amount of unnecessary and / or redundant information within the LLM agent tool prompt, thereby improving tool prompt processing by the LLM agent.

[0078] Figure 4 is an exemplary block diagram illustrating the primary operating components of an LLM agent tool prompt compressor according to one exemplary embodiment. The operating components shown in Figure 4 may be implemented as dedicated computer hardware components, computer software running on computer hardware configured to perform specific computer operations attributable to those components, or any combination of dedicated computer hardware and computer software-configured computer hardware. It should be understood that these operating components perform automatically assigned operations without human intervention, even if the input may be provided by a human, such as a search query, and the resulting output may be useful to a human. The present invention particularly targets automatically operating computer components that aim to improve the way LLM agent tool prompts are formulated in order to improve the efficiency of LLM agent operations by compressing tool prompts and removing unnecessary or redundant information. The exemplary embodiments provide a mechanism for performing tool prompt compression by vector space modeling of tool prompts, importance assessment of text chunks of tool prompts, and pruning logic using association trees, which cannot actually be performed by a human as a mental process and are not intended to organize any human activity.

[0079] As shown in Figure 4, the LLM agent tool prompt compressor 400 includes a vector space modeler 410, an importance assessor 420, and a compressed tool prompt generator 430. In some exemplary embodiments, the LLM agent tool prompt compressor may be part of the same computing system to which one or more LLM agents 440 and / or LLM 450 are provided. In other exemplary embodiments, one or more of components 440 and 450 may be provided on a separate computing system from the other components. For example, LLM 450 may be provided by one or more LLM computing systems 470, and LLM agent 440 may be associated with another computing system 480. In any desired implementation, these components communicate with each other either internally within the same computing system or via one or more data networks 490 in a distributed environment implementation. LLM agent 440 employs LLM450 and performs computational operations according to the logic and tools of LLM agent 440 using appropriately constructed LLM prompts, some of which may include LLM agent tool prompts such as those discussed above.

[0080] Accordingly, according to exemplary embodiments, the LLM agent tool prompt compressor operates based on LLM agent tool prompts developed for LLM agents 440 and LLM 450. These LLM agent tool prompts may be developed by one or more subject experts and / or automated tools associated with the LLM, and each LLM agent tool prompt may identify one or more functional tools available to LLM agents 440 and LLM 450 by functional tool definitions following a predetermined structure, e.g., functional tool identifier, functional tool description, functional tool parameter description, and functional tool response description. This particular structure is used for illustrative purposes only and is not intended to limit the scope. Any preferred predetermined structure may be used without departing from the spirit and scope of the invention.

[0081] The operation of the LLM agent tool prompt compressor can be initiated, for example, in response to the generation and / or modification of an LLM agent tool prompt, or in response to an automated or manually entered request to compress a given LLM agent tool prompt. The original LLM agent tool prompt is input to the LLM agent tool prompt compressor, which is then processed by the vector space modeler 410, importance assessor 420, and compressed tool prompt generator 430, thereby generating a new compressed LLM agent tool prompt 435, which is then stored in the LLM agent tool prompt storage 445 associated with the LLM agent 440 for later use.

[0082] The vector space modeler 410 comprises computer-executable logic that segments the functional tool descriptions of the original LLM agent tool prompt, thereby separating the text of these functional tool descriptions into multiple text chunks. Segmenting the text into text chunks can be implemented using any preferred text parsing and segmenting algorithm 412 configured to specifically handle the functional tool descriptions of the LLM agent tool prompt. For example, the text parsing and segmenting algorithm 412 may be configured to identify specific tags, keywords, phrases, structural elements, and the like that are specific to the functional tool descriptions in the LLM agent tool prompt. For example, recognizing that the structure of the functional tool description has parts 140-170 in Figure 1B, the segmenting algorithm may be configured to identify these different parts based on structure, identifying key terms, e.g., "param" or "return", and the like. Based on this configuration and knowledge, the segmentation algorithm 412 is capable of segmenting the description of such functional tools within the LLM agent tool prompt, as well as identifying other elements of the LLM agent tool prompt, such as the LLM specification persona, instructions to the LLM regarding the functions it should perform, and other information that identifies how the LLM agent should behave, what actions it should perform, and the type of response the LLM should provide, and segmenting them into text chunks.

[0083] According to exemplary embodiments, some of these text chunks are assumed to contain unnecessary or redundant information, which can be removed from the LLM agent tool prompt without affecting the functionality of the original LLM agent tool prompt. To determine which text chunks contain such unnecessary or redundant information, the text chunks are input to an LLM encoder 414 that encodes the text chunks into corresponding semantic vector representations. The semantic vector is a high-dimensional vector model derived from a word-document matrix and is used to determine the semantic similarity between parts of a text based on their contextual relationships in a corpus of text, where the corpus can be considered to be the LLM agent tool prompt. The semantic vector embedding of parts of text is performed using a trained deep learning model trained on a knowledge base to generate semantic vector embeddings that represent the semantic relationships in the text based on the knowledge base. LLM encoding of parts of text into semantic vector representations is generally known in the art, and therefore a more detailed description of this process is not included herein. Such LLM encoding is applied to specific text chunks generated from the segmentation of the LLM agent tool prompt.

[0084] It should be understood that by inputting each text chunk into the LLM encoder 414, multiple semantic vectors are generated, with one semantic vector for each text chunk. These semantic vectors of the text chunks can then be input into a semantic vector comparator 416, which compares the semantic vector representations of the text chunks to determine semantic vector similarity and identify which text chunks contain redundant and unnecessary information, such as information that is semantically similar to other chunks or parts of other chunks, or information that has little semantic value. That is, for example, in some exemplary embodiments, semantic vectors having a sufficiently high semantic vector similarity exceeding a predetermined threshold can be determined to be sufficiently similar with respect to at least one authorization elimination among the text chunks of the compared semantic vector representations.

[0085] In other exemplary embodiments, it is recognized that such vector comparisons of semantic vectors to themselves do not need to take into account all variations in a manner in which text chunks may represent similar concepts within an LLM agent tool prompt. Therefore, to make the identification of unnecessary or redundant parts of an LLM agent tool prompt more robust, the vector space modeler 410 may further include a perturbation network 418 and a clustering module 419 to represent the semantic distribution of the original LLM agent tool prompt. In some embodiments, the clustering module performs clustering on the semantic vectors to produce an output such as a Gaussian Mixture Model (GMM) where the parameters are parametric representations of the original LLM agent tool prompt in semantic space. In some exemplary embodiments, the perturbation network 418, which may be a fully-connected (FC) network with dropout (randomly selected abandonment of neurons, e.g., neuron values ​​being set to 0), for example, randomly perturbs the original vectors in vector space to generate multiple different homogeneous vectors in the semantic vector space. Homogeneous vectors are virtually similar, or share a common structure or space, but may differ slightly while still representing the same type of information or concept. This is the opposite of heterogeneous vectors, which may be fundamentally different, represent different types of information, or originate from different vector spaces. In some exemplary embodiments, portions of a vector are dropped, and the significance of these dropped portions is determined in comparison to the entire meaning of the original semantic vector. If the dropped portions do not significantly alter the meaning of the original semantic vector, they may be determined to be unnecessary.

[0086] Therefore, using an FC network, for example, an FC network applies a linear transformation to the input vector to map the input to another vector space, but without significantly altering the nature or meaning of the input. Dropout introduces randomness by randomly deactivating one or more nodes in the network, which acts as a kind of regularization. This results in a slight change to the output vector, but does not destroy the core information from the input. Dropout adds a small random perturbation to these vectors, but since it is a controlled partial modification (by deactivating random nodes), dropout creates a small deformation. These changes introduce randomness without altering the core meaning of the vector. Thus, the output vector still represents the same semantic meaning, but with a slight change or perturbation. This is because these resulting vectors are considered homogeneous, as they are still virtually essentially the same despite the small random differences.

[0087] After generating various homogeneous vectors, the original LLM agent tool prompt can be transformed into a higher-dimensional embedding in a larger vector distribution. The clustering module 419 acts on this larger vector distribution, i.e., the set of homogeneous vectors generated by the perturbation network 418, so that the semantic distribution of the vectors is mathematically represented. The clustering module 419 receives a set of homogeneous vectors as input and clusters these vectors to produce a model, e.g., a GMM, where the parameters are a parametric representation of the original LLM agent tool prompt in semantic space. Thus, the original LLM agent tool prompt can be quantified by the parameters of the GMM, and the produced GMM serves as a spatial identifier. Subsequently, importance tests can be performed on each text chunk.

[0088] In some exemplary embodiments, in addition to the vector similarity comparison operation described above, the importance assessor 420 may perform testing the importance of text chunks using a larger distribution of vectors from perturbations and semantic distribution quantification of GMM 419. In some exemplary embodiments, testing importance involves sequentially extracting each text chunk from the original LLM agent tool prompt. After the text chunks are extracted, they are modeled using the perturbation network 418 and clustering module 419 to yield another GMM distribution, which is shown as GMM-cmp (compare). At this point, GMM-cmp is compared with the original GMM distribution at the data distribution level, and the degree of difference is recorded. If the result of the comparison between the comparison GMM distribution, GMM-cmp, and the original GMM distribution is minimal at the data distribution level, for example below a given threshold, this indicates that the text chunk is not important in its potential meaning and can be removed to reduce prompt length and complexity.

[0089] In some exemplary embodiments, a hybrid approach can be used to compare the difference between two GMM distributions, for example, the original GMM distribution and the GMM-cmp distribution, at the data distribution level. In this hybrid approach, the KS test is combined with the Jensen-Shannon divergence. This method comprehensively captures both similarity and dissimilarity between two GMM distributions. The KS test method uses the Kolomogorov-Smirnov test (KS test) to measure the maximum difference in the cumulative distribution function (CDF) between the two GMM distributions. The KS test statistic determines whether the two distributions are derived from the same population. A smaller statistic suggests greater similarity between the distributions.

[0090] The Jensen-Shannon divergence measures the similarity between two probability distributions. The Jensen-Shannon divergence is the mean of the Kullback-Leibler divergence, taking into account the relative entropy between the two distributions. A smaller Jensen-Shannon divergence indicates a greater similarity between the distributions.

[0091] Therefore, the KS test measures the maximum difference between two paths (distributions). The Jensen-Shannon divergence measures how similar or different two maps (distributions) are. Using the KS test, the Kolmogorov-Smirnov test (KS test) is first used to measure the maximum difference in the cumulative distribution function (CDF) between two GMM distributions. The KS test statistic helps determine whether the two distributions are derived from the same population. A smaller statistic indicates greater similarity between the distributions. Next, the Jensen-Shannon divergence is used to measure the similarity between two probability distributions. The Jensen-Shannon divergence is the mean of the Kullback-Leibler divergence, taking into account the relative entropy between the two distributions. A smaller Jensen-Shannon divergence indicates greater similarity between the distributions.

[0092] By combining these two metrics, the KS test and the Jensen-Shannon divergence, a comprehensive dissimilarity index can be defined to quantify the difference between two distributions. For example, a weighted mean of the KS test statistic and the Jensen-Shannon divergence provides a more complete reflection of the difference between two GMM distributions at the data distribution level. It should be understood that the KS test and the Jensen-Shannon divergence are merely examples of mechanisms for comparing two distributions to determine their similarity / dissimilarity, and the present invention is not limited to such examples. Rather, other mechanisms that may be used exist, such as the Earth-Moving Distance, the Chi-Squared Test, the Kullback-Leibler divergence, and similar ones, which may be used in addition to or instead of one or more of the KS test and the Jensen-Shannon divergence without departing from the spirit and scope of the present invention.

[0093] A pruned association tree data structure is generated by a compressed tool prompt generator 430 based on the results of a comparison of vectors and / or distributions performed by a semantic vector comparator 416, which essentially scores each path between nodes in the tree data structure. The pruned association tree data structure contains a tree data structure in which nodes correspond to text chunks of the original LLM agent tool prompt, but some nodes and paths have been excluded. That is, when generating the pruned association tree data structure, nodes and / or paths with low relative semantic information, or low self-information, are pruned from the complete association tree data structure. Paths with high semantic information, or high self-information nodes, are retained. Paths with both low and high semantic information nodes are retained in the final association tree data structure. Whether a path from one node to another has "low" or "high" self-information can be determined based on a comparison of predetermined rules and thresholds and scoring performed on these rules and thresholds by a semantic vector comparator 416. For example, a score greater than 0.8 indicates low self-information (high similarity), while a score less than 0.5 indicates high self-information (dissimilar nodes), or vice versa if different scoring is used. These threshold values ​​and rules are implementation-specific and can be set to any suitable values ​​or rules without departing from the spirit and scope of the present invention. The resulting final association tree data structure has the remaining paths interconnected within the association tree data structure.

[0094] Once the final association tree data structure is generated, the compressed tool prompt generator 430 performs a depth-first traversal of the tree to reconstruct node semantic segments or text chunks to generate a new compressed LLM agent tool prompt 435. Since low semantic information paths and nodes are excluded from the association tree data structure, the new compressed LLM agent tool prompt 435 does not include the text chunks or semantic segments of these excluded nodes and paths. Therefore, the new compressed LLM agent tool prompt 435 contains only the parts of the original LLM agent tool prompt that are not unnecessary or redundant.

[0095] As described above, the exemplary embodiments implement vector space modeling by the vector space modeler 410 of the LLM agent tool prompt compressor 400. Figure 5 is an exemplary figure illustrating the operation for performing vector space modeling of a tool prompt by one or more exemplary embodiments. Given the original LLM agent tool prompt, as shown in block 510, the prompt is segmented into text chunks, e.g., text chunks 1 to N in Figure 5. Each text chunk is input to the LLM encoder 512, which generates a semantic vector 514 for that text chunk. In some embodiments, an embedding layer of a machine learning model is used instead of the LLM encoder 512 to generate the respective semantic vectors 514. As described above, in some exemplary embodiments, these semantic vectors 514 can be compared using a vector similarity metric so that semantic vectors that are similar enough to indicate that one corresponding text chunk is unnecessary or redundant with respect to the other text chunk can be identified.

[0096] In another exemplary embodiment, as shown in block 520, each semantic vector 514 generated by the LLM encoder 512 may be submitted to a perturbation network 522, generating a plurality of semantic vectors 524. This set of semantic vectors 524 may then be submitted to a clustering module 532, as shown in block 530, to generate a semantic distribution 534 of the original LLM agent tool prompt. A similar technique may be used to generate a GMM-cmp distribution after the original LLM agent tool prompt has been perturbed and then submitted to blocks 510-530. An importance assessment may then be performed on these distributions. This may be done with respect to each text chunk within the plurality of text chunks generated from the original LLM agent tool prompt.

[0097] Figure 6 provides an exemplary diagram illustrating the operation for performing a text chunk importance assessment according to one exemplary embodiment. As shown in Figure 6, the original LLM agent tool prompt is the basis for generating a set of text chunks 1-N 610, and semantic vector representations of these text chunks are generated. The operations corresponding to blocks 510-530 are performed on the semantic vectors to generate a GMM distribution 620. Similarly, perturbations of the text chunk semantic vectors are performed, for example, by removing the semantic vector corresponding to text chunk 2, and the modified set of semantic vectors is submitted back to the process of blocks 510-530 to generate a GMM-cmp distribution 630. These two distributions are then compared at the data distribution level (640) so that the similarity between the distributions can be determined as described above. Based on this comparison 640, the text chunks corresponding to the perturbations, e.g., the text chunk of the removed semantic vector, are either retained when constructing the related tree data structure or excluded from further consideration (650). This can be done for each text chunk in the original LLM agent tool prompt to determine which text chunks should be retained and which should not be retained when generating the final associated tree data structure. Therefore, if one of the text chunks is removed in this embodiment, the remaining text chunks are used to generate the tree data structure and, ultimately, the compressed tool prompt.

[0098] Figure 7 presents a flowchart outlining exemplary operation of elements of the present invention relating to one or more exemplary embodiments. It should be understood that the operations outlined in Figure 7 are performed automatically by improved computer tools of the exemplary embodiments and are not intended, and indeed impossible, to be performed by humans, either as mental processes or by organizing human activities. On the contrary, humans may, in some cases, initiate the execution of the operations described in Figure 7, and in some cases utilize the results produced as a consequence of the operations described in Figure 7, but the operations in Figure 7 themselves are performed in an automated manner, particularly by improved computing tools.

[0099] Figure 7 is a flowchart outlining exemplary operation for compressing an LLM agent tool prompt by one exemplary embodiment. As shown in Figure 7, the operation begins by receiving an LLM agent tool prompt for compression (step 702). The LLM agent tool prompt is segmented into multiple text chunks (step 704), and each text chunk is encoded into a semantic vector (step 706). Each semantic vector is submitted to a perturbation network to generate a set of vectors perturbed with respect to each semantic vector (step 708). The set of perturbed vectors is input to a GMM for generating a data distribution (step 710). Next, the set of text chunks of the original LLM agent tool prompt may be perturbed (step 712), and the process is repeated to generate a comparison distribution (GMM-cmp) (step 714). The two distributions may then be compared to determine the level of similarity between the distributions (step 716). Based on the similarity of the distributions, perturbed text chunks are identified for either retention or discarding during the generation of the related tree data structure (step 718). Steps 712–718 may be performed for each text chunk in the original LLM agent tool prompt set to evaluate the similarity distribution for each perturbation.

[0100] Next, the association tree data structure is generated based on the identification of distribution similarity in step 718, or pruned if it has already been generated from a complete set of text chunks (step 720). For nodes (text chunks) determined to be highly similar, for example, having low semantic importance, from the distribution similarity determination, paths containing such nodes are excluded from the association tree data structure, while paths containing nodes with high semantic importance, as determined from the distribution similarity determination, are retained (step 722). The resulting association tree data structure is then traversed in a depth-first manner to generate a new LLM agent tool prompt (step 724). The operation then terminates.

[0101] The description of the present invention is presented for illustrative and explanatory purposes, but is not intended to be comprehensive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The embodiments are selected and described to best illustrate the principles and practical applications of the invention and to enable those skilled in the art to understand the invention with respect to various embodiments with various modifications suitable for the particular use intended. The terminology used herein has been selected to best illustrate the principles, practical applications, or technical improvements to the technology available on the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer implementation method, The initial tool prompt for the generative machine learning model has been received; The next step is to segment the original tool prompt into multiple text chunks; A step of generating at least one semantic vector representation of the plurality of text chunks; A step of generating a first semantic distribution based on the at least one semantic vector representation; A step of generating a perturbed semantic vector representation based on a subset of the plurality of text chunks, wherein the subset is generated by excluding at least one text chunk from the plurality of text chunks; A step of generating a second semantic distribution based on a semantic vector representation given the aforementioned perturbation; A step of performing a comparison of the first semantic distribution and the second semantic distribution to generate at least one similarity metric; and A step in which, in response to at least one similarity metric exceeding a threshold similarity value, a compressed tool prompt is generated based on the subset of the plurality of text chunks. A method that includes [a certain feature].

2. The method according to claim 1, further comprising the step of storing the compressed tool prompt in data storage accessible to an artificial intelligence agent communicating with the generative machine learning model.

3. The step of adding the compressed tool prompt to the task prompt; The step of inputting the aforementioned task prompt into the machine learning model; and In response to the input step, a step of receiving task output from the generative machine learning model. The method according to claim 1, further comprising:

4. The aforementioned original tool prompt is useful for defining function tools, as follows: A function declaration that identifies the identifier of the aforementioned function tool, A function description that describes what the aforementioned function tool does, A parameter description that describes the parameters used by the aforementioned function tool, and A return description that describes the type of output provided by the function tool in response to the function tool being invoked. The method according to claim 1, having one or more defining elements selected from the group consisting of the following.

5. The method according to claim 4, wherein the original tool prompt has a first number of defining elements, and the compressed tool prompt has a second number of defining elements, the second number being less than the first number.

6. The method according to claim 1, further comprising the step of generating an association tree data structure based on the plurality of text chunks and at least one similarity metric, wherein the at least one similarity metric has a plurality of similarity metrics, and the connections between nodes of the association tree data structure have corresponding similarity metrics from the plurality of similarity metrics, and the connections identify similarities between nodes connected by the corresponding connections.

7. The method according to claim 6, wherein the step of generating the compressed tool prompts comprises pruning the association tree data structure by removing only nodes and paths whose corresponding similarity metrics satisfy a predetermined criterion, thereby generating a pruned association tree data structure.

8. The method according to claim 7, wherein the step of generating the compressed tool prompt comprises traversing the pruned associated tree data structure to reconstruct a tool prompt containing smaller text content than the original tool prompt.

9. The method according to claim 1, wherein the at least one similarity metric is generated by performing at least one of a first algorithm for measuring the maximum difference between the first semantic distribution and the second semantic distribution, and a second algorithm for measuring how much the first semantic distribution and the second semantic distribution are similar or different.

10. The method according to claim 9, wherein the first algorithm is a K-S test algorithm and the second algorithm is a Jensen-Shannon divergence algorithm.

11. The method according to claim 1, wherein the step of segmenting the original tool prompt into multiple text chunks comprises the steps of parsing the original tool prompt and generating text chunks based on the identification of at least one of tags, keywords, expressions, or structural elements specific to the description of a functional tool within the tool prompt.

12. The method according to claim 1, wherein the step of generating the first semantic distribution comprises processing the at least one semantic vector representation by a Gaussian mixture model (GMM), and the step of generating the second semantic distribution comprises processing the semantic vector representation perturbed by the GMM.

13. The method according to claim 1, wherein the step of generating at least one semantic vector representation of the plurality of text chunks includes the step of generating a separate semantic vector representation for each text chunk in the plurality of text chunks, and the step of generating a perturbed semantic vector representation includes the step of generating a separate perturbed semantic vector representation for each text chunk in the plurality of text chunks, excluding the at least one excluded text chunk.

14. To the processor set: Steps to receive the original tool prompt for a generative machine learning model; The procedure for segmenting the aforementioned original tool prompt into multiple text chunks; A procedure for generating at least one semantic vector representation of the plurality of text chunks; A procedure for generating a first semantic distribution based on the aforementioned at least one semantic vector representation; A procedure for generating a perturbed semantic vector representation based on a subset of the plurality of text chunks, wherein the subset is generated by excluding at least one text chunk from the plurality of text chunks; A procedure for generating a second semantic distribution based on a semantic vector representation given the aforementioned perturbation; A procedure for performing a comparison of the first semantic distribution and the second semantic distribution to generate at least one similarity metric; and A procedure for generating a compressed tool prompt based on the subset of the plurality of text chunks in response to at least one similarity metric exceeding a threshold similarity value. A computer program designed to execute something.

15. The computer program according to claim 14, further comprising the step of causing the processor set to store the compressed tool prompt in data storage accessible to an artificial intelligence agent communicating with the generative machine learning model.

16. To the aforementioned processor set: Steps to add the compressed tool prompt to the Task Prompt; The procedure for inputting the aforementioned task prompt into the machine learning model; and A procedure for receiving task output from the generative machine learning model in response to the input procedure. The computer program according to claim 14, which further executes the following.

17. The aforementioned original tool prompt is useful for defining function tools, as follows: A function declaration that identifies the identifier of the aforementioned function tool, A function description that describes what the aforementioned function tool does, A parameter description that describes the parameters used by the aforementioned function tool, and A return description that describes the type of output provided by the function tool in response to the function tool being invoked. The computer program according to claim 14, having one or more defining elements selected from the group consisting of the following.

18. The computer program according to claim 17, wherein the original tool prompt has a first number of the defining elements, and the compressed tool prompt has a second number of the defining elements, the second number being less than the first number.

19. The computer program according to claim 14, wherein the processor set further performs a procedure for generating an association tree data structure based on the plurality of text chunks and at least one similarity metric, the at least one similarity metric having a plurality of similarity metrics, and the connections between nodes of the association tree data structure having a corresponding similarity metric among the plurality of similarity metrics, and the connections identify the similarity between nodes connected by the corresponding connections.

20. Processor set; One or more computer-readable storage media; and Program instructions stored on one or more computer-readable storage media, wherein the processor set: Steps to receive the original tool prompt for a generative machine learning model; The procedure for segmenting the aforementioned original tool prompt into multiple text chunks; A procedure for generating at least one semantic vector representation of the plurality of text chunks; A procedure for generating a first semantic distribution based on the aforementioned at least one semantic vector representation; A procedure for generating a perturbed semantic vector representation based on a subset of the plurality of text chunks, wherein the subset is generated by excluding at least one text chunk from the plurality of text chunks; A procedure for generating a second semantic distribution based on a semantic vector representation given the aforementioned perturbation; A procedure for performing a comparison of the first semantic distribution and the second semantic distribution to generate at least one similarity metric; and A procedure for generating a compressed tool prompt based on the subset of the plurality of text chunks in response to at least one similarity metric exceeding a threshold similarity value. Program instructions for causing an operation to be performed A computer system equipped with the following features.

Images