Method and system for extreme optimization of large model agent output unit

Abstract

The invention provides a method and system for extreme optimization of an intelligent agent output unit of a large model, and the method comprises the steps: compressing the content generated by the large model through a compression algorithm, and directly generating a compressed code; optimizing the number of tokens output by the large model, and performing customized coding on the tokens, including tool name and parameter name compression, JSON structure compression and special symbol conjunction compression; the method focuses on data in a specific field, finely adjusts the large model, trains the large model to identify the data in the specific field, and generates a more concise coding mode. According to the method, the large model tool calling operation speed is remarkably increased; the simplified Token output reduces the processing time, and is particularly suitable for being applied to a system requiring low time delay; an optimized character string processing mode enables tool calling to be quicker and more accurate, and various problems caused by too long analysis time are reduced; the operation efficiency is improved by improving the operation efficiency, and the user experience is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of large-model intelligent agent technology; specifically, to a method and system for extreme optimization of a large-model intelligent agent output unit. Background Art

[0002] With the rapid evolution of artificial intelligence technology, large-scale model applications in fields such as natural language processing, machine translation, and intelligent question answering have received widespread attention and use.

[0003] However, in these large model applications, large model tool calling methods face significant efficiency challenges, especially when low-latency response is required.

[0004] Advances in natural language processing (NLP) technology have enabled large models to perform various language tasks with higher accuracy and speed. These models have demonstrated powerful capabilities in understanding and generating language through large-scale pre-training and fine-tuning.

[0005] However, in traditional methods, in tasks where intelligent agents use tools, large models output tool lists in a specific form (jsonized tool generation), wait for the tools to be generated, and then process these tools, which greatly increases the response delay. When this method is applied in systems that require low latency, it directly leads to a significant reduction in user experience.

[0006] In the prior art, large models usually use JSON format ({'name': 'device_control', 'parameters': {'device': 'Air conditioner', 'command': 'Close'}}) to output tool lists. Although JSON format has the advantages of clear structure and easy parsing, and performs well in simple tasks, it will cause parsing delays and reduced processing efficiency in complex tasks. The redundant information in JSON format and the direct output of the original function name further increase the number of tokens, which also increases the response time. The redundant tokens brought by the direct output of the original function name using JSON format significantly increase the parsing and processing time, and reduce system performance.

[0007] Through in-depth analysis of the technical problems of existing technologies, it is found that the performance bottlenecks of current large models at high token output rates are mainly concentrated in two aspects: one is the redundant information processing in JSON format, and the other is the excessive number of tokens caused by the direct output of the original function name.

[0008] Therefore, how to reduce the number of tokens and optimize the output format to improve system operation efficiency and user experience has become a difficult problem that needs to be solved urgently. Summary of the invention

[0009] In view of this, the purpose of the present invention is to design a new large model tool calling method and system, which reduces the number of Token outputs by improving the output format of the large model tool call, effectively reduces the number of Tokens by encoding the original function name and removing redundant JSON format, streamlines the output Tokens, directly processes the output format, optimizes the tool information output format, and reduces the output time of each tool to achieve the purpose of rapid response, solves the problem of inefficiency caused by excessive Token output in the current large model tool calling system, and is suitable for application in systems requiring low latency; by reducing the number of output units and optimizing the output format, the running speed and overall running efficiency of the large model are significantly improved, and a better user experience can also be brought.

[0010] The present invention provides a method for optimizing the output unit of a large model intelligent agent, comprising the following steps: S1. Use a compression algorithm to compress the content generated by the large model and directly generate compressed code; S2. Optimize the number of tokens output by large models and perform customized encoding on tokens, including: compression of tool names and parameter names, compression of JSON structures, and compression of special symbol connectors; The present invention makes the word segmentation and generation process more efficient by defining the unique tokens in the specific field through a custom tokenization tool (such as Byte-Pair Encoding, BPE method); S3. Focus on data in a specific field, fine-tune the large model (GPT, etc.), train the large model to recognize data in that specific field, and generate a more concise encoding method to meet the compression requirements of steps S1 and S2. Improve processing efficiency.

[0011] The present invention further reduces the number of tokens and response time through multi-level compression and optimization combined with encoding and training.

[0012] Specifically, the multi-level compression includes the following levels: compression algorithm level, vocabulary level, structure level, symbol level, and model level; The compression algorithm layer uses compression algorithms such as gzip and Brotli to compress the content generated by the language model as a whole; The vocabulary layer is used to customize field name compression and reduce the overhead caused by long field names; The structural layer is used to optimize structures such as JSON and reduce format redundancy; The symbol layer reduces the number of tokens by using compression and representation of special symbols; The model layer optimizes the generation method by fine-tuning the model to adapt to the compression requirements of the above levels.

[0013] Furthermore, the method of fine-tuning the large model in step S3 comprises the following steps: S31. Collect and prepare a large amount of data in the specific field, including abbreviations, coding methods, and sentence structures specific to the specific field; S32. Use the collected data to fine-tune the large model so that it can understand and use the concise encoding method in this specific field.

[0014] Furthermore, the method for compressing the tool name and parameter name in step S2 includes: establishing short codes for tool names and parameters to reduce the overhead caused by long field names; for example, `getWeather` can be abbreviated as `GW`.

[0015] An example is: `getWeather` ->`GW` An example is: `getDeviceStatus` ->`GDS` Furthermore, the method for compressing the JSON structure in step S2 includes: encoding field names and common values ​​to reduce redundant information in the JSON structure and make the JSON structure more concise; Example: `{'name': 'device_control', 'parameters': {'device': 'Air Conditioner', 'command': 'Off'}}` -> `DC|d=Air Conditioner,c=Off` An example is: `{'status': 'active', 'data': [123, 456, 789]}` ->`A|D[123,456,789]`.

[0016] Furthermore, the special symbol connector compression in step S2 includes: optimizing special symbols (such as equal signs and separators) so that the special symbols become single tokens in the domain, thereby reducing the number of tokens; preferably, the effect can be further enhanced by fine-tuning the training model; For example: `DC|d=Air Conditioner,c=Off` -> `DC|is a token`.

[0017] Further, the compression algorithm of step S1 includes: gzip compression algorithm, Brotli compression algorithm or Lempel-Ziv-Welch (LZW) compression algorithm; For example, the generated content is compressed using a compression algorithm such as gzip, Lempel-Ziv-Welch (LZW), or Brotli.

[0018] The gzip compression algorithm is a popular compression algorithm that is widely used to reduce file size.

[0019] Brotli compression algorithm is a compression algorithm developed by Google, which has a higher compression ratio and faster decompression speed.

[0020] The present invention also provides a system for optimizing the output unit of a large model intelligent agent, and executes the method for optimizing the output unit of a large model intelligent agent as described above, including: Compression coding module: used to compress the content generated by the large model using a compression algorithm and directly generate compressed codes; Vertical field encoding module: used to optimize the number of tokens output by large models and perform customized encoding of tokens, including: tool name and parameter name compression, JSON structure compression, and special symbol connector compression; Fine-tuning training module: used to focus on data in a specific field, fine-tune the large model, train the large model to recognize data in that specific field, and generate a more concise encoding method to meet the compression requirements of steps S1 and S2.

[0021] Furthermore, the fine-tuning training module includes: Fine-tuning training data preparation unit: used to collect and prepare a large amount of data in the specific field, including abbreviations, encoding methods, and sentence structures specific to the specific field; Fine-tuning model training unit: used to fine-tune the large model using the collected data, so that the large model can understand and use the concise encoding method in the specific field.

[0022] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for extreme optimization of the output unit of a large model intelligent agent as described above.

[0023] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for extreme optimization of the output unit of a large model intelligent agent as described above are implemented.

[0024] Compared with the prior art, the present invention has the following beneficial effects: The method and system for extreme optimization of the large model intelligent body output unit provided by the present invention significantly improves the running speed of large model tool calls by reducing the number of tokens and optimizing string processing; the streamlined Token output reduces the processing time, improves the processing efficiency, and significantly improves the overall efficiency of the system, which is particularly suitable for application in systems requiring low latency; the optimized string processing method makes tool calls faster and more accurate, enhances the accuracy of tool calls, and reduces various problems caused by long parsing time; the improvement in running efficiency also improves operational efficiency, and users have faster response speeds and smoother interactive experiences in actual operations, which significantly improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0026] In the attached picture: Figure 1 A schematic diagram of a process of controlling a smart home system device through natural language commands by a user of an application example of the present invention; Figure 2 A flowchart of a method for extreme optimization of the output unit of a large model intelligent agent according to the present invention; Figure 3 A flow chart of the method for fine-tuning a large model of the present invention; Figure 4 The figure is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.

[0028] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0030] The embodiments of the present invention are described in further detail below.

[0031] The embodiment of the present invention provides a method for optimizing the output unit of a large model agent, see Figure 2 As shown, the following steps are included: S1. Use a compression algorithm to compress the content generated by the large model and directly generate compressed code; Specifically, the generated content is compressed using the gzip compression algorithm.

[0032] S2. Optimize the number of tokens output by large models and perform customized encoding on tokens, including: compression of tool names and parameter names, compression of JSON structures, and compression of special symbol connectors; Specifically, by defining the unique tokens in this specific field through the custom tokenization tool BPE method, the word segmentation and generation process is more efficient; Methods for compressing tool names and parameter names include: creating short codes for tool names and parameters to reduce the overhead caused by long field names; The examples in this embodiment are: `getWeather` ->`GW` `getDeviceStatus` ->`GDS` The methods of JSON structure compression include: encoding field names and common values ​​to reduce redundant information in the JSON structure and make the JSON structure more concise; The examples in this embodiment are: `{'name': 'device_control', 'parameters': {'device': 'Air conditioner', 'command': 'Close'}}` ->`DC|d=Air conditioner,c=Close` `{'status': 'active', 'data': [123, 456, 789]}` ->`A|D[123,456,789]`.

[0033] Special symbol connector compression includes: special optimization of special symbols (such as equal signs and separators) to make special symbols a single token in the field, reducing the number of tokens; preferably, the effect can be further enhanced by fine-tuning the training model; The example in this embodiment is: `DC|d=air conditioning, c=off` -> `DC|is a token`.

[0034] S3. Focus on data in a specific field, fine-tune the large model, train the large model to recognize data in that specific field, and generate a more concise encoding method to meet the compression requirements of steps S1 and S2.

[0035] The method for fine-tuning a large model consists of the following steps (see Figure 3 shown): S31. Collect and prepare a large amount of data in the specific field, including abbreviations, coding methods, and sentence structures specific to the specific field; S32. Use the collected data to fine-tune the large model so that it can understand and use the concise encoding method in this specific field.

[0036] This embodiment further reduces the number of tokens and response time through multi-level compression and optimization combined with encoding and training.

[0037] Specifically, multi-level compression includes the following levels: compression algorithm layer, vocabulary layer, structure layer, symbol layer, and model layer; the compression algorithm layer uses compression algorithms such as gzip and Brotli to compress the content generated by the language model as a whole; the vocabulary layer is used to customize field name compression to reduce the overhead caused by long field names; the structure layer is used to optimize structures such as JSON to reduce format redundancy; the symbol layer uses the compression and representation of special symbols to reduce the number of tokens; the model layer optimizes the generation method by fine-tuning the model to meet the compression requirements of the above levels.

[0038] The embodiment of the present invention further provides a system for optimizing the output unit of a large model intelligent agent, and executes the method for optimizing the output unit of a large model intelligent agent as described above, including: Compression coding module: used to compress the content generated by the large model using a compression algorithm and directly generate compressed codes; Vertical field encoding module: used to optimize the number of tokens output by large models and perform customized encoding of tokens, including: tool name and parameter name compression, JSON structure compression, and special symbol connector compression; Fine-tuning training module: used to focus on data in a specific field, fine-tune the large model, train the large model to recognize data in that specific field, and generate a more concise encoding method to meet the compression requirements of steps S1 and S2.

[0039] The fine-tuning training module includes: Fine-tuning training data preparation unit: used to collect and prepare a large amount of data in the specific field, including abbreviations, encoding methods, and sentence structures specific to the specific field; Fine-tuning model training unit: used to fine-tune the large model using the collected data, so that the large model can understand and use the concise encoding method in the specific field.

[0040] An application example of the embodiment of the present invention is (see Figure 1 shown): Assume that the user has a smart home system where the user can control the devices through natural language commands: Input command: `Raise the temperature in the living room to 25 degrees` Standardized command: `{'device': 'thermostat', 'location': 'living_room', 'command': 'set_temperature', 'value': 25}` Compressed output: `T|lr|st|25` Through the above method, the number of output characters is significantly reduced, the number of tokens is reduced by two-thirds, the latency is increased by 1.5 times, and the processing speed and response speed are effectively improved.

[0041] By streamlining the number of output tokens, the number of tokens in the generated text is reduced, which can reduce the number of output tokens, speed up the response time, reduce the model latency, reduce the computing cost, and improve the user experience.

[0042] An embodiment of the present invention further provides a computer device, Figure 4 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 4 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the method for extreme optimization of the output unit of the large model intelligent body as provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 4 The example of connecting through bus is taken in the following.

[0043] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs, computer executable programs, such as the program instructions corresponding to the method for the extreme optimization of the output unit of the large model intelligent body described in the embodiment of the present invention; the memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 can further include a memory remotely arranged relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0044] The input system 23 may be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device; the output system 24 may include display devices such as display screens.

[0045] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, that is, realizes the above-mentioned method of extreme optimization of the output unit of the large model intelligent body.

[0046] The computer device provided above can be used to execute the method for extreme optimization of the output unit of the large model intelligent agent provided in the above embodiment, and has corresponding functions and beneficial effects.

[0047] The embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to perform the method for optimizing the output unit of a large model intelligent agent as provided in the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape system; computer system memory or random access memory, such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or a combination thereof; in addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (for example, in different computer systems connected via a network). The storage medium can store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.

[0048] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention is not limited to the method for extreme optimization of the output unit of a large model intelligent agent as described in the above embodiment, and the computer executable instructions can also execute the relevant operations in the method for extreme optimization of the output unit of a large model intelligent agent provided by any embodiment of the present invention.

[0049] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for optimizing the output unit of a large model agent, characterized in that: The following steps are involved: S1. Use a compression algorithm to compress the content generated by the large model and directly generate compressed code; S2. Optimize the number of tokens output by large models and perform customized encoding on tokens, including: compression of tool names and parameter names, compression of JSON structures, and compression of special symbol connectors; S3. Focus on data in a specific field, fine-tune the large model, train the large model to recognize data in that specific field, and generate a more concise encoding method to meet the compression requirements of steps S1 and S2.

2. The method for optimizing the output unit of a large model agent according to claim 1, characterized in that: The method of fine-tuning the large model in step S3 comprises the following steps: S31. Collect and prepare a large amount of data in the specific field, including abbreviations, coding methods, and sentence structures specific to the specific field; S32. Use the collected data to fine-tune the large model so that it can understand and use the concise encoding method in this specific field.

3. The method for optimizing the output unit of a large model agent according to claim 1, characterized in that: The method for compressing the tool name and parameter name in step S2 includes: establishing short codes for tool names and parameters to reduce the overhead caused by long field names.

4. The method for optimizing the output unit of a large model agent according to claim 1, characterized in that: The method for compressing the JSON structure in step S2 includes: encoding field names and common values, reducing redundant information in the JSON structure, and making the JSON structure more concise.

5. The method for optimizing the output unit of a large model agent according to claim 1, characterized in that: The special symbol connector compression in step S2 includes: specially optimizing the special symbol so that the special symbol becomes a single token in the field, thereby reducing the number of tokens.

6. The method for optimizing the output unit of a large model agent according to claim 1, characterized in that: The compression algorithm in step S1 includes: gzip compression algorithm, Brotli compression algorithm or Lempel-Ziv-Welch compression algorithm.

7. A system for optimizing the output unit of a large model intelligent agent, executing the method for optimizing the output unit of a large model intelligent agent as described in any one of claims 1 to 6, characterized in that: include: Compression coding module: used to compress the content generated by the large model using a compression algorithm and directly generate compressed codes; Vertical field encoding module: used to optimize the number of tokens output by large models and perform customized encoding of tokens, including: tool name and parameter name compression, JSON structure compression, and special symbol connector compression; Fine-tuning training module: used to focus on data in a specific field, fine-tune the large model, train the large model to recognize data in that specific field, and generate a more concise encoding method to meet the compression requirements of steps S1 and S2.

8. The system for optimizing the output unit of a large model agent according to claim 7, characterized in that: The fine-tuning training module includes: Fine-tuning training data preparation unit: used to collect and prepare a large amount of data in the specific field, including abbreviations, encoding methods, and sentence structures specific to the specific field; Fine-tuning model training unit: used to fine-tune the large model using the collected data, so that the large model can understand and use the concise encoding method in the specific field.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for extreme optimization of the output unit of a large model intelligent agent as described in any one of claims 1-6 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for extreme optimization of the output unit of a large model intelligent agent are implemented as described in any one of claims 1-6.

Images