A method and apparatus for improving the response speed of AI large model tool calls

Abstract

This invention provides a method and apparatus for improving the response speed of AI large-scale model tool calls. The method includes: adjusting the output strategy of the AI ​​large-scale model and designing the output logic of the AI ​​large-scale model. When processing user requests, the AI ​​large-scale model first generates a response message to the user, and then generates structured function call information. The structured function call information adopts a JSON format data structure, containing the function name and corresponding parameters. This invention achieves a new response mechanism by adjusting the output strategy of the AI ​​large-scale model. The AI ​​large-scale model first outputs the response message to the user, and then the structured function call information. The AI ​​large-scale model can stream a response to the user while generating the response message, and the structured data of the function call information is generated simultaneously, effectively shortening the user's waiting time. Optimizing the output strategy of the AI ​​large-scale model reduces user waiting time and significantly improves the response speed of AI large-scale model tool calls.

Description

Technical Field

[0001] This invention relates to the fields of human-computer interaction, artificial intelligence, large AI models, and the invocation of large AI model tools; specifically, it relates to a method and apparatus for improving the response speed of invoking large AI model tools. Background Technology

[0002] In the field of artificial intelligence, particularly in Natural Language Processing (NLP) technology, large language models have become a significant milestone. These NLP models, through massive training data and a large number of neural network parameters, demonstrate a powerful ability to understand natural language, images, and even videos. A notable characteristic of large language models is that knowledge is pre-trained into the model, enabling it to understand and respond to user input.

[0003] However, this pre-training method also brings some problems. First, due to the huge number of model parameters and the slow speed of knowledge updates, the model cannot acquire the latest knowledge and information in real time, easily producing outdated or inaccurate information. To address this, researchers have proposed enabling large AI models to call tools to compensate for this deficiency. The function call capability of large AI models can achieve this function, and the function call process consists of three steps:

[0004] Step 1: The user's context information and current command are input into the AI ​​model. The AI ​​model outputs the function name and corresponding parameters, such as... Figure 1 As shown;

[0005] Step 2: Based on the function name and parameters obtained from the AI ​​large model parsing in Step 1, call the local function and get the returned result;

[0006] Step 3: Input the results obtained in step 2 into the AI ​​big model, and the AI ​​big model will summarize and provide feedback to the user.

[0007] Function calls are typically formatted as structured data such as JSON or XML. The function name and parameters can only be parsed after the large AI model has finished outputting its data; in other words, function calls from the large AI model do not return data in a streaming manner. While this mechanism ensures accuracy during the call process, it also introduces significant response latency. Traditionally, when processing user requests and calling external tools, the large AI model must wait until it has fully processed the request before parsing the output structured data (such as JSON or XML), calling the function interface, and finally generating a response message for the user. This results in long waiting times for users.

[0008] Therefore, in the current technological context, although large AI models can obtain real-time information and enhance their functionality by calling tools, the non-streaming function call mechanism has become a major bottleneck in improving response speed and user experience. Summary of the Invention

[0009] Therefore, the purpose of this invention is to propose a method and apparatus for improving the response speed of AI large-scale model tool calls. By adjusting the output strategy of the AI ​​large-scale model, a new response mechanism is achieved. Under the new mechanism, the AI ​​large-scale model first outputs the reply text to the user, and then the structured function call information. As the AI ​​large-scale model generates the reply text, it can stream a reply to the user. At the same time, the structured data of the function call information is also generated, which can effectively shorten the user's waiting time. By optimizing the output strategy of the AI ​​large-scale model and reducing the user's waiting time, the response speed of AI large-scale model tool calls is significantly improved.

[0010] This invention provides a method for improving the response speed of AI large model tool calls, including:

[0011] The output strategy of the AI ​​big model was adjusted and the output logic of the AI ​​big model was redesigned. When processing user requests, the AI ​​big model first generates a response message to the user, and then generates structured function call information. The structured function call information adopts a JSON data structure, which includes the function name and corresponding parameters.

[0012] The design of the JSON data structure in this invention optimizes the response speed of the AI ​​model when processing user requests, enabling the AI ​​model to generate responses to users more quickly and simultaneously generate function call information to respond to user requests rapidly.

[0013] The method for generating response scripts for users includes the following steps:

[0014] S01. The AI ​​big model determines the user's intent based on the user's dialogue context information, current instructions, and the user's historical behavior, so as to more accurately understand the specific meaning of the user's request.

[0015] S02. Based on the understanding of user intent and context, the AI ​​large model uses its pre-trained language generation capabilities to construct response scripts, which involves selecting appropriate vocabulary, constructing sentence structures, and ensuring the fluency and naturalness of the scripts.

[0016] S03. The AI ​​model personalizes the dialogue based on the user's characteristics (such as age, interests, past interaction style, etc.) and improves the relevance and satisfaction of the dialogue through internal optimization algorithms.

[0017] S04. Output the generated response to the user, notify the user that the request is being processed, and provide necessary information or guidance to maintain a smooth conversation and a good user experience.

[0018] Throughout the process, the goal of the AI ​​big data model is to quickly generate accurate, relevant, and user-friendly responses to reduce user wait times and improve the overall interactive experience. In this way, the AI ​​big data model can maintain real-time communication with users while processing complex requests.

[0019] The response script is generated by the AI ​​model analyzing and understanding the user's request. This process involves the AI ​​model learning and predicting language patterns and user intent.

[0020] Furthermore, the method for generating structured function call information includes the following steps:

[0021] S1. The AI ​​model receives and parses the user's input request to understand the user's intent and the type of information required.

[0022] S2. Based on parsing the user's request, the AI ​​model generates a response to the user; for example, "Okay, we are checking the weather information for Beijing tomorrow for you."

[0023] S3. Based on the user's request, the AI ​​model determines the name of the function to be called (e.g., `weather_query`) and the corresponding parameters (e.g., location `Beijing` and time `tomorrow`);

[0024] S4. Construct a structured data object from the determined function name and its corresponding parameters in JSON format;

[0025] S5. The AI ​​big model first outputs the generated response script to the user so that the user can get a response in time. Then, it outputs structured function call information, which can be used by the system to call the actual function interface.

[0026] S6. While the AI ​​large model outputs the response script, it processes the structured function call information in parallel and calls the corresponding function interface to obtain the required data.

[0027] S7. When the function interface returns a result, the AI ​​big model processes the returned result and generates a final response for the user, completing the entire interaction process.

[0028] This invention improves the parsing process of function call information and optimizes the data parsing process. Under the new mechanism, the output of the AI ​​big data model and the function call information are continuous, using specific identifiers (such as "@") to distinguish the end of the speech and the beginning of the function call information. In this way, the speech part of the AI ​​big data model can be returned to the user immediately, while the background can continue to generate function call information, and then parse and execute the function call after generation is complete.

[0029] By implementing an adjusted output strategy and achieving a parallel processing mechanism, while the AI ​​model responds to users using the generated reply script, function call information is generated simultaneously. After the structured data is generated, the function interface can be called to return the results, and finally the query results are returned to the user.

[0030] Furthermore, the corresponding parameters in the structured function call information include the following categories: query conditions, operation type, user preferences, and authentication;

[0031] The query conditions are the time, location, and keywords that a user needs to provide when requesting to query a certain type of information; for example, in a request to query the weather, "Beijing" and "tomorrow" are query condition parameters.

[0032] The operation type refers to the specific operation that some functions need to know the user wants to perform; such as adding, deleting, updating, or querying. These operation type parameters guide the function to perform the correct operation.

[0033] The user preferences refer to the user's specific preferences or requirements, such as language selection, result sorting method, and format requirements. These parameters help the AI ​​model adjust its output according to the user's personalized needs.

[0034] The authentication process involves passing authentication information (such as username, password, token, etc.) as parameters if the request involves an operation that requires user authentication, such as logging in or accessing protected data.

[0035] For example, in the example above, "corresponding parameters" specifically refers to:

[0036] -`location`: The location parameter for querying the weather; here it is set to "Beijing".

[0037] -`data`: The time parameter for querying the weather, here set to "tomorrow".

[0038] The parameters in structured function call messages are the specific information or data that needs to be passed to the function during its execution. These parameters are crucial factors for function execution, determining the function's behavior and output. In the context of large-scale AI models, these parameters are typically closely related to the information requested by the user, guiding the model on how to correctly process the request and retrieve or generate the corresponding data. These parameters are encapsulated in structured function call messages and output as part of a JSON object by the large-scale AI model, allowing subsequent systems to perform corresponding queries or operations based on these parameters.

[0039] Furthermore, the structured data object in step S4 includes a field for the function name (`Action`) and sub-objects for all parameters (`Action_Input`).

[0040] This invention also provides an apparatus for improving the response speed of AI large model tool calls, executing the method for improving the response speed of AI large model tool calls as described above, including: a response script generation system and a structured function call information generation system; wherein, the response script generation system includes:

[0041] Semantic understanding module: This module is used by the AI ​​model to determine the user's intent based on the context of the user's dialogue, the current instruction, and the user's historical behavior, so as to more accurately understand the specific meaning of the user's request.

[0042] The dialogue generation module is used to construct response dialogues based on the understanding of user intent and context. The AI ​​model uses its pre-trained language generation capabilities to do so. This involves selecting appropriate vocabulary, constructing sentence structure, and ensuring the fluency and naturalness of the dialogue.

[0043] Personalization and optimization module: Used by AI large model to personalize the dialogue based on the user's characteristics, and improve the relevance and satisfaction of the dialogue through internal optimization algorithms;

[0044] Output script module: Used to output the generated response script to the user, notify the user that the request is being processed, and provide necessary information or guidance to maintain a smooth conversation and a good user experience.

[0045] Furthermore, the system for generating structured function call information includes:

[0046] User Request Parsing Module: This module is used to receive and parse user input requests from the large AI model, and to understand the user's intent and the type of information required.

[0047] The response script generation module is used to generate response scripts for users based on parsing user requests, using a large AI model.

[0048] Function name and parameter determination module: This module is used to determine the name of the function to be called and the corresponding parameters based on the user's request from the large AI model.

[0049] Constructing a structured data module: This module is used to construct a structured data object from a given function name and its corresponding parameters in JSON format.

[0050] Output response script and structured data module: This module is used to first output the generated response script to the user so that the user can receive an immediate response, and then output structured function call information.

[0051] Parallel processing module: Used to process structured function call information in parallel while the AI ​​large model outputs the response script, and call the corresponding function interface to obtain the required data;

[0052] The query results return module is used when the function interface returns results. The AI ​​big model processes the returned results and generates a final response for the user, completing the entire interaction process.

[0053] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method described above for improving the response speed of AI large model tool calls.

[0054] The present invention also provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method for improving the response speed of AI large model tool calls as described above.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] The method and apparatus for improving the response speed of AI large-scale model tool calls provided by this invention achieve a new response mechanism by adjusting the output strategy of the AI ​​large-scale model. The adjusted output strategy includes two parts: the response text generated by the AI ​​large-scale model to the user and the function information to be called. The function information to be called includes the function name and corresponding parameters. The first part can be streamed to shorten the user's waiting time, while the second part is structured function call information, which is parsed after the AI ​​large-scale model finishes outputting. Under the new mechanism, the AI ​​large-scale model first outputs the response text to the user, and then the structured function call information. The AI ​​large-scale model can stream the response text to the user while simultaneously generating the structured data of the function call information, effectively shortening the user's waiting time. This invention achieves rapid response to user requests by adjusting the output strategy of the AI ​​large-scale model, significantly reducing user waiting time and significantly improving the response speed of AI large-scale model tool calls. The instant response feature greatly enhances the user experience during interaction, especially in interactive application scenarios, reducing the user experience degradation caused by response delays and improving the overall user experience. Attached Figure Description

[0057] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0058] In the attached diagram:

[0059] Figure 1 This is a flowchart of the AI ​​large model output function call in an embodiment of the present invention;

[0060] Figure 2 The flowchart of the method for generating response scripts for users according to the present invention is shown below;

[0061] Figure 3 A flowchart illustrating the method for generating structured function call information according to this invention;

[0062] Figure 4 This is a schematic diagram of the configuration of a computer device according to an embodiment of the present invention. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and products consistent with some aspects of this disclosure as detailed in the appended claims.

[0064] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0065] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0066] The embodiments of the present invention will be described in further detail below.

[0067] This invention provides a method for improving the response speed of AI large model tool calls, including:

[0068] The output strategy of the AI ​​big model was adjusted and the output logic of the AI ​​big model was redesigned. When processing user requests, the AI ​​big model first generates a response message to the user, and then generates structured function call information. The structured function call information adopts a JSON data structure, which includes the function name and corresponding parameters.

[0069] The design of this JSON data structure optimizes the response speed of the AI ​​model when processing user requests, enabling the AI ​​model to generate responses to users more quickly and simultaneously generate function call information in order to respond to user requests rapidly.

[0070] The following is a concrete example of a structured function call information structure:

[0071]

[0072] The method for generating response scripts for users can be found in [reference needed]. Figure 2 As shown, it includes the following steps:

[0073] S01, The AI ​​model determines the user's dialogue context and current instructions (see...). Figure 1 As shown in the figure, and information such as the user's historical behavior, we can judge the user's intent in order to more accurately understand the specific meaning of the user's request;

[0074] S02. Based on the understanding of user intent and context, the AI ​​large model uses its pre-trained language generation capabilities to construct response scripts, which involves selecting appropriate vocabulary, constructing sentence structures, and ensuring the fluency and naturalness of the scripts.

[0075] S03. The AI ​​model personalizes the dialogue based on the user's characteristics, and improves the relevance and satisfaction of the dialogue through internal optimization algorithms.

[0076] S04. Output the generated response to the user, notify the user that the request is being processed, and provide necessary information or guidance to maintain a smooth conversation and a good user experience.

[0077] Throughout the process, the goal of the AI ​​big data model is to quickly generate accurate, relevant, and user-friendly responses to reduce user wait times and improve the overall interactive experience. In this way, the AI ​​big data model can maintain real-time communication with users while processing complex requests.

[0078] The response script is generated by the AI ​​model analyzing and understanding the user's request. This process involves the AI ​​model learning and predicting language patterns and user intent.

[0079] The method for generating structured function call information is described in [reference needed]. Figure 3 As shown, it includes the following steps:

[0080] S1. The AI ​​model receives and parses the user's input request to understand the user's intent and the type of information required.

[0081] S2. Based on parsing the user's request, the AI ​​model generates a response to the user; for example, "Okay, we are checking the weather information for Beijing tomorrow for you."

[0082] S3. Based on the user's request, the AI ​​big data model determines the name of the function to be called and the corresponding parameters;

[0083] S4. Construct a structured data object from the determined function name and its corresponding parameters in JSON format;

[0084] The structured data object contains a field for the function name and sub-objects for all parameters.

[0085] S5. The AI ​​big model first outputs the generated response script to the user so that the user can get a response in time. Then, it outputs structured function call information, which can be used by the system to call the actual function interface.

[0086] S6. While the AI ​​large model outputs the response script, it processes the structured function call information in parallel and calls the corresponding function interface to obtain the required data.

[0087] S7. When the function interface returns a result, the AI ​​big model processes the returned result and generates a final response for the user, completing the entire interaction process.

[0088] This invention improves the parsing process for function call information and optimizes the data parsing process. Under the new mechanism, the output of the AI ​​big data model and the function call information are continuous, using specific identifiers (such as "@") to distinguish the end of the speech and the beginning of the function call information. In this way, the speech part of the AI ​​big data model can be returned to the user immediately, while the background can continue to generate function call information, and then parse and execute the function call after generation is complete.

[0089] By implementing an adjusted output strategy and achieving a parallel processing mechanism, while the AI ​​model responds to users using the generated reply script, function call information is generated simultaneously. After the structured data is generated, the function interface can be called to return the results, and finally the query results are returned to the user.

[0090] The parameters in the structured function call information include the following categories: query conditions, operation type, user preferences, and authentication.

[0091] The query conditions are the time, location, and keywords that a user needs to provide when requesting to query a certain type of information; for example, in a request to query the weather, "Beijing" and "tomorrow" are query condition parameters.

[0092] The operation type refers to the specific operation that some functions need to know the user wants to perform; such as adding, deleting, updating, or querying. These operation type parameters guide the function to perform the correct operation.

[0093] The user preferences refer to the user's specific preferences or requirements, such as language selection, result sorting method, and format requirements. These parameters help the AI ​​model adjust its output according to the user's personalized needs.

[0094] The authentication process involves passing authentication information (such as username, password, token, etc.) as parameters if the request involves an operation that requires user authentication, such as logging in or accessing protected data.

[0095] In this embodiment, the corresponding parameters are specifically:

[0096] -`location`: The location parameter for querying the weather; here it is set to "Beijing".

[0097] -`data`: The time parameter for querying the weather, here set to "tomorrow".

[0098] The parameters in structured function call messages are the specific information or data that needs to be passed to the function during its execution. These parameters are crucial factors for function execution, determining the function's behavior and output. In the context of large-scale AI models, these parameters are typically closely related to the information requested by the user, guiding the model on how to correctly process the request and retrieve or generate the corresponding data. These parameters are encapsulated in structured function call messages and output as part of a JSON object by the large-scale AI model, allowing subsequent systems to perform corresponding queries or operations based on these parameters.

[0099] This invention also provides an apparatus for improving the response speed of AI large model tool calls, executing the method for improving the response speed of AI large model tool calls as described above, including: a response script generation system and a structured function call information generation system; wherein, the response script generation system includes:

[0100] Semantic understanding module: This module is used by the AI ​​model to determine the user's intent based on the context of the user's dialogue, the current instruction, and the user's historical behavior, so as to more accurately understand the specific meaning of the user's request.

[0101] The dialogue generation module is used to construct response dialogues based on the understanding of user intent and context. The AI ​​model uses its pre-trained language generation capabilities to do so. This involves selecting appropriate vocabulary, constructing sentence structure, and ensuring the fluency and naturalness of the dialogue.

[0102] Personalization and optimization module: Used by AI large model to personalize the dialogue based on the user's characteristics, and improve the relevance and satisfaction of the dialogue through internal optimization algorithms;

[0103] Output script module: Used to output the generated response script to the user, notify the user that the request is being processed, and provide necessary information or guidance to maintain a smooth conversation and a good user experience.

[0104] The system for generating structured function call information includes:

[0105] User Request Parsing Module: This module is used to receive and parse user input requests from the large AI model, and to understand the user's intent and the type of information required.

[0106] The response script generation module is used to generate response scripts for users based on parsing user requests, using a large AI model.

[0107] Function name and parameter determination module: This module is used to determine the name of the function to be called and the corresponding parameters based on the user's request from the large AI model.

[0108] Constructing a structured data module: This module is used to construct a structured data object from a given function name and its corresponding parameters in JSON format.

[0109] Output response script and structured data module: This module is used to first output the generated response script to the user so that the user can receive an immediate response, and then output structured function call information.

[0110] Parallel processing module: Used to process structured function call information in parallel while the AI ​​large model outputs the response script, and call the corresponding function interface to obtain the required data;

[0111] The query results return module is used when the function interface returns results. The AI ​​big model processes the returned results and generates a final response for the user, completing the entire interaction process.

[0112] Application examples

[0113] A user requests a weather forecast from an AI model. Under traditional mechanisms, the AI ​​model needs to complete the entire processing flow before outputting structured data, requiring the user to wait a considerable amount of time. However, the process under the technical solution of this invention is as follows:

[0114] -A user commented: "Beijing's weather tomorrow."

[0115] - AI Big Data Model Response: The AI ​​big data model outputs "Okay, querying Beijing's weather information for tomorrow. @{"Action":"weather_query","Action_Input":{"location":"Beijing","data":"tomorrow"}}", where "Okay, querying Beijing's weather information for tomorrow" is the AI ​​big data model's response to the user, which the system can stream to the user. "@" is a special character used as a separator between the response and function call information. {"Action":"weather_query","Action_Input":{"location":"Beijing","data":"tomorrow"}} represents the function name and parameters to be called by the AI ​​big data model. While the system is reading the AI ​​big data model's response, the system can call the corresponding function interface in parallel to obtain the required results, and then continue to provide the query results to the user.

[0116] Therefore, it can be seen that the implementation of this invention can significantly reduce user waiting time and improve the response speed of calling AI large model tools.

[0117] This invention also provides a computer device. Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; see the accompanying drawings. Figure 4 As shown, the computer device includes: an input device 23, an output device 24, a memory 22, and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the method for improving the response speed of AI large model tool calls as provided in the above embodiments; wherein the input device 23, the output device 24, the memory 22, and the processor 21 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0118] The memory 22, as a read / write storage medium for a computing device, can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the method for improving the response speed of AI large model tool calls described in this embodiment of the invention. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device. Furthermore, the memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 22 may further include memory remotely located relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0119] Input device 23 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the device; output device 24 may include display devices such as a display screen.

[0120] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, thereby realizing the above-mentioned method to improve the response speed of AI large model tool calls.

[0121] The computer equipment provided above can be used to execute the method for improving the response speed of AI large model tool calls provided in the above embodiments, and has corresponding functions and beneficial effects.

[0122] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the method for improving the response speed of AI large model tool calls as provided in the above embodiments. The storage medium can be any type of memory device or storage device, including: mounting media such as CD-ROM, floppy disk, or magnetic tape; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or combinations thereof; furthermore, the storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet); the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0123] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method for improving the response speed of AI large model tool calls as described in the above embodiments, but can also execute related operations in the method for improving the response speed of AI large model tool calls provided in any embodiment of the present invention.

[0124] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for improving the response speed of AI large model tool calls, characterized in that, include: Adjust the output strategy of the AI ​​big model and redesign the output logic of the AI ​​big model. Use specific identifiers to distinguish the position of the end of the speech and the beginning of the function call information. When processing user requests, the AI ​​big model first generates a reply speech for the user and generates structured function call information at the same time as responding to the user. The structured function call information uses a JSON data structure, which includes the function name and its corresponding parameters; The method for generating response scripts for users includes the following steps: S01. The AI ​​big model determines the user's intent based on the user's dialogue context information, current instructions, and the user's historical behavior information, so as to more accurately understand the specific meaning of the user's request. S02. Based on the understanding of user intent and context, the AI ​​large model uses its pre-trained language generation capabilities to construct response scripts, which involves selecting appropriate vocabulary, constructing sentence structures, and ensuring the fluency and naturalness of the scripts. S03. The AI ​​model personalizes the dialogue based on the user's characteristics, and improves the relevance and satisfaction of the dialogue through internal optimization algorithms. S04. Output the generated response to the user, notify the user that the request is being processed, and provide necessary information or guidance to maintain a smooth conversation and a good user experience. The method for generating structured function call information includes the following steps: S1. The AI ​​model receives and parses the user's input request to understand the user's intent and the type of information required. S2. Based on parsing the user's request, the AI ​​model generates the response to the user; S3. Based on the user's request, the AI ​​big data model determines the name of the function to be called and the corresponding parameters; S4. Construct a structured data object from the determined function name and its corresponding parameters in JSON format; S5. The AI ​​model first outputs the generated response script to the user so that the user can receive a response in time, and then outputs structured function call information. S6. While the AI ​​large model outputs the response script, it processes the structured function call information in parallel and calls the corresponding function interface to obtain the required data. S7. When the function interface returns a result, the AI ​​big model processes the returned result and generates a final response for the user, completing the entire interaction process. The parameters in the structured function call information include the following categories: query conditions, operation type, user preferences, and authentication. The query conditions are the time, location, and keywords that a user needs to provide when requesting to query a certain type of information. The operation type refers to certain functions that need to know the specific operation that the user wants to perform. The user preferences are the user's specific preferences or requirements; The authentication is performed if the request involves an operation that requires verifying the user's identity. The structured data object in step S4 includes a field for the function name and sub-objects for all parameters.

2. An apparatus for improving the response speed of AI large model tool calls, executing the method for improving the response speed of AI large model tool calls as described in claim 1, characterized in that, include: A system for generating response scripts and a system for generating structured function call information; wherein, the system for generating response scripts includes: Semantic understanding module: This module is used by the AI ​​model to determine the user's intent based on the context of the user's dialogue, the current instruction, and the user's historical behavior information, so as to more accurately understand the specific meaning of the user's request. The dialogue generation module is used to construct response dialogues based on the understanding of user intent and context. The AI ​​model uses its pre-trained language generation capabilities to do so. This involves selecting appropriate vocabulary, constructing sentence structure, and ensuring the fluency and naturalness of the dialogue. Personalization and optimization module: Used by AI large model to personalize the dialogue based on the user's characteristics, and improve the relevance and satisfaction of the dialogue through internal optimization algorithms; Output script module: Used to output the generated response script to the user, notify the user that the request is being processed, and provide necessary information or guidance to maintain a smooth conversation and a good user experience.

3. The apparatus for improving the response speed of AI large model tool calls according to claim 2, characterized in that, The system for generating structured function call information includes: User Request Parsing Module: This module is used to receive and parse user input requests from the large AI model, and to understand the user's intent and the type of information required. The response script generation module is used to generate response scripts for users based on parsing user requests, using a large AI model. Function name and parameter determination module: This module is used to determine the name of the function to be called and the corresponding parameters based on the user's request from the large AI model. Constructing a structured data module: This module is used to construct a structured data object from a given function name and its corresponding parameters in JSON format. Output response script and structured data module: This module is used to first output the generated response script to the user so that the user can receive an immediate response, and then output structured function call information. Parallel processing module: Used to process structured function call information in parallel while the AI ​​large model outputs the response script, and call the corresponding function interface to obtain the required data; The query results return module is used when the function interface returns results. The AI ​​big model processes the returned results and generates a final response for the user, completing the entire interaction process.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for improving the response speed of AI large model tool calls as described in claim 1.

5. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for improving the response speed of AI large model tool calls as described in claim 1.

Images