Control method of interactive audio middleware, electronic equipment, medium and product

Abstract

The invention discloses a control method of interactive audio middleware, electronic equipment, a medium and a product. The control method comprises the following steps: acquiring demand information input by a user; based on the demand information, obtaining a tool capable of being called in a model context protocol server and tool information corresponding to the tool, and determining a currently called target function and corresponding description information; determining a called target tool and related parameters in the model context protocol server, and initiating a calling request to the target tool through the model context protocol host; and judging the received execution result in combination with the demand information, and sending the execution result to the user through the model context protocol host, or determining a next tool called in the model context protocol server and related parameters corresponding to the next tool. According to the method, intelligent operation is carried out on the Wwise editor through the natural language, the technical threshold of Wwise automatic processing is lowered, and a Wwise operation tool with high universality and reusability is achieved.

Description

Control methods, electronic devices, media, and products for interactive audio middleware Technical Field

[0001] This application relates to the field of audio development technology, specifically to control methods, electronic devices, media, and products for interactive audio middleware. Background Technology

[0002] In audio development, interactive audio middleware solutions, such as Wwise (Wave Works Interactive Sound Engine), are widely used. Among the complete set of design and development tools provided by Wwise, the Wwise editor offers users rich audio production and management functions, enabling the construction of complex audio projects. However, traditional usage of Wwise relies primarily on manual operation through its graphical interface, which is inefficient and prone to errors when dealing with large-scale audio resource management, content querying and inspection, batch processing tasks, and repetitive workflows.

[0003] To address these issues, Wwise provides the WAAPI (Wwise Authoring API), allowing developers to automate operations such as creation, modification, querying, and management programmatically, thereby achieving batch processing and workflow automation. However, using WAAPI has a high technical barrier; developers need relevant programming skills and familiarity with the API's structure and parameter formats. Furthermore, typical WAAPI tools can only solve specific needs, requiring the development of corresponding scripts or tools for each specific requirement to implement its specific business logic. Scripts and tools are difficult to reuse across different projects or requirements, leading to increased development costs. Summary of the Invention

[0004] In view of this, this application provides a control method, electronic device, medium, and product for interactive audio middleware to solve the problems of high technical barriers to the use of WAAPI in related technologies, difficulty in reusing scripts and tools between different projects or needs, and increased development costs.

[0005] Firstly, this application provides a control method for interactive audio middleware. The method includes: acquiring user-inputted requirement information; acquiring callable tools and their corresponding tool information in a Model Context Protocol (MCP) server based on the requirement information, determining the currently invoked target function based on the requirement information and tool information, and obtaining description information corresponding to the target function; determining the invoked target tool and related parameters in the MCP server based on the requirement information, tool information, and target function description information, and initiating a call request to the target tool through the MCP host, wherein the target tool is used to send a network request about the target function to the interactive audio middleware; the related parameters include the description information corresponding to the target function and the environment parameters of the target tool, the environment parameters describing the session to which the operation belongs and the current communication environment; judging the received execution result in conjunction with the requirement information, sending the execution result to the user through the MCP host, or determining the next invoked tool and its corresponding related parameters in the MCP server based on the execution result and requirement information, wherein the execution result is obtained after the interactive audio middleware performs an operation on the target function.

[0006] Secondly, this application provides a control method for interactive audio middleware. The method includes: receiving a call request from a Model Context Protocol (MCP) host to a target tool; responding to the call request, invoking the target tool in a MCP server to integrate relevant parameters input to the target tool into a network request, wherein the relevant parameters include descriptive information corresponding to the target function and environmental parameters of the target tool, the environmental parameters describing the session to which the operation belongs and the current communication environment, and the target function being determined based on user-input requirement information and tool information corresponding to the callable tools determined based on the requirement information; the target tool in the MCP server sending the network request to the interactive audio middleware; the target tool in the MCP server receiving the execution result fed back by the interactive audio middleware, and the MCP server sending the execution result to the MCP host, wherein the execution result is obtained by the interactive audio middleware after executing the operation corresponding to the target function based on the relevant parameters.

[0007] Thirdly, this application provides a control method for interactive audio middleware. The method includes: parsing a received network request to obtain description information corresponding to a target function, wherein the network request is a response from a model context protocol server to a call request for a target tool, and the target tool integrates the description information corresponding to the target function with the target tool's environment parameters. The environment parameters describe the session to which the operation belongs and the current communication environment. The target function is determined based on user-input requirements and tool information corresponding to the callable tools determined based on the requirements; executing the operation corresponding to the target function based on the description information to obtain an execution result; and sending the execution result to the target tool in the model context protocol server.

[0008] Fourthly, this application provides a control method for interactive audio middleware, the method comprising: receiving user-inputted requirement information; sending the requirement information to a large language model; receiving a call request sent by the large language model to a target tool of a model context protocol server, and sending the call request to the model context protocol server; receiving an execution result sent by the model context protocol server, wherein the execution result is obtained by the interactive audio middleware after executing the operation corresponding to the target function based on relevant parameters, the relevant parameters including the description information corresponding to the target function and the environment parameters of the target tool, the environment parameters being used to describe the session to which the operation belongs and the current communication environment, the target function being determined based on the user-input requirement information and the tool information corresponding to the callable tool determined based on the requirement information; and sending the execution result to the large language model.

[0009] Fifthly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the control method of the interactive audio middleware of any of the embodiments corresponding to the first, second, third, or fourth aspects described above.

[0010] In a sixth aspect, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute a control method for an interactive audio middleware according to any of the embodiments corresponding to the first, second, third, or fourth aspects described above.

[0011] In a seventh aspect, this application provides a computer program product, including computer instructions for causing a computer to execute a control method for an interactive audio middleware according to any of the embodiments corresponding to the first, second, third, or fourth aspects described above.

[0012] This application provides a control method, electronic device, medium, and product for interactive audio middleware. A large language model acquires user-input requirement information, determines the callable tools and their corresponding tool information in a Model Context Protocol (MCP) server based on this requirement information, determines the currently invoked target function based on the requirement and tool information, and obtains the description information corresponding to the target function. Then, it determines the target tool and related parameters to be invoked in the MCP server, and initiates a call request to the target tool through the MCP host. Based on the target tool, it sends a network request about the target function to the interactive audio middleware. The large language model then receives the execution result obtained after the interactive audio middleware performs an operation on the target function, judges the execution result, and sends the execution result to the user through the MCP host, or determines the next invoked tool and its corresponding parameters in the MCP server based on the execution result and requirement information. Through the above embodiments of this application, automatic invocation of the corresponding target function is achieved, enabling intelligent operation of the interactive audio middleware, lowering the technical threshold for automated processing of interactive audio middleware, and realizing a highly versatile and reusable interactive audio middleware operation tool, significantly saving time and labor costs. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0014] Figure 1 is a flowchart illustrating a control method for an interactive audio middleware according to an embodiment of this application; Figure 2 is a flowchart illustrating a control method for another interactive audio middleware according to an embodiment of this application; Figure 3 is a flowchart illustrating a control method for yet another interactive audio middleware according to an embodiment of this application; Figure 4 is a flowchart illustrating a control method for yet another interactive audio middleware according to an embodiment of this application; Figure 5 is a system block diagram illustrating a communication diagram illustrating the control of an interactive audio middleware according to an embodiment of this application; Figure 6 is a structural block diagram illustrating a control device for an interactive audio middleware according to an embodiment of this application; Figure 7 is a structural block diagram illustrating a control device for another interactive audio middleware according to an embodiment of this application; Figure 8 is a structural block diagram illustrating a control device for yet another interactive audio middleware according to an embodiment of this application; Figure 9 is a structural block diagram illustrating a control device for yet another interactive audio middleware according to an embodiment of this application; Figure 10 is a hardware structure diagram illustrating an electronic device according to an embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0017] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0018] The Wwise editor (an interactive audio middleware) provides users with rich audio production and management features, which can be used to build complex game audio projects. However, the traditional way of using Wwise mainly relies on its graphical interface for manual operation, which is inefficient and prone to errors when dealing with large-scale audio resource management, content querying and inspection, batch processing tasks, and repetitive workflows.

[0019] To address the aforementioned issues, Wwise provides the WAAPI interface, allowing developers to programmatically automate operations such as creation, modification, querying, and management, thereby achieving batch processing and workflow automation. However, it suffers from the following problems: To resolve these issues, according to an embodiment of this application, a control method embodiment for interactive audio middleware is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that presented here.

[0020] This embodiment provides a control method for an interactive audio middleware, which can be applied to a large language model. Figure 1 is a flowchart of the control method for the interactive audio middleware according to an embodiment of this application. As shown in Figure 1, the process includes the following steps: Step S101, obtaining user input requirement information.

[0021] Optionally, since this application embodiment applies to LLM (Large Language Model), the LLM will currently obtain the user's input requirements on the terminal side. These requirements can also be understood as input prompts for the large language model. The main purpose is to enable querying, creating, modifying, and controlling various objects. For example, "Use the MCP tool to set the volume attribute of all Sound SFX objects under the \Actor-Mixer Hierarchy\SFX\Footsteps object whose names contain the word 'wood' to 2dB, and check if the modification was successful." Another example is "Query which speech objects under the \Actor-Mixer Hierarchy\VO\Alice object do not contain English resources."

[0022] Here, several prompt words can be pre-defined in the MCP (Model Context Protocol) server. These prompt words are predefined text templates with acceptable parameters. Users can select prompt words according to their purpose to provide more accurate guidance information to large language models.

[0023] Step S102: Based on the requirement information, obtain the callable tools and corresponding tool information in the model context protocol server, determine the currently called target function based on the requirement information and tool information, and obtain the description information corresponding to the target function.

[0024] Optionally, the large language model analyzes the user's input requirements. Based on the requirements, it can determine which tools are currently available for invocation in the model context protocol server. After determining the currently available tools, it obtains the tool information corresponding to these tools, such as tool description information and tool annotation information.

[0025] It should be noted that the tools currently available for invocation in the Model Context Protocol Server (i.e., the MCP server) refer to the MCP tools.

[0026] MCP tools are tools provided by the MCP server. These tools are written as executable program units, accepting several parameters, performing corresponding operations, and returning execution results. Here, "executable program unit" means that the MCP tool is edited through program functions. For example, in a certain programming language, an MCP tool can be implemented using the following code: `@mcp.tool(description="Add two integers and return their sum") def add(a: int, b: int) ->int:return a + b`. MCP tools should include corresponding descriptions and annotations for the large language model's reference. For example, the description should include the tool's specific function, the format and meaning of the input parameters, the structure and type of the output results, usage scenarios and limitations, error handling mechanisms, etc. The large language model implements various operations by calling the tools.

[0027] After the large language model is established, the candidate tools to be invoked are determined based on the requirements and tool information. That is, from the available tools (e.g., 4), the candidate tools to be invoked (e.g., 2) are analyzed.

[0028] If the candidate tool contains a corresponding specified function (i.e., a function that is already bound to the candidate tool), then the specified function will be used as the target function to be called.

[0029] If the candidate tool does not contain the corresponding specified function, then the large language model needs to judge and determine the target function to be called when processing the operation behavior involved in the requirement information. For example, for the requirement information "set the volume property of all Sound SFX objects with the word wood in their names under the object \Actor-MixerHierarchy\SFX\Footsteps to 2dB", the operation involved in the requirement information is analyzed and the target functions to be called are determined to be: (1) call the ak.wwise.core.object.get function to get all Sound SFX objects under the object \Actor-MixerHierarchy\SFX\Footsteps from the Wwise editor; (2) call the ak.wwise.core.object.setProperty function to adjust the volume property of the objects that meet the name condition; (3) call the ak.wwise.core.object.get function to get the volume property of each object from the Wwise editor and determine whether the setting is successful.

[0030] In other words, for a large language model, the role of candidate tools is to determine the target function: if the current candidate tool has already been bound to a called function, the bound called function is directly used as the target function, and then in subsequent calculations and inferences, it is further determined whether to use a batch calling tool or a single calling tool for the target function; if the current candidate tool has not been bound to a called function, then the large language model participates in the decision-making process to determine the called target function, and then in subsequent calculations and inferences, it is further determined whether to use a batch calling tool or a single calling tool for the target function.

[0031] Once the target function being called is determined, its description information can be obtained, such as its purpose, parameters, and return value. It should be noted that the target function here refers to any WAAPI function. WAAPI is an application programming interface that allows other processes to communicate with the Wwise editor. Therefore, calling WAAPI functions can manipulate audio resources in the interactive audio middleware (i.e., the Wwise editor mentioned later).

[0032] Step S103: Based on the requirement information, tool information, and description information of the objective function, determine the target tool and related parameters to be invoked in the Model Context Protocol server, and initiate an invocation request to the target tool through the Model Context Protocol host. The target tool is used to send a network request about the objective function to the interactive audio middleware. The related parameters include the description information corresponding to the objective function and the environment parameters of the target tool. The environment parameters are used to describe the session to which the operation belongs and the current communication environment.

[0033] Optionally, the large language model analyzes the requirement information, tool information, and description information of the target function, and then further determines what kind of tool is being called, such as a batch calling tool for the target function or a single calling tool for the target function.

[0034] It should be noted that the target function has been determined in the preceding description. For each target function, there is a series of related parameters. These parameters include descriptive information about the target function, such as its location information and the function parameters and option parameters obtained based on the requirement information when performing operations on the target function. The location information of the target function can be its function identifier, which can be a Uniform Resource Identifier (URI), such as "ak.wwise.core.object.setProperty", or it can be understood as the name of the target function. In addition, the description information also includes the function parameters and option parameters obtained based on the requirement information when the target function performs the operation. For example, for the requirement information "set the volume property of all Sound SFX objects with the word 'wood' in their names under the \Actor-Mixer Hierarchy\SFX\Footstep object to 2dB and check whether the setting is successful", the operation involved in the requirement information is analyzed and the target function to be called is determined to be: (1) call the ak.wwise.core.object.get function to obtain all Sound SFX objects with the word 'wood' in their names under the \Actor-Mixer Hierarchy\SFX\Footsteps object from the Wwise editor; specifically, its function parameter is {"waql": "from object\"\\Actor-Mixer Hierarchy\\Default Work Unit\\Footsteps\" select descendantswhere type = \"Sound\" and name : \"*wood*\""}, and the option parameter is {"return": ["id", "name", (2) Batch call the ak.wwise.core.object.setProperty function to adjust the volume properties of each object; specifically, its function parameter is {"object": ID, "property": "Volume", "value": 2}, where ID is the object ID obtained in (1), and the option parameter is {}; (3) Batch call the ak.wwise.core.object.get function to obtain the volume properties of each object from the Wwise editor and determine whether the setting is successful; specifically, the function parameter is {"waql": "from object ID"}, and the option parameter is {"return":["id", "@Volume"]}.

[0035] The function parameters and option parameters are business data and configuration options required to perform the specified operation, and are organized in the form of structured data.

[0036] Among them, the function identifier, function parameters, and option parameters are all information used to describe the target function being called.

[0037] The large language model currently also needs to obtain the environment parameters required when calling the target tool, including the session to which the operation belongs (represented by a session identifier) ​​and the current communication environment (represented by communication connection parameters). The session identifier is a unique identifier for the session, used to distinguish different user sessions and associate complete operation records of multiple operations within the same session. The communication connection parameters include information such as the communication IP address and port corresponding to the interactive audio middleware, representing the current communication environment.

[0038] Step S104: Combine the received execution result with the requirement information to judge, and send the execution result to the user through the Model Context Protocol host, or determine the next tool called in the Model Context Protocol server and the relevant parameters of the next tool based on the execution result and the requirement information. The execution result is obtained by the interactive audio middleware performing operations on the target function.

[0039] Optionally, the large language model evaluates the execution result returned by the interactive audio middleware through the model context protocol host. Here, the execution result is the information obtained by the interactive audio middleware after performing operations on the target function. If the large language model considers the execution result to be an error, such as using unsupported parameters, it will re-determine the next tool to be called in the model context protocol server and its corresponding parameters based on the error message, and then call the next tool again. If the returned result is correct, the large language model sends the execution result to the user through the model context protocol host, or executes the next step based on the requirement information.

[0040] This application provides a control method, electronic device, medium, and product for interactive audio middleware. A large language model acquires user-input requirement information, determines the callable tools and their corresponding tool information in a Model Context Protocol (MCP) server based on this requirement information, determines the currently invoked target function based on the requirement and tool information, and obtains the description information corresponding to the target function. Then, it determines the target tool and related parameters to be invoked in the MCP server, and initiates a call request to the target tool through the MCP host. Based on the target tool, it sends a network request about the target function to the interactive audio middleware. The large language model then receives the execution result obtained after the interactive audio middleware performs an operation on the target function, judges the execution result, and sends the execution result to the user through the MCP host, or determines the next invoked tool and its corresponding parameters in the MCP server based on the execution result and requirement information. Through the above embodiments of this application, automatic invocation of the corresponding target function is achieved, enabling intelligent operation of the interactive audio middleware, lowering the technical threshold for automated processing of interactive audio middleware, and realizing a highly versatile and reusable interactive audio middleware operation tool, significantly saving time and labor costs.

[0041] As an optional embodiment, the method further includes: obtaining an index document introducing all functions, wherein the index document contains location information of all functions, uses of all functions, and document information introducing the usage of each function; learning the content in the index document, and obtaining a preset number of reference documents introducing the specific usage of a particular function based on the content in the index document and the requirement information; and learning the content in the reference documents.

[0042] Optionally, at the start of each session, an index file introducing all functions will be obtained first. This index file contains the location information of all functions, the uses of all functions, and documentation information introducing the usage of each function.

[0043] Users can be prompted to access the WAAPI function index documents provided in the MCP server to obtain information on available WAAPI functions.

[0044] Furthermore, a WAAPI function indexing method is provided to obtain information on available WAAPI functions. This resource is implemented by listing the location information of all WAAPI functions in a single file, such as the Uniform Resource Identifier (URI), the specific purpose of the function, and its usage documentation. A description of the file content and hints for the large language model are provided at the beginning of the file. The file can be in Markdown format or other text formats. When the large language model accesses this resource, the MCP server reads the WAAPI function index file and returns its contents, allowing the large language model to learn from all the resources in the index file.

[0045] For learning a specific WAAPI function, the current approach is to obtain a preset number of reference documents on the specific usage of that WAAPI function from the content and requirements information in the index documents (for example, selecting 4 reference documents based on the content and requirements information in the index documents). At the beginning of each session, the user can be prompted to access the reference documents provided by the MCP server for learning a specific WAAPI function.

[0046] For example, a user might input the following request: "Use the MCP tool to set the volume property of all Sound SFX objects under the \Actor-Mixer Hierarchy\SFX\Footsteps object whose names contain the word 'wood' to 2dB, and check if the modification was successful. You can refer to the WAAPI function reference documentation first." Or, another user might input: "Use the MCP tool to find which Sound SFX objects under the \Actor-Mixer Hierarchy\SFX\Footsteps object have a volume property of 0dB. You can refer to the ak.wwise.core.object.get function reference documentation first; the documentation path can be found in the index."

[0047] In this embodiment, a specific usage reference method for a particular WAAPI function is provided: before performing specific operations using tools, the large language model can access this resource to obtain detailed usage reference information for a certain WAAPI function. This resource is implemented by creating an independent reference document for each WAAPI function. The reference document contains all descriptive information for the specific WAAPI function, including: detailed description, parameter description, return information format, usage examples, and precautions.

[0048] When a large language model needs to access reference resources for a specific function for learning, it can obtain the path to access the reference resource by embedding the filename of the reference document for that specific function into an identifier. The Uniform Resource Identifier (URI) for the resource uses a templated format. Reference documents can be in Markdown or other text formats. When the large language model accesses a reference resource, the MCP server locates and reads the corresponding document file based on the provided filename and returns its content. For each WAAPI function, in addition to the main reference document, there may be accompanying parameter pattern reference documents and usage example documents, which are referenced in the main reference document. The large language model can also access this data when needed. Furthermore, there are some general reference documents, such as WAQL syntax and Wwise object references, which are also referenced in the main reference document and can be retrieved when necessary.

[0049] The knowledge acquisition mechanism implemented in this application enables the model to learn index documents related to functions before performing operations, which can significantly improve the accuracy of large language model operations. At the same time, it enables the model to obtain reference documents and usage instructions for specific functions before performing operations, which can also significantly improve the accuracy of large language model operations.

[0050] As an optional embodiment, the method further includes: obtaining guidance information for accessing the target operation record file contained in the demand information, wherein the target operation record file stores operation process records executed in each historical session; obtaining the path to access the target operation record file based on the filename corresponding to the target operation record file; and learning the target operation record file based on the path of the target operation record file.

[0051] Optionally, at the start of each session, the user can prompt the large language model to refer to a previously saved operation record from a previous session to execute the same or similar workflow. The operation record file is saved by the large language model, as instructed by the user, using an operation record saving tool, and can be saved in JSON format or other formats. It should be noted that in the above embodiments, the session identifier included in the environment parameters is used to distinguish different user sessions and associate multiple operation processes involved in the same session, recording the entire operation process corresponding to that session and saving it as a specific operation record file.

[0052] Therefore, all process information can be obtained by retrieving the operation log file corresponding to the session identifier. This operation log file records the session identifier, the target tool called, the call time, the target function and parameters called, whether it was successful, and the return information.

[0053] When a large language model needs to learn from reference resources recorded in a target operation log file based on requirement information, it can do the following: The large language model uses the guidance information for accessing a target operation log file (such as the prompt "currently refer to operation log file Y") input by the user. Based on the filename corresponding to the target operation log file, the large language model embeds the filename into a Uniform Resource Identifier to obtain the path to access the target operation log file, thereby enabling access to the target operation log file, reading the corresponding operation log file, and returning its content.

[0054] The operation flow saving mechanism implemented in this application embodiment enables the model to obtain the operation records saved in the historical session before performing the operation, thereby realizing the reusability of the workflow and greatly improving the accuracy of large language model operations.

[0055] As an optional embodiment, the target tool to be invoked in the Model Context Protocol server is determined based on requirement information, tool information, and description information of the objective function. This includes: determining the number of objective functions based on requirement information, tool information, and description information of the objective function; determining the tool type of the invoked target tool based on requirement information, tool information, description information of the objective function, and the number of objective functions; if the tool type of the target tool is not a specific type, determining the invoked target tool based on requirement information, description information of the objective function, tool information, and the number of objective functions; and if the tool type of the target tool is a specific type, determining the invoked target tool based on requirement information, description information of the objective function, tool information, and the number of objective functions.

[0056] Optionally, in the embodiments of this application, the target tool is divided into two types: a general calling tool that is not set for a specific function (i.e., non-specific type) and a special calling tool that is set for a specific function (i.e., specific type).

[0057] The current large language model can automatically analyze the number of target functions that can be invoked based on requirement information, tool information, and description information of the target function. Furthermore, the number of target functions invoked is related to the user's input requirement information, the function's purpose, and the target function's functionality. For example, if the requirement information involves more operations, more target functions will be invoked; if the requirement information involves fewer operations, fewer target functions will be invoked. These are part of the large language model's own reasoning process and will not be elaborated further.

[0058] Then, the large language model determines the tool type of the target tool to be called based on the requirement information, tool information, description information of the target function, and the number of target functions. Generally speaking, if the target function has a corresponding dedicated calling tool, that is, a specific type of tool, then the specific type of tool is used; otherwise, a non-specific type of tool is used.

[0059] When the target tool's tool type is not specific, the target tool is determined as either a general batch invocation tool or a general single invocation tool based on the number of target functions, combined with the requirement information, the description information of the target functions, and the tool information (including the first tool description information and the first tool annotation information). For example, if the number of target functions is a preset number (i.e., 1), the target tool is a general single invocation tool, initiating a single network request to the interactive audio middleware to invoke the target functions. If the number of target functions is greater than the preset number, the target tool is a general batch invocation tool, initiating multiple network requests to the interactive audio middleware to invoke the target functions in batches.

[0060] When the target tool is of a specific type, based on the number of target functions, combined with the requirement information, the description information of the target functions, and the tool information (including the description information and annotation information of the second tool), it is determined whether the target tool is a dedicated batch calling tool or a dedicated single calling tool. For example, when the number of target functions is a preset number (i.e., 1), the target tool is a dedicated single calling tool, which initiates a single network request to the interactive audio middleware to call the target functions. When the number of target functions is greater than the preset number, the target tool is a dedicated batch calling tool, which initiates multiple network requests to the interactive audio middleware to call the target functions in batches.

[0061] This application embodiment designs and develops an MCP server, providing MCP tools such as a general single-call tool for WAAPI functions, a general batch-call tool for WAAPI functions, a dedicated single-call tool for a specific WAAPI function, and a dedicated batch-call tool for a specific WAAPI function. The system automatically invokes these tools to perform operations on the Wwise editor, including various operations such as querying, creating, editing, and setting.

[0062] This embodiment also provides a control method for interactive audio middleware, as shown in Figure 2. This method can be applied to a Model Context Protocol server. The method includes: step S201, receiving a call request from the Model Context Protocol host for the target tool.

[0063] Step S202: In response to the call request, the target tool in the Model Context Protocol server integrates the relevant parameters input to the target tool into a network request. These parameters include the description information corresponding to the objective function and the environment parameters of the target tool. The environment parameters describe the session to which the operation belongs and the current communication environment. The objective function is determined based on the user's input requirements and the tool information corresponding to the callable tools determined based on those requirements. Step S203: The target tool in the Model Context Protocol server sends the network request to the interactive audio middleware. Step S204: The target tool in the Model Context Protocol server receives the execution result fed back by the interactive audio middleware, and the Model Context Protocol server sends the execution result to the Model Context Protocol host. The execution result is obtained by the interactive audio middleware after executing the operation corresponding to the objective function based on the relevant parameters.

[0064] Optionally, in this embodiment, the execution entity is an MCP server. The MCP server contains multiple MCP tools, which, in response to a Model Context Protocol (MCP) host's request to invoke a target tool, integrate the relevant parameters input to the target tool into a network request.

[0065] Specifically, (a) when the target tool is a general-purpose single-call tool for WAAPI functions, the information input to the target tool includes the description information corresponding to the target function and the environment parameters of the target tool. The description information includes the function identifier (which may vary depending on the case, as explained in the above embodiments and will not be repeated), function parameters, and option parameters. The environment parameters include the session identifier and communication connection parameters. The target tool will organize all received parameters into a network request and send the network request to the interactive audio middleware (i.e., the Wwise editor), and perform the corresponding operation through communication with the Wwise editor. The specific sending and receiving methods of the request depend on the communication method between the MCP server and the Wwise editor.

[0066] (b) When the target tool is a general-purpose batch calling tool for WAAPI functions, compared to (a), it avoids unnecessary repeated connection establishment, reduces network communication overhead, and facilitates unified processing. The input parameters of this tool should include a list of function identifiers, a list of function parameters, and a list of option parameters. The elements in these three lists correspond one-to-one in order to form a complete operation request. Similar to (a), this tool organizes the received parameters into a network request and performs the corresponding operation through communication with the Wwise editor. The main difference is that it sends and processes multiple requests in a single connection. This tool is generally used to perform batch and similar operations. Users can instruct the system to perform an operation first for verification before performing batch operations.

[0067] (c) When the target tool is a dedicated single-call tool for a specific WAAPI function, such tools are optimized for the parameter characteristics and business logic of the specific function. For example, important parameters are elevated to independent input parameters, parameter verification and construction processes are added, and more detailed descriptions and guidance are provided.

[0068] For example, when the model uses the `ak.wwise.core.audio.import` function, there are instances where key parameters are missing or the parameter format is incorrect, such as not passing language information or not adding an object type identifier before the audio object. Therefore, a separate tool is implemented for this function, listing each required parameter directly as the parameters the tool needs to receive, along with detailed descriptions. A checking mechanism is also added to automatically add an object type identifier if it is missing from the audio object list.

[0069] The target tool will organize all received parameters into a network request and send the network request to an interactive audio middleware (such as the Wwise editor), and perform the corresponding operation through communication with the Wwise editor.

[0070] (d) When the target tool is a dedicated batch calling tool for a specific WAAPI function, this type of tool combines the advantages of (b) batch calling and (c) dedicated function calling. It can execute the same type of operations in batches within a single connection, while also being optimized for a specific function. Some input parameters of this type of tool can take multiple forms; for example, a single element indicates that all operations use the same parameter, while a list indicates that each operation uses the parameters sequentially. This type of tool can improve the efficiency of model operation execution while also improving operation accuracy.

[0071] For example, by implementing the above audio import tool as a batch audio import tool, multiple import operations can be completed in a single connection. Parameters that do not need to be modified can also be reused; for example, setting the "importOperation" parameter to "useExisting" will cause all batch import operations to use the "useExisting" logic.

[0072] The target tool will organize all received parameters into a network request and send the network request to an interactive audio middleware (such as the Wwise editor). Through communication with the Wwise editor, it can manipulate objects such as sound objects, container objects, event objects, and audio package objects in the interactive audio middleware.

[0073] This application embodiment, by setting input parameters for various types of MCP tools, can automatically select and call appropriate WAAPI functions based on the user's natural language commands. This avoids the problem of developers needing to write scripts and be familiar with complex API interfaces in the traditional way, and greatly reduces the technical threshold for automated operations in the Wwise editor.

[0074] As an optional embodiment, step S203 above includes: sending a network request to a proxy program, so that after establishing a connection with the model context protocol server, the proxy program sends the network request to the interactive audio middleware via a communication protocol and receives feedback information from the interactive audio middleware, wherein the proxy program is used to manage the network request; or, creating a child process, wherein the child process creates a connection with the interactive audio middleware according to the communication protocol; sending the network request to the interactive audio middleware based on the child process and receiving feedback information from the interactive audio middleware; or, creating a temporary connection, sending the network request to the interactive audio middleware based on the temporary connection and receiving feedback information from the interactive audio middleware, and closing the temporary connection after successful sending and receiving; or, creating a global connection, sending the network request to the interactive audio middleware based on the global connection and receiving feedback information from the interactive audio middleware, and maintaining and reusing the global connection after successful sending and receiving.

[0075] Optionally, the data interaction between the MCP server and the Wwise editor can be implemented in three ways: (1) Implement a WAAPI proxy program as a data interaction relay station between the MCP server and the Wwise editor, and manage and record requests.

[0076] Upon startup, the WAAPI proxy establishes a global connection with the Wwise editor via a communication protocol (such as WebSocket) and maintains the connection state. It then starts a TCP server listening on a specified port, waiting for connection requests from the MCP server. Once the MCP server and WAAPI proxy establish a TCP connection, the WAAPI proxy receives the request from the MCP server, parses and converts the request to obtain the associated information and environment parameters of the target function, and then sends this information to the Wwise editor via the established WebSocket connection. The Wwise editor receives the target function call, processes it, and returns the execution result to the WAAPI proxy via the WebSocket connection. The proxy receives the response data from Wwise, parses and repackages it, and then returns it to the MCP server. Multiple operation requests can be sent and processed within a single TCP connection. After completing all operations on this TCP connection, the MCP server will actively close the connection. The proxy and editor will maintain the connection until the proxy is closed. This process enables data interaction between the MCP server and the Wwise editor. Furthermore, the proxy saves all received information to a log for later review. The WAAPI proxy supports multithreading and features state monitoring mechanisms, enabling stable and efficient data management, including data processing and forwarding (i.e., relaying), avoiding the repeated creation of network connections and achieving unified information management. Compared to directly creating a global connection to the Wwise editor in the MCP server, this solution avoids complex thread management and potential event loop conflicts.

[0077] (2) Through a subprocess: Sending requests to the Wwise editor and receiving responses through a subprocess. Specifically, each time the MCP server needs to call a WAAPI function, it creates and starts an independent subprocess. The subprocess communicates with the Wwise editor through the WebSocket communication protocol, executes the specified WAAPI function call, and returns the execution result to the parent process (i.e., the MCP server) through standard output or other inter-process communication methods.

[0078] (3) Direct Connection: The MCP server directly creates a WebSocket connection internally to establish communication with the Wwise editor. This solution can be implemented in two ways: creating a temporary connection and creating a global connection. In the temporary connection method, each target function temporarily creates an independent WAAPI client connection when needed and closes the connection after execution. In the global connection method, a global WAAPI client connection is created when the MCP server starts and is maintained, allowing all target functions to reuse this connection for function calls. To avoid conflicts with the MCP server's own asynchronous event loop, thread isolation or other related technologies are required.

[0079] This application implements various methods for communication between MCP servers and Wwise editors, which can establish a stable and efficient data interaction mechanism between the MCP server and Wwise editor, enabling rapid operation and feedback, and providing complete operation traceability.

[0080] This embodiment also provides a control method for interactive audio middleware, as shown in Figure 3. This method can be applied to interactive audio middleware and includes: Step S301, parsing the received network request to obtain the description information corresponding to the target function, wherein the network request is the information obtained by the model context protocol server in response to the call request for the target tool, and integrating the description information corresponding to the target function and the environment parameters of the target tool. The environment parameters are used to describe the session to which the operation belongs and the current communication environment. The target function is determined based on the user's input requirement information and the tool information corresponding to the callable tool determined based on the requirement information; Step S302, executing the operation corresponding to the target function based on the description information to obtain the execution result; Step S303, sending the execution result to the target tool in the model context protocol server.

[0081] Optionally, in this embodiment, the execution entity is an interactive audio middleware. The interactive audio middleware contains a target function for processing audio resources. Therefore, the interactive audio middleware parses the received network request, obtains the associated information corresponding to the target function to be called, and then executes the operation corresponding to the target function based on the associated information to obtain the execution result. The execution result is then sent to the model context protocol host, which in turn sends the execution result to the large language model, allowing the large language model to determine whether the execution result is correct and decide whether to continue to determine the next tool to be called or to end the process.

[0082] It should be noted that for interactive audio middleware, when parsing network requests, it only needs to obtain the description information of the target function in the body of the network request. This is because the environment parameters used when calling the tool are used to describe the communication IP address and port information of the interactive audio middleware. Therefore, other execution entities need to determine the interactive audio middleware to communicate with based on the environment parameters. However, for interactive audio middleware, it only needs to parse out the description information of the target function.

[0083] This embodiment also provides a control method for interactive audio middleware, as shown in Figure 4. This method can be applied to a Model Context Protocol (MCP) host and includes: step S401, receiving user-inputted requirement information; step S402, sending the requirement information to a large language model; step S403, receiving a call request sent by the large language model to the target tool of the MCP server, and sending the call request to the MCP server; step S404, receiving the execution result sent by the MCP server, wherein the execution result is obtained by the interactive audio middleware after executing the operation corresponding to the target function based on relevant parameters. The relevant parameters include the description information corresponding to the target function and the environment parameters of the target tool. The environment parameters are used to describe the session to which the operation belongs and the current communication environment. The target function is determined based on the user-input requirement information and the tool information corresponding to the callable tool determined based on the requirement information; step S405, sending the execution result to the large language model.

[0084] Optionally, in the overall control system of the interactive audio middleware, the Model Context Protocol (MGP) host primarily plays a role in data transmission and communication coordination, interacting with the user, the large language model, and the MGP server. The entire interaction is implemented by the MGP host according to the MGP standard. The interaction process mainly involves: receiving user-inputted requirements; sending the requirements to the large language model; receiving the call request from the large language model to the target tool on the MGP server, and sending the call request to the MGP server; receiving the execution result from the MGP server, where the execution result is obtained after the interactive audio middleware performs operations on the target function, the target function being determined based on the user-inputted requirements and the tool information corresponding to the callable tools determined based on the requirements, and the target tool being determined by the large language model based on the requirements; and sending the execution result back to the large language model.

[0085] The specific implementation process can be found in the above embodiments, and will not be repeated here. As an application embodiment of this application, as shown in Figure 5, Figure 5 is a system block diagram communication schematic diagram of the control of the interactive audio middleware according to the embodiment of this application. The system includes a user, an LLM model (i.e., a large language model), an MCP host (i.e., a model context protocol host), an MCP server (model context protocol server), a WAAPI proxy, and a Wwise editor (i.e., interactive audio middleware).

[0086] The user sends a request to the MCP host to "set the IsStreamingEnabled property of the object \Actor-Mixer Hierarchy\VO\Alice\vo_alice_0001 to true". The MCP host then sends this request to the LLM model. The LLM model analyzes the WAAPI function to be called and sends an automatic call to the MCP tool to the MCP server via the MCP host. The parameters input to the MCP tool include: waapi_function_url is "ak.wwise.core.object.setProperty", waapi_function_args is {"object": "\\Actor-Mixer Hierarchy\\VO\\Alice\\vo_alice_0001", "property": "IsStreamingEnabled", "value": true}, and waapi_function_opts is {} (in the current case, the opts parameter does not need to be set, so it is set to {}).

[0087] After receiving these parameters, the MCP tool encapsulates them into a network request, such as a TCP request, and sends it to the WAAPI proxy. The WAAPI proxy then forwards it to the Wwise editor via the WebSocket protocol. The Wwise editor parses this request, sets the specified property of the specified object to the specified value, and then returns the execution result to the MCP server. The MCP server then sends the execution result back to the MCP host, which forwards the execution result to the LLM model. The LLM model automatically determines whether the operation was successful based on the returned result and decides on the next step (either sending the result to the user via the MCP host or sending a request to the MCP host to call the next MCP tool in the MCP server).

[0088] The method by which the MCP host interacts with the user, the LLM model, and the MCP server is implemented by the MCP host in accordance with the MCP protocol standard, and this application embodiment does not elaborate or limit it.

[0089] This system implements an intelligent method for operating the Wwise editor through natural language interaction. It can improve the automation level of Wwise editor operation, enhance game audio development efficiency, reduce development costs, lower the technical threshold for automated operation, fill the technical gap in intelligent control of audio middleware editors, and support the intelligent development of audio production.

[0090] This solution can be directly configured and used on any AI application that supports the MCP protocol, making it easy to use and universal. Furthermore, its modular design gives it strong scalability.

[0091] This system can be widely used in the audio development process of various game products such as mobile games, PC games, and console games, providing a technical solution for intelligent audio development and possessing significant industrial application value and industry promotion value.

[0092] This embodiment also provides a control device for an interactive audio middleware, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0093] This embodiment provides a control device for an interactive audio middleware. The device is a large language model, as shown in Figure 6. The device includes: a first acquisition module 601, used to acquire user-inputted requirement information; a first determination module 602, used to acquire callable tools and their corresponding tool information in a model context protocol server based on the requirement information, and to determine the currently invoked target function based on the requirement information and tool information, and obtain the description information corresponding to the target function; and a second determination module 603, used to determine the invoked target tool and related parameters in the model context protocol server based on the requirement information, tool information, and the description information of the target function, and to determine the target tool and related parameters through the model context. The protocol host initiates a call request to the target tool, whereby the target tool sends a network request for the target function to the interactive audio middleware. Relevant parameters include the description information corresponding to the target function and the environment parameters of the target tool, which describe the session to which the operation belongs and the current communication environment. The judgment module 604 is used to judge the received execution result in conjunction with the requirement information, and send the execution result to the user through the model context protocol host, or determine the next tool to be called and its corresponding parameters in the model context protocol server based on the execution result and the requirement information. The execution result is obtained after the interactive audio middleware performs the operation on the target function.

[0094] In some optional implementations, the first determining module 602 is used to obtain an index document introducing all functions, wherein the index document contains the location information of all functions, the uses of all functions, and document information introducing the usage of each function; to learn the content in the index document, and to obtain a preset number of reference documents introducing the specific usage of the functions based on the content in the index document and the requirement information; and to learn the content in the reference documents.

[0095] In some optional embodiments, the apparatus further includes: obtaining guidance information for accessing a target operation log file contained in the demand information, wherein the target operation log file stores operation process records executed in each historical session; obtaining the path to access the target operation log file based on the filename corresponding to the target operation log file; and learning the target operation log file based on the path of the target operation log file.

[0096] In some optional implementations, the first determining module 602 is used to obtain the callable tools and the tool information corresponding to each tool based on the demand information. The tool information includes tool description information and tool annotation information. When the tool type is not a specific type, the tool description information and tool annotation information corresponding to the tool information are first tool description information and first tool annotation information. When the tool type is a specific type, the tool description information and tool annotation information corresponding to the tool information are second tool description information and second tool annotation information.

[0097] In some optional implementations, the second determining module 603 is used to determine the number of target functions based on requirement information, tool information, and description information of target functions; determine the tool type of the target tool to be called based on requirement information, tool information, description information of target functions, and the number of target functions; if the tool type of the target tool is not a specific type, determine the target tool to be called based on requirement information, description information of target functions, tool information, and the number of target functions; if the tool type of the target tool is a specific type, determine the target tool to be called based on requirement information, description information of target functions, tool information, and the number of target functions.

[0098] In some optional implementations, the second determining module 603 is used to select a target tool based on requirement information, description information of the target function, description information of the first tool, and annotation information of the first tool when the number of target functions is a preset number, wherein the target tool is used to send one network request; and to select a target tool based on requirement information, description information of the target function, description information of the first tool, and annotation information of the first tool when the number of target functions is greater than the preset number, wherein the target tool is used to send multiple network requests.

[0099] In some optional implementations, the second determining module 603 is used to select a target tool based on requirement information, description information of the target function, description information of the second tool, and annotation information of the second tool when the number of target functions is a preset number, wherein the target tool is used to send one network request; and to select a target tool based on requirement information, description information of the target function, description information of the second tool, and annotation information of the second tool when the number of target functions is greater than the preset number, wherein the target tool is used to send multiple network requests.

[0100] This embodiment provides a control device for interactive audio middleware. The device is a Model Context Protocol (MCP) server, as shown in Figure 7. The device includes: a first receiving module 701, used to receive a call request from the MCP host to a target tool; an integration module 702, used to respond to the call request by calling the target tool in the MCP server to integrate relevant parameters input to the target tool into a network request. The relevant parameters include description information corresponding to the target function and environmental parameters of the target tool. The environmental parameters describe the session to which the operation belongs and the current communication environment. The target function is determined based on user-input requirements and tool information corresponding to the callable tools determined based on the requirements; a first sending module 703, used by the target tool in the MCP server to send the network request to the interactive audio middleware; and a second sending module 704, used by the target tool in the MCP server to receive the execution result fed back by the interactive audio middleware, and the MCP server to send the execution result to the MCP host. The execution result is obtained by the interactive audio middleware after executing the operation corresponding to the target function based on the relevant parameters.

[0101] In some optional implementations, the first sending module 703 is used to send network requests to a proxy program, so that after establishing a connection with the Model Context Protocol server, the proxy program sends the network requests to the interactive audio middleware via a communication protocol and receives feedback information from the interactive audio middleware, wherein the proxy program is used to manage network requests; or, a child process is created, wherein the child process establishes a connection with the interactive audio middleware according to the communication protocol; the child process sends network requests to the interactive audio middleware and receives feedback information from the interactive audio middleware; or, a temporary connection is created, the network requests are sent to the interactive audio middleware based on the temporary connection, and feedback information from the interactive audio middleware is received, and the temporary connection is closed after successful sending and receiving; or, a global connection is created, the network requests are sent to the interactive audio middleware based on the global connection, and feedback information from the interactive audio middleware is received, and the global connection is maintained and reused after successful sending and receiving.

[0102] This embodiment provides a control device for an interactive audio middleware. As shown in Figure 8, the device includes: a parsing module 801, used to parse received network requests to obtain description information corresponding to a target function. The network request is information obtained by the Model Context Protocol server responding to a call request for a target tool, integrating the description information corresponding to the target function and the environmental parameters of the target tool. The environmental parameters describe the session to which the operation belongs and the current communication environment. The target function is determined based on user-input requirements and tool information corresponding to the callable tools determined based on the requirements. A obtaining module 802 is used to execute the operation corresponding to the target function based on the description information to obtain the execution result. A third sending module 803 is used to send the execution result to the target tool in the Model Context Protocol server.

[0103] This embodiment provides a control device for interactive audio middleware. This device is a Model Context Protocol (MCP) host, as shown in Figure 9. The device includes: a second receiving module 901 for receiving user-inputted requirement information; a fourth sending module 902 for sending the requirement information to a large language model; a third receiving module 903 for receiving a call request sent by the large language model to the target tool of the MCP server and sending the call request to the MCP server; a fourth receiving module 904 for receiving the execution result sent by the MCP server, wherein the execution result is obtained by the interactive audio middleware after executing the operation corresponding to the target function based on relevant parameters. The relevant parameters include the description information corresponding to the target function and the environment parameters of the target tool. The environment parameters describe the session to which the operation belongs and the current communication environment. The target function is determined based on the user-inputted requirement information and the tool information corresponding to the callable tool determined based on the requirement information; and a fifth sending module 905 for sending the execution result to the large language model.

[0104] The control device for the interactive audio middleware provided in this disclosure can execute the control method for the interactive audio middleware provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0105] Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0106] Referring specifically to FIG10, a schematic diagram of a structure suitable for implementing an electronic device according to an embodiment of the present disclosure is shown below. The electronic device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a memory 10010 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device. The processor 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0107] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication devices 1009 allow the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 10 shows an electronic device with various devices, it should be understood that it is not required to implement or possess all the devices shown, and more or fewer devices may be implemented alternatively.

[0108] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from memory 1008, or installed from ROM 1002. When the computer program is executed by processor 1001, it performs the functions defined in the control method of the interactive audio middleware of embodiments of this disclosure.

[0109] The electronic device shown in Figure 10 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0110] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the control method of the interactive audio middleware shown in the above embodiments is implemented.

[0111] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0112] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A control method for interactive audio middleware, characterized in that, The method includes: acquiring user-inputted requirement information; acquiring callable tools and corresponding tool information in a Model Context Protocol (MCP) server based on the requirement information, determining the currently invoked target function based on the requirement information and the tool information, and obtaining description information corresponding to the target function; determining the invoked target tool and related parameters in the MCP server based on the requirement information, the tool information, and the description information of the target function, and initiating a call request to the target tool through a MCP host, wherein the target tool is used to send a network request about the target function to the interactive audio middleware; the related parameters include the description information corresponding to the target function and the environment parameters of the target tool, the environment parameters being used to describe the session to which the operation belongs and the current communication environment; judging the received execution result in conjunction with the requirement information, sending the execution result to the user through the MCP host, or determining the next invoked tool and its corresponding related parameters in the MCP server based on the execution result and the requirement information, wherein the execution result is obtained after the interactive audio middleware performs an operation on the target function.

2. The method according to claim 1, characterized in that, The step of determining the target function to be called based on the requirement information and the tool information includes: determining the candidate tool to be called based on the requirement information and the tool information; when the candidate tool contains a corresponding specified function, using the specified function as the target function to be called; and when the candidate tool does not contain a corresponding specified function, determining the target function to be called based on the requirement information.

3. The method according to claim 1, characterized in that, The method further includes: obtaining an index document introducing all functions, wherein the index document contains location information of all functions, uses of all functions, and document information introducing the usage of each function; learning the content in the index document, and obtaining a preset number of reference documents introducing the specific usage of a particular function based on the content in the index document and the requirement information; and learning the content in the reference documents.

4. The method according to claim 1, characterized in that, The method further includes: obtaining guidance information for accessing a target operation record file contained in the requirement information, wherein the target operation record file stores operation process records executed in each historical session; obtaining the path to access the target operation record file based on the filename corresponding to the target operation record file; and learning the target operation record file based on the path of the target operation record file.

5. The method according to claim 1, characterized in that, The step of obtaining the callable tools and corresponding tool information in the Model Context Protocol server based on the requirement information includes: obtaining the callable tools and corresponding tool information based on the requirement information, wherein the tool information includes tool description information and tool annotation information. When the tool type is not a specific type, the tool description information and tool annotation information corresponding to the tool information are first tool description information and first tool annotation information. When the tool type is a specific type, the tool description information and tool annotation information corresponding to the tool information are second tool description information and second tool annotation information.

6. The method according to claim 5, characterized in that, Based on the requirement information, the tool information, and the description information of the objective function, determining the target tool to be invoked in the Model Context Protocol server includes: determining the number of objective functions based on the requirement information, the tool information, and the description information of the objective function; determining the tool type of the invoked target tool based on the requirement information, the tool information, the description information of the objective function, and the number of objective functions; if the tool type of the target tool is not a specific type, determining the invoked target tool based on the requirement information, the description information of the objective function, the tool information, and the number of objective functions; if the tool type of the target tool is a specific type, determining the invoked target tool based on the requirement information, the description information of the objective function, the tool information, and the number of objective functions.

7. The method according to claim 6, characterized in that, When the tool type of the target tool is not a specific type, the target tool to be invoked is determined based on the requirement information, the description information of the target function, the tool information, and the number of target functions. This includes: when the number of target functions is a preset number, selecting the target tool based on the requirement information, the description information of the target function, the first tool description information, and the first tool annotation information, wherein the target tool is used to send one network request; when the number of target functions is greater than the preset number, selecting the target tool based on the requirement information, the description information of the target function, the first tool description information, and the first tool annotation information, wherein the target tool is used to send multiple network requests.

8. The method according to claim 6, characterized in that, When the tool type of the target tool is a specific type, determining the target tool to be invoked based on the requirement information, the description information of the target function, the tool information, and the number of target functions includes: when the number of target functions is a preset number, selecting the target tool based on the requirement information, the description information of the target function, the description information of the second tool, and the annotation information of the second tool, wherein the target tool is used to send one network request; when the number of target functions is greater than the preset number, selecting the target tool based on the requirement information, the description information of the target function, the description information of the second tool, and the annotation information of the second tool, wherein the target tool is used to send multiple network requests.

9. A control method for interactive audio middleware, characterized in that, The method includes: receiving a call request from a Model Context Protocol (MCP) host to a target tool; responding to the call request, invoking the target tool in the MCP server to integrate relevant parameters input to the target tool into a network request, wherein the relevant parameters include descriptive information corresponding to the target function and environmental parameters of the target tool, the environmental parameters describing the session to which the operation belongs and the current communication environment, and the target function being determined based on user-input requirement information and tool information corresponding to the callable tools determined based on the requirement information; the target tool in the MCP server sending the network request to the interactive audio middleware; the target tool in the MCP server receiving the execution result fed back by the interactive audio middleware, and the MCP server sending the execution result to the MCP host, wherein the execution result is obtained by the interactive audio middleware after executing the operation corresponding to the target function based on the relevant parameters.

10. The method according to claim 9, characterized in that, The target tool in the Model Context Protocol (MCP) server sends the network request to the interactive audio middleware, including: sending the network request to a proxy program, so that after establishing a connection with the MCP server, the proxy program sends the network request to the interactive audio middleware via a communication protocol and receives feedback information from the interactive audio middleware, wherein the proxy program is used to manage the network request; or, creating a subprocess, wherein the subprocess creates a connection with the interactive audio middleware according to the communication protocol; sending the network request to the interactive audio middleware based on the subprocess and receiving feedback information from the interactive audio middleware; or, creating a temporary connection, sending the network request to the interactive audio middleware based on the temporary connection, and receiving feedback information from the interactive audio middleware, closing the temporary connection after successful sending and receiving; or, creating a global connection, sending the network request to the interactive audio middleware based on the global connection, and receiving feedback information from the interactive audio middleware, maintaining and reusing the global connection after successful sending and receiving.

11. A control method for interactive audio middleware, characterized in that, The method includes: parsing a received network request to obtain description information corresponding to a target function, wherein the network request is information obtained by the Model Context Protocol server responding to a call request for a target tool, and integrating the description information corresponding to the target function and the environment parameters of the target tool, wherein the environment parameters are used to describe the session to which the operation belongs and the current communication environment, and the target function is determined based on user-input requirement information and tool information corresponding to the callable tools determined based on the requirement information; executing the operation corresponding to the target function based on the description information to obtain an execution result; and sending the execution result to the target tool in the Model Context Protocol server.

12. A control method for interactive audio middleware, characterized in that, The method includes: receiving user-inputted requirement information; sending the requirement information to a large language model; receiving a call request sent by the large language model to a target tool of a model context protocol server, and sending the call request to the model context protocol server; receiving an execution result sent by the model context protocol server, wherein the execution result is obtained by the interactive audio middleware performing an operation corresponding to a target function based on relevant parameters, the relevant parameters including descriptive information corresponding to the target function and environmental parameters of the target tool, the environmental parameters being used to describe the session to which the operation belongs and the current communication environment, the target function being determined based on the user-input requirement information and tool information corresponding to the callable tool determined based on the requirement information; and sending the execution result to the large language model.

13. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the control method of the interactive audio middleware according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the control method of the interactive audio middleware according to any one of claims 1 to 12.

15. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform a control method for the interactive audio middleware according to any one of claims 1 to 12.

Images