Intelligent interaction method, computing device, storage medium and program product

Abstract

The invention provides an intelligent interaction method, computing equipment, a storage medium and a program product. The method relates to the field of artificial intelligence, and comprises the steps of obtaining an execution condition and a historical behavior path of a first skill after the execution of the first skill is finished; dynamically deciding a second skill associated with the first skill according to the execution condition and the historical behavior path of the first skill by using the powerful thinking and reasoning capabilities of the large model; according to the technical scheme of the invention, the skill (namely the second skill) associated with the previous skill can be customized and recommended for the user according to the execution condition and historical behavior path of the previous skill (namely the first skill) of the user, and the personalized recommendation of the associated skill / ability can be realized, so that the interactive process of thousands of people and thousands of aspects in the real sense can be realized. And the goal of thousands of users' interaction can be achieved, so that the intelligent interaction system is more intelligent.

Description

Technical Field

[0001] The present application relates to computer technology, and in particular to an intelligent interaction method, a computing device, a storage medium and a program product. Background Art

[0002] In the field of intelligent interaction, users have different goals and needs for intelligent interaction with the system, which is different for each user. Current intelligent interaction systems usually only provide fixed capabilities / functions, such as page content summary, word search, etc. These fixed capabilities cannot achieve the interactive goals of different users. The intelligent interaction system has a low degree of intelligence and cannot achieve the interactive goals of different users. Summary of the invention

[0003] The present application provides an intelligent interaction method, a computing device, a storage medium and a program product, which are used to solve the problem that the intelligent interaction system has a low intelligence level and cannot achieve the interaction goal of thousands of users with different faces.

[0004] In a first aspect, the present application provides an intelligent interaction method, comprising: in response to the completion of execution of a first skill, obtaining the execution status and historical behavior path of the first skill; using a large model to dynamically decide a second skill associated with the first skill based on the execution status and historical behavior path of the first skill; and outputting a trigger component of the second skill.

[0005] In a second aspect, the present application provides a computing device, comprising: a memory and a processor; the memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, and when the computer programs / instructions are executed by the processor, the method described in any of the preceding aspects is implemented.

[0006] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method provided in any of the aforementioned aspects is implemented.

[0007] In a fourth aspect, the present application provides a computer program product, including a computer program, which implements the method provided in any of the above aspects when executed by a processor.

[0008] The intelligent interaction method, computing device, storage medium and program product provided by the present application obtain the execution status and historical behavior path of the first skill after the execution of the first skill is completed; use the powerful thinking and reasoning ability of the large model to dynamically decide the second skill associated with the first skill according to the execution status and historical behavior path of the first skill; and output the trigger component of the second skill, which can customize and recommend the skill (i.e., the second skill) associated with the previous skill for the user based on the execution status and historical behavior path of the previous skill (i.e., the first skill), realize personalized recommendation of related skills / abilities, thereby realizing a truly personalized interaction process for thousands of people, achieving the interactive goals of thousands of users, and making the intelligent interaction system more intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0010] Figure 1 A schematic diagram of an intelligent interaction scenario provided in an embodiment of the present application;

[0011] Figure 2 A flowchart of an intelligent interaction method provided for an exemplary embodiment of the present application;

[0012] Figure 3 A flowchart of an intelligent interaction method based on a finite state machine provided by an exemplary embodiment of the present application;

[0013] Figure 4 An example diagram of state transition information of a finite state machine model provided by an exemplary embodiment of the present application;

[0014] Figure 5 A flowchart of the second stage training provided for an exemplary embodiment of the present application;

[0015] Figure 6 A flowchart of searching for a target behavior path that meets user needs provided by an exemplary embodiment of the present application;

[0016] Figure 7 It is a structural block diagram of a computing device according to an embodiment of the present application.

[0017] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0018] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0019] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0020] First, the terms involved in this application are explained:

[0021] Skills: refers to the basic capability units that can be achieved by intelligent interactive systems, which can be functional modules, service components, data interfaces, tools, etc.

[0022] In-Context Learning (ICL): A machine learning method in which the model reasoning and generation is performed given a context. Unlike traditional supervised learning, ICL does not require a large amount of labeled data, but relies on the model's ability to learn and apply knowledge in a specific context. In this solution, ICL is used to enhance the adaptability and generalization ability of the model, especially when dealing with dynamic and diverse user needs, it can quickly understand and respond to user intentions and provide more personalized services.

[0023] Key-Value Cache (KVCache): is an efficient data storage structure used to cache key-value pairs. It significantly improves the speed and efficiency of data retrieval by quickly accessing and updating key-value pairs. In this solution, KVCache is used to accelerate data processing and response time, especially in scenarios with high concurrency and large data volumes. It can effectively reduce the burden on the backend system and improve overall performance.

[0024] Minimal Inference (Minference): is a lightweight reasoning method that aims to achieve efficient reasoning and decision-making in resource-constrained environments. By optimizing the structure and calculation process of the model, Minference can reduce the consumption of computing resources while maintaining a high accuracy rate. In this solution, Minference is used to optimize the operating efficiency of the model, especially on edge devices or mobile devices. It can ensure that the model can still provide high-quality services under the requirements of low power consumption and low latency.

[0025] Finite State Machine (FSM): is a mathematical model used to model and analyze systems with a finite number of states and transitions between these states. Widely used in computer science and software engineering, it decomposes complex system behaviors into a series of discrete states, each of which represents a stable configuration of the system, and describes how the system responds to different inputs or events by defining transition rules between states. In this technical solution, FSM is used to infer the next operation to be performed based on the current state and historical operations. Especially in simple application scenarios, it helps to build user behavior path planning.

[0026] Monte Carlo Tree Search (MCTS): is a heuristic search algorithm that is particularly suitable for dealing with problems with randomness, uncertainty or incomplete information, such as game strategy formulation or complex decision-making processes. The algorithm simulates possible behavior paths by constructing a probabilistic search tree and optimizes decisions through repeated "select-expand-simulate-backpropagation" cycles. In this solution, MCTS is used to search for user behavior paths in complex scenarios, especially when it is difficult to enumerate all possible operations or predict all emergencies. It uses a large model to evaluate the value of state nodes to find the optimal or approximately optimal decision path.

[0027] Multimodal pre-trained model: refers to a pre-trained model whose input and output data involve multiple modal data such as images and text. After fine-tuning and training, it can be applied to multimodal task processing.

[0028] Pre-trained language model: A pre-trained model obtained by pre-training a large-scale language model (LLM).

[0029] A large model refers to a deep learning model with large-scale model parameters, usually containing hundreds of millions, tens of billions, or even hundreds of billions of model parameters. A large model can also be called a foundation model / foundation model (FM). It is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as large-scale language models and multi-modal pre-training models.

[0030] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0031] In response to the technical problem that the intelligent interaction system has a low level of intelligence and cannot achieve the interactive goal of adapting to different users, the present application provides an intelligent interaction method, which obtains the execution status and historical behavior path of the first skill after the execution of the first skill is completed; uses the powerful thinking and reasoning capabilities of the large model to dynamically decide on a second skill associated with the first skill based on the execution status and historical behavior path of the first skill; and outputs a trigger component for the second skill, which can customize and recommend a skill (i.e., the second skill) associated with the previous skill for the user based on the execution status and historical behavior path of the previous skill (i.e., the first skill), and achieve personalized recommendation of associated skills / abilities, thereby realizing a truly personalized interactive process, achieving the interactive goal of adapting to different users, and making the intelligent interaction system more intelligent.

[0032] Among them, the historical behavior path of the first skill includes: skills (including the first skill) that have been executed to meet the current user needs (that is, the user needs corresponding to the first skill), and the skills in the historical behavior path are arranged in the order of execution.

[0033] The execution status of any skill may include, but is not limited to: the execution log of the skill, the execution result, and the feedback information on the execution result. Among them, the execution log of the skill records the operation log of the skill execution process, including the time of skill execution, the information of the knowledge base used, the execution status (success or failure), etc. The execution result refers to the structured information of the result data generated by the skill execution. For example, for the skill of building an AI (artificial intelligence) assistant, its execution result may include relevant information of a built intelligent body, including the large model called, the tools used, etc.; the execution result may also include the evaluation dimension of the intelligent body, etc., which can be set according to the actual application requirements and is not specifically limited here. Feedback information on the execution result refers to the feedback information given by the user on the execution result of the skill, including but not limited to whether the execution result meets the user's needs, error information, etc.

[0034] Figure 1 A schematic diagram of an intelligent interaction scenario provided in an embodiment of the present application. Figure 1 As shown in the figure, the interaction process of the intelligent interaction platform includes:

[0035] The user submits user demand information to the intelligent interaction platform through the terminal device. The user demand information reflects the goals and intentions that the user wants to achieve. In response to the user's needs, the intelligent interaction platform executes at least one skill (called the first skill) that matches the user's needs. After the first skill is executed, the execution result may well meet the user's needs, or it may not meet the user's needs.

[0036] In this embodiment, in order to better meet user needs, after the first skill is executed, the intelligent interaction platform can obtain the execution status and historical behavior path of the first skill; using the big model, according to the execution status and historical behavior path of the first skill, dynamically decide the second skill associated with the first skill, and output the trigger component of the second skill, so as to customize and recommend the skill associated with the previous skill (i.e., the second skill) for the user. By executing the second skill, user needs can be better met, and personalized recommendations of related skills / abilities can be achieved, thereby realizing a truly personalized interaction process for thousands of people, making the intelligent interaction system more intelligent.

[0037] In an example scenario, a user can submit a product description document to the intelligent interactive platform and ask the intelligent interactive platform to build a product usage description assistant based on the product description document. The product usage description assistant has the ability to ask and answer questions, and can automatically give correct answers to questions related to the use of the product submitted by the customer based on the product description document. The intelligent interactive platform completes the construction of the product usage description assistant by executing at least one skill.

[0038] Furthermore, the intelligent interactive platform can determine the AI ​​assistant automatic evaluation skill associated with the product instruction assistant based on the execution status and historical behavior path of the skill used to build the product instruction assistant. By executing the AI ​​assistant automatic evaluation skill, the built product instruction assistant can be automatically evaluated to obtain the evaluation results, so as to automatically evaluate whether the built product instruction assistant meets user needs.

[0039] Furthermore, the intelligent interaction platform can determine the associated AI assistant automatic optimization skills based on the execution status and historical behavior path of the AI ​​assistant automatic evaluation skills. By executing the AI ​​assistant automatic optimization evaluation skills, the built product instruction assistant can be automatically optimized, thereby improving the performance of the built product instruction assistant, so that the built product instruction assistant can better meet user needs.

[0040] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0041] Figure 2 This is a flow chart of an intelligent interaction method provided by an exemplary embodiment of the present application. The execution subject of this embodiment is a computing device that implements a real-time intelligent interaction system, such as servers of various interactive intelligent platforms / systems, intelligent entities that implement intelligent interaction functions, etc. Figure 2 As shown, the specific steps of this method are as follows:

[0042] Step S201: In response to the completion of execution of the first skill, the execution status and historical behavior path of the first skill are obtained.

[0043] Among them, the first skill refers to one or more skills matched by the intelligent interaction system based on user demand information. The intelligent interaction system executes the first skill to obtain results that meet user needs.

[0044] In actual applications, the intelligent interaction platform can analyze the user's intention based on the input user demand information, select at least one skill that matches the user's intention as the first skill that matches the user demand information, and execute the first skill to obtain the execution result of the first skill. However, the execution result of the first skill may not meet the user's demand. The execution result may meet the user's demand well, or may not meet the user's demand well, or may even fail to meet the user's demand. In this embodiment, there is no specific limitation on how to determine the matching skills based on the user demand information.

[0045] The solution of this embodiment can dynamically recommend a second skill associated with the first skill to the user based on the execution status and historical behavior path of the first skill that has been executed, in order to better meet the user's needs that are not well met by the first skill, in order to better meet the user's needs through the execution of the second skill.

[0046] The solution of this embodiment can also target user needs that can be better met by the first skill, based on the execution status and historical behavior path of the first skill that has been executed, explore user potential needs, and dynamically recommend a second skill associated with the first skill to the user, hoping to better meet the user's potential needs through the execution of the second skill.

[0047] In this step, the execution status and historical behavior path of the first skill are first obtained to provide a basis for dynamically deciding the data of the second skill associated with the first skill.

[0048] Among them, the historical behavior path of the first skill includes: skills (including the first skill) that have been executed to meet the current user needs (that is, the user needs corresponding to the first skill), and the skills in the historical behavior path are arranged in the order of execution.

[0049] The execution status of any skill may include but is not limited to: the skill execution log, execution results, and feedback information on the execution results. The skill execution log records the operation log of the skill execution process, including the time of skill execution, information about the knowledge base used, execution status (success or failure), etc. The execution result refers to the structured information of the result data generated by the skill execution.

[0050] For example, for the skill of building an AI (artificial intelligence) assistant, the execution result may include relevant information of a built intelligent agent, including the called large model, the tools used, etc.; the execution result may also include the evaluation dimensions of the intelligent agent, etc., which can be set according to actual application needs and are not specifically limited here.

[0051] Feedback information on execution results refers to feedback information given by users on the execution results of skills, including but not limited to whether the execution results meet user requirements, error information, etc.

[0052] Step S202: Use the big model to dynamically decide the second skill associated with the first skill based on the execution status of the first skill and the historical behavior path.

[0053] After obtaining the execution status and historical behavior path of the first skill, the powerful thinking and reasoning capabilities of the large model are utilized. With user personalized tasks as the center, user needs are analyzed and understood based on the execution status and historical behavior path of the first skill used to complete user personalized tasks. The second skill associated with the first skill is dynamically decided to achieve personalized recommendations of related skills / abilities, which can better meet user needs.

[0054] Among them, the large model can be various pre-trained models, or large models that have been fine-tuned in the field, including but not limited to various large language models LLM and multimodal large models, which are not specifically limited in this embodiment.

[0055] Exemplarily, in some relatively simple intelligent interaction scenarios, a finite state machine (FSM) model corresponding to the intelligent interaction system can be pre-built, and the state transition information of the finite state machine (FSM) model, such as a state transition diagram, a state transition table, etc., can be determined. The execution status and historical behavior path of the first skill, as well as the state transition information of the finite state machine model, are input into the large model for dynamic decision-making, and the large model outputs the second skill associated with the first skill. At the same time, the large model can also output the value of the entry parameter of the second skill to provide the parameter value required to execute the second skill. Optionally, the large model can also output the reasoning process information of the associated second skill, such as the thinking / reasoning steps of the large model. The specific implementation of this solution is described in detail in subsequent embodiments.

[0056] For example, in some more complex intelligent interaction scenarios, or in scenarios where experience information is missing (cold start), the FSM model is difficult to cover all interaction scenarios, and the limitations of the FSM model will limit the intelligence of the intelligent interaction system. Therefore, the Monte Carlo Tree Search (MCTS) algorithm is used to treat each skill (or operation) in the behavior path as a node on the search tree; the large model is used as a value function to evaluate whether the current search results meet user needs based on the execution of the first skill and the historical behavior path, and the evaluation results are used to determine the reward value of the node; the feedback information in the historical behavior path in the intelligent interaction system and the reward value and number of visits of the node are used to continuously adjust the confidence information of the nodes in the search tree, thereby adjusting the search strategy (the strategy of selecting nodes based on confidence information) to search for the associated second skill, which can improve the decision-making planning ability of the system in complex interaction scenarios. The specific implementation of this solution is described in detail in the subsequent embodiments.

[0057] Step S203: output the trigger component of the second skill.

[0058] After determining the second skill associated with the first skill, the trigger component of the second skill is output through the front-end page to recommend the second skill associated with the first skill to the user, so that the user can trigger the execution of the second skill, thereby improving the intelligence of the intelligent interaction platform and better meeting user needs.

[0059] The user can trigger the execution of the second skill by performing a trigger operation on the trigger component of the second skill displayed on the front-end page, such as clicking the trigger component of the second skill. In response to any trigger component of the second skill being triggered, the intelligent interactive system executes the second skill.

[0060] This embodiment scheme obtains the execution status and historical behavior path of the first skill after the execution of the first skill is completed; uses the powerful thinking and reasoning ability of the large model to dynamically decide the second skill associated with the first skill according to the execution status and historical behavior path of the first skill, and outputs the trigger component of the second skill. It can customize and recommend skills (i.e., the second skill) associated with the previous skill for the user based on the execution status and historical behavior path of the previous skill (i.e., the first skill), realize personalized recommendation of related skills / abilities, thereby realizing a truly personalized interaction process for each user, achieving the interactive goal of each user, and making the intelligent interaction system more intelligent.

[0061] Figure 3 The flowchart of the intelligent interaction method based on the finite state machine is provided for an exemplary embodiment of the present application. Based on the above embodiment, in an optional embodiment, Figure 3 As shown, in the aforementioned step S202, a large model is used to dynamically decide the second skill associated with the first skill according to the execution status of the first skill and the historical behavior path, which can be specifically implemented in the following manner:

[0062] Step S301: Obtain a finite state machine model corresponding to the intelligent interactive system, where the finite state machine model includes at least one skill.

[0063] In this embodiment, a finite state machine model corresponding to the intelligent interactive system may be pre-built based on the instruction document, expert experience information, etc. in the intelligent interactive system.

[0064] In this embodiment, key elements involved in the finite state machine (FSM) model include: operation, state, transition, event, state context, and operation flow.

[0065] Among them, operation refers to a skill in the intelligent interaction system, specifically a series of actions that can realize intelligent interaction functions, which are usually triggered by operable elements on the corresponding page, such as clicking the "knowledge access" button, configuring training parameters, etc. The operations here can be divided into two types, which are distinguished by whether external user input is required. The operation space that does not require external input can be enumerated, while the operation space that requires external input cannot be fully enumerated.

[0066] States refer to different pages that are redirected after an operation is performed. The execution of an operation (i.e., a skill) causes the state of the intelligent interactive system to change.

[0067] Transition means that after performing different operations on the current page (i.e. the current state), it may cause the current page to jump to other pages or the content in the current page to change dynamically, that is, a state transition occurs.

[0068] An event refers to a state transition condition. Any state can be transferred to a new state after satisfying certain events (or state transition conditions).

[0069] State context refers to the information displayed on a page. The optional operations and the displayed state context on each page at different stages in the intelligent interaction system are defined. The state context displayed on these pages is of great significance for determining the associated skills.

[0070] In this step, the usage rules of at least one skill in the intelligent interactive system are determined based on the usage instructions and expert experience information in the intelligent interactive system, and a skill rule library is constructed. The skill usage rules include the page that provides the skill (i.e., the state), the rules for skill execution, and the page that jumps to after the skill is executed. Based on the skill rule library, a finite state machine model is constructed. The constructed finite state machine model contains the skills in the skill rule library.

[0071] Specifically, the big model can be used to construct a finite state machine model corresponding to the intelligent interactive system. Specifically, the instruction document, skill rule library and prompt words for generating the FSM model in the intelligent interactive system are input into the big model together, and the big model automatically constructs the FSM model based on the prompt words for generating the FSM model and the instruction document and skill rule library in the intelligent interactive system.

[0072] Optionally, considering that the length of the instruction document and the skill rule library in the intelligent interaction system is relatively long, it will affect the speed of building the FSM model of the large model. The process of building the FSM model of the large model includes two stages: prefilling and decoding. During the prefilling process of the large model, all input tokens need to be calculated. In order to speed up the prefilling speed, caching technologies such as KVCache can be used to prefill the tokens corresponding to the instruction document and the skill rule library in the intelligent interaction system, and store the calculation results in the cache. When building the FSM model, the large model can directly use the pre-cached calculation results for prefilling, thereby improving the speed of prefilling. In addition, Minference technology can also be used to speed up the prefilling process.

[0073] In addition, the constructed finite state machine model can describe the operation flow of using various skills in the intelligent interactive system to realize specific interactive functions, and can also be constructed by relevant technical personnel based on the instruction documents and expert experience information in the intelligent interactive system.

[0074] After the finite state machine model is determined, state transition information of the finite state machine model, such as a state transition diagram, a state transition table, etc., may also be determined.

[0075] For example, Figure 4 This is an example diagram of state transition information of a finite state machine model provided in this embodiment. Figure 4 As shown, when the intelligent interactive system is started, it enters the corresponding interactive page, and the system is in the starting state S0. After executing the e0 operation (such as entering the knowledge access function), the corresponding page of the knowledge access function is entered, and the system is in the knowledge access S1 state; after executing the e1 operation (executing the knowledge access skill, such as executing the a1 operation of document upload), the knowledge access result display interface is entered, and the system is in the knowledge access S2 state, indicating that the knowledge access is completed; after executing the e2 operation (such as entering the knowledge base creation function), the corresponding page of the knowledge base creation function is entered, and the system is in the knowledge base creation S3 state; after executing the e3 operation (executing the knowledge base creation skill, such as executing the a2 operation of creating a knowledge base), the knowledge base creation result display interface is entered, and the system is in the knowledge base creation S4 state, indicating that the knowledge base creation is completed. After executing the e4 operation (such as ending the interaction), the system enters the ending state S5.

[0076] It should be noted that in practical applications, as the use of instruction documents, interaction data and other experience information in the intelligent interaction system increases, the finite state machine model corresponding to the intelligent interaction system and the state transition information of the finite state machine model can be continuously updated.

[0077] For example, relevant technicians can develop new skills according to actual application requirements and add the rule information of the new skills to the skill rule library. For example, the intelligent interaction system can update the skill rule library at intervals according to the newly added usage instruction documents and interaction data during this period.

[0078] Further, in response to the update of the skill rule base, the finite state machine model and the state transition information of the finite state machine model are updated.

[0079] Exemplarily, the instruction document, updated skill rule library and prompt words for generating the FSM model in the current intelligent interactive system can be input into the big model together, and the new FSM model can be constructed by the big model based on the prompt words for generating the FSM model. It should be noted that in the prefill process, for the tokens of the previously existing instruction document and skill rule library, the calculation results of these tokens that are pre-cached can be used. It is only necessary to calculate the tokens of the newly added instruction document and the updated part of the skill rule library, and store the calculation results of the tokens of the newly added instruction document and the updated part of the skill rule library in the cache, which can improve the speed of prefill.

[0080] Step S302: Based on the user demand information in the execution status of the first skill, determine whether the finite state machine model contains a target skill that matches the user demand information.

[0081] Among them, the execution status of the first skill includes user demand information and other relevant information. Please refer to the description of the aforementioned embodiment for details, which will not be repeated here.

[0082] After acquiring the finite state machine model corresponding to the intelligent interactive system, it is determined whether the finite state machine model includes a target skill that matches the user demand information based on the user demand information in the execution of the first skill.

[0083] Specifically, the user intent corresponding to the user demand information can be determined according to the execution status of the first skill, and the rule information of the skill in the skill rule library includes the user intent matched by the skill, that is, the user intent that the skill can satisfy. Determining the user intent corresponding to the user demand information can be achieved by any method for identifying the user intent based on the user demand information, which is not specifically limited here.

[0084] Further, based on the skills included in the finite state machine model and the user intents matched by the various skills, it can be determined whether the finite state machine model includes skills that match the user intent corresponding to the user demand information. If the finite state machine model includes skills that match the user intent corresponding to the user demand information, it is determined that the finite state machine model includes target skills that match the user demand information. If the finite state machine model does not include skills that match the user intent corresponding to the user demand information, it is determined that the finite state machine model does not include target skills that match the user demand information.

[0085] According to the judgment result, if it is determined that the finite state machine model contains the target skill that matches the user's intention, then execute step S303, use the large model, based on the finite state machine (FSM) model corresponding to the intelligent interaction system, to dynamically decide the associated second skill and the value of the entry parameter of the second skill.

[0086] According to the judgment result, if it is determined that the finite state machine model does not contain the target skill that matches the user's intention, then execute steps S304-S305, adopt a search algorithm, use the large model as the value function, search for the target behavior path that meets the user's needs, determine the second skill associated with the first skill based on the target behavior path, and obtain the value of the entry parameter of the second skill.

[0087] Step S303: If it is determined that the finite state machine model contains a target skill that matches the user's intention, the execution status and historical behavior path of the first skill, as well as the state transition information of the finite state machine model, are input into the decision model for dynamic decision-making, and the second skill associated with the first skill and the value of the entry parameter of the second skill are output.

[0088] When it is determined that the finite state machine model contains the target skill that matches the user's intention, the execution status and historical behavior path of the first skill, as well as the state transition information of the finite state machine model, are input into the decision model for dynamic decision making, and the decision model outputs the second skill associated with the first skill, as well as the value of the entry parameter of the second skill. In addition, the large model can also output the reasoning process information of the second skill that is determined to be associated, such as the thinking / reasoning steps of the large model.

[0089] Among them, the decision model can be various large models, specifically various pre-trained models, or large models that have been fine-tuned in the field, including but not limited to various large language models LLM, multimodal large models, etc., which are not specifically limited in this embodiment.

[0090] Although the decision-making and planning capabilities of the current large model are already very good, in order to be closer to the capabilities required by specific application scenarios and to improve the balance of resource consumption, in this embodiment, the decision model used can be obtained by training a moderately sized pre-trained model in the following two stages:

[0091] Phase 1: Use the instruction fine-tuning dataset to perform instruction fine-tuning training on the pre-trained model to obtain an intermediate model to enhance the model's instruction understanding and compliance capabilities.

[0092] The second stage: Use the intelligent agent trajectory dataset to train the intermediate model and obtain the decision model to improve the decision-making planning ability of the decision model.

[0093] By performing the above two stages of training on a moderately sized pre-trained model, the command understanding and compliance capabilities, and decision-making planning capabilities of the moderately sized model can be comprehensively improved, and then the ultra-large-scale model can be replaced for decision reasoning, thereby improving its usage efficiency.

[0094] Optionally, in the first stage, the currently public high-quality instruction fine-tuning dataset can be used to perform instruction fine-tuning training on the pre-trained model to obtain an intermediate module to enhance the model's instruction understanding and compliance capabilities.

[0095] Optionally, in the first stage, based on the currently public high-quality instruction fine-tuning data set, some instruction fine-tuning data can be selected from the instruction fine-tuning data set for ICL transformation, sorted according to the similarity of the input information (i.e., query) in the instruction fine-tuning data, and constructed into ICL format data. Using the remaining instruction fine-tuning data and ICL format data to perform instruction fine-tuning training on the pre-trained model can better enhance the model's instruction understanding and compliance capabilities.

[0096] In the second stage, the public agent trajectory data set and the usage log data of the intelligent interactive system can be combined for fine-tuning. Specifically, the usage log data of the intelligent interactive system is cleaned and preprocessed to construct the agent trajectory data. According to the constructed agent trajectory data and the public agent trajectory data set, the intermediate model obtained in the first stage is trained to obtain a decision model to improve the decision-making planning ability of the decision model. The specific training process of the second stage is described in detail in the following embodiments.

[0097] Through two stages of training, a data-driven learning method is adopted to train the decision model using a large amount of usage data, so that the decision model can capture richer operation modes, enhance the automatic decision-making planning ability of the decision model, and make the decision model closer to the level of human experts in complex scenarios. At the same time, combining the FSM model with the decision model and using the state transition information of the FSM model to assist the decision model in making dynamic decisions can improve the decision-making reasoning ability of the model, thereby improving the accuracy of the associated skills of the decision.

[0098] Step S304: If it is determined that the finite state machine model does not include the target skill, a search algorithm is used to use the large model to search for a target behavior path that meets the user's needs based on the execution status of the first skill and the historical behavior path, and the target behavior path includes at least one skill.

[0099] When it is determined that the finite state machine model does not contain the target skill, a search algorithm is used to use the large model as the value function to search for a target behavior path that meets the user's needs based on the execution of the first skill and the historical behavior path. The target behavior path contains at least one skill.

[0100] The search algorithm used to search for the target behavior path may be a Monte Carlo tree search algorithm or other algorithms with similar functions, which is not specifically limited in this embodiment.

[0101] For example, taking the Monte Carlo Tree Search (MCTS) algorithm as an example, each skill (or operation) in the behavior path is used as a node on the search tree (i.e., Monte Carlo tree); a large model is used as a value function (i.e., a reward model), and the current search results are evaluated based on the execution of the first skill and the historical behavior path to determine whether they meet the user's needs. The evaluation results are used to determine the reward value of the node; the feedback information in the historical behavior path in the intelligent interactive system and the algorithm calculate the self-reflection data (including the reward value and the number of visits of the node) to continuously adjust the confidence information of the nodes in the search tree, thereby adjusting the search strategy (the strategy of selecting nodes based on the confidence information) to search for the associated second skill, which can improve the decision-making planning ability of the system in complex interactive scenarios. The specific implementation of this solution is described in detail in the subsequent embodiments.

[0102] Step S305: Determine the second skill associated with the first skill according to the target behavior path, and obtain the value of the entry parameter of the second skill.

[0103] After searching for a target behavior path that meets the user's needs, at least one skill in the target behavior path may be used as a second skill associated with the first skill.

[0104] Optionally, the first skill in the target behavior path can be used as the second skill associated with the first skill to recommend the first skill in the target behavior path to the user. After the recommended first skill is executed, the associated skills of the recommended first skill can be dynamically determined based on the execution status of the recommended first skill and the historical behavior path. By analogy, new associated skills can be recommended to the user multiple times to achieve personalized recommendations of associated skills / abilities, thereby achieving a truly personalized interaction process for each person, making the intelligent interaction system more intelligent.

[0105] Optionally, all the skills included in the target behavior path can be used as the second skill associated with the first skill to recommend to the user a target behavior path that meets the user's needs. This can not only achieve personalized recommendation of associated skills / abilities, thereby realizing a truly personalized interaction process for each individual, but also meet user needs more quickly by recommending the entire target behavior path, making the intelligent interaction system more intelligent.

[0106] After determining the second skill associated with the first skill, the value of the entry parameter of the second skill may be further obtained.

[0107] Exemplarily, the intelligent interactive system can extract the value of the entry parameter required for the second skill from the execution result of the first skill and the historical interaction data according to the entry parameter of the second skill. Specifically, according to the type or name of the entry parameter required for the second skill, the value of the corresponding entry parameter is obtained from the execution result of the first skill and the historical interaction data, and the value of the entry parameter required for the second skill can be obtained. According to the extraction result, if it is determined that the value of at least one entry parameter is missing, the intelligent interactive system can output the type of the entry parameter of the second skill with these missing values, the filling requirements (such as the data type of the parameter, the value range, etc.), etc. through the interactive page, so that the user can enter the value of these entry parameters in the interactive interface.

[0108] In an optional embodiment, the intelligent interactive system supports the user to edit the entry parameter of the second skill. Specifically, the intelligent interactive system can output the value of at least one entry parameter of the second skill through the front-end interactive page. The user can edit the value of the entry parameter of the second skill displayed on the page. In response to the editing operation of any entry parameter of the second skill, the intelligent interactive system updates the value of the entry parameter of the second skill according to the editing result of any entry parameter of the second skill.

[0109] In this embodiment, the user manual and expert experience of the intelligent interactive system are systematized to build a skill rule library for common scenarios, and a finite state machine (FSM) model is built based on the skill rule library. In simple application scenarios, the FSM model can effectively abstract and execute these rules to help the intelligent agent make quick and accurate decisions.

[0110] As the complexity of application scenarios increases, the FSM model that relies on empirical data gradually shows its limitations and is difficult to cope with complex and changing environments. By introducing data-driven learning methods and using a large amount of usage data to train the decision model, the decision model can capture richer operation modes, enhance the automatic decision-making planning ability of the decision model, and make the decision model closer to the level of human experts in complex scenarios. At the same time, combining the FSM model with the decision model and using the state transition information of the FSM model to assist the decision model in making dynamic decisions can improve the decision-making reasoning ability of the model, thereby improving the accuracy of the associated skills determined.

[0111] In order to deal with the problem that the FSM model cannot meet user needs in cold start scenarios without prior knowledge, a more powerful Monte Carlo Tree Search (MCTS) algorithm is adopted, and the large model is used as the value function to optimize the decision-making process. The Monte Carlo Tree Search algorithm can perform efficient strategy exploration in dynamic and uncertain environments, helping intelligent agents find high-quality behavior paths.

[0112] Through the above solution, the system can not only efficiently and accurately recommend related skills in simple scenarios, but also flexibly respond to complex and changing environments, providing intelligent related skills recommendation solutions. This fusion of rule-based FSM and data-driven learning methods makes the related skills recommendation of the intelligent interactive system more accurate and reliable, thereby realizing a truly personalized interactive process, making the intelligent interactive system more intelligent and bringing a better experience to users.

[0113] Figure 5 A flowchart of the second stage training provided for an exemplary embodiment of the present application. In an optional embodiment, in the second stage training, the intermediate model is trained using the agent trajectory data set to obtain the decision model, which can be implemented by the following steps:

[0114] Step S501: Determine the agent trajectory data set.

[0115] Among them, the agent trajectory data refers to the process record of the interaction between the agent and the external environment. The agent trajectory dataset reflects the behavior path (or trajectory) of the agent in a certain period of time, and is a collection of data generated by the interaction between the agent and the environment under a specific strategy.

[0116] The data of the agent generally includes four types: normal conversation without tools, normal conversation with tools, conversation without tools but requiring tools, and conversation with tools using tools. The tools can be understood as skills.

[0117] Among them, normal dialogue without providing tools refers to inputting instructions (such as query) to the agent but not providing tools. The decision result made by the agent is that there is no need to use tools and the response can be directly generated based on the instructions.

[0118] Providing tools for normal dialogue means inputting commands to the agent and providing tools at the same time. The decision result made by the agent is the response to the command generated by the tool. In this case, the decision result of the agent includes the tool to be used and the value of the tool's input parameters.

[0119] Not providing tools but requiring them in the conversation means inputting instructions to the agent but not providing tools, and the agent making a decision that requires tools to generate a response to the instruction.

[0120] Using tools in a tool-providing dialogue means inputting commands to the agent and providing tools at the same time. The decision result made by the agent is the response to the tool-generating command. In this case, the decision result of the agent includes the tool to be used, the value of the tool's input parameters, and the reasoning process information.

[0121] The input instructions of the agent in the agent trajectory data are used as input samples, and the skills in the intelligent interaction system are used as tools provided to the agent. The tools required to be used in the decision results output by the agent are used as associated skills, and the values ​​of the tool's input parameters are used as the values ​​of the associated skill's input parameters, so that reference results including associated skills, reference values ​​of the associated skill's input parameters, and reasoning process information can be obtained.

[0122] In this embodiment, the agent trajectory data set includes input samples and reference results corresponding to the input samples. The input samples include: historical execution status of skills and historical behavior paths, and the reference results include associated skills, reference values ​​of entry parameters of associated skills, and reasoning process information.

[0123] Exemplarily, in this step, the publicly available agent trajectory dataset and the usage log data of the intelligent interaction system can be combined to construct the agent trajectory dataset used in this embodiment.

[0124] Specifically, after cleaning the usage log data of the intelligent interactive system, any skill used in a complete interaction between the intelligent interactive system and the user is used as an associated skill, and the value of the entry parameter of the associated skill in the actual execution process and the reasoning process information of the associated skill are obtained to obtain the reference result. The historical execution status and historical behavior path of the skills executed before the associated skill in this complete interaction process are obtained as input samples. The input samples and reference results constitute the intelligent agent trajectory data. In this way, a large amount of intelligent agent trajectory data can be constructed based on the usage log data of the intelligent interactive system.

[0125] The constructed agent trajectory data and the public agent trajectory data set are merged as the agent trajectory data set used in this embodiment.

[0126] Currently, in the public agent trajectory dataset, there is usually a lack of two types of data: "normal conversation with tools provided" and "tools not provided but required in conversation", which will to some extent cause the model to have knowledge hallucinations during planning. Therefore, these two types of trajectory data can be constructed based on the existing data in the agent trajectory dataset and added to the agent trajectory dataset, which can make the agent trajectory dataset richer and more complete, avoid the problem of knowledge hallucinations in the model, and improve the decision-making and planning capabilities of the model.

[0127] Exemplarily, based on the trajectory data of "normal conversation without providing tools" in the agent trajectory data set, at least one skill is added to the trajectory data as a tool provided to the agent to construct trajectory data of the "normal conversation with providing tools" type.

[0128] Exemplarily, based on the trajectory data of "tools are provided but tools are needed in the conversation" in the agent trajectory data set, the tools provided to the agent in the trajectory data are deleted to construct trajectory data of the "tools are not provided but tools are needed in the conversation" type.

[0129] Step S502: Input the input sample, the state transition information of the finite state machine model and the reference result corresponding to the input sample are input into the intermediate model for dynamic decision making, and the prediction result of the reference result is output.

[0130] In the second stage of training, based on the agent trajectory dataset, the input samples and the state transition information of the finite state machine model corresponding to the intelligent interaction system, as well as the reference results corresponding to the input samples, are input into the intermediate model together, and dynamic decisions are made through the intermediate model to output the prediction results of the reference results. Among them, the prediction results of the reference results output by the intermediate model include , that is, the probability of predicting the tth token based on the first t-1 tokens in the text sequence corresponding to the reference result, where Indicates the tth token in the corresponding text sequence, Represents the first t-1 tokens in the corresponding text sequence.

[0131] Step S503: According to the prediction result of the reference result, the parameters of the intermediate model are adjusted to obtain a decision model.

[0132] In this step, the loss function value is calculated based on the prediction result of the reference result, and the reverse gradient propagation is performed based on the loss function value to adjust the parameters of the intermediate model. After the second stage of training is completed, the trained decision model can be determined.

[0133] In an optional embodiment, a cross entropy loss function may be used in this step to calculate the cross entropy loss function value based on the prediction result of the reference result.

[0134] For example, the cross entropy loss function value can be calculated using the following formula (1):

[0135] Formula (1)

[0136] Among them, Loss_1 represents the value of the cross entropy loss function, T represents the length of the text sequence corresponding to the reference result (that is, the number of tokens), It indicates the probability of predicting the tth token based on the first t-1 tokens in the text sequence corresponding to the reference result.

[0137] In another optional embodiment, a weighted cross entropy loss function may be used in this step. Specifically, the weighted cross entropy loss function value is calculated based on the prediction result of the reference result, and the weight coefficients corresponding to the associated skills, the entry parameters of the associated skills, and the reasoning process information.

[0138] Among them, the weight coefficient of the associated skill and the weight coefficient of the entry parameter of the associated skill are both greater than the weight coefficient of the reasoning process information, which can be configured according to actual application requirements and are not specifically limited here. For example, the weight coefficient of the associated skill and the weight coefficient of the entry parameter of the associated skill can be 2.0, and the weight coefficient of the reasoning process information can be 1.0.

[0139] For example, the weighted cross entropy loss function value may be calculated using the following formula (2):

[0140] Formula (2)

[0141] in, represents the value of the weighted cross entropy loss function, and T represents the length of the text sequence corresponding to the reference result (i.e. the number of tokens). Indicates the weight coefficient of the tth token in the text sequence corresponding to the reference result. It indicates the probability of predicting the tth token based on the first t-1 tokens in the text sequence corresponding to the reference result.

[0142] In this embodiment, by introducing a weighted loss mechanism and increasing the weight coefficients of associated skills and entry parameter values ​​of associated skills in trajectory data, the learning of associated skills and entry parameter values ​​of associated skills can be emphasized during the second stage of training, thereby enhancing the dynamic decision-making planning ability of the model.

[0143] It should be noted that the training strategy used for the second stage fine-tuning training of the intermediate model, including the learning rate, optimization algorithm, etc., can be designed and adjusted according to actual application requirements and are not specifically limited here.

[0144] The method of this embodiment constructs an intelligent agent trajectory data set, inputs the input samples in the trajectory data, the state transition information of the finite state machine model, and the reference results corresponding to the input samples into the intermediate model for dynamic decision-making, and outputs the prediction results of the reference results; according to the prediction results of the reference results, the parameters of the intermediate model are adjusted to obtain the decision model. Through the data-driven learning method, the decision model is trained based on a large amount of usage data of the intelligent interactive system, which can greatly enhance the automatic decision-making planning ability of the decision model, so that the decision-making planning ability of the decision model is close to the level of human experts.

[0145] Figure 6 A flowchart of searching for a target behavior path that meets user needs is provided for an exemplary embodiment of the present application. Based on any of the above embodiments, in this embodiment, in a cold start scenario where there is a lack of experience information, or in a complex or even difficult to perceive complete state information (i.e., an imperfect FSM), a search algorithm and a large model can be combined to determine the associated skills. Figure 6 As shown, a search algorithm is used to use a large model to search for a target behavior path that meets user needs based on the execution status of the first skill and the historical behavior path. The specific steps are as follows:

[0146] Step S601: Determine a skill search space.

[0147] The skill search space refers to the scope of the skill search. In this embodiment, all skills in the intelligent interaction system can be added to the skill search space as searchable skills.

[0148] Optionally, in order to narrow the skill search space and improve search efficiency, skills not included in the finite state machine model can be added to the skill search space based on the skills included in the FSM model corresponding to the intelligent interaction system, so that the skill search space includes skills not included in the finite state machine model.

[0149] In addition, the skills included in the finite state machine model may be recommended by the method of recommending related skills based on the FSM model in the aforementioned embodiment.

[0150] Step S602: construct an initial search tree, where the initial search tree includes a root node.

[0151] In this embodiment, a Monte Carlo Tree Search (MCTS) algorithm may be used to search for a target behavior path that meets user needs.

[0152] Here, we first define the information of each node on the Monte Carlo tree. In this embodiment, each node on the Monte Carlo tree corresponds to a skill, and stores the information of the corresponding skill, the corresponding behavior path, the number of visits during the search process, and the reward value. Among them, the behavior path corresponding to the node includes the skills corresponding to each node on the path from the root node to the extension node. The reward value of the node is the evaluation value of the node using the reward model during the search process of the node.

[0153] The core of the Monte Carlo Tree Search algorithm lies in search, which is used to explore user behavior paths that may currently meet user needs. In each round of search, the current search tree is processed through selection, expansion, evaluation, simulation, and back propagation until the target behavior path that can meet user needs is found.

[0154] First, construct an initial search tree (i.e., Monte Carlo tree). The initial search tree contains only the root node. The root node can store the execution status and historical behavior path of the first skill.

[0155] Step S603: Use the root node of the search tree as the access node, and perform a subsequent search process on the access node in the skill search space.

[0156] In the first round of search, the root node of the search tree is selected as the access node, and the subsequent search process is performed on the access node in the skill search space.

[0157] Step S604: Add an extension node of the access node.

[0158] In this step, the currently selected access node is expanded. Specifically, available skills are selected from the skill search space, nodes corresponding to the available skills are created as extension nodes of the access node, and the extension nodes are added to the search tree as child nodes of the root node. Available skills refer to skills that can be executed after the skills corresponding to the access node are accessed.

[0159] In this embodiment, the extended node of the access node corresponds to a skill and has a corresponding behavior path. The behavior path corresponding to the extended node includes the skills corresponding to each node on the path from the root node to the extended node.

[0160] Next, through steps S605-S607, the extended node of the access node is evaluated to determine the reward value of the access node.

[0161] Step S605: Using the reward model, based on the execution status of the skills corresponding to the access nodes and the historical behavior paths, evaluate whether the behavior paths corresponding to the extension nodes meet the user needs, and obtain evaluation results.

[0162] Among them, the reward model is a large model, which can be various pre-trained models, or large models that have been fine-tuned in the field, including but not limited to various large language models LLM, which are not specifically limited here.

[0163] In this step, the large model is used as the reward model, and the execution status and historical behavior path of the skills corresponding to the access node are input into the reward model. The reward model simulates the energy-saving execution results corresponding to the extension node based on the execution status and historical behavior path of the skills corresponding to the access node, as well as the behavior path corresponding to the extension node, and evaluates whether the energy-saving execution results corresponding to the extension node meet user needs, so as to achieve the purpose of evaluating whether the behavior path corresponding to the extension node meets user needs and obtain the evaluation results of the extension node.

[0164] Step S606: Determine whether the evaluation result meets the user's needs.

[0165] In this step, the evaluation result of the extended node is used to determine whether the evaluation result meets the user's needs. If the evaluation result meets the user's needs, that is, the behavior path corresponding to the extended node meets the user's needs, then step S609 is executed to use the behavior path corresponding to the extended node as the target behavior path that meets the user's needs.

[0166] If the evaluation result is that the user demand is not met, that is, the behavior path corresponding to the extended node does not meet the user demand, then steps S607-S608 are executed for back propagation to update the reward value of each node on the behavior path corresponding to the extended node according to the evaluation result.

[0167] Step S607: If the evaluation result shows that the behavior path corresponding to the expansion node does not meet the user's needs, the reward value of each node on the behavior path corresponding to the expansion node is updated according to the evaluation result.

[0168] When the behavior path corresponding to the extended node does not meet the user's needs, in this step, back propagation is performed according to the evaluation results to update the reward value of each node on the behavior path corresponding to the extended node.

[0169] Specifically, the reward value of the extended node is generated according to the evaluation result. When the evaluation result shows that the behavior path corresponding to the extended node does not meet the user's needs, the reward value of the extended node is a first value (such as 0). When the evaluation result shows that the behavior path corresponding to the extended node meets the user's needs, the reward value of the extended node is a second value (such as 1). The first value and the second value are different, and can be set and adjusted according to actual application requirements, and are not specifically limited here.

[0170] Further, according to the reward value of the expanded node, the reward values ​​of the remaining nodes on the behavior path corresponding to the expanded node are updated. Any back propagation method of the value function value in the Monte Carlo tree search algorithm can be used here, and no specific limitation is made here.

[0171] Step S608: Select one of the extended nodes as a new access node according to the reward value and the number of visits of the extended node and the number of visits of the access node.

[0172] In this step, the confidence information of the extended node is calculated according to the reward value and the number of visits of the extended node and the number of visits of the visited node; according to the confidence information of the extended node, one of the extended nodes with higher confidence information is selected as the next visited node. For example, the extended node with the highest confidence information is selected as the next visited node.

[0173] The method for calculating the confidence information of the extended node may be implemented by using any method for calculating the upper confidence bound (UCB) of a child node or the upper confidence bound (UCT) of a tree in a Monte Carlo tree search algorithm, and is not specifically limited here.

[0174] For example, the following formula (3) may be used to calculate the UCT of the extended node as the confidence information of the extended node:

[0175] Formula (3)

[0176] in, Represents an expansion node, Represents an extended node The parent node of . Represents an extended node of UCT. Represents an extended node The reward value, Represents an extended node The number of visits, Represents the parent node c is a balance control parameter, which can be set according to actual application requirements and experience.

[0177] After the new access node is determined, steps S604-S608 are continued to perform the above search process for the new access node.

[0178] Step S609: If the evaluation result shows that the behavior path corresponding to the extended node meets the user's needs, the behavior path that meets the user's needs is used as the target behavior path.

[0179] When the behavior path corresponding to the extended node meets the user's needs, the search is terminated, and the behavior path that meets the user's needs is used as the target behavior path.

[0180] In order to cope with cold start scenarios with a lack of prior knowledge, and complex or even difficult to perceive complete state information (i.e., incomplete FSM), the method of this embodiment adopts the Monte Carlo Tree Search (MCTS) algorithm, and uses a large model as a reward model to evaluate the reward value of the node to search for the target behavior path that meets user needs. The MCTS algorithm can perform efficient strategy exploration in a dynamic and uncertain environment, and can find a better behavior path faster. Through the above scheme, the intelligent interactive system can not only perform efficient skill recommendation (navigation) based on the FSM model in simple scenarios, but also can flexibly respond in complex and changing environments, and provide intelligent skill recommendation / navigation solutions.

[0181] Figure 7 is a structural block diagram of a computing device according to an embodiment of the present application. Figure 7 As shown, the computing device may include: one or more (only one is shown in the figure) processors 701 and a memory 702. The memory 702 is used to store computer programs / instructions, and the processor 701 is used to execute computer programs / instructions. When the computer programs / instructions are executed by the processor 701, the technical solutions provided by any of the aforementioned method embodiments are implemented. The specific functions and technical effects that can be achieved are similar and will not be described in detail here.

[0182] The above-mentioned computing device can be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a PC (Personal Computer), a model all-in-one machine, a mobile phone, a tablet computer or other portable intelligent terminal, etc. Optionally, the computing device may be pre-installed with the model in the above-mentioned embodiment of the present application.

[0183] Specifically, the computing device can pre-set multiple types of models, including but not limited to models in the fields of natural language processing, visual processing, speech processing, code processing, multimodal task processing, etc., so as to provide a variety of model choices. In different product forms, the computing device can support one or more model usage methods, including but not limited to model training, model calling, model fine-tuning, model deployment, model reasoning and application, etc. In some product forms, the computing device also supports model management, including but not limited to multi-type model management (supporting the management of multiple types of models such as discriminants and generative models), model version control (supporting the control of different model versions), model evaluation (based on model evaluation tools, evaluating the performance and effect of the model), etc. In other product forms, the computing device can also create applications based on the model, provide API (Application Programming Interface, application programming interface) calling capabilities, and can call the model to the created application through the API interface, while providing application management tools to achieve management and monitoring of the application.

[0184] Furthermore, the computing device can also include data management (supporting the creation and management of model tuning data sets), training center (providing rich training resources to help users learn and master AI technology), and basic management and control capabilities (providing enterprise-level basic management and control capabilities to ensure the security and efficient operation of the system). Through the above functions, a comprehensive, integrated AI development, training, deployment and application device is provided.

[0185] The embodiment of the present application also provides a server of an intelligent interactive system, the server comprising: a memory and a processor. The memory is used to store computer-executable instructions and can be configured to store various other data to support operations on the server. The processor is communicatively connected to the memory and is used to execute the computer-executable instructions stored in the memory to implement the technical solution provided by any of the above method embodiments, and its specific functions and technical effects that can be achieved are similar and will not be repeated here.

[0186] An embodiment of the present application also provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the method of any of the aforementioned embodiments is implemented. The specific functions and technical effects that can be achieved are not repeated here.

[0187] The present application also provides a computer program product, including a computer program, which implements the method of any of the above embodiments when executed by a processor. The computer program is stored in a readable storage medium, and at least one processor of a computing device can read the computer program from the readable storage medium. At least one processor executes the computer program so that the computing device executes the technical solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0188] The embodiment of the present application provides a chip, including: a processing module and a communication interface, the processing module can execute the technical solution provided by the aforementioned method embodiment. Optionally, the chip also includes a storage module (such as a memory), the storage module is used to store instructions, the processing module is used to execute the instructions stored in the storage module, and the execution of the instructions stored in the storage module enables the processing module to execute the technical solution provided by any of the aforementioned method embodiments.

[0189] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0190] It should be understood that the above processor can be a processing unit (Central Processing Unit, referred to as CPU), a graphics processing unit (graphics processing unit, referred to as GPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application-specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in at least one processor.

[0191] The memory may include high-speed random access memory (RAM), and may also include non-volatile storage, such as at least one disk storage, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0192] The above-mentioned storage may be an object storage service (OSS). The above-mentioned storage may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0193] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a mobile hotspot (WiFi), a second-generation mobile communication system (2G), a third-generation mobile communication system (3G), a fourth-generation mobile communication system (4G) / Long Term Evolution (LTE), a fifth-generation mobile communication system (5G) and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared technology, ultra-wide band (UWB) technology, Bluetooth technology and other technologies.

[0194] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0195] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0196] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in a dedicated integrated circuit. Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0197] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0198] The order of the above-mentioned embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. In addition, in some of the processes described in the above-mentioned embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed or executed in parallel in the order in which they appear in this article, and are only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or less operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit "first" and "second" to different types. The meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0199] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0200] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0201] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An intelligent interaction method, characterized in that: include: In response to the first skill being executed and finished, acquiring the execution status and historical behavior path of the first skill; Using the big model, dynamically deciding a second skill associated with the first skill according to the execution status and historical behavior path of the first skill; Output the trigger component of the second skill.

2. The method according to claim 1, characterized in that The execution status of the first skill includes user demand information, and the use of the big model dynamically determines the second skill associated with the first skill according to the execution status of the first skill and the historical behavior path, including: Acquire a finite state machine model corresponding to the intelligent interactive system, wherein the finite state machine model includes at least one skill; If it is determined that the finite state machine model contains a target skill that matches the user demand information, the execution status and historical behavior path of the first skill, as well as the state transition information of the finite state machine model, are input into the decision model for dynamic decision-making, and the second skill associated with the first skill and the value of the input parameter of the second skill are output, wherein the decision model is a large model.

3. The method according to claim 2, characterized in that Also includes: Building a skill rule library according to the usage rules of at least one skill in the intelligent interactive system; According to the skill rule library, the finite state machine model is constructed, and the state transition information of the finite state machine model is determined, wherein the finite state machine model includes states and operations, the states correspond to the interactive pages in the intelligent interactive system, the operations correspond to the skills in the intelligent interactive system, and the execution of the operations causes the state of the intelligent interactive system to change.

4. The method according to claim 3, characterized in that Also includes: In response to the updating of the skill rule base, the finite state machine model and the state transition information of the finite state machine model are updated.

5. The method according to claim 2, characterized in that: Also includes: Use the instruction fine-tuning dataset to perform instruction fine-tuning training on the pre-trained model to obtain an intermediate model; The intermediate model is trained using the agent trajectory dataset to obtain the decision model.

6. The method according to claim 5, characterized in that The using the intelligent agent trajectory data set to train the intermediate model and obtain the decision model includes: Determine an agent trajectory data set, the agent trajectory data set including an input sample and a reference result corresponding to the input sample, the input sample including: a historical execution status of a skill and a historical behavior path, the reference result including an associated skill, a reference value of an entry parameter of the associated skill, and reasoning process information; Inputting the input sample, the state transition information of the finite state machine model and the reference result corresponding to the input sample into the intermediate model for dynamic decision making, and outputting a prediction result of the reference result; According to the prediction result of the reference result, the parameters of the intermediate model are adjusted to obtain the decision model.

7. The method according to claim 6, characterized in that The step of adjusting the parameters of the intermediate model according to the prediction result of the reference result to obtain the decision model includes: Calculate a weighted cross entropy loss function value according to the prediction result of the reference result, and the weight coefficients corresponding to the associated skill, the entry parameter of the associated skill, and the reasoning process information, wherein the weight coefficient of the associated skill and the weight coefficient of the entry parameter of the associated skill are both greater than the weight coefficient of the reasoning process information; According to the weighted cross entropy loss function value, the parameters of the intermediate model are adjusted to obtain the decision model.

8. The method according to claim 2, characterized in that: Also includes: If it is determined that the finite state machine model does not include the target skill, a search algorithm is used to use the large model to search for a target behavior path that meets the user's needs based on the execution status of the first skill and the historical behavior path, where the target behavior path includes at least one skill; According to the target behavior path, a second skill associated with the first skill is determined, and a value of an entry parameter of the second skill is obtained.

9. The method according to claim 8, characterized in that The adopting of a search algorithm and using a large model to search for a target behavior path that meets the user's needs according to the execution status of the first skill and the historical behavior path includes: Using a Monte Carlo tree search algorithm, construct an initial search tree, wherein the initial search tree includes a root node; The root node of the search tree is used as an access node, and the following search process is performed on the access node in the skill search space: Adding an extension node of the access node, wherein the extension node corresponds to a skill and has a corresponding behavior path, and the behavior path corresponding to the extension node includes the skills corresponding to each node on the path from the root node to the extension node; Using the reward model, according to the execution status of the skills corresponding to the access node and the historical behavior path, it is evaluated whether the behavior path corresponding to the extension node meets the user needs, and an evaluation result is obtained, wherein the reward model is a large model; If the evaluation result is that the user demand is not met, then according to the evaluation result, the reward value of each node on the behavior path corresponding to the extended node is updated, and according to the reward value and the number of visits of the extended node and the number of visits of the visit node, one of the extended nodes is selected as a new visit node; Performing the search process on the new access node; When the evaluation result obtained is that the behavior path corresponding to the extended node meets the user demand, the behavior path that meets the user demand is used as the target behavior path.

10. The method according to claim 9, characterized in that The step of updating the reward value of each node on the behavior path corresponding to the extended node according to the evaluation result includes: Generating a reward value for the extension node according to the evaluation result; According to the reward value of the extended node, the reward values ​​of the remaining nodes on the behavior path corresponding to the extended node are updated, wherein the remaining nodes refer to nodes other than the extended node.

11. The method according to claim 9, characterized in that The adding of the extension node of the access node comprises: Selecting an available skill from the skill search space, creating an extension node corresponding to the available skill, and using the extension node as a child node of the root node; The available skills refer to skills that can be executed after the skill corresponding to the access node is accessed.

12. The method according to claim 9, characterized in that The step of selecting one of the extended nodes as the next access node according to the reward value and the number of visits of the extended node and the number of visits of the access node comprises: Calculating the confidence information of the extended node according to the reward value and the number of visits of the extended node and the number of visits of the visited node; According to the confidence information of the extended nodes, one of the extended nodes is selected as the next access node.

13. The method according to claim 8, characterized in that Also includes: outputting a value of at least one of the input parameters of the second skill; In response to an editing operation on any of the entry parameters of the second skill, the value of the entry parameter of the second skill is updated according to the editing result.

14. The method according to any one of claims 1 to 13, characterized in that Also includes: In response to any trigger component of the second skill being triggered, the second skill is executed according to the value of the entry parameter of the second skill.

15. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the method according to any one of claims 1 to 14 is implemented.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 14 is implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images