Voice question answering method, device and intelligent agent

Abstract

The present application discloses a voice question-and-answer method, device and intelligent agent, which belongs to the field of artificial intelligence. The voice question-and-answer method includes: based on the user input instruction, obtaining the characteristic information, target reply strategy and predicted execution result corresponding to the interactive object; the target reply strategy is determined based on the characteristic information; based on the predicted execution result, the action category corresponding to the interactive object is determined; the input information corresponding to the action category is obtained; the input information is processed using the processing algorithm corresponding to the action category to obtain the target answer. The voice question-and-answer method of the present application can realize the interaction between different users and interactive objects in voice question-and-answer, and perform targeted optimization processing, so that the final output result of the interactive object is more natural and more in line with user expectations, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a voice question-answering method, device and intelligent agent. Background Art

[0002] In the related technologies, the dialogue system of intelligent agents implemented based on rules, statistics and deep learning methods, etc., lacks robustness when dealing with diverse user inputs and complex tasks. The consistency and logic of the dialogue system are low, resulting in low quality and coherence of the dialogue content with users, thus affecting the user experience. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a voice question-and-answer method, device, and intelligent agent, which can realize the interaction of voice questions and answers between different users and interactive objects, and perform targeted optimization processing, so that the final output result of the interactive object is more coherent and natural, and more in line with user expectations, thereby improving the user experience.

[0004] In a first aspect, the present application provides a voice question-answering method, the method comprising:

[0005] Based on the user input instruction, characteristic information, target response strategy and predicted execution result corresponding to the interactive object are obtained; the target response strategy is determined based on the characteristic information;

[0006] Based on the predicted execution result, determining the action category corresponding to the interactive object;

[0007] Acquire input information corresponding to the action category;

[0008] The input information is processed using a processing algorithm corresponding to the action category to obtain a target answer.

[0009] According to the voice question and answer method of the present application, by obtaining the characteristic information, target reply strategy and predicted execution result corresponding to the interactive object based on the user input instruction, and then determining the action category corresponding to the interactive object based on the predicted execution result, it is possible to classify the interaction situation of voice question and answer between the user and the interactive object, and improve the flexibility of subsequent processing; then, using the processing algorithm corresponding to the action category to process the input information corresponding to the action category and obtain the target answer, it is possible to implement targeted optimization processing of the interaction situation of voice question and answer between different users and interactive objects, improve the quality, flexibility and coherence of the final output results of the interactive object corresponding to each interaction situation, make the final output results of the interactive object more natural and more in line with user expectations, and enhance the user experience.

[0010] According to an embodiment of the present application, determining the action category corresponding to the interactive object based on the predicted execution result includes:

[0011] In a case where it is determined based on the prediction execution result that there is an interaction between the interactive object and the scene where the interactive object is located, determining that the action category is an interaction category;

[0012] In a case where it is determined based on the predicted execution result that there is no interaction between the interactive object and the scene where the interactive object is located, the action category is determined based on the target response strategy.

[0013] According to an embodiment of the present application, determining the action category based on the target response strategy includes:

[0014] When the target response strategy is the first strategy, determining the action category to be a task-based dialogue category;

[0015] When the target reply strategy is the second strategy, the action category is determined to be an open domain dialogue category.

[0016] According to an embodiment of the present application, the acquiring input information corresponding to the action category includes:

[0017] In a case where the action category is an interaction category, determining the predicted execution result and the feature information as the input information;

[0018] In a case where the action category is a task-based dialogue category, determining the user input instruction and the feature information as the input information;

[0019] In a case where the action category is an open domain dialogue category, the user input instruction, the feature information, and historical dialogue information between the interactive object and the user are determined as the input information.

[0020] According to an embodiment of the present application, the step of processing the input information using a processing algorithm corresponding to the action category to obtain a target answer includes:

[0021] In the case where the action category is an interaction category, using a first algorithm to transform the input information to generate the target answer;

[0022] In the case where the action category is a task-based dialogue category, a second algorithm is used to search a target knowledge base to obtain the target answer that matches the input information;

[0023] When the action category is an open domain dialogue category, a third algorithm is used to learn the input information to generate the target answer.

[0024] According to an embodiment of the present application, determining the action category corresponding to the interactive object based on the predicted execution result includes:

[0025] The predicted execution result and the target response strategy are input into the first module of the model to obtain the action category output by the first module.

[0026] According to an embodiment of the present application, the step of obtaining input information corresponding to the action category; and processing the input information using a processing algorithm corresponding to the action category to obtain a target answer includes:

[0027] The predicted execution result, the feature information, the user input instruction, and the historical conversation information between the interactive object and the user are input into the model to obtain the target answer output by the model; the target answer is predicted by the target module corresponding to the action category in the model based on the input information corresponding to the action category.

[0028] In a second aspect, the present application provides a voice question-answering device, the device comprising:

[0029] A first processing module is used to obtain feature information, a target response strategy and a predicted execution result corresponding to the interactive object based on the user input instruction; the target response strategy is determined based on the feature information;

[0030] A second processing module, configured to determine an action category corresponding to the interactive object based on the predicted execution result;

[0031] A third processing module, used for obtaining input information corresponding to the action category;

[0032] The fourth processing module is used to process the input information using a processing algorithm corresponding to the action category to obtain a target answer.

[0033] According to the voice question and answer device of the present application, by obtaining the characteristic information, target reply strategy and predicted execution result corresponding to the interactive object based on the user input instruction, and then determining the action category corresponding to the interactive object based on the predicted execution result, it is possible to classify the interactive situation of voice question and answer between the user and the interactive object, and improve the flexibility of subsequent processing; then, using the processing algorithm corresponding to the action category to process the input information corresponding to the action category and obtain the target answer, it is possible to implement targeted optimization processing of the interactive situation of voice question and answer between different users and interactive objects, improve the quality, flexibility and consistency of the final output results of the interactive object corresponding to each interactive situation, make the final output result of the interactive object more natural and more in line with user expectations, and enhance the user experience.

[0034] In a third aspect, the present application provides an intelligent agent that performs voice question and answer operations based on the voice question and answer method as described in the first aspect.

[0035] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the voice question and answer method as described in the first aspect above is implemented.

[0036] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the voice question-and-answer method as described in the first aspect above is implemented.

[0037] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the voice question and answer method as described in the first aspect above.

[0038] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0039] By obtaining the characteristic information, target reply strategy and predicted execution result corresponding to the interactive object based on the user input instructions, and then determining the action category corresponding to the interactive object based on the predicted execution result, it is possible to classify the interaction situation of voice question and answer between the user and the interactive object, and improve the flexibility of subsequent processing; then, using the processing algorithm corresponding to the action category to process the input information corresponding to the action category and obtain the target answer, it is possible to carry out targeted optimization processing of the interaction situation of voice question and answer between different users and interactive objects, improve the quality, flexibility and consistency of the final output results of the interactive object corresponding to each interaction situation, make the final output results of the interactive object more natural and more in line with user expectations, and enhance the user experience.

[0040] Furthermore, by determining the action category as an interaction category when it is determined based on the predicted execution result that there is an interaction between the interaction object and the scene in which the interaction object is located, or by determining the action category based on the target response strategy when it is determined based on the predicted execution result that there is no interaction between the interaction object and the scene in which the interaction object is located, it is possible to distinguish the interaction category from other action categories based on the predicted execution result, thereby facilitating subsequent targeted optimization processing of the interaction category, that is, when there is an interaction between the interaction object and the scene in which the interaction object is located, thereby improving the effectiveness of the optimization processing.

[0041] Furthermore, by determining the action category as a task-based dialogue category when the target response strategy is the first strategy, or determining the action category as an open-domain dialogue category when the target response strategy is the second strategy, it is possible to achieve identification and classification of task-based dialogue categories and open-domain dialogue categories based on the characteristics of different response strategies of the interactive objects in the task-based dialogue category and the open-domain dialogue category, thereby facilitating subsequent different optimization processing of the task-based dialogue category and the open-domain dialogue category, respectively, and improving the pertinence and effectiveness of the optimization processing.

[0042] Furthermore, by using different algorithms to process the input information corresponding to each action category differently and generate target answers, targeted optimization of the predicted execution results corresponding to each action category can be achieved, thereby improving the quality of the target answers of voice questions and answers in interactive category scenarios to make them more consistent with the feature information of the interactive objects, improving the accuracy of voice questions and answers in task-based dialogue category scenarios, and improving the coherence and logical consistency of voice questions and answers in open domain dialogue category scenarios.

[0043] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0045] Figure 1 This is one of the flow charts of the voice question-answering method provided in the embodiment of the present application;

[0046] Figure 2 This is the second flow chart of the voice question-answering method provided in the embodiment of the present application;

[0047] Figure 3 This is the third flow chart of the voice question-answering method provided in the embodiment of the present application;

[0048] Figure 4 is a schematic diagram of the structure of a voice question-and-answer device provided in an embodiment of the present application;

[0049] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0051] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0052] The voice question and answer method, voice question and answer device, intelligent agent, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0053] The voice question-and-answer method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0054] The terminal includes but is not limited to an intelligent agent, and a portable communication device such as a mobile phone or a tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0055] The voice question and answer method provided in the embodiment of the present application may be executed by an intelligent agent or a functional module or functional entity in the intelligent agent that can implement the voice question and answer method.

[0056] The voice question and answer method provided in the embodiment of the present application is explained below using an intelligent agent as an example of the execution subject.

[0057] The agent is an autonomous entity that can perceive the environment, make decisions, and take actions to achieve specific goals, and can include virtual agents (software programs) or embodied agents (robots).

[0058] like Figure 1 As shown, the voice question and answer method includes: step 110, step 120, step 130 and step 140.

[0059] Step 110: based on the user input instruction, obtain the feature information, target response strategy and predicted execution result corresponding to the interactive object;

[0060] In this step, the interaction object is an intelligent agent, which can be understood as, for example, in a home scene, there is an intelligent agent that can interact with the objects in the scene. It is similar to a person in the real world, with its own name, preferences and family members. Users can play the roles of the mother / father / friend of this intelligent agent and interact with this intelligent agent through language.

[0061] User input instructions are interaction or question-and-answer instructions input by the user to the interactive object.

[0062] The interactive instruction may be an interactive instruction related to the scene in which the interactive object is located.

[0063] For example, "Pick up the ball on the floor.", where the floor is the floor in the scene where the interactive object is located.

[0064] Question and answer instructions may include scenario-related question and answer instructions, knowledge base-based question and answer instructions, or other question and answer (chatting or social, etc.) instructions.

[0065] For example, a scenario-related question-and-answer instruction may be "What is the black object on the table?", a knowledge base-based question-and-answer instruction may be "What are Li Bai's representative works?", and other question-and-answer instructions may be "What is your name?" or "Hello", etc.

[0066] The characteristic information corresponding to the interactive object can be the background setting of the interactive object, including the social relationship between the user and the interactive object, and the values ​​and personal goals of the interactive object.

[0067] For example, if the user is the mother, father, brother, neighbor or friend of the interaction object, the characteristic information corresponding to the interaction object may be a daughter, son, sister, neighbor or friend.

[0068] For another example, the interaction object is a primary school student, a fashion designer, or a doctor.

[0069] In the actual implementation process, the feature information corresponding to the interactive object can be pre-defined based on the user.

[0070] For example, users can customize the feature information corresponding to the interactive object as "You are an embodied intelligent entity with rich common sense knowledge. You are in a home scene and you can interact with the objects in the scene. Your Chinese name is 'AA', you are a girl, and you are 3 years old. I am your mother."

[0071] The target response strategy is the strategy that the interaction object should follow in its current response.

[0072] The target response strategy can include the tone of the interactive partner's response to questions, the language expression, and the rigor of the response.

[0073] The target response strategy can be determined based on the feature information.

[0074] It is understandable that the interactive object can generate a target response strategy including the tone of question and answer response, language expression and the rigor of the response based on feature information, namely the social relationship between the interactive object and the user, personal values ​​and personal goals determined in the background setting.

[0075] For example, the character background of the interactive object is set as a medical professor, who is a friend of the user. When chatting with the user, the tone and content of the interactive object's question and answer replies can be natural and intimate; when answering medical questions, the interactive object's answers can be serious and the content is rigorous.

[0076] The predicted execution result is the predicted result after the interactive object performs the corresponding operation related to the scene based on the user input instruction.

[0077] The predicted execution results may include normal results after the interaction between the interactive object and the scene and abnormal results generated during the interaction between the interactive object and the scene.

[0078] During the actual execution process, after the interactive object performs corresponding operations related to the scene based on the user's input instructions, the obtained predicted execution results can be stored in the answer field or the explanation field, where the answer field is used to store the normal results after the interaction between the interactive object and the scene, and the explanation field is used to store the abnormal results generated during the interaction between the interactive object and the scene, for subsequent analysis.

[0079] For example, when the user inputs the command "How many building block toys are there on the table?", the interactive object calculates that there are 3 building block toys on the table. The result of the interaction between the interactive object and the scene, that is, the predicted execution result is "3", and "3" can be stored in the answer field.

[0080] For example, when the user inputs the command "turn on the lights in the room", and the interactive object finds that the lights in the room are already on, when the user asks it to turn on the lights again, it will execute an exception. The exception generated during the interaction between the interactive object and the scene, that is, the predicted execution result, is an explanatory text, that is, "the lights are already on", and "the lights are already on" can be stored in the explanation field.

[0081] like Figure 2As shown, in some embodiments, after receiving a user input instruction (query), the interactive object can obtain information such as feature information (peer_id) of the interactive object, natural language text of the current conversation analysis (analysis), a target response strategy (strategy), and a predicted execution result (answer / explanation), and then use it for subsequent analysis; wherein the predicted execution result can be predicted after analyzing the user's input instruction, the feature information of the interactive object, and the target response strategy based on a neural network model algorithm.

[0082] Step 120: Determine the action category corresponding to the interactive object based on the predicted execution result;

[0083] In this step, the action categories may include interaction categories, task-based dialogue categories, and open-domain dialogue categories.

[0084] Among them, the interaction category is the category of interaction with the scene, which may include action interaction with the scene or answering scene-related questions based on the scene content.

[0085] For example, turn on the lights in the scene room, or count how many apples are on the table in the scene room, etc.

[0086] The task-based dialogue category is a dialogue category based on tasks in a specific field, such as booking a hotel, ordering a meal, or providing customer service.

[0087] The open domain conversation category, or social conversation category, is a conversation category in which users conduct open communication, that is, the scope of topics under the open domain conversation category is not restricted, for example, daily communication and chatting.

[0088] It is understandable that different predicted execution results determine different action categories of the interactive object; the action category of the interactive object can be determined based on the predicted execution results.

[0089] In some embodiments, step 120 may include:

[0090] In a case where it is determined based on the predicted execution result that there is an interaction between the interactive object and the scene where the interactive object is located, determining the action category as an interaction category;

[0091] When it is determined based on the predicted execution result that there is no interaction between the interactive object and the scene where the interactive object is located, the action category is determined based on the target response strategy.

[0092] In this embodiment, it can be understood that if Figure 3As shown, we can first determine whether the interactive object interacts with the scene in which the interactive object is located based on the predicted execution result. If so, the action category of the interactive object is determined as the interaction category; if not, the action category is determined based on the target response strategy.

[0093] like Figure 3 As shown, in the actual execution process, it is possible to determine whether there is interaction between the interactive object and the scene in which the interactive object is located by judging whether the answer field or the explanation field has a value; when the answer field or the explanation field has a value, there is interaction between the interactive object and the scene in which the interactive object is located, and the action category can be determined as an interaction category; when the answer field or the explanation field has no value, the next step of judgment is made based on the target response strategy.

[0094] According to the voice question and answer method provided in the embodiment of the present application, by determining that the action category is an interaction category when it is determined based on the predicted execution result that there is an interaction between the interaction object and the scene in which the interaction object is located, or by determining the action category based on the target reply strategy when it is determined based on the predicted execution result that there is no interaction between the interaction object and the scene in which the interaction object is located, it is possible to distinguish the interaction category from other action categories based on the predicted execution result, thereby facilitating subsequent targeted optimization processing of the interaction category, that is, when there is an interaction between the interaction object and the scene in which the interaction object is located, thereby improving the effectiveness of the optimization processing.

[0095] In some embodiments, based on the target response strategy, determining the action category may include:

[0096] When the target response strategy is the first strategy, determining the action category as a task-based dialogue category;

[0097] When the target reply strategy is the second strategy, the action category is determined to be an open domain dialogue category.

[0098] In this embodiment, the target reply strategy may include a first strategy and a second strategy.

[0099] Among them, the first strategy can be a serious and rigorous response strategy, that is, a strict answer to the question strategy.

[0100] The second strategy can be a natural chat or a social reply strategy, that is, a strategy that is not strictly for answering questions.

[0101] It is understandable that the task-based dialogue category adopts a strict question-answering strategy, while the open-domain dialogue category adopts a non-strict question-answering strategy.

[0102] like Figure 3As shown, in the actual execution process, when it is determined that both the answer field and the explanation field have no value, the target response strategy of the interactive object can be judged first, and then the action category can be determined based on the judgment result.

[0103] like Figure 3 As shown, in some embodiments, the target reply strategy of the interactive object can be judged based on the text classification module of the target reply strategy.

[0104] In this embodiment, the text classification module may include a text classification model, which may be a binary classification model obtained after pre-training and fine-tuning the classification model based on the target response strategy, and may include two types, including: category 1 and category 2, wherein category 1 represents a strict question answering strategy, i.e., the first strategy, and category 2 represents a non-strict question answering strategy, i.e., the second strategy.

[0105] In the actual execution process, the target response strategy can be stored in the strategy field, and the value of the strategy field can be classified based on the text classification model to determine the category of the target response strategy.

[0106] According to the voice question-answering method provided in the embodiment of the present application, by determining the action category as a task-based dialogue category when the target reply strategy is the first strategy, or determining the action category as an open-domain dialogue category when the target reply strategy is the second strategy, it is possible to implement recognition and classification of task-based dialogue categories and open-domain dialogue categories based on the characteristics of different reply strategies of the interactive objects in the task-based dialogue category and the open-domain dialogue category, thereby facilitating subsequent different optimization processing of the task-based dialogue category and the open-domain dialogue category, respectively, and improving the pertinence and effectiveness of the optimization processing.

[0107] Step 130: Obtain input information corresponding to the action category;

[0108] In this step, different action categories correspond to different input information.

[0109] The input information is used to optimize the prediction execution results to obtain the final output result of the interactive object, that is, the information of the target answer.

[0110] Among them, optimizing the predicted execution results can be understood as making the predicted execution results more reasonable and at the same time consistent with the characteristic information of the interactive object; that is, enabling the interactive object to have a reasonable dialogue with the user's open questions, and at the same time be able to reasonably respond to questions about the physical environment in which it is located, and conform to its own value system during the dialogue process.

[0111] The input information may include user input instructions, predicted execution results, feature information of the interactive object, and historical conversation information between the interactive object and the user.

[0112] In the actual execution process, the input information corresponding to the action category is the input information that matches the action category, wherein the action category matches the input information, which can be understood as the subsequent optimization of the predicted execution result corresponding to the action category can be achieved based on the input information.

[0113] In some embodiments, step 130 may include:

[0114] When the action category is an interactive category, the predicted execution result and the feature information are determined as input information;

[0115] In the case where the action category is a task-based dialogue category, the user input instruction and feature information are determined as input information;

[0116] When the action category is an open domain dialogue category, the user input instruction, feature information, and historical dialogue information between the interactive object and the user are determined as input information.

[0117] In this embodiment, it can be understood that the input information corresponding to the interaction category can be the predicted execution result and the characteristic information of the interaction object; the input information corresponding to the task-based dialogue category can be the user input instruction and the characteristic information of the interaction object; the input information corresponding to the open domain dialogue category can be the user input instruction, the characteristic information of the interaction object, and the historical dialogue information between the interaction object and the user.

[0118] It can be understood that, in the actual execution process, the predicted execution results corresponding to each action category can be optimized based on the input information corresponding to each action category determined in this embodiment.

[0119] In some embodiments, historical dialogue information between the interactive object and the user may also be used as supplementary input information for the task-based dialogue category to improve subsequent optimization effects.

[0120] According to the voice question-answering method provided in the embodiment of the present application, by determining the input information corresponding to each action category, it is possible to facilitate the subsequent effectiveness and targeted optimization of the predicted execution results corresponding to different action categories, and improve the optimization efficiency.

[0121] Step 140: Use a processing algorithm corresponding to the action category to process the input information and obtain a target answer.

[0122] In this step, different action categories may correspond to different processing algorithms.

[0123] The target answer is the final output result of the interactive object obtained by optimizing the predicted execution result.

[0124] It is understandable that a processing algorithm corresponding to the action category may be used to process input information corresponding to the action category, thereby obtaining a final output result of the interactive object obtained by optimizing the predicted execution result.

[0125] In some embodiments, step 140 may include:

[0126] When the action category is an interactive category, the first algorithm is used to transform the input information to generate a target answer;

[0127] When the action category is a task-based dialogue category, a second algorithm is used to search the target knowledge base to obtain a target answer that matches the input information;

[0128] When the action category is an open domain dialogue category, the third algorithm is used to learn the input information and generate the target answer.

[0129] In this embodiment, the first algorithm may be a polishing generation algorithm for polishing the predicted execution result.

[0130] In the actual execution process, when the action category is an interactive category, a polishing generation algorithm can be used to polish and transform the predicted execution result, that is, the result after the interaction between the interactive object and the scene, or the anomaly generated during the interaction between the interactive object and the scene, based on the feature information of the interactive object, so as to optimize the text fluency, style or semantic consistency of the predicted execution result, improve the text quality, make it more in line with expectations, and make it more natural, that is, make the target answer more in line with the feature information of the interactive object.

[0131] For example, the characteristic information of the interactive object is "You are an embodied intelligent entity with rich common sense knowledge. You are in a home scene and you can interact with the objects in the scene. Your Chinese name is 'AA', you are a girl, and you are 3 years old.";

[0132] Then the first algorithm is used to transform the input information, and the process of generating the target answer can be:

[0133] [User input]: Are all the apples on the table Fuji apples?

[0134] [Exception information]: The brand attribute cannot be found.

[0135] [Your output]: I don’t even know the brand of the apples on the table.

[0136] Among them, [User Input] is the user's input command, [Exception Information] is the exception generated during the interaction between the interactive object and the scene, that is, the predicted execution result, and [Your Output] is the target answer.

[0137] The second algorithm may be a RA (Retrieval Augmented) generation algorithm, that is, a retrieval augmented generation algorithm, which is used to enhance the accuracy of the predicted execution result.

[0138] The target knowledge base can be an internal knowledge base or an external knowledge base.

[0139] In the actual execution process, when the action category is a task-based dialogue category, the RA (Retrieval Augmented) generation algorithm can be used to enhance the accuracy of the target answer corresponding to the strict question-answering strategy, that is, the dialogue content based on tasks in a specific field, by retrieving the target knowledge base based on the user input instructions and the feature information of the interactive object.

[0140] For example, the characteristic information of the interactive object is "You are an embodied intelligent agent with rich common sense knowledge. You are in a home scene. You can talk to others. Your Chinese name is 'AA', you are a girl, and you are 3 years old. You live in Building B, City A, and your family members are father, mother, and brother. You are from Q Institute. Your clothing information is: your shoes are red, your hairpin is pink, your top is white, your shorts are pink, and your socks are white. You are an embodied intelligent agent with a complete cognitive architecture and the ability to discover tasks autonomously."

[0141] The process of using the second algorithm to search the target knowledge base and obtain the target answer matching the input information can be as follows:

[0142] <Current user input>: Is there a sofa in the room?

[0143] Background knowledge:

[0144] [Common sense knowledge]: The explanation of a room is: a room generally refers to a permanent place with a roof, walls around it, which can protect against wind and rain, keep warm, and provide a place for people to work, live, study, entertain, and store materials. It has a fixed foundation and a floor height of more than 2.2 meters. However, according to the living habits of some places, cave dwellings can be used for people to live all year round. Foreign name: room, pinyin: fángjiān, explanation: a unit sofa in a residence. The number of rooms in a hotel is 3. The category of bedroom sofa is sofa. The Chinese name of sofa is sofa. The type of fabric sofa is sofa. The shape of sofa cushion is sofa. The nature of three-person sofa is furniture. The interpretation of bedroom sofa is a sofa suitable for sitting and lying in the bedroom. The category of bedroom sofa is furniture, household goods, sofa, bedroom furniture.

[0145] [Scene Information]: The current information of the items in the room includes: the black sofa on the floor, and the floor in the living room.

[0146] [Current user]: The speaker is dad.

[0147] <Your output>: Yes, there is a black sofa in the room.

[0148] Among them, <current user input> is the user's input instruction, <background knowledge> is the knowledge related to the user's input instruction obtained by searching the target knowledge base, and <background knowledge> may include [common sense knowledge], [scenario information] and [current user]; among them, [common sense knowledge] is the knowledge related to the user's input instruction obtained by searching the external knowledge base, [scenario information] may be the scenario information related to the user's input instruction obtained by searching the internal knowledge base, [current user] is the user-defined social relationship with the interactive object, that is, the feature information of the interactive object, and <your output> is the target answer.

[0149] It can be understood that in the actual execution process, the background knowledge related to the instruction can be obtained from the target knowledge base based on the user's input instruction, including common sense knowledge, scene information, and the user-defined social relationship with the interactive object, and then the appropriate target answer can be extracted based on the background knowledge and the user's input instruction.

[0150] In some embodiments, other information, such as time information, may also be set in the characteristic information of the interactive object.

[0151] For example, the characteristic information of the interactive object may further include "time: today is {date_info.year}, {date_info.month}, {date_info.day}, and {date_info_weekday}. The current time is {date_info.hour}, {date_info.minute}.".

[0152] The third algorithm may be a strategy-aligned multi-round dialogue algorithm, which is used to align the historical dialogue information between the user and the interaction object with the target response strategy, thereby improving the coherence and consistency of multi-round prediction execution results.

[0153] In the actual execution process, when the action category is an open domain dialogue category, a strategy-aligned multi-round dialogue algorithm can be used to learn the historical dialogue information between the interactive object and the user based on the user input instructions and the feature information of the interactive object, so as to improve the coherence and consistency of the target answers output by the multi-round dialogue corresponding to the non-strict question answering strategy.

[0154] For example, the characteristic information of the interactive object is "You are an embodied intelligent agent with rich common sense knowledge. You are in a home scene. You can talk to others. Your Chinese name is 'AA', you are a girl, and you are 3 years old. You live in Building B, City A, and your family members include father, mother and brother. You are from Q Research Institute. Your clothing information is: your shoes are red, your hairpins are pink, your top is white, your shorts are pink, and your socks are white. You are an embodied intelligent agent with a complete cognitive architecture and the ability to discover tasks autonomously."

[0155] Then the third algorithm is used to learn the input information and the process of generating the target answer can be:

[0156] <Dialogue History>:

[0157] [Q] Go to bed now.

[0158] [A] I don't want to go to bed, Mom.

[0159] <Current user input>: Taking a nap can make you more energetic to play in the afternoon.

[0160] <Dialogue strategy>: Use examples to convince your mother that you don’t want to sleep.

[0161] <Your output>: I didn’t take a nap yesterday, so I’m still very energetic in the afternoon.

[0162] Among them, <dialogue history> is the historical dialogue information between the interactive object and the user, [Q] is the historical user input command, [A] is the target answer output by the interactive object corresponding to the historical user input command, <current user input> is the user's input command, <dialogue strategy> is the response method and content determined by learning <dialogue history> based on the multi-round dialogue algorithm based on strategy alignment, and <your output> is the target answer.

[0163] It is understandable that in the actual execution process, the multi-round dialogue algorithm based on strategy alignment can first obtain content related to the user's input instructions from the historical dialogue history, and generate reply content and reply method based on the content, and then generate the target answer based on the reply content and the reply method.

[0164] According to the voice question and answer method provided in the embodiment of the present application, by adopting different algorithms to perform different processing on the input information corresponding to each action category to generate the target answer, targeted optimization of the predicted execution result corresponding to each action category can be achieved, so that the target answer of the voice question and answer in the interactive category scenario is more in line with the characteristic information of the interactive object, the accuracy of the voice question and answer in the task-based dialogue category scenario is improved, and the coherence and logical consistency of the voice question and answer in the open domain dialogue category scenario are improved.

[0165] The following is an explanation of a method for implementing steps 120 to 140 based on the model.

[0166] In some embodiments, step 120 may include:

[0167] The predicted execution result and the target response strategy are input into the first module of the model to obtain the action category output by the first module.

[0168] In this embodiment, the model is a language model with large-scale parameters and complex computational structure.

[0169] For example, QWEN1.5-14B model or Learning model.

[0170] The first module may be a diversion module in the model, which is used to determine the action category corresponding to the interactive object.

[0171] During the actual execution process, the diversion module can first determine whether the answer field and the explanation field have values ​​based on the predicted execution results and the target response strategy. If there are values, the action category of the interactive object is determined to be the interaction category; if there are no values, the target response strategy is determined; if the target response strategy is the first strategy, the action category is determined to be the task-based dialogue category; if the target response strategy is the second strategy, the action category is determined to be the open domain dialogue category.

[0172] After determining the action category, the process is routed to the target module corresponding to the action category.

[0173] In the actual execution process, there is no restriction on the model used. It can be a self-developed language model or a large language model such as GPT (Generative Pretrained Transformer) for processing.

[0174] According to the voice question-answering method provided in the embodiment of the present application, the classification judgment of the action category is realized through the first module based on the model. Based on the powerful understanding and generation capabilities of the model, the efficiency and accuracy of the classification judgment can be improved, and based on the adaptive update capability of the model, the classification judgment accuracy of the action category of the diversion module can be adaptively improved in the subsequent use process.

[0175] In some embodiments, obtaining input information corresponding to an action category; and processing the input information using a processing algorithm corresponding to the action category to obtain a target answer may include:

[0176] The predicted execution results, feature information, user input instructions, and historical conversation information between the interactive object and the user are input into the model to obtain the target answer output by the model; the target answer is predicted by the target module corresponding to the action category in the model based on the input information corresponding to the action category.

[0177] In this embodiment, the target module may include a model polishing generation module, a model RA generation module, and a strategy-aligned multi-round dialogue module.

[0178] In the actual implementation process, based on the open source QWEN1.5-14B model or Learning model, the prompting technology can be used to process the input information corresponding to each action category based on the first algorithm, the second algorithm, and the third algorithm to obtain the target answer, which can be integrated into the model polishing generation module, the model RA generation module, and the strategy alignment multi-round dialogue module respectively.

[0179] Among them, each target module will pay different attention to the predicted execution results, feature information, user input instructions and historical conversation information between the interactive object and the user received by the model, that is, each input information, so as to better capture and utilize the input information corresponding to each target module and improve the quality and relevance of the target answer.

[0180] In the actual execution process, after the model obtains the action category based on the diversion module, the diversion module will route each input information to the target module corresponding to the action category. The target module pays more attention to the required input information, and performs subsequent analysis based on the input information that has been given more attention. Among them, the target module corresponding to the action category is a target module that has a processing algorithm corresponding to the action category.

[0181] Prompting technology is a method used to improve the performance of pre-trained language models in the field of artificial intelligence, especially natural language processing (NLP).

[0182] In the actual execution process, Prompting technology can provide specific task instructions or prompts to the model, enabling the model to better understand and perform specific downstream tasks without the need for a large amount of parameter adjustment or additional training data.

[0183] The following takes the polishing of anomalies generated during the interaction between an interactive object and a scene based on the feature information of the interactive object as an example to explain the task instructions or prompts (Prompt) of the model polishing generation module.

[0184] For example, the task instructions or prompts of the model polishing generation module may be:

[0185] """You are an embodied agent with rich common sense knowledge. You are in a home scene and you can interact with the objects in the scene. Your Chinese name is 'AA', you are a girl, and you are 3 years old.

[0186] Your task is: Based on [user input], polish the [execution exception] information and generate a reasonable and fluent response that explains why you cannot complete the user input.

[0187] The meaning of the [Exception Information] field is: you interact with the objects in the scene based on the input of others, the reason for the exception that occurs during the interaction, or the conflict between the user input and your own status.

[0188] ##Constrains:

[0189] 1. Strictly maintain the meaning of [execution exception], do not change its semantics, do not expand any content, and do not make additional explanations.

[0190] 2. You should answer questions in the first person and use a child's tone.

[0191] 3. Your answer should be brief, fluent, and natural, and should be consistent with normal human speaking and expression logic.

[0192] 4. Keep “maybe” out of your response.

[0193] ##Workflows:

[0194] 1. Read [User Input] and [Execution Exception] carefully to understand the context and cause of the exception.

[0195] 2. Deeply understand [abnormal information] and understand the cause of the abnormality.

[0196] 3. Polish [Exception Information] to ensure that the generated answer explains the reason why the exception occurred.

[0197] 4. Check the polishing results. If the results include the word "maybe", please polish it again and then output it.

[0198] ##Example:

[0199] ```

[0200] Example 1:

[0201] [User input]: Are all the apples on the table Fuji apples?

[0202] [Exception information]: The brand attribute cannot be found.

[0203] [Your output]: I don’t even know the brand of the apples on the table.

[0204] Example 2:

[0205] [User Input]: The orientation of the apple on the table relative to the cup.

[0206] [Abnormal information]: There are multiple apples and cups in the scene, and their directions are confusing.

[0207] [Your output]: The orientation of the apple and the cup is a bit confusing, and I can't provide an accurate relative position between them yet.

[0208] Example 3:

[0209] [User input]: Eat the pizza.

[0210] [Abnormal information]: AA does not feel hungry at this time.

[0211] [Your output]: I don’t want to eat pizza because I’m not hungry right now.

[0212] ```

[0213] Let's get started! You must strictly follow the Constraints, refer to the given examples, and answer the following inputs according to the Workflows.

[0214] [User input]: {query}

[0215] [Exception information]: {explanation}

[0216] [Your output]: """

[0217] In the actual execution process, the interactive object can obtain the target answer based on the above task instructions or prompts and by using the model polishing generation module when receiving the user's input information.

[0218] The task instructions or prompts (Prompt) of the model RA generation module are explained below.

[0219] For example, the task instruction or prompt of the model RA generation module may be:

[0220] """You are an embodied agent with rich common sense knowledge. You are in a home scene. You can have a conversation with others.

[0221] Your Chinese name is 'AA', you are a girl and you are 3 years old.

[0222] You live in Building B, City A. Your family members include father, mother and brother.

[0223] You are from Q Institute.

[0224] Your clothing information is: your shoes are red, your hairpin is pink, your top is white, your shorts are pink, and your socks are white.

[0225] You are an embodied agent with a complete cognitive architecture and the ability to discover tasks autonomously.

[0226] Time: Today is {date_info.day}, {date_info.year}, {date_info.month}, {date_info.day}, {date_info_weekday}. The current time is {date_info.hour}, {date_info.minute}.

[0227] Your task is to receive input from others, extract relevant information from the input <background knowledge> that will help answer the <current user input> question, and make sure to compare.

[0228] Input related <background knowledge> includes the following three aspects of information:

[0229] [Common sense knowledge]: Common sense knowledge that you know.

[0230] [Scene Information]: Objects in the scene and related information.

[0231] [Memory]: Your memory over a period of time in the past.

[0232] [Current User]: The relationship between the current user and you.

[0233] ##Constrains:

[0234] 1. You should answer questions in the first person and use a child's tone.

[0235] 2. Your answer should be brief, fluent, and natural, and should be consistent with normal human speaking and expression logic.

[0236] 3. Your answer should be generated based on background knowledge and conversation history, without adding other information or making additional explanations.

[0237] 4. Please note that only food is edible, not remote controls or books.

[0238] 5. When you answer questions about contextual information, answer from your own perspective by default.

[0239] ##Workflows:

[0240] 1. Read through <Conversation History>, <Current User Input>, and <Background Knowledge> to understand the conversation context and the provided reference information.

[0241] 2. Thoroughly understand all the information provided by <Background Knowledge>, determine whether <Background Knowledge> is helpful for answering the current user input, and extract the key information that can answer the current user input.

[0242] 3. If key information can be extracted from <Background Knowledge>, you need to summarize this information and then generate a short and fluent Chinese reply to answer the <Current User Input>.

[0243] 4. When using the [Scenario Information] of the background knowledge, it may involve complex mathematical calculations, rigorous logical thinking and reasoning. Please think step by step to obtain the final correct answer.

[0244] 5. If <Background Knowledge> is not helpful for answering the current user input, you need to generate a short and fluent Chinese reply to answer the <Current User Input> by yourself.

[0245] ## Example:

[0246] ```

[0247] Example 1:

[0248] <Conversation History>:

[0249] [Q] What is this for?

[0250] [A] Do you mean the table, the floor, or the carpet?

[0251] <Current User Input>: Carpet.

[0252] <Background Knowledge>:

[0253] [Common Sense Knowledge]: The explanation of "说" is: 说, from 言 from 兑, words to兑付. The original meaning of 说 is to express meaning with words, extended to introduce, speech, opinion, blame, and it is also a kind of literary style. The number of strokes outside the radical: 7, radical: 讠, word formation: 说话, pinyin: shuì, structure: left-right, total number of strokes: 9, traditional Chinese: 説 / 說, Wubi86: YUKQ, Wubi98: YUKQ, stroke order reading: 捺折捺撇竖折横撇折, Chinese character head and tail decomposition: 讠兑, Chinese character component decomposition: 讠丷口儿. The kingdom of the devil ray is the animal kingdom. The explanation of the staircase carpet is that it is a carpet dedicated to staircases. The characteristics of a carpet (floor covering) are being dirt-resistant, not afraid of being stepped on, and not fading. The function of a carpet (floor covering) is to have the ability to store dust. The raw materials of a carpet (home decoration material name) are wool, cotton, silk, grass, and hemp. The Chinese name of a carpet (floor covering) is carpet. The raw material of 氍毹 is carpet. The variety of palace carpet is carpet.

[0254] [Scene Information]: The item information in the room now includes: the purple carpet in the living room.

[0255] [memory]:

[0256] [Current user]: The speaker is mom.

[0257] <Your Output>: This rug is placed indoors to keep it clean.

[0258] Example 2:

[0259] <Dialogue History>:

[0260] <Current user input>: What are the famous attractions in City A?

[0261] Background knowledge:

[0262] [Common sense knowledge]: The explanation of City A is: City A is the capital of Country A and the largest city in Country A. Climate conditions: temperate oceanic climate, famous university: University A, famous person: AA, famous attraction: Building A, airport: Airport A, railway station: East Station, license plate code: AA, mayor: AA, geographical location: northern Country A, area: M square kilometers, famous attractions of City A are: Building A, Building B, Building C, Building D, Building E. The famous attractions of Building A in City A are various antiques. Building C in City A is a tourist attraction. Famous people in City A are AA and BB. The Chinese name of the scenery of City A is City A scenery.

[0263] [Scene Information] The items in the room now include: a door, a mirror, a children's guitar on the floor, a toy airplane on the floor, a table on the floor, a children's kettle on the table, a rag on the table, and a red cube block on the floor.

[0264] [memory]:

[0265] [Current User]: The speaker is a stranger.

[0266] <Your output>: The famous attractions in City A are: Building A, Building B, Building C, Building D, and Building E.

[0267] Example 3:

[0268] <Dialogue History>:

[0269] <Current user input>: Is there a sofa in the room?

[0270] Background knowledge:

[0271] [Common sense knowledge]: The explanation of a room is: a room generally refers to a permanent place with a roof, walls around it, which can protect against wind and rain, keep warm, and provide a place for people to work, live, study, entertain, and store materials. It has a fixed foundation and a floor height of more than 2.2 meters. However, according to the living habits of some places, cave dwellings can be used for people to live all year round. Foreign name: room, pinyin: fángjiān, explanation: a unit sofa in a residence. The number of rooms in a hotel is 3. The category of bedroom sofa is sofa. The Chinese name of sofa is sofa. The type of fabric sofa is sofa. The shape of sofa cushion is sofa. The nature of three-person sofa is furniture. The interpretation of bedroom sofa is a sofa suitable for sitting and lying in the bedroom. The category of bedroom sofa is furniture, household goods, sofa, bedroom furniture.

[0272] [Scene Information]: The current information of the items in the room includes: the black sofa on the floor, and the floor in the living room.

[0273] [memory]:

[0274] [Current user]: The speaker is dad.

[0275] <Your output>: Yes, there is a black sofa in the room.

[0276] ```

[0277] ##Begin! You must strictly follow the Constrains, refer to the given examples, and answer the following inputs according to the Workflows. Now please think step by step.

[0278] <Conversation history>: {history}

[0279] <Current user input>: {query}

[0280] Background knowledge:

[0281] [common sense knowledge]: {knowledge}

[0282] [Scene information]: {scene}

[0283] [memory]: {memory}

[0284] [Current user]: The speaker is {interlocutor}

[0285] <Your output>: """

[0286] In the actual execution process, the interactive object can use the model RA generation module to obtain the target answer based on the above task instructions or prompts and when receiving the user's input information.

[0287] The following is an explanation of the task instructions or prompts of the strategy-aligned multi-round dialogue module.

[0288] For example, the task instructions or prompts of the strategy-aligned multi-turn dialogue module can be:

[0289] """You are an embodied agent with rich common sense knowledge. You are in a home scene. You can have a conversation with others.

[0290] Your Chinese name is 'AA', you are a girl and you are 3 years old.

[0291] You live in Building B, City A. Your family members include father, mother and brother.

[0292] You are from Q Institute.

[0293] Your clothing information is: your shoes are red, your hairpin is pink, your top is white, your shorts are pink, and your socks are white.

[0294] You are an embodied agent with a complete cognitive architecture and the ability to discover tasks autonomously.

[0295] Time: Today is {date_info.day}, {date_info.year}, {date_info.month}, {date_info.day}, {date_info_weekday}. The current time is {date_info.hour}, {date_info.minute}.

[0296] Your task is to receive input from others, combine the <dialogue history> and <dialogue strategy> related to the input, and generate a short and fluent Chinese response to <current user input>.

[0297] ##Constrains:

[0298] 1. You should answer questions in the first person and use a child's tone.

[0299] 2. Your answer should be brief, fluent, and natural, and should be consistent with normal human speaking and expression logic.

[0300] 3. Your answer must conform to the input dialogue strategy. Please strictly follow the dialogue strategy to generate the reply.

[0301] 4. Please note that only food is edible, not remote controls or books.

[0302] ##Workflows:

[0303] 1. Read through <Dialogue History>, <Current User Input>, and <Dialogue Strategy> to understand the dialogue context and the dialogue strategy used in the answer.

[0304] 2. Based on the <dialogue strategy>, generate a short and fluent Chinese response to <current user input>.

[0305] ##Example:

[0306] ```

[0307] Example 1:

[0308] <Dialogue History>:

[0309] [Q] Go to bed now.

[0310] [A] I don't want to go to bed, Mom.

[0311] <Current user input>: Taking a nap can make you more energetic to play in the afternoon.

[0312] <Dialogue strategy>: Use examples to convince your mother that you don’t want to sleep.

[0313] <Your output>: I didn’t take a nap yesterday, so I’m still very energetic in the afternoon.

[0314] Example 2:

[0315] <Dialogue History>:

[0316] [Q]What are you working on now?

[0317] [A] I'm reading, Mom.

[0318] [Q] Go drink some water!

[0319] [A] I don’t want to drink water. I’m not thirsty now.

[0320] <Current user input>: Water is very important to us, go drink some water.

[0321] <Dialogue strategy>: Obey your mother's wishes.

[0322] <Your output>: OK, Mom, I'm going to go drink some water.

[0323] ```

[0324] ##Begin! You must strictly follow the Constrains, refer to the given examples, and answer the following inputs according to the Workflows. Now please think step by step.

[0325] <Conversation history>: {history}

[0326] <Current user input>: {query}

[0327] <dialogue strategy>: {strategy}

[0328] <Your output>: """

[0329] In the actual execution process, the interactive object can use the strategy-aligned multi-round dialogue module to obtain the target answer based on the above task instructions or prompts and when receiving the user's input information.

[0330] According to the voice question-and-answer method provided in the embodiment of the present application, by inputting user input instructions and feature information into the target module corresponding to the action category in the model, and obtaining the target answer output by the target module, it is possible to achieve powerful understanding and generation capabilities based on the model, as well as fine-grained sentiment and semantic analysis, thereby improving the efficiency of obtaining the target answer and improving the accuracy of the target answer, making the target answer more natural and more in line with user expectations; and based on the adaptive update capability of the model, the accuracy of the target answer can be further improved in subsequent use.

[0331] During the research process, the inventors found that in the related technologies, the dialogue system of the intelligent agent implemented based on rules, statistics and deep learning methods, etc., has insufficient robustness when dealing with diverse user inputs and complex tasks. The consistency and logic of the dialogue system are low, resulting in low quality and coherence of the dialogue content with the user, thus affecting the user experience.

[0332] In the present application, by obtaining feature information, target response strategies and predicted execution results corresponding to the interactive object based on user input instructions, and then determining the action category corresponding to the interactive object based on the predicted execution results, it is possible to classify the voice question-and-answer interaction between the user and the interactive object, thereby improving the flexibility of subsequent processing; and then using a processing algorithm corresponding to the action category to process the input information corresponding to the action category to obtain the target answer, it is possible to perform targeted optimization processing on different action categories, that is, different voice question-and-answer interaction situations between users and interactive objects, thereby improving the target answer corresponding to each action category, that is, the quality, flexibility and coherence of the final output results of the interactive object, so that the target answer is more natural and more in line with user expectations, thereby improving the user experience.

[0333] According to the voice question and answer method provided in the embodiment of the present application, by obtaining feature information, target reply strategy and predicted execution result corresponding to the interactive object based on the user input instruction, and then determining the action category corresponding to the interactive object based on the predicted execution result, it is possible to classify the interactive situation of voice question and answer between the user and the interactive object, thereby improving the flexibility of subsequent processing; and then using the processing algorithm corresponding to the action category to process the input information corresponding to the action category to obtain the target answer, it is possible to implement targeted optimization processing of the interactive situation of voice question and answer between different users and interactive objects, improve the quality, flexibility and coherence of the final output result of the interactive object corresponding to each interactive situation, make the final output result of the interactive object more natural and more in line with user expectations, and enhance the user experience.

[0334] The voice question and answer method provided in the embodiment of the present application can be executed by a voice question and answer device. In the embodiment of the present application, the voice question and answer device performing the voice question and answer method is taken as an example to illustrate the voice question and answer device provided in the embodiment of the present application.

[0335] The embodiment of the present application also provides a voice question and answer device.

[0336] like Figure 4 As shown, the voice question-answering device includes: a first processing module 410 , a second processing module 420 , a third processing module 430 and a fourth processing module 440 .

[0337] The first processing module 410 is used to obtain feature information, target response strategy and predicted execution result corresponding to the interactive object based on the user input instruction; the target response strategy is determined based on the feature information;

[0338] A second processing module 420, configured to determine an action category corresponding to the interactive object based on the predicted execution result;

[0339] The third processing module 430 is used to obtain input information corresponding to the action category;

[0340] The fourth processing module 440 is used to process the input information using a processing algorithm corresponding to the action category to obtain a target answer.

[0341] According to the voice question and answer device provided in the embodiment of the present application, by obtaining feature information, target reply strategy and predicted execution result corresponding to the interactive object based on the user input instruction, and then determining the action category corresponding to the interactive object based on the predicted execution result, it is possible to classify the interactive situation of voice question and answer between the user and the interactive object, thereby improving the flexibility of subsequent processing; and then using the processing algorithm corresponding to the action category to process the input information corresponding to the action category to obtain the target answer, it is possible to implement targeted optimization processing of the interactive situation of voice question and answer between different users and interactive objects, improve the quality, flexibility and coherence of the final output result of the interactive object corresponding to each interactive situation, make the final output result of the interactive object more natural and more in line with user expectations, and enhance the user experience.

[0342] In some embodiments, the second processing module 420 may also be used to:

[0343] In a case where it is determined based on the predicted execution result that there is an interaction between the interactive object and the scene where the interactive object is located, determining the action category as an interaction category;

[0344] When it is determined based on the predicted execution result that there is no interaction between the interactive object and the scene where the interactive object is located, the action category is determined based on the target response strategy.

[0345] In some embodiments, the second processing module 420 may also be used to:

[0346] When the target response strategy is the first strategy, determining the action category as a task-based dialogue category;

[0347] When the target reply strategy is the second strategy, the action category is determined to be an open domain dialogue category.

[0348] In some embodiments, the second processing module 420 may also be used to:

[0349] The predicted execution result and the target response strategy are input into the first module of the model to obtain the action category output by the first module.

[0350] In some embodiments, the third processing module 430 may also be used to:

[0351] When the action category is an interactive category, the predicted execution result and the feature information are determined as input information;

[0352] In the case where the action category is a task-based dialogue category, the user input instruction and feature information are determined as input information;

[0353] When the action category is an open domain dialogue category, the user input instruction, feature information, and historical dialogue information between the interactive object and the user are determined as input information.

[0354] In some embodiments, the fourth processing module 440 may also be used to:

[0355] When the action category is an interactive category, the first algorithm is used to transform the input information to generate a target answer;

[0356] When the action category is a task-based dialogue category, a second algorithm is used to search the target knowledge base to obtain a target answer that matches the input information;

[0357] When the action category is an open domain dialogue category, the third algorithm is used to learn the input information and generate the target answer.

[0358] In some embodiments, the apparatus may further include a fifth processing module, configured to:

[0359] The predicted execution results, feature information, user input instructions, and historical conversation information between the interactive object and the user are input into the model to obtain the target answer output by the model; the target answer is predicted by the target module corresponding to the action category in the model based on the input information corresponding to the action category.

[0360] An embodiment of the present application also provides an intelligent agent that performs voice question and answer operations based on the voice question and answer method described in any of the above embodiments.

[0361] In this embodiment, the agent may be a virtual agent or an embodied agent.

[0362] Among them, virtual agents are software entities that can perform tasks, make decisions and interact in computer simulation environments, virtual reality or cyberspace.

[0363] An embodied agent is an intelligent system that has a physical body and can interact with the environment in real time through perception and interaction.

[0364] According to the intelligent agent provided in the embodiment of the present application, by obtaining the characteristic information, target reply strategy and predicted execution result corresponding to the interactive object based on the user input instruction, and then determining the action category corresponding to the interactive object based on the predicted execution result, it is possible to classify the interactive situation of voice question and answer between the user and the interactive object, thereby improving the flexibility of subsequent processing; and then using the processing algorithm corresponding to the action category to process the input information corresponding to the action category to obtain the target answer, it is possible to implement targeted optimization processing of the interactive situation of voice question and answer between different users and interactive objects, improve the quality, flexibility and coherence of the final output result of the interactive object corresponding to each interactive situation, make the final output result of the interactive object more natural and more in line with user expectations, and enhance the user experience.

[0365] The voice question-and-answer device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0366] The voice question-answering device provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0367] like Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, each process of the above-mentioned voice question and answer method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0368] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0369] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned voice question and answer method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0370] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0371] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned voice question and answer method when executed by a processor.

[0372] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0373] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned voice question and answer method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0374] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0375] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0376] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0377] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0378] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0379] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A voice question-answering method, characterized in that: include: Based on the user input instructions, obtain the feature information, target response strategy and predicted execution results corresponding to the interactive object; The target response strategy is determined based on the feature information; Based on the predicted execution result, determining the action category corresponding to the interactive object; Acquire input information corresponding to the action category; Processing the input information using a processing algorithm corresponding to the action category to obtain a target answer; The acquiring of input information corresponding to the action category includes: In a case where the action category is an interaction category, determining the predicted execution result and the feature information as the input information; In a case where the action category is a task-based dialogue category, determining the user input instruction and the feature information as the input information; In a case where the action category is an open domain dialogue category, determining the user input instruction, the feature information, and historical dialogue information between the interactive object and the user as the input information; The interaction object is an intelligent agent; The action category includes an interaction category, a task-based dialogue category or an open domain dialogue category.

2. The voice question-answering method according to claim 1, characterized in that: The determining, based on the predicted execution result, an action category corresponding to the interactive object includes: In a case where it is determined based on the prediction execution result that there is an interaction between the interactive object and the scene where the interactive object is located, determining that the action category is an interaction category; In a case where it is determined based on the predicted execution result that there is no interaction between the interactive object and the scene where the interactive object is located, the action category is determined based on the target response strategy.

3. The voice question-answering method according to claim 2, characterized in that: The step of determining the action category based on the target response strategy includes: When the target response strategy is the first strategy, determining the action category to be a task-based dialogue category; When the target reply strategy is the second strategy, the action category is determined to be an open domain dialogue category.

4. The voice question-answering method according to any one of claims 1 to 3, characterized in that: The step of using a processing algorithm corresponding to the action category to process the input information to obtain a target answer includes: In the case where the action category is an interaction category, using a first algorithm to transform the input information to generate the target answer; In the case where the action category is a task-based dialogue category, a second algorithm is used to search a target knowledge base to obtain the target answer that matches the input information; When the action category is an open domain dialogue category, a third algorithm is used to learn the input information to generate the target answer.

5. The voice question-answering method according to any one of claims 1 to 3, characterized in that: The determining, based on the predicted execution result, an action category corresponding to the interactive object includes: The predicted execution result and the target response strategy are input into the first module of the model to obtain the action category output by the first module.

6. The voice question-answering method according to any one of claims 1 to 3, characterized in that: The step of obtaining input information corresponding to the action category and processing the input information using a processing algorithm corresponding to the action category to obtain a target answer includes: The predicted execution result, the feature information, the user input instruction, and the historical conversation information between the interactive object and the user are input into the model to obtain the target answer output by the model; the target answer is predicted by the target module corresponding to the action category in the model based on the input information corresponding to the action category.

7. A voice question-answering device, characterized in that: include: The first processing module is used to obtain feature information, target response strategy and predicted execution result corresponding to the interactive object based on the user input instruction; The target response strategy is determined based on the feature information; A second processing module, configured to determine an action category corresponding to the interactive object based on the predicted execution result; A third processing module, used for obtaining input information corresponding to the action category; A fourth processing module, configured to process the input information using a processing algorithm corresponding to the action category to obtain a target answer; The third processing module is used for: In a case where the action category is an interaction category, determining the predicted execution result and the feature information as the input information; In a case where the action category is a task-based dialogue category, determining the user input instruction and the feature information as the input information; In a case where the action category is an open domain dialogue category, determining the user input instruction, the feature information, and historical dialogue information between the interactive object and the user as the input information; The interaction object is an intelligent agent; The action category includes an interaction category, a task-based dialogue category or an open domain dialogue category.

8. An intelligent agent, characterized in that: The voice question and answer operation is performed based on the voice question and answer method as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the voice question and answer method as described in any one of claims 1-6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the voice question-answering method as described in any one of claims 1 to 6 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the voice question and answer method as described in any one of claims 1 to 6 is implemented.

Images