Voice interaction method and device, equipment, storage medium and program product

Abstract

The invention discloses a voice interaction method and device, equipment, a storage medium and a program product, which are used for realizing multi-object and multi-task-oriented voice interaction and improving the flexibility, convenience and application universality of voice interaction on the premise of ensuring the interaction safety and accuracy. The voice interaction method comprises the following steps: in response to a received first voice instruction, determining whether the first voice instruction comes from an authorized first object; if the first voice instruction comes from a first object, whether a first task indicated by the first voice instruction is executed or not is determined based on environment information sending the first voice instruction and first configuration information, and the first configuration information comprises a task authorized for the first object; and if it is determined that the first task is executed, responding to the first voice instruction based on an execution result of the first task.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a speech interaction method, device, equipment, storage medium and program product. Background Art

[0002] Voice interaction robots can be applied to different jobs and industries such as customer service, education, companionship, and business support. As the technology and related products required for voice intelligent interaction continue to mature, the development and progress of voice interaction robots are changing with each passing day. Current voice interaction robots are only for scenarios with a single speaker and a single task, which is not flexible and convenient enough, and has application limitations. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a voice interaction method, device, equipment, storage medium and program product for realizing multi-object and multi-task voice interaction, and to improve the flexibility, convenience and wide application of voice interaction while ensuring the security and accuracy of the interaction.

[0004] In order to achieve the above objectives, the present application embodiment adopts the following technical solutions: In a first aspect, an embodiment of the present application provides a voice interaction method, comprising: In response to receiving a first voice instruction, determining whether the first voice instruction originates from an authorized first object; If the first voice instruction originates from the first object, determining whether to execute a first task indicated by the first voice instruction based on environment information in which the first voice instruction is issued and first configuration information, wherein the first configuration information includes tasks authorized for the first object; If it is determined to execute the first task, the first voice instruction is responded to based on the execution result of the first task.

[0005] In a second aspect, an embodiment of the present application provides a voice interaction device, including: A first determination module, configured to determine, in response to receiving a first voice instruction, whether the first voice instruction originates from an authorized first object; a second determination module, configured to determine whether to execute a first task indicated by the first voice instruction based on environment information of issuing the first voice instruction and first configuration information if the first voice instruction originates from the first object, wherein the first configuration information includes tasks authorized for the first object; A response module is used to respond to the first voice instruction based on the execution result of the first task if it is determined to execute the first task.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including: processor; a memory for storing instructions executable by the processor; Among them, the processor is configured to execute the instructions to implement the voice interaction method provided in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the voice interaction method provided in the first aspect.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the voice interaction method provided in the first aspect.

[0009] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: By configuring the authorized first object and authorizing the corresponding task for the first object, the objects allowed to participate in the voice interaction and the tasks that the objects can indicate through voice interaction are controlled, so as to provide data support for realizing voice interaction for multiple objects and multiple tasks. Based on this, when the first voice instruction is received, it is first determined whether the first voice instruction comes from the first object, and if the first voice instruction comes from the first object, it is determined whether to execute the first task indicated by the first voice instruction based on the environmental information of issuing the first voice instruction and the task authorized for the first object, and then when the first task is determined, the first voice instruction is responded to based on the execution result of the first task; in this way, as long as the received voice instruction is issued by an object in the object set, the object has the right to indicate the task indicated by the voice instruction, and the object does need to execute the task, the voice instruction can be responded to. On the one hand, multiple objects can participate in the same conversation and indicate different tasks, realizing voice interaction for multiple objects and multiple tasks, greatly improving the flexibility, convenience and wide application of voice interaction; on the other hand, it can effectively avoid erroneous responses to voices issued by unconfigured objects, tasks that configured objects have no right to indicate, and tasks that configured objects do not need to execute, thereby improving the security and accuracy of voice interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application; Figure 2 A flowchart of a voice interaction method provided for one embodiment of the present application; Figure 3 A flowchart of a voice interaction method provided for another embodiment of the present application; Figure 4 A schematic diagram of a flow chart of a cache management method provided for an embodiment of the present application; Figure 5 A schematic diagram of the structure of a voice interaction device provided for one embodiment of the present application; Figure 6 A schematic diagram of the structure of an electronic device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0012] The terms "first", "second", etc. in this specification and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in this specification and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0013] Explanation of relevant terms: TTS (Text-to-Speech) technology: Given an input text and a selected timbre, the technology can synthesize speech for the selected text using the selected timbre.

[0014] ASR (Automatic Speech Recognition) technology: technology that recognizes the speech content of a given speech and transcribes it into text.

[0015] NLP (Natural Language Processing): NLP in a narrow sense refers to the general term for technologies that perform various tasks such as classification, recognition, conversion, prediction, translation, and generation of text-based language information.

[0016] Speaker Recognition: An identity authentication technology that builds a voiceprint database based on the independence of human physiological pronunciation characteristics and determines the identity of the speaker based on voiceprint comparison.

[0017] KWS (Keyword Spotting): A technology for identifying key information in speech, that is, determining whether a certain speech segment contains a specific keyword.

[0018] IR (Intent Recognition): Generally speaking, it refers to a natural language processing technology that uses a domain intent tag library built by experts and a classification model trained with relevant text to identify the intent of the text.

[0019] Domain knowledge base: A structured knowledge storage system used to organize and store professional knowledge in a certain field. Domain knowledge keyword table is a minimalist domain knowledge base.

[0020] Slot: In a multi-round conversation, intent recognition and slot completion are used together. The robot needs to complete the slot based on the context information to get the true and clear intent, and then give the correct answer. For example, in a multi-round conversation, if the user asks "how much is it", the robot cannot directly determine the specific intent of the question and answer. However, based on the context information, the robot knows that the user is asking "how much is XX brand eggs" because the customer just asked "where are XX brand eggs produced". This completed information is composed of each slot.

[0021] Knowledge graph: attribute management and value management of entities in the domain, a tool for structured representation. Slot completion cannot be achieved without the help of knowledge graph.

[0022] As mentioned above, for the sake of service security and consistency management of conversation content, current voice interaction robots are only oriented to scenarios with a single speaker and a single task. For example, in a family life scenario, the male host issues a voice command to the voice interaction robot "Xiaozhi", "Xiaozhi, find the hot pot restaurant with the highest rating recently." After "Xiaozhi" recognizes that the male host is speaking, it gives the address of the hot pot restaurant. At this time, if the hostess interrupts and asks "how to get there the fastest", "Xiaozhi" finds that the voice command is not issued by the previous male host, and ignores the voice command. In this case, the hostess can only interact with "Xiaozhi" after the male host's voice interaction with "Xiaozhi" ends. As a result, voice interaction is not flexible and convenient enough, and there are application limitations. It cannot be applied in some special scenarios, such as scenarios where multiple speakers want to interact with voice interaction robots.

[0023] To this end, a multi-object multi-task voice interaction solution is needed to overcome the above technical problems. In order to achieve this purpose, the embodiment of the present application proposes a voice interaction method, which adopts the following inventive concept: by configuring an authorized first object and authorizing the corresponding task for the first object, the objects allowed to participate in the voice interaction and the tasks that the objects can indicate through voice interaction are controlled, so as to provide data support for realizing multi-object multi-task voice interaction; based on this, when a voice instruction is received, as long as the voice instruction is issued by an object in the object set, the object has the right to indicate the task indicated by the voice instruction, and the object does need to perform the task, the voice instruction can be responded to. On the one hand, multiple objects can participate in the same conversation and indicate different tasks, realizing multi-object multi-task voice interaction, which greatly improves the flexibility, convenience and wide application of voice interaction; on the other hand, it can effectively avoid erroneous responses to voices issued by unconfigured objects, tasks that configured objects have no right to indicate, and tasks that configured objects do not need to perform, thereby improving the security and accuracy of voice interaction.

[0024] The voice interaction method provided in the embodiments of the present application can be executed by an electronic device or software installed in the electronic device. The so-called electronic device here may include a terminal device that supports the voice interaction function, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a smart watch, a vehicle terminal, an aircraft, etc.

[0025] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0026] In order to facilitate the understanding of the technical solution provided in the embodiment of the present application, firstly, Figure 1 , the application scenario of the technical solution provided in the embodiment of the present application is described. It should be understood that the technical solution provided in the embodiment of the present application is applied to Figure 1 The scenario shown is only an exemplary description and should not be understood as a limitation on the application scenario of the technical solution.

[0027] like Figure 1 As shown, an application scenario of the technical solution provided in the embodiment of the present application includes: n objects 1 and a terminal device 2 supporting voice interaction. Among them, the n objects may include, for example, but are not limited to: users, terminal devices supporting voice output functions, etc., and the terminal devices here may be devices of the same type as the terminal device 2, such as but not limited to smart phones, tablet computers, laptop computers, desktop computers, intelligent voice interaction devices, smart home appliances, smart watches, vehicle terminals, aircraft, etc.

[0028] Any of the n objects can issue a voice command to instruct the terminal device 2 to perform a corresponding task by voice. In the related art, the terminal device 2 is only oriented to a single object and a single task, that is, in a conversation process, the terminal device 2 can only respond to a task instructed by an object. For example, after the object user1 initiates a conversation process with the terminal device 2, the terminal device 2 only responds to the task instructed by the voice command issued by the object user1. During this period, the voice command issued by any of the objects user2~object usern cannot be responded by the terminal device 2; only after the object user1 ends the conversation with the terminal device 2, the terminal device 2 can respond to the voice commands issued by other objects.

[0029] To this end, the voice interaction method proposed in the embodiment of the present application configures an object set and a task authorized for each object in the object set in the terminal device 2. The object set includes objects that are allowed to interact with the terminal device 2 by voice, and authorizing a task for each object means allowing the task indicated by the object. Based on this, after receiving a voice instruction, the terminal device 2 can respond to the voice instruction as long as it recognizes that the voice instruction is issued by an object in the object set, the object has the right to indicate the task indicated by the voice instruction, and the object does need to perform the task.

[0030] For example, the object set configured in the terminal device 2 includes the object user1 and the object user2. The tasks authorized for the object user1 include consulting some topics about entertainment, and the tasks authorized for the object user2 include consulting some topics about education. The object user1 issues a voice command "Is there any concert recently?" The terminal device 2 recognizes that the voice command is issued by the object user1, and the task indicated by the voice command is the task authorized by the object user1, and then responds to the voice command and outputs the recent concert information; then, the object user2 issues a voice command "Can you help me check the math book?" The terminal device 2 recognizes that the voice command is issued by the object user2, and the task indicated by the voice command belongs to the task authorized by the object user2, and then responds to the voice command and outputs information related to the math book; then, the object user3 issues a voice command "What new songs does Zhang San have recently?" The terminal device 2 recognizes that the voice command is not issued by the object in the object set, and then does not respond to the voice command; then, the object user1 issues a voice command "How about this textbook XX?" The terminal device 2 recognizes that the voice command is issued by the object user1, but the task indicated by the voice command does not belong to the task authorized for the object use1 or the object user1 does need to perform the task, and then does not respond to the voice command. In this way, multiple objects can participate in a conversation process, and each object can instruct the terminal device 2 to perform a corresponding task, so that the terminal device 2 can achieve multi-object and multi-task voice interaction.

[0031] Please refer to Figure 2 , is a flow chart of a voice interaction method provided by an embodiment of the present application, which can be applied to a terminal device supporting a voice interaction function. Figure 2 As shown, the method comprises the following steps: S202: In response to receiving a first voice instruction, determine whether the first voice instruction comes from an authorized first object.

[0032] The first object refers to an object that allows voice interaction with the terminal device. The number of first objects can be multiple. Each first object can have a corresponding voice feature, and the voice feature of each first object can be pre-configured in the terminal device. As an example, the voice feature of each first object is the voiceprint feature of the first object, so as to accurately distinguish different first objects. For example, before the first object interacts with the terminal device by voice, the terminal device displays a text and receives the voice of the first object reading the text, and obtains the voiceprint feature of the first object by extracting the voiceprint feature of the voice; then, the terminal device generates a first data table based on the voiceprint feature of each first object. Among them, the first data table includes multiple rows of data, each row of data corresponds to a first object, including the identity (ID) of the first object, the voiceprint feature, and the role (such as administrator or non-administrator). For example, the following Table 1 shows an example of a first data table.

[0033] Table 1

[0034] In this case, the above S202 includes the following steps: performing voiceprint recognition on the received first voice command to obtain a first voiceprint feature; matching the first voiceprint feature with the voiceprint feature of the first object; if the two match, determining that the first voice command comes from the authorized first object.

[0035] For example, a pre-trained voiceprint recognition model is used to perform voiceprint recognition on the first voice command to obtain a first voiceprint feature; then, for each first object, the voiceprint feature of the first object is matched with the first voiceprint feature. If the match is successful, such as the similarity between the two is greater than a similarity threshold, it is determined that the first voiceprint feature belongs to the first object, and then the first voice command is determined to be from the first object, and the following step S204 is further performed. If the first voiceprint feature fails to match the voiceprint features of all first objects, it is determined that the first voice command is not issued by the first object, and then the first voice command is not responded to. The voiceprint recognition model may include, for example, but is not limited to: UBM-GMM, i-vector, VGG-16, ECAPA-TDNN, etc.

[0036] In another embodiment, in order to avoid the influence of environmental noise on the voice interaction process, before the above S202, the voice interaction method provided in the embodiment of the present application also includes: performing noise reduction processing on the first voice instruction, that is, removing the noise in the first voice instruction, thereby enhancing the recognition effect of the first voice instruction and improving the recognition accuracy.

[0037] The first voice command may be subjected to noise reduction processing by various noise reduction technologies, which are not limited in the embodiments of the present application. For example, the first voice command is input into a pre-trained noise reduction model for noise reduction processing to obtain the first voice command after noise reduction. The noise reduction model may include, but is not limited to: RNNoise, DC-CRN, CRUSE, DeepFilterNet, etc.

[0038] In another embodiment, in order to further avoid the influence of environmental noise on the voice interaction process, reduce the computational complexity of noise reduction processing, and reduce the probability of the terminal device being activated by mistake, before performing noise reduction on the first voice command, the voice interaction method provided in the embodiment of the present application further includes: identifying whether the first voice command is noise. Further, if the first voice command is not noise, noise reduction processing is performed on the first voice command.

[0039] As an example, the energy amplitude of the first voice instruction is extracted. If the duration for which the energy value of the first voice instruction is continuously lower than the energy threshold is greater than the duration prediction, it is determined that the first voice instruction is noise; otherwise, it is determined that the first voice instruction is not noise.

[0040] As another example, a first voice instruction is input into a pre-trained classification model to obtain the probability that the first voice instruction is noise; if the probability is greater than a probability threshold, it is determined that the first voice instruction is noise; if the probability is less than or equal to the probability threshold, it is determined that the first voice instruction is not noise.

[0041] S204: If the first voice instruction comes from the first object, determine whether to execute the first task indicated by the first voice instruction based on the environment information of issuing the first voice instruction and the first configuration information. The first configuration information includes the tasks authorized for the first object.

[0042] The first task indicated by the first voice instruction can be obtained by parsing the first voice instruction in various appropriate ways.

[0043] In one embodiment, the first task is determined by: Step A1: convert a first voice instruction into a first text.

[0044] The first voice command is converted into the first text, which can be achieved by various ASR technologies in the art, and the embodiments of the present application are not limited to this. As an example, the first voice command is input into a pre-trained speech recognition model to obtain the first text. The speech recognition model may include, but is not limited to: QuartzNet, ContextNet, Conformer-CTC, Wav2vec2.0, Whisper, etc.

[0045] Step A2: performing intent recognition on the first text to obtain a first intent of the first text.

[0046] The intent recognition of the first text may be obtained through various intent recognition methods, which are not limited in the embodiments of the present application.

[0047] As an example, the first text is input into an intent recognition model to obtain a first intent of the first text. The intent recognition model may include, but is not limited to, BERT+LSTM, IntentNet, InteRACT, EdgeDIPN and other NLP-based models.

[0048] Considering that in a multi-round dialogue for multiple objects and multiple tasks, the text of the current voice command may be missing content, and the true and clear intention of the object cannot be accurately identified, and thus the correct response cannot be given. Since these intentions are usually hidden in the context information, it is necessary to complete the missing information based on the context information to accurately identify the intention of the text of the current voice command. For example, in a multi-round dialogue, the object user1 asked "how much", and the terminal device cannot directly determine the specific intention question and answer. But according to the context information, the object user1 just asked "where are XX brand eggs produced", so it is known that the object user1 is asking "how much is XX brand eggs".

[0049] Based on this, as another example, in order to improve the accuracy of intent recognition, intent recognition is performed on the first text, including: step A21, based on the first configuration information, searching for a session record of a second task, wherein the second task includes a task authorized for the first object, and the session record includes a second text converted from a second voice instruction indicating the second task, and the second voice instruction is a voice instruction received before the first voice instruction; step A22, determining the context of the first text from the session record; step A23, performing intent recognition on the first text based on the context to obtain the intent of the first text.

[0050] In order to accurately identify the intention of the voice command issued by each object in the multi-object multi-task voice interaction scenario, the cache stores the session records of multiple tasks, which are generated and written to the cache during the session. For example, during the session, each time the terminal device receives a voice command, after determining the task indicated by the voice command, it can generate the session record of the task based on the text of the voice command and write it to the cache.

[0051] The conversation record of each task includes the text involved in the task, such as the text converted from the voice instructions indicating the task. During the conversation, these texts constitute the contextual information of the text obtained later, which contains rich information and helps to identify the intention of the subsequent text. In actual applications, the conversation record of each task can also contain more information, such as the identifier of the object to which the text in the conversation record of the task belongs (referred to as object ID), the intention of the text, etc., which is not limited in this embodiment of the present application.

[0052] In the above step A21, since the first configuration information includes the task authorized for the first object, based on the identifier of the task (referred to as task ID), the session record of the task is searched in the cache. In the above step A22, if the session record of the task is found, all texts in the session record can be used as the context of the first text, or texts adjacent to the first text in terms of generation time can be selected as the context of the first text.

[0053] In the above step A23, the first text and the context can be spliced ​​and input into the intention recognition model. The intention recognition model uses the semantics of the context to parse the semantics of the first text, and the intention output by the intention recognition model is used as the first intention of the first text.

[0054] In the above step A23, the first text and the context can also be spliced ​​and input into the intention recognition model to obtain the second intention and the probability of the second intention; the first text can be input into the intention recognition model to obtain the third intention and the probability of the third intention; further, the intention with the highest probability between the second intention and the third intention can be determined as the first intention of the first text.

[0055] Exemplarily, when outputting the intent, the intent recognition model also outputs the probability of the intent; the terminal device selects the one with a greater probability from the second intent and the third intent as the first intent of the first text.

[0056] For example, the first text is "how much", and the second text is "where are XX brand eggs produced". The first text and the second text are concatenated and input into the intent recognition model, and the second intent "ask about the price of XX brand eggs" is obtained, with a probability of 0.9; the first text is input into the intent recognition model, and the third intent "ask about the price" is obtained, with a probability of 0.6; further, it is determined that the first intention of the first text is "ask about the price of XX brand eggs".

[0057] In the above method, the intent of the first text is not only recognized in combination with the context of the first text, but also only the first text is recognized, and the first intent of the first text is determined by combining the two intent recognition results, thereby improving the accuracy of intent recognition.

[0058] Of course, if the conversation record of the second task is not found, the intent recognition is directly performed on the first text to obtain the first intent of the first text.

[0059] Step A3: query the second configuration information based on the first intent to obtain the first task corresponding to the first intent.

[0060] The second configuration information includes a set of tasks supported by voice interaction and the intent corresponding to each task in the task set. The second configuration information can be stored in the terminal device in the form of a data table. Exemplarily, the second configuration information includes a second data table and a third data table. As shown in Table 2 below, the second data table has multiple rows of data, each row of data corresponds to a task, including the identifier of the task (ie, the task ID), the name of the task, and the intent corresponding to the task. As shown in Table 3 below, the third data table includes multiple rows of data, each row of data corresponds to an intent, including the identifier of the intent, the name, the identifier of the corresponding task, and the response template.

[0061] Table 2

[0062] Table 3

[0063] For example, if the first intention is "search for celebrity works", which is identified as I2, then by querying the first configuration information, it can be obtained that the task corresponding to the first intention is entertainment information recommendation, which is identified as T1, and then the task is used as the first task indicated by the first voice command.

[0064] In the above embodiment, since the first intention of the first text can accurately reflect the semantics of the first text, based on the correspondence between the task and the intention, the intention corresponding to the first intention is determined as the first task indicated by the first voice instruction, which can improve the accuracy of task recognition and help to accurately decide whether to respond to the first voice instruction.

[0065] In another embodiment, the above-mentioned determination of the first task indicated by the first voice instruction includes the following steps: performing word segmentation processing on the first text to obtain keywords of the first text; matching the keywords of the first text with the keywords of each task in the task set supported by voice interaction; and determining the task to which the keywords matching the keywords of the first text belong as the first task indicated by the first voice instruction. In this way, the first task indicated by the first voice instruction can be quickly determined through keyword detection technology.

[0066] For example, the first text is "Where are XX brand eggs produced?", and its keywords include "eggs" and "production", and the keywords of Task 1 "Product Information Consultation" in the task set include "eggs", "place of production", "price", etc. Since the keywords of the first text match the keywords of Task 1, Task 1 is determined to be the first task indicated by the first voice instruction.

[0067] The embodiment of the present application shows a partial implementation method of the above-mentioned determination of the first task indicated by the first voice instruction. Of course, it should be understood that the above-mentioned step of determining the first task indicated by the first voice instruction can also be implemented in other ways, and the embodiment of the present application does not limit this.

[0068] In another embodiment, after determining the first task indicated by the first voice instruction, the voice interaction method provided in the embodiment of the present application also includes: generating a session record of the first task based on the first text; and writing the session record of the first task into a cache.

[0069] In this way, the conversation record of the first task can be used as the context of subsequent text related to the first task, so that the terminal device can accurately identify the intention of the text, so as to accurately decide whether to respond to the received voice command.

[0070] In another embodiment, the session records of each task in the cache may also be periodically maintained to save cache resources and ensure the real-time nature of the session records in the cache. Specifically, the terminal device stores third configuration information, which includes the maximum number of tasks that can be stored in the cache and the maximum cache duration of each task. Based on this, the terminal device periodically queries the number of tasks stored in the cache. If the number of tasks reaches the maximum number of tasks, the tasks and their session records that were generated earlier will be deleted; and / or, for each task, the session records whose cache duration exceeds the maximum cache duration under the task will be deleted. In actual applications, the processing of voice commands and the maintenance of the cache may be performed by different threads.

[0071] In order to avoid false responses and improve the accuracy of voice interaction, when it is determined that the first object has the right to instruct the first task, the voice interaction method provided in the embodiment of the present application also combines the environmental information of issuing the first voice command to determine whether the first object really needs to perform the first task.

[0072] Specifically, in one embodiment, the above S204 includes the following steps: S241, identifying whether the environment information in which the first voice command is issued satisfies the execution condition of the first task.

[0073] As an example, the environmental information includes object information in the environment where the first voice command is issued and sound features in the environment. For example, in response to receiving the first voice command, the terminal device collects images of the environment through a camera, performs target detection on the collected images, and obtains objects contained in the environment. The terminal device also collects sounds in the environment, and obtains sound features in the environment by extracting features from the collected sounds.

[0074] In this case, the above S241 includes the following steps: Step B1: determining whether a first object exists in the environment based on the object information.

[0075] Step B2: If the first object does not exist in the environment, determine that the environmental information does not meet the execution condition of the first task.

[0076] If the first object does not exist in the environment, the first object may not want to perform the first task at this time. For example, the first voice command may be issued by someone else using a recording of the first object, or the first voice command is generated when other objects talk to the first object through a speaker, and then it is determined that the environmental information does not meet the execution conditions of the first task, so there is no need to respond to the first voice command.

[0077] Step B3: if there is a first object in the environment, classify the sound features to obtain the category of the sound features, and determine whether the environmental information meets the execution condition of the first task based on the category of the sound features.

[0078] If there is a first object in the environment, the first object may not want to perform the first task at this time. For example, the first object is triggered by mistake when talking with other users. Therefore, it is necessary to combine the differences in timestamps of different sound features in the environment to further determine whether the first object wants to perform the first task at this time.

[0079] Specifically, if there are at least two categories of sound features in the environment, and the difference between the timestamps of the sound features of different categories is less than the duration threshold, it is determined that the environmental information does not meet the execution condition of the first task.

[0080] For example, the timestamp of a sound feature may include a start timestamp. If there are two categories of sound features in the environment, and the difference between the start timestamps of the two categories of sound features is less than a duration threshold, it can be determined that the sound in the environment comes from two objects, and the two objects may be talking, and the first voice command is mistakenly triggered during the conversation between the two objects. Therefore, it is determined that the environmental information does not meet the execution conditions of the first task, and no response is given to the first voice command.

[0081] If there is one category of sound features in the environment, or there are at least two categories of sound features in the environment and the difference between the timestamps of the sound features of different categories is greater than or equal to the duration threshold, it is determined that the environmental information meets the execution condition of the first task.

[0082] For example, if there is a first object in the environment, and there is a type of sound feature in the environment, it can be determined that there is only the first object in the environment, the first voice command is issued by the first object, and the first object needs to execute the first task at this time, and then it is determined that the environmental information meets the execution conditions of the first task. If there is the first object in the environment, but there are two types of sound features in the environment, and the difference between the start timestamps of the two types of sound features is greater than or equal to the duration threshold, it can be determined that the sound in the environment comes from two objects, and the two objects are not talking, and the first object does need to execute the first voice command at this time, and then it is determined that the environmental information meets the execution conditions of the first task.

[0083] S242: If the environment information satisfies the execution condition, the first task is matched with the task authorized for the first object.

[0084] S243: If the match is successful, determine to execute the first task.

[0085] The first configuration information includes tasks authorized for the first object. The tasks authorized for different first objects may be at least partially the same, so that different first objects may indicate the same task.

[0086] Exemplarily, the first configuration information may be stored in the terminal device in the form of a data table. For example, as shown in Table 4 below, the first configuration information includes a fourth data table, which includes multiple rows of data, each row of data corresponds to a first object, including an identifier of the first object (i.e., an object ID) and an identifier of a task authorized for the first object.

[0087] Table 4

[0088] Assuming that the first object is user1, if the first task indicated by the first voice instruction is T2, since the tasks authorized for user1 include T2, the two successfully match, and it is determined that the first object has the right to indicate the first task T2, and then it is determined that the first task T2 is executed. If the first task indicated by the first voice instruction is T3, since the tasks authorized for user1 do not include T3, the two fail to match, and it is determined that the first object has no right to indicate the first task T3, and then it is determined that the first task T3 is not executed.

[0089] In the above implementation, it is first identified whether the environmental information for issuing the first voice command satisfies the execution conditions of the first task, and it can be determined whether the first object needs to execute the first task in the environment. If so, it is then authenticated whether the first object has the right to execute the first task. If so, it is determined to execute the first task. In this way, it is possible to effectively avoid erroneous responses in situations where the configured object has no right to indicate a task and the configured object does not need to execute a task, thereby improving the security and accuracy of voice interaction.

[0090] In another embodiment, the above S204 includes the following steps: matching the first task with the task authorized for the first object; if the match is successful, identifying whether the environmental information for issuing the first voice command meets the execution condition of the first task; if the environmental information meets the execution condition, determining to execute the first task.

[0091] The above describes some implementation methods of the above S204. Of course, it should be understood that the above S204 can also be implemented in other ways, which are not limited in the present embodiment.

[0092] S206: If it is determined to execute the first task, respond to the first voice instruction based on the execution result of the first task.

[0093] Specifically, the first task is executed based on the first text to achieve a response to the first voice command. In one embodiment, in order to achieve an accurate response to the first voice command so that the first object can quickly obtain the required information, the above S206 includes the following steps: Step C1, executing a first task based on a first text, and obtaining an execution result of the first task.

[0094] For example, the first task is identified as "entertainment information consultation" in the above manner, and the first intention of the first text is "inquiring about the nearest concerts in the city". Based on this, a search engine or application (Application, APP) interface is called to query the city's concert release information from the Internet; according to the hyperparameters configured in the terminal device, such as the "message validity period" is 15 days, and according to the time of the searched concert release information, the concert release information within 15 days from the current time is determined as the execution result of the first task.

[0095] Step C2, determining a first template that matches the first text.

[0096] The first template can be determined from a template set or generated in real time. The template set is a collection of multiple templates pre-configured in the terminal device. Each template includes a question text and an answer text template. The first text is matched with the question text of each template to obtain a first template that matches the first text. The matching method can be based on edit distance sorting or similarity scoring sorting based on a large language model (LLM) to obtain the first template that is most similar to the first text.

[0097] Exemplarily, as shown in Table 3 above, each intent has at least one corresponding template. Templates corresponding to the first intent are taken from the template set; the question texts of these templates are matched with the first text, and a first template matching the first text is determined from these templates.

[0098] For example, the first intention of the first text is "the time of the nearest concert". By matching the first text with the question text of each template corresponding to the first intention (i.e., "Is there a concert in [this city] recently?", "Concert ticket price?"), the first template is obtained as "{Is there a concert in [this city] recently? - Hello, there are {|there are} concerts scheduled in {this city} in the last {} days, [the concert will be held on {} day at {}, and the lead singer is {}]}".

[0099] Step C3, filling in the first template based on the execution result of the first task to obtain a response text of the first text.

[0100] Exemplarily, the execution result is filled into the response text template of the first template to obtain the response text of the first text. For example, the first template is "{Is there a concert in [this city] recently? - Hello, there is {|there is not} a concert scheduled in {this city} in the last {} days, [The concert was held on {} on {}, and the lead singer is {}]}", which contains the question text "Is there a concert in [this city] recently?" and the response text template "Hello, there is {|there is not} a concert scheduled in {this city} in the last {} days, [The concert was held on {} on {}, and the lead singer is {}]". If the execution result of the first task is: XX singer held a concert in this city on July 5, then the execution result is filled into the response text template to obtain the response text of the first text: Hello, there is a concert scheduled in this city within the last 15 days, the concert was held on July 5, and the lead singer was XX singer].

[0101] Step C4, converting the response text into response voice and then outputting it.

[0102] Exemplarily, the response text is converted into a response voice and then outputted through a pre-trained TTS model, thereby realizing voice interaction with the first object. The TTS model may include, for example, but is not limited to: Tacotron, FastSpeech2, VITS, Glow-TTS, NaturalSpeech, etc.

[0103] Of course, if the first object has no right to instruct the first task, the first voice instruction is ignored, that is, no response is given to the first voice instruction.

[0104] In another embodiment, in a multi-object and multi-task voice interaction scenario, in order to achieve flexible control of different objects and different tasks, before the above S202, the voice interaction method provided in the embodiment of the present application may also include: displaying a configuration interface; in response to a first configuration operation input by an object of the first role in the configuration interface, determining an object of the second role, and generating an object set based on the object of the first role and the object of the second role; in response to a second configuration operation of the object of the first role for each object in the object set, determining an authorized task to be configured for each object, and generating first configuration information based on the authorized task for each object in the object set.

[0105] The first role may be a role with management authority, such as an administrator; the second role may be a role without management authority, such as a non-administrator.

[0106] For example, the administrator can configure objects that are allowed to interact with the terminal device through voice through the configuration interface, and set corresponding roles for these objects, so that the terminal device generates an object set based on these objects and roles. In addition, the administrator can also configure the permissions of each object through the configuration interface, such as for each object, inputting the tasks authorized for the object, so that the terminal device generates the first configuration information based on the tasks authorized for each object.

[0107] For example, in family life, the mother is an administrator, and the mother can set the father and the child as objects that are allowed to interact with the terminal device through voice through the configuration interface, and configure the father's role as an administrator and the child's role as a non-administrator, so as to allow the father to modify the object set and the first configuration information, but not allow the child to modify the object set and the first configuration information. In addition, the mother can also configure corresponding tasks for herself, the father, and the child through the configuration interface. For example, the tasks configured for herself and the father include entertainment information recommendation and learning information consultation, and the tasks configured for the child only include learning information consultation, so that the terminal device only responds to the voice commands issued by the father and the mother related to entertainment information recommendation and learning information consultation, and only responds to the voice commands issued by the child related to learning information consultation.

[0108] It is worth noting that the object of the first role configures the above information through the configuration interface, which can be a configuration operation input through the control displayed on the configuration interface, or a physical button (such as a mechanical button) provided by the terminal device. The configuration interface can be a web page or a mobile APP page, etc. The form of the configuration interface in the embodiment of the present application is not limited.

[0109] The voice interaction method provided by one or more embodiments of the present application controls the objects allowed to participate in the voice interaction and the tasks that the objects can indicate through voice interaction by configuring an authorized first object and authorizing corresponding tasks for the first object, thereby providing data support for realizing multi-object and multi-task voice interaction. Based on this, when a first voice instruction is received, it is first determined whether the first voice instruction comes from the first object, and if the first voice instruction comes from the first object, it is determined whether to execute the first task indicated by the first voice instruction based on the environmental information of issuing the first voice instruction and the task authorized for the first object, and then when the first task is determined, the first voice instruction is responded to based on the execution result of the first task; in this way, as long as the received voice instruction is issued by an object in the object set, the object has the right to indicate the task indicated by the voice instruction, and the object does need to execute the task, the voice instruction can be responded to. On the one hand, multiple objects can participate in the same conversation and indicate different tasks, realizing voice interaction for multiple objects and multiple tasks, greatly improving the flexibility, convenience and wide application of voice interaction; on the other hand, it can effectively avoid erroneous responses to voices issued by unconfigured objects, tasks that configured objects have no right to indicate, and tasks that configured objects do not need to execute, thereby improving the safety and accuracy of voice interaction.

[0110] In order to facilitate the understanding of the voice interaction method provided in the above embodiment, Figure 3 and Figure 4 Provide detailed explanation.

[0111] During the configuration phase, the terminal device displays a configuration interface. The administrator configures the object set, the first configuration information, the second configuration information, and the third configuration information through the configuration interface. Among them, the object set includes multiple first objects allowed to interact with the terminal device and the role of each first object. The first configuration information includes the tasks authorized for each first object. The second configuration information includes the task set supported by voice interaction and the intent corresponding to each task in the task set. The third configuration information includes the maximum number of tasks that can be stored in the cache of the terminal device and the maximum cache duration of each task.

[0112] During the registration phase, the terminal device displays a registration interface, which includes text for registering voiceprints. Each first subject reads the text. The terminal device receives the voice of each first subject, and obtains the voiceprint features of each first subject by performing voiceprint recognition on the voice, which serves as the basis for subsequent authentication.

[0113] In the voice interaction stage, such as Figure 3 As shown, the terminal device receives a first voice command, obtains a first voiceprint feature by performing noise reduction processing and voiceprint recognition on the first voice command; then, the first voiceprint feature is matched with the voiceprint feature of each first object in the object set; if there is no second voiceprint feature matching the first voiceprint feature, no response is given to the first voice command; if there is a second voiceprint feature matching the first voiceprint feature, the first object to which the second voiceprint feature belongs is determined as the first object from which the first voice command originated.

[0114] Further, the terminal device converts the first voice instruction into a first text, queries the cache for a session record of the task authorized for the first object, obtains the context of the first text from the session record, and performs intent recognition on the first text based on the context to obtain a first intention of the first text; if the first intention is to end the task, the current session process is ended; if the first intention is not to end the session, the second configuration information is queried based on the first intention to obtain a task related to the first intention as the first task indicated by the first voice instruction.

[0115] Furthermore, the terminal device generates a session record of the first task based on the first text and writes the session record into the cache.

[0116] Further, the terminal device determines from the first configuration information whether the first object has the right to indicate the first task; if the first object does not have the right to indicate the first task, it does not respond to the first voice instruction; if the first object has the right to indicate the first task, it determines whether the first object needs the first task based on the environmental information of issuing the first voice instruction; if the first object does not need to perform the first task, it does not respond to the first voice instruction; if the first object needs to perform the first task, it performs the first task and obtains the execution result of the first task; determines a first template matching the first text from the template set; fills the first template based on the execution result of the first task to obtain a response text of the first text, and converts the response text into a response voice for output.

[0117] In addition, the terminal device also starts another independent thread to periodically maintain the session records of each task in the cache. Figure 4As shown, based on the third configuration information, the terminal device periodically detects whether there is a session record under each task whose cache duration exceeds the maximum cache duration, and if so, deletes such session record. In addition, the terminal device also detects whether the number of tasks in the cache exceeds the maximum number of tasks, and if so, deletes the tasks and their session records that were generated earlier.

[0118] The following uses a simple family life scenario as an example to illustrate the above voice interaction process. Assume that the object set includes mother, father and child, and the first configuration information includes: the tasks authorized for mother and father include entertainment information recommendation (identified as T1) and learning information consultation (identified as T2), and the tasks authorized for children include learning information consultation. The second configuration information is shown in Table 2 and Table 3 above.

[0119] The terminal device receives the voice command 1, and through voiceprint recognition, determines that the voice command 1 is issued by the mother, and then converts the voice command 1 into the corresponding text 1, and obtains "Is there any concert recently?"

[0120] Next, the terminal device does not find the session records of task T1 and task T2 in the cache, and directly performs intent recognition on text 1 to obtain intent I1 "recent concert"; by querying the second configuration information, it is determined that intent I1 corresponds to task T1. Since the tasks authorized for the mother include task T1, that is, the mother has the right to instruct task T1, and the environmental information of the voice instruction is used to determine that the mother needs to perform task T1, the search engine is called to query the recent concert information from the Internet, and the first template matching text 1 is determined from the templates corresponding to intent I1, and the query result is filled into the first template to obtain the response text 1 of text 1 "Hello, there is a concert scheduled in this city within the next 15 days. The concert will be held on July 5th, and the lead singer is singer XX", and then based on the speech synthesis technology, the response text 1 is converted into speech and output.

[0121] Further, the terminal device generates a session record of task T1 based on text 1 and response text 1, and writes it into the cache.

[0122] The terminal device receives the voice command 2 again, and through voiceprint recognition, determines that the voice command 2 is issued by the father, and then converts the voice command 2 into the corresponding text 2, and obtains "What new songs does singer XX have?".

[0123] Next, the terminal device finds the session records of task T1 and task T2 in the cache, obtains text 1 related to text 2 from these session records, uses text 2 to perform intent recognition on text 1, and obtains intent I2 "star work search"; by querying the second configuration information, it is determined that intent I1 is associated with task T1. Since the tasks authorized for Dad include task T1, that is, Dad has the right to instruct task T1, and the environmental information of issuing the voice command determines that Dad needs to perform task T1, and then calls the search engine to query the works of XX singer from the Internet, and determines the first template that matches text 2 from the template corresponding to intent I2, fills the query results into the first template, and obtains text 2 response text 2 "Hello, according to the search results, XX singer has no new works in the past three months." Then, based on speech synthesis technology, the response text 2 is converted into speech and output.

[0124] Further, the terminal device generates a session record of task T1 based on text 2 and response text 2, and writes it into the cache.

[0125] The terminal device receives the voice instruction 3 again, and through voiceprint recognition, determines that the voice instruction 3 is issued by the child, and then converts the voice instruction 3 into the corresponding text 3 to obtain "Can I go to the concert?".

[0126] Next, the terminal device does not find the session record of task T2 in the cache, and then directly performs intent recognition on text 3, and obtains intent I3 "consulting about the concert"; by querying the second configuration information, it is determined that intent I3 corresponds to task T1. Since the tasks authorized for the child do not include task T1, the child has no right to instruct task T1, and thus does not respond to voice instruction 3.

[0127] Further, the terminal device generates a session record of task T2 based on text 3 and writes it into the cache.

[0128] The terminal device receives the voice instruction 4 again, and through voiceprint recognition, determines that the voice instruction 4 is issued by the child, and then converts the voice instruction into the corresponding text 4, obtaining "by the way, buy another math book."

[0129] Next, the terminal device does not find the session record of task T2 in the cache, and then directly performs intent recognition on text 4 to obtain intent J1; by querying the second configuration information, it is determined that intent J1 is associated with task T2. Since the tasks authorized for the child include task T2, that is, the child has the right to instruct task T2, and the environmental information of the voice instruction is used to determine that the child needs to perform task T2, the search engine is called to query the teaching materials related to mathematics books from the Internet, and the first template matching text 4 is determined from the templates corresponding to intent J1, and the query result is filled into the first template to obtain the response text 4 of text 4 "OK, based on your geographical location, Xinhua Bookstore on Wanshou Road in this city may have the teaching materials you are consulting." Then, based on speech synthesis technology, the response text 4 is converted into speech and output.

[0130] Further, the terminal device generates a session record of task T2 based on text 4 and response text 4, and writes it into the cache.

[0131] The terminal device receives the voice instruction 5 again, and through voiceprint recognition, determines that the voice instruction 5 is issued by the mother, and then converts the voice instruction 5 into the corresponding text 5, and obtains "Forget it, I won't go if there is no new song."

[0132] Next, the terminal device searches for the session records of task T1 and task T2 in the cache, and obtains the context of text 5 from these session records, including text 1, text 2, and response text 2. Then, these contexts are used to identify the intent of text 5, and the intent I100 "Entertainment information recommendation task is over" is obtained; by querying the second configuration information, it is determined that intent I100 is associated with task T1. Since the tasks authorized for the mother include task T1, that is, the mother has the right to instruct task T1, and intent I100 is the intent to end the task, the first template matching text 5 is determined from the templates corresponding to intent I100, and the response text 5 of text 5 is obtained based on the first template, "Okay, I am happy to serve you, and the consultation service for this concert is over." Then, based on the speech synthesis technology, the response text 5 is converted into speech and output.

[0133] Further, the terminal device generates a session record of task T1 based on text 5 and response text 5, and writes it into the cache.

[0134] At the same time, the terminal device also starts an independent thread to check the session records of each topic in the cache every 5 seconds. Assuming that the maximum cache time is 20 minutes, the session records of each topic that have been cached for more than 20 minutes will be deleted.

[0135] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] Based on the same inventive concept, the present application also provides a voice interaction device. Figure 5 , is a structural diagram of a voice interaction device 500 provided in an embodiment of the present application, the device 500 includes: a first determination module 510, a second determination module 520 and a response module 530.

[0137] The first determination module 510 is configured to determine, in response to receiving a first voice instruction, whether the first voice instruction originates from an authorized first object.

[0138] A second determination module 520 is configured to determine whether to execute a first task indicated by the first voice instruction based on environment information of issuing the first voice instruction and first configuration information if the first voice instruction originates from the first object, wherein the first configuration information includes tasks authorized for the first object; The response module 530 is configured to respond to the first voice instruction based on the execution result of the first task if it is determined to execute the first task.

[0139] In another embodiment, the first task is determined by: Converting the first voice instruction into a first text; Performing intent recognition on the first text to obtain a first intent of the first text; Based on the first intent, second configuration information is queried to obtain a first task corresponding to the first intent, wherein the second configuration information includes a task set supported by voice interaction and an intent corresponding to each task in the task set.

[0140] In another embodiment, performing intent recognition on the first text to obtain the first intent of the first text includes: Based on the first configuration information, searching for a session record of a second task, where the second task includes a task authorized for the first object, and the session record includes a second text converted from a second voice instruction indicating the second task, where the second voice instruction is a voice instruction received before the first voice instruction; determining a context of the first text from the conversation record; Intent recognition is performed on the first text based on the context to obtain a first intent of the first text.

[0141] In another embodiment, the performing intent recognition on the first text based on the context to obtain the first intent of the first text includes: splicing the first text and the context and inputting them into an intent recognition model to obtain a second intent and a probability of the second intent; Inputting the first text into the intent recognition model to obtain a third intent and a probability of the third intent; The intent with the highest probability between the second intent and the third intent is determined as the first intent of the first text.

[0142] In another embodiment, the second determining module is used to: Identify whether the environment information in which the first voice command is issued meets the execution condition of the first task; If the environment information satisfies the execution condition, matching the first task with a task authorized for the first object; If the match is successful, it is determined to execute the first task.

[0143] In another embodiment, the environmental information includes object information in the environment where the first voice command is issued and sound characteristics in the environment; When the second determination module identifies whether the environment information in which the first voice instruction is issued meets the execution condition of the first task, the second determination module performs the following steps: Based on the object information, determining whether the first object exists in the environment; If the first object does not exist in the environment, determining that the environment information does not meet the execution condition of the first task; If the first object exists in the environment, the sound feature is classified to obtain the category of the sound feature, and based on the category of the sound feature, it is determined whether the environmental information meets the execution condition of the first task.

[0144] In another embodiment, when the second determination module determines whether the environmental information meets the execution condition of the first task based on the category of the sound feature, the second determination module performs the following steps: If there are at least two types of sound features in the environment, and the difference between the timestamps of the sound features of different types is less than the duration threshold, it is determined that the environment information does not meet the execution condition of the first task; If there is one category of sound features in the environment, or there are at least two categories of sound features in the environment and the difference between the timestamps of the sound features of different categories is greater than or equal to the duration threshold, it is determined that the environmental information meets the execution condition of the first task.

[0145] In another embodiment, the response module is used to: executing the first task based on the first text to obtain an execution result of the first task, wherein the first text is obtained by converting the first voice instruction; Determining a first template that matches the first text; Filling the first template based on the execution result to obtain a response text of the first text; The response text is converted into a response voice and then outputted.

[0146] Obviously, the voice interaction device provided in the embodiment of the present application can be used as Figure 2 The execution subject of the voice interaction method shown is, for example Figure 2 In the voice interaction method shown in FIG. 1 , step S202 may be performed by Figure 5 The first determining module 510 in the voice interaction device shown in FIG. 1 is executed, and step S204 can be performed by Figure 5 The second determining module 520 in the voice interaction device shown in FIG. 1 is executed, and step S206 can be performed by Figure 5 The response module 530 in the voice interaction device shown is executed.

[0147] According to another embodiment of the present application, Figure 5 The various modules in the voice interaction device shown can be separately or completely combined into one or several other modules to constitute, or one (some) of the modules can be further divided into multiple smaller modules in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In practical applications, the functions of one module can also be implemented by multiple modules, or the functions of multiple modules can be implemented by one module. In the embodiments of the present application, the voice interaction device may also include other modules. In practical applications, these modules can also be implemented with the assistance of other modules, and can be implemented by the collaboration of multiple modules.

[0148] For example, the first determination module includes an identity identification interface, and the second determination module includes an authority identification interface. The first determination module determines the first object from the object set that is the source of the received first voice instruction by calling the identity identification interface, and the second determination module determines whether the first object has the authority to instruct the first task from the first configuration information by calling the authority identification interface.

[0149] The second determination module also includes a speech recognition module and an intention recognition module. The speech recognition module is configured with a speech recognition interface, and the intention recognition module is configured with an intention recognition interface. The determination module can convert the first voice instruction into a first text through the speech recognition module, and perform intention recognition on the first text through the intention recognition module to obtain the first intention of the first text, thereby determining the first task indicated by the first voice instruction based on the first intention.

[0150] The response module includes a speech synthesis module, and a speech synthesis interface is configured in the speech synthesis module. After obtaining the response text of the first text, the response module calls the speech synthesis interface through the speech synthesis module to synthesize the response text into speech and then output it.

[0151] According to another embodiment of the present application, the program can be executed on a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM), and other processing elements and storage elements. Figure 2 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Figure 5 The voice interaction device shown in the figure is used to implement the voice interaction method of the embodiment of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and transferred to an electronic device through the computer-readable storage medium and run therein.

[0152] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0153] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0154] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0155] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a voice interaction device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: In response to receiving a first voice instruction, determining whether the first voice instruction originates from an authorized first object; If the first voice instruction originates from the first object, determining whether to execute a first task indicated by the first voice instruction based on environment information in which the first voice instruction is issued and first configuration information, wherein the first configuration information includes tasks authorized for the first object; If it is determined to execute the first task, the first voice instruction is responded to based on the execution result of the first task.

[0156] The above application Figure 2The method performed by the voice interaction device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0157] The electronic device may also perform Figure 2 The method and the voice interaction device are implemented in Figures 2 to 4 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0158] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0159] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 2 The method of the embodiment shown is specifically used to perform the following operations: In response to receiving a first voice instruction, determining whether the first voice instruction originates from an authorized first object; If the first voice instruction originates from the first object, determining whether to execute a first task indicated by the first voice instruction based on environment information in which the first voice instruction is issued and first configuration information, wherein the first configuration information includes tasks authorized for the first object; If it is determined to execute the first task, the first voice instruction is responded to based on the execution result of the first task.

[0160] An embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps in the voice interaction method provided in the embodiment of the present application.

[0161] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0162] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0163] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0165] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A voice interaction method, characterized in that: include: In response to receiving a first voice instruction, determining whether the first voice instruction originates from an authorized first object; If the first voice instruction originates from the first object, determining whether to execute a first task indicated by the first voice instruction based on environment information in which the first voice instruction is issued and first configuration information, wherein the first configuration information includes tasks authorized for the first object; If it is determined to execute the first task, the first voice instruction is responded to based on the execution result of the first task.

2. The method according to claim 1, characterized in that The first task is determined by: Converting the first voice instruction into a first text; Performing intent recognition on the first text to obtain a first intent of the first text; Based on the first intent, second configuration information is queried to obtain a first task corresponding to the first intent, wherein the second configuration information includes a task set supported by voice interaction and an intent corresponding to each task in the task set.

3. The method according to claim 1, characterized in that The determining whether to execute the first task indicated by the first voice instruction based on the environment information of issuing the first voice instruction and the first configuration information, wherein the first configuration information includes the task authorized for the first object, comprises: Identify whether the environment information in which the first voice command is issued meets the execution condition of the first task; If the environment information satisfies the execution condition, matching the first task with a task authorized for the first object; If the match is successful, it is determined to execute the first task.

4. The method according to claim 3, characterized in that The environmental information includes object information in the environment where the first voice command is issued and sound characteristics in the environment; The identifying whether the environment information in which the first voice command is issued meets the execution condition of the first task includes: Based on the object information, determining whether the first object exists in the environment; If the first object does not exist in the environment, determining that the environment information does not meet the execution condition of the first task; If the first object exists in the environment, the sound feature is classified to obtain the category of the sound feature, and based on the category of the sound feature, it is determined whether the environmental information meets the execution condition of the first task.

5. The method according to claim 4, characterized in that The determining, based on the category of the sound feature, whether the environmental information meets the execution condition of the first task includes: If there are at least two types of sound features in the environment, and the difference between the timestamps of the sound features of different types is less than the duration threshold, it is determined that the environment information does not meet the execution condition of the first task; If there is one category of sound features in the environment, or there are at least two categories of sound features in the environment and the difference between the timestamps of the sound features of different categories is greater than or equal to the duration threshold, it is determined that the environmental information meets the execution condition of the first task.

6. The method according to claim 1, characterized in that The responding to the first voice instruction based on the execution result of the first task includes: executing the first task based on the first text to obtain an execution result of the first task, wherein the first text is obtained by converting the first voice instruction; Determining a first template that matches the first text; Filling the first template based on the execution result to obtain a response text of the first text; The response text is converted into a response voice and then outputted.

7. A voice interaction device, characterized in that: include: A first determination module, configured to determine, in response to receiving a first voice instruction, whether the first voice instruction originates from an authorized first object; a second determination module, configured to determine whether to execute a first task indicated by the first voice instruction based on environment information of issuing the first voice instruction and first configuration information if the first voice instruction originates from the first object, wherein the first configuration information includes tasks authorized for the first object; A response module is used to respond to the first voice instruction based on the execution result of the first task if it is determined to execute the first task.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the voice interaction method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the voice interaction method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the voice interaction method as described in any one of claims 1 to 6.

Images