A voice interaction method, intelligent device and system

Abstract

The embodiments of the present application provide a voice interaction method, intelligent device and system, the method comprising: obtaining a voice request statement input by a user, performing semantic analysis on the voice request statement, and obtaining semantic recognition information, wherein the semantic recognition information includes business type information, user intent information and user slot information. Search for a preset process in a process database of a business type corresponding to the business type information; when the corresponding preset process is found, fill the user slot information into the preset process; execute the preset process after filling the user slot information. The voice interaction method provided in the present application is based on a pre-configured preset process. On the one hand, in the process of developing new voice services and skills, there is no need to rely on writing business codes, thereby shortening the overall development cycle. On the other hand, using the preset process, users can also develop new voice services more easily without the need for development intervention, thereby saving manpower and time costs.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, intelligent device and system. Background Art

[0002] With the development of intelligent voice interaction technology, voice interaction function has gradually become a standard configuration of intelligent terminal products. Users can use the voice interaction function to realize voice control of intelligent terminal products and perform a series of operations such as watching videos, listening to music, checking the weather, and controlling the TV.

[0003] The specific process of voice control of intelligent terminal products is that the voice recognition module recognizes the voice input by the user as text. Then the semantic analysis module performs lexical, syntactic and semantic analysis on the text to understand the user's intention. Finally, the control end controls the intelligent terminal product to perform corresponding operations based on the understanding results.

[0004] The development of new voice services and skills relies on writing business codes. The overall development cycle is relatively long and requires development intervention, which results in very high labor and time costs. Summary of the invention

[0005] The present application provides a voice interaction method, intelligent device and system for solving the problem that the development of new voice services and skills depends on writing business codes, the overall development cycle is relatively long, and development intervention is required, and the manpower cost and time cost are very high.

[0006] In a first aspect, an embodiment of the present application provides a voice interaction method, the method comprising: obtaining a voice request sentence input by a user;

[0007] Performing semantic analysis on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information, and user slot information;

[0008] Searching for a preset process in a process database of a business type corresponding to the business type information, wherein the preset intention information of the preset process corresponds to the user intention information, and in the process database, the preset process and the preset intention information have a mapping relationship;

[0009] When the corresponding preset process is found, the user slot information is filled into the preset process;

[0010] Execute the preset process after filling the user slot information.

[0011] In a second aspect, an embodiment of the present application provides a voice interaction system, including:

[0012] The human-computer interaction interface is configured to receive a user voice request statement;

[0013] A speech recognition engine is configured to perform semantic analysis on the speech request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information and user slot information;

[0014] A process database is configured to store preset intention information, preset processes, and a mapping relationship between the preset intention information and the preset processes;

[0015] A process configuration engine is configured to search for a preset process in a process database of a business type corresponding to the business type information, wherein the preset intention information of the preset process corresponds to the user intention information, and when the preset process corresponding to the preset process is found, fill the user slot information into the preset process;

[0016] The process execution engine is configured to execute the preset process after filling the user slot information.

[0017] In a third aspect, an embodiment of the present application provides an intelligent device for voice interaction, the intelligent device comprising:

[0018] A user input unit is configured to obtain a voice request sentence input by a user;

[0019] A semantic analysis processing unit, configured to perform semantic analysis on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information and user slot information;

[0020] A process search unit is configured to search for a preset process in a process database of a business type corresponding to the business type information, wherein the preset intention information of the preset process corresponds to the user intention information, and in the process database, the preset process and the preset intention information have a mapping relationship, and when the preset process corresponding to the preset process is found, the user slot information is filled into the preset process;

[0021] The first process execution unit is configured to execute the preset process after filling the user slot information.

[0022] The technical solution provided by the present application includes the following beneficial effects: obtaining the voice request statement input by the user, performing semantic analysis and processing on the voice request statement, and obtaining semantic recognition information, wherein the semantic recognition information includes business type information, user intent information and user slot information. Searching for a preset process in a process database of a business type corresponding to the business type information, wherein the preset intent information of the preset process corresponds to the user intent information, and in the process database, the preset process and the preset intent information have a mapping relationship; when the corresponding preset process is found, the user slot information is filled into the preset process; and the preset process after the user slot information is filled is executed. The voice interaction method provided by the present application is based on a pre-configured preset process. On the one hand, in the process of developing new voice services and skills, there is no need to rely on writing business codes, thereby shortening the overall development cycle. On the other hand, using the preset process, users can also develop new voice services more easily without the need for development intervention, thereby saving manpower costs and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A schematic diagram exemplarily illustrates the voice interaction principle according to some embodiments;

[0025] Figure 2 A schematic diagram of a voice interaction system framework according to some embodiments is exemplarily shown;

[0026] Figure 3 A schematic diagram of a framework for using a preset process configuration file according to some embodiments is exemplarily shown;

[0027] Figure 4 The following is a flow chart of a voice interaction method according to some embodiments;

[0028] Figure 5 A signaling diagram of a voice interaction method according to some embodiments is exemplarily shown;

[0029] Figure 6 The following is a schematic diagram of a configuration protocol conversion process according to some embodiments;

[0030] Figure 7 Schematically shows a schematic diagram of a process engine executing a business logic according to some embodiments;

[0031] Figure 8 A schematic diagram of a task node executing a task according to a protocol according to some embodiments is exemplified;

[0032] Fig. 9 A schematic diagram of a user interface according to some embodiments is exemplarily shown;

[0033] Fig.10 A schematic diagram of a voice interaction process framework according to some embodiments is exemplarily shown;

[0034] Fig.11 A schematic diagram of a flow chart of another voice interaction method according to some embodiments is exemplarily shown;

[0035] Fig.12 A schematic diagram of a smart device framework for voice interaction according to some embodiments is exemplarily shown. DETAILED DESCRIPTION

[0036] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0037] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0038] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0039] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0040] To clearly illustrate the embodiments of the present application, Figure 1 A speech recognition network architecture provided in an embodiment of the present application is described.

[0041] See also Figure 1 , Figure 1 A schematic diagram of a speech recognition network architecture provided for an embodiment of the present application. Figure 1In the embodiment, the intelligent device is used to receive input information and output the processing result of the information. The speech recognition service device is a terminal device deployed with a speech recognition service, the semantic service device is a terminal device deployed with a semantic service, and the business service device is a terminal device deployed with a business service. The terminal devices here may include servers, computers, etc., and the speech recognition service, semantic service (also called semantic engine) and business service here are web services that can be deployed on the terminal devices, wherein the speech recognition service is used to recognize audio as text, the semantic service is used to perform semantic analysis on the text, and the business service is used to provide specific services such as weather query service, music query service, etc. In one embodiment, Figure 1 In the illustrated architecture, there may be multiple physical service devices deployed with different business services, or one or more functional services may be integrated into one or more physical service devices.

[0042] In some embodiments, the following Figure 1 The process of processing the information input into the smart device by the architecture shown is described by way of example. Taking the information input into the smart device as a query statement input by voice as an example, the above process may include the following three processes:

[0043] Voice recognition: After receiving a query statement input by voice, the smart device may upload the audio of the query statement to the voice recognition service device, so that the voice recognition service device recognizes the audio as text through the voice recognition service and returns it to the smart device. In one embodiment, before uploading the audio of the query statement to the voice recognition service device, the smart device may perform denoising on the audio of the query statement, and the denoising process here may include steps such as removing echoes and environmental noise.

[0044] Semantic understanding: The smart device uploads the text of the query statement recognized by the speech recognition service to the semantic service device, which then performs semantic analysis on the text through the semantic service to obtain the business domain, intent, etc. of the text.

[0045] Semantic response: The semantic service device sends a query instruction to the corresponding business service device based on the semantic analysis result of the query statement text to obtain the query result given by the business service. The smart device can obtain the query result from the semantic service device and output it. As an embodiment, the semantic service device can also send the semantic analysis result of the query statement to the smart device, so that the smart device can output the feedback statement in the semantic analysis result.

[0046] It should be noted that Figure 1 The architecture shown is only an example and does not limit the protection scope of the present application. In the embodiments of the present application, other architectures may also be used to implement similar functions. For example, all or part of the three processes may be completed by smart devices, which will not be described in detail here.

[0047] In some embodiments, Figure 1 The smart device shown can be a display device, such as a smart TV. The function of the voice recognition service device can be implemented by the sound collector and controller set on the display device. The functions of the semantic service device and the business service device can be implemented by the controller of the display device or by the server of the display device.

[0048] In some embodiments, the smart terminal product supports voice interaction function, which can be used as a standard configuration of the smart terminal product. Users can use the voice interaction function to realize voice control of the smart terminal product to perform a series of operations such as watching videos, listening to music, checking the weather, and controlling the TV.

[0049] The process of voice control of smart terminal products is usually as follows: the voice recognition module recognizes the user's input voice as text. Then the semantic analysis module performs lexical, syntactic and semantic analysis on the text to understand the user's intention. Finally, the control end controls the smart terminal product to perform corresponding operations based on the understanding results. Intent recognition is to determine what the user wants to do. For example, if a user asks a question to the voice recognition device, the voice recognition device needs to determine whether the user is asking about the weather, travel or information about a movie. Intent recognition is equivalent to a text classification problem.

[0050] With respect to the voice control process in the above embodiment, if new voice services and new voice skills are to be developed, they all need to rely on business code writing, resulting in a relatively long overall development cycle and low development efficiency.

[0051] In order to solve the above problems, the present application provides a voice interaction method, which can call a preset voice interaction process and directly execute the process during the voice interaction process. Therefore, when developing new voice services and new voice skills, there is no need to rely on business code writing, thereby shortening the development cycle and improving development efficiency.

[0052] Figure 2The block diagram of the voice interaction system on the terminal device is shown. The voice interaction system mainly includes: a human-computer interaction interface 101, a processing module 102 and a process database 103. The user can input a voice request statement through the human-computer interaction interface 101. The input of the processing module 102 is the output of the human-computer interaction interface 101. The processing module 102 is interconnected with the human-computer interaction interface 101, and can receive the voice data input by the user through the human-computer interaction interface 101, and output the interaction data to the user through the human-computer interaction interface, for example, the dialogue data fed back to the user, the task execution process and results. The human-computer interaction interface 101 also provides a visual process configuration interface, and the user can configure the business process through the definition of the web interface or the xml (eXtensible Markup Language) configuration file.

[0053] In some embodiments, a configuration file template library can be pre-set, and the configuration file template library contains multiple configuration items and format specifications of each configuration item; at least one configuration item is selected from the configuration file template library, and the corresponding project content is set for it according to the format specification of the selected configuration item. The configuration file template library can be provided by a tool chain (such as application compilation software, etc.). For example, when a developer creates an application through a specific application compilation software, the application compilation software automatically generates a configuration file template for the application according to the pre-stored configuration file template library. Among them, the application compilation software can be various types of software such as Go language compilation software and VC compilation software. The configuration file template automatically generated by the application compilation software contains multiple configuration items, and the developer only needs to select several configuration items from them and set the corresponding project content for them according to the format specification of the selected configuration items.

[0054] The configuration items included in the configuration file can be either environment variables related to the Hero framework or environment variables related to the game application. For example, the environment variables related to the Hero framework include: service loading path, Go language path (also called GoPass), etc. The environment variables related to the game application include: information about the dynamic link library or database that the game application needs to call during operation, etc., where the database information includes the database port, database username and password, etc. In addition, developers can also flexibly configure a variety of other configuration items.

[0055] Each configuration item in the configuration file can be stored in the form of Key-Value, the project name of the configuration item is represented by Key, and the specific project content of the configuration item is represented by Value. The Key-Value storage method can save storage space and improve query efficiency. In this embodiment, the configuration file is written in YAML language. In other embodiments of the present invention, the configuration file can also be written in other languages ​​such as XML and JS.

[0056] like Figure 3 As shown in the configuration file diagram, in the configuration file of the embodiment of the present invention, the mapped key is the preset intent information, and the value is the preset process. After the preset intent information and the preset process are mapped, the preset process can be called from the database according to the intent information. It should be noted that the preset process of the present application includes all processes after identifying the user intent information and slot information during the voice interaction process.

[0057] The processing module 102 further includes a speech recognition engine 1021, a process configuration engine 1022, and a process execution engine 1023. The speech recognition engine 1021 is used to recognize the user's voice command as text and parse the semantic recognition information from the text; the process configuration engine 1022 is used to store the process configuration file defined by the user through the human-computer interaction interface 101, and is also used to query the process configuration file; the process execution engine 1023 is used to control the execution of the process. The process configuration engine 1022 can store the process configuration file in the process database 103.

[0058] The speech recognition engine 1021 determines the user intention (i.e., determines the task), determines the key knowledge data corresponding to each slot associated with the user intention, fills the key knowledge data into the corresponding slot, and then executes the task based on the filled one or more slots.

[0059] In some embodiments, the voice assistant system is an interactive voice application developed and executed using software running on a general-purpose computer system. In other embodiments, the voice interaction method can be implemented by using dedicated hardware instead of software or by combining hardware and software. Therefore, the embodiments of the present invention are not limited to any specific combination of hardware circuits or software.

[0060] like Figure 4 The voice interaction method flow chart shown is based on Figure 2 The voice assistant system shown, the voice interaction method of the present application includes the following steps:

[0061] In step S101, a voice request sentence input by a user is obtained.

[0062] In step S102, semantic analysis is performed on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information and user slot information.

[0063] A voice request is a request sent in the form of voice, that is, a request issued by a user in the form of speaking, etc. A voice request sentence is a voice text converted according to a user input voice signal. In a specific application scenario, a voice request may be an audio signal including the content "What is the weather like today?", and the converted voice request sentence is the text content of "What is the weather like today?".

[0064] Specifically, the user inputs a voice signal within the distance range where the terminal device equipped with the voice assistant receives the signal. The terminal device may collect the voice signal input by the user through a microphone, and then recognize the voice text from the voice signal. In the embodiment of the present application, the voice text may be recognized by the voice recognition server. The semantic server performs semantic analysis on the voice text. It should be noted that the semantic server performs semantic analysis on the voice text.

[0065] The semantic analysis process is as follows: first, the speech text is segmented. The forward maximum matching method can be used based on the word library to perform word segmentation. For example, "Xiao Ming's pencil" is segmented to obtain the segmentation "Xiao Ming, of, pencil" after word segmentation. Then the segmented words are tagged with parts of speech. Specifically, for example, the LAC (Lexical Analysis of Chinese) lexical analysis tool can be used to perform Chinese word segmentation and part-of-speech tagging on the media asset title. The LAC lexical analysis tool is a joint lexical analysis model that can complete Chinese word segmentation and part-of-speech tagging as a whole. A custom dictionary can also be added to identify proper names. The input of the LAC lexical analysis task is a string, and the output is the word boundaries and parts of speech in the media asset title. It should be noted that the word segmentation and part-of-speech tagging tools used in this application are not limited to the LAC lexical analysis tool, and other lexical analysis tools can also be used. The result after part-of-speech tagging is {Xiao Ming-Xiao Ming [actor]}, {of-of [funcwordstructaux]}, pencil-pencil [title]}.

[0066] Then, we use dependency syntax to extract the user intent in the speech text based on the results of part-of-speech tagging. Dependency syntax analysis uses global search to generate multiple dependency trees. Each sentence corresponds to one or more dependency trees, and the system outputs the tree with the highest probability. Finally, the intent corresponding to the dependency tree with the highest probability is taken as the user intent in the speech text.

[0067] There are three main ways to represent natural language semantics: distribution semantics, framework semantics, and model-theoretic semantics. This application is related to intelligent dialogue and uses model-theoretic semantics. For example, domain parameters (business type information), intent parameters (for intent information), and user slot information (user slot information) are used to represent the semantic structure. That is, the semantic text is processed through semantic analysis and includes at least three parameters: domain parameters, intent parameters, and user slot information.

[0068] The voice text "Xiao Ming's pencil" is processed through semantic analysis to obtain three parameters: the service type information is audio service, the intent information is to play audio, and the user slot information is Xiao Ming, of, and pencil.

[0069] In some embodiments, when understanding the user's intention, if the understanding is only based on the voice currently input by the user, the understanding of the user's intention will often be biased. Therefore, the user's true intention can be understood in detail according to the context.

[0070] Specifically, two voice texts that are in a sequential relationship in time sequence are obtained. The former voice text is parsed to obtain the former domain parameter, the former intent parameter, and the former user slot information. The latter voice text is parsed to obtain the latter domain parameter, the latter intent parameter, and the latter user slot information. If the latter user slot information is a replacement, that is, the user has a slot replacement intention, then the latter user slot information is replaced with other user slot information. Finally, the user's true intention is determined based on the replaced user slot information.

[0071] Exemplarily, the previous domain parameter, the previous intent parameter, and the previous user slot information obtained by the previous speech text analysis are: audio service, play audio, Xiao Ming, of, pencil. The subsequent user slot information is: change, person, of. At this time, the subsequent user slot information includes the keyword "person", indicating that the user slot information related to the person in the previous user slot information needs to be replaced. In the previous user slot information, "Xiao Ming" is the "author" and is related to the person. And "Pencil" is a title, not related to the person. It means that the previous user slot information "Xiao Ming" needs to be replaced. For example, the user slot information "Xiao Ming" related to the person is replaced with "Xiao Hong", and finally the user's real intention is to play the audio "Xiao Hong" "Pencil".

[0072] In step S103, a preset process is searched in a process database of a business type corresponding to the business type information, wherein the preset intention information of the preset process corresponds to the user intention information, and in the process database, the preset process and the preset intention information have a mapping relationship.

[0073] This application configures different process databases according to different business types. For example, the audio business is configured with a corresponding audio business process database, and the video business is configured with a corresponding video business process database. Different process databases store the preset processes and preset intent information of their respective business types. In the process database, the preset processes and preset intent information also have a mapping relationship.

[0074] In some embodiments, the functions of the voice assistant are relatively simple and generalized. All voice interactions use a general voice recognition module for voice recognition processing. It cannot perform personalized recognition for the scene, and the recognition accuracy cannot be guaranteed, especially in special scenes, where recognition errors are more likely to occur. Therefore, when configuring resources for voice recognition, the preset processes associated with each scene are stored for scene configuration according to the needs and the scenes that the voice product may involve. The scenes here can be different usage scenarios, such as the distinction between public scenes and private scenes. The voice recognition module used in private scenes can be customized according to personal usage habits, thereby streamlining the voice recognition module. Public scenes need to expand the interaction range of the voice recognition module to meet the usage needs of different groups of people.

[0075] In some embodiments, speech recognition resources can also be configured according to the scenario. Speech recognition generally includes an acoustic model and a language model. In the actual configuration process, audio is generally provided for training the acoustic model, and corpus is provided for training the language model. Therefore, during configuration, audio and corpus of different scenarios are configured according to the scenario to train and obtain speech recognition resources for different scenarios (including trained acoustic models and language models. The training method is to pre-configure the audio and corpus of the corresponding scenario and input them into the training model for training to obtain the required speech recognition resources). Through this step, the acoustic model (AM) and language model (LM) can be customized for different processes and scenarios, that is, scene speech recognition resources for multiple scenarios can be obtained.

[0076] In response to the received user voice request, the scene voice recognition resource associated with the intended scene is obtained for voice recognition. When configuring the voice recognition resource based on the scene, the user's voice request can be configured separately as a scene due to the similarity in the voice interaction. Since the user's voice request is generally to express the incoming call intention, this scene can be defined as the intended scene, and the scene voice recognition resource is configured for it in the semantic analysis and processing process. In this way, when the user's voice request is received, the scene voice recognition resource corresponding to the intended scene can be obtained from the scene voice recognition resources associated with the scene stored in the configuration for voice recognition. In the dialogue response process, the associated scene is determined according to the current user intention, and the corresponding scene voice recognition resource is obtained according to the determined associated scene to perform voice recognition on the monitored user audio. Since the user's voice request generally starts the first valid voice instruction of the voice product, the voice recognition resource of the intended scene can be directly classified according to the scene of the voice request during the incoming call, and the voice recognition resource can be obtained for voice recognition. After the voice product is started, the voice interaction process will be entered, and there will be interaction and dialogue between the voice product and the user. In some embodiments, recognition resources can be switched during a conversation. Since the user's intention and the response content to the user's audio (i.e., the conversation content) can be obtained during the conversation, the current scene can be determined conveniently and accurately. Therefore, the current scene is determined based on the conversation process, and the speech recognition resources are switched based on the determined scene. This will improve the fit between the selected speech recognition resources and the scene, and further improve the accuracy.

[0077] When receiving a user voice request, load the scene voice recognition resources associated with the voice product. After selecting the scene resources for the product in the product development stage, when receiving a voice request, you can first load the voice recognition resources configured for the product.

[0078] In this way, when performing speech recognition, including the recognition of user voice requests and the recognition of user feedback audio in dialogue interaction, the corresponding scene speech recognition resources can be directly obtained from the loaded scene speech recognition resources to perform speech recognition on user voice requests or monitored user audio. Since the capacity of the configured stored speech recognition resources is relatively large, and when configuring scenes for different products or the same product, each product may not involve so many scenes in a specific application, preloading can narrow the scope of recognized resources and improve recognition efficiency. Moreover, since preloading does not require resources to be obtained from distributed servers, it can also effectively shorten the recognition response time.

[0079] In step S104, when the corresponding preset process is found, the user slot information is filled into the preset process.

[0080] In step S105, the preset process after filling the user slot information is executed.

[0081] User slot information is used to indicate associated information for executing a preset process.

[0082] The following is Example 1 of the above-mentioned embodiment. First, a voice request sentence input by the user is received, "What will the weather be like in Hangzhou tomorrow?" After semantic analysis and processing, semantic recognition information is obtained. The semantic recognition information includes business type information of query, and user intent information of "query weather." Then, from the process database whose business type is query, the preset process is searched according to the user intent information "query weather." Specifically, the preset intent information of "query weather" is searched, and then the corresponding preset process of querying weather is found. Finally, the preset process of querying weather is executed according to the semantic recognition information.

[0083] In a further solution of Example 1, the semantic recognition information also includes user slot information, specifically the location slot information "Hangzhou" and the time slot information "tomorrow". The location slot information "Hangzhou" and the time slot information "tomorrow" are filled into the preset process. The filling here means directly filling the slot into the code of the preset process for the code to run. Finally, the preset process of querying the weather in Hangzhou tomorrow is executed.

[0084] The following is Example 2 of the above embodiment. First, a voice request sentence "Play Xiao Ming's pencil" input by the user is received. After semantic analysis and processing, semantic recognition information is obtained. The semantic recognition information includes that the business type is audio and video, and the user intention information is "play audio or video". After that, in the process database whose business type is audio and video, the preset process is searched according to the user intention information "play audio or video". Specifically, the preset intention information of "play audio or video" is searched, and then the corresponding preset process of playing audio or video is found. Finally, the preset process of playing audio or video is executed according to the semantic recognition information.

[0085] In a further solution of Example 2, the semantic recognition information also includes user slot information, specifically including character slot information "Xiao Ming" and title slot information "Pencil". The character slot information "Xiao Ming" and the title slot information "Pencil" are filled into the preset process, where filling means directly filling the slot into the code of the preset process for the code to run. Finally, the preset process of playing the audio or video of "Xiao Ming's Pencil" is executed.

[0086] based on Figure 2 The voice assistant system shown in FIG. 1 and the signaling diagram of the voice interaction method of the present application are as follows: Figure 5 shown.

[0087] First, the user inputs a voice request statement through the human-computer interaction interface 101, and the human-computer interaction interface 101 sends the voice request statement to the voice recognition engine 1021. The voice recognition engine 1021 parses the voice request statement. For the specific parsing process, see the above embodiment. After parsing the voice request statement, voice recognition information is obtained. The voice recognition information includes business type information, user intent information, and user slot information.

[0088] After parsing the speech recognition information, the speech recognition engine 1021 can send the speech recognition information to the process execution engine 1022. The process execution engine 1022 then sends the user intent information in the speech recognition information to the process configuration engine 1023. The process configuration engine 1023 searches for a preset process from the process database 103 of the corresponding business type according to the user intent information. Specifically, a mapping relationship between the preset process and the preset intent information is created in the process database 103.

[0089] In the process database 103, the preset intention information corresponding to the user intention information is first searched, and then the preset process with a mapping relationship is searched according to the preset intention information. The process database 103 feeds back the found preset process to the process configuration engine 1023. The process configuration engine 1023 fills the user slot information into the preset process, and sends the preset process filled with the user slot information to the process execution engine 1022. Finally, the process execution engine 1022 executes the preset process after filling the user slot information. In addition, the process execution engine 1022 can also render the result of executing the preset process on the human-computer interaction interface 101.

[0090] In some embodiments, the specific implementation process of configuring the preset process in the above embodiments may be the following steps:

[0091] The process execution engine 1022 is initialized. When the process execution engine 1022 is initialized, it reads all available preset processes from the process configuration engine 1023 and loads the preset processes into the global cache (ProcessCache). ProcessCache uses a Map data structure to store data. An exemplary structure is shown in Table 1:

[0092] key value volume.up ProcessObject volume.set ProcessObject volume.down ProcessObject alarm.set ProcessObject weather ProcessObject

[0093] Table 1 Global cache data structure example

[0094] At the same time, the preset process will be parsed into objects in the process execution engine, and each preset process will be parsed into a Node object. When a voice request enters the process execution engine, the voice recognition engine is called first. The voice recognition engine analyzes the user intent information and returns the user intent information and user slot information.

[0095] User intent (intentCode) indicates what the user wants to do, for example, "What will the weather be like in Hangzhou tomorrow?" The user intent information is encoded as weather.query.

[0096] The user slot information represents the key information used to perform the task in the current user's intent. The user slot of "What will the weather be like in Hangzhou tomorrow" is: time slot information time = tomorrow, location slot information city = Hangzhou. The process configuration engine searches for the corresponding process information based on the intentCode. When a matching preset process is found, it parses and executes the process. Finally, the process execution result is returned to the terminal device for rendering.

[0097] In some embodiments, the user intention information and the preset intention information can be encoded using a phonetic code algorithm, and only matching and searching can be performed according to the encoding. The present application can encode the user intention information and the preset intention information based on the existing Metaphone algorithm. Metaphone is a phonetic code algorithm for English, which encodes texts by English pronunciation rules and encodes texts with similar pronunciations into the same phonetic code. The Metaphone algorithm is designed for English and can also be extended to similar phonetic languages ​​(such as English, Spanish, Russian, etc.). Therefore, the present application can utilize the Metaphone algorithm for different languages ​​to encode the error correction entity.

[0098] In addition to using the Metaphone algorithm for phonetic characters, it is also possible to develop phonetic coding algorithms for pictographic languages ​​based on pronunciation rules. For example, Chinese is encoded based on pinyin rules, and Japanese is encoded through the gospel of fifty sounds. Figure 4 This is an example of Chinese phonetic code encoding. For the text to be encoded, the PyPinyin tool is used to convert the text into the corresponding pinyin, and the initial consonants and finals in the pinyin are encoded separately. The similarity of the pinyin pronunciation is taken into account during the encoding process. For example, the initial consonant n and the initial consonant l, and the finals "an" and "ang" can be encoded into the same code.

[0099] A large amount of preset intention information is pre-stored in the database of the present application, and the preset intention information in the database is also generated through the same encoding process, so the preset intention information in the phonetic code database is also in encoded form. This step matches the encoded form of the user intention information with the encoded form data in the database, so as to find the preset intention information that matches the user intention information. It should be noted that the matching of the user intention information and the preset intention information here can be that the encoding of the user intention information is the same as the encoding of the preset intention information, or the encoding of the user intention information contains the encoding of the preset intention information.

[0100] For example, if the user intent information is to query the weather and the preset intent information is also to query the weather, then the two are matches of the same encoding type. If the user intent information is to play music and the preset intent information is to play audio and video, then the preset intent information contains the user intent information, and at this time the preset intent information also matches the user intent information.

[0101] In some embodiments, the process protocol may be expressed as follows:

[0102] The start node represents the starting node of the process. The protocol is as follows: start (start node identifier); start.id (unique identifier of the node); start.intentCode (intent code, when the user intent information is the same as the preset intent information of the process, the process is entered); start.childId (execute script); params.param.name (the name of the intent slot necessary for executing the process); params.param.message (when the intent slot is empty, a prompt message is sent to the user).

[0103] Gateway node, representing a process branch node, when the set condition is met, it will go to branch A, and when it is not met, it will go to branch B. gateway.id (gateway node id); condition.value (Boolean expression): when the return result is true, it jumps to the process node pointed to by ifTrue.childId; when the return result is false, it jumps to the process node pointed to by ifFalse.childId.

[0104] The local method node indicates calling the local built-in method of the application. nativeMethod.id (node ​​id); nativeMethod.childId (id of the next execution node); type (local method type (static, springBean)); className (fully qualified name of the local method, through which the local method can be called); method (local method); inputParam.type (input type of local parameter); inputParam.value (input value (# indicates a reference to the value in the session cache, otherwise it is a constant)).

[0105] http method node, indicating the call to the external HTTP interface. httpMethod.id (node ​​id); httpMethod.childId (id of the next execution node); type (GET / POST); url (address of the http interface); inputParam.type (input parameter type); inputParam.value (input parameter value (the # at the beginning indicates the reference value in the session cache, otherwise it is a constant)).

[0106] Script execution node, which means executing the script language locally. executeScript.id (node ​​id); executeScript.childId (id of the next execution node); type (script type, python, javascript, lua, etc.); script (executable script); inputParam.type (input parameter type); inputParam.value (input parameter value (the # at the beginning indicates the reference value in the session cache, otherwise it is a constant)).

[0107] End node. end.id (current node id); result.key (key of the returned result); result.value (value of the returned result (the # at the beginning indicates the reference value in the session cache, otherwise it is a constant)).

[0108] The output of the final node will be assembled into a json (JavaScript Object Notation) data.

[0109] Taking the completion process protocol of the weather query process as an example, the configuration protocol can be converted into Figure 6 The process shown.

[0110] After starting the process at the start node, determine whether the time slot is empty. If it is empty, obtain the current system time, add the current system time to the time slot, and proceed to the process of the next node. If the time slot is not empty, proceed to the process of the next node. The next node of the start node is the gateway node. Query the weather through the gateway interface node. If the query result is empty, end the process and give a failure prompt message. If the query result is not empty, assemble tts (Text To Speech) in the script node. Finally, end the process at the end node.

[0111] Based on the above process protocol, the design of the process execution engine can be expressed as follows:

[0112] Session cache design:

[0113] Session: When a user requests to enter a process, the process execution engine creates a session for the user request until the process execution is completed, indicating that the session is complete. During a session, a session cache (SessionCache) is created for the current session. The session cache mainly stores process execution information and is deleted after the process ends. The session includes but is not limited to the following information:

[0114] Process execution information, currentNode (currently executed node); intent information, intent.intenCode (intent code); intent.slot.[slotName] (intent slot); request parameters, reqParams.[paramName]: (parameter name); engine node execution result, executeResult.[nodeId] (nodeId is a variable that identifies the node ID); lastExexuteResult (the execution result of the previous node). The above session cache can be obtained through the parameter name at any node in the current process execution. For example, if you need to obtain the intent code in the configuration process, it is expressed as #intent.intentCode.

[0115] Specifically, the process engine executes the business logic diagram as follows Figure 7 As shown:

[0116] If no matching process is found according to intentCode (user intent information), the default process is executed, and a prompt is prompted that the current function is not supported. If a matching process is found, a session cache is created, and information such as user process, request parameters, intent code, slot, etc. are placed in the session cache. Parse the start node to determine whether the necessary slots are missing. If the necessary slots are missing, prompt the user that the necessary message is missing and delete the session cache. After the parameter verification is completed, obtain the child node childId, find and load the corresponding node information, and determine whether the node is the end node at that time. If it is an end node, assemble the parameters into json format according to the params (variable method parameters) definition, end the process, and delete the session cache. If it is not an end node, execute the task according to the protocol and cache the node execution information. After the execution is completed, jump to the step of obtaining the child node task.

[0117] like Figure 8 As shown in the figure, the specific process of the task node executing the task according to the protocol is as follows: Find and load the node information according to the child node childId. First determine whether the current node is a gateway node. If it is not a gateway node, parse the node and splice the request parameters according to the node definition. Call the task according to the definition (httpMethod (hypertext transfer request method), executeScript (execute script), nativeMethod (java calls non-java code interface)). Put the execution result into the session cache. Get the child node childId and jump to the step of finding and loading the node information according to the child node childId. If the current node is a gateway node, execute the gateway expression. Determine the child node childId corresponding to the executed branch process according to the expression result (true, false), and then jump to the step of splicing the request parameters according to the node definition.

[0118] In some embodiments, if first semantic recognition information and second semantic recognition information are obtained after semantic analysis of the voice request statement, the first preset process and the second preset process corresponding to the first semantic recognition information and the second semantic recognition information are respectively searched in the process database.

[0119] Exemplarily, the voice request sentence input by the user is "I want to sing and dance". After semantic analysis of the voice request sentence, two semantic recognition information are obtained. The first semantic recognition information includes the service type information "audio service" and the user intention information "play songs". The second semantic recognition information includes the service type information "video service" and the user intention information "play dance videos". The first preset process for playing songs corresponding to the user intention information "play songs" can be searched from the audio service process database, and the second preset process for playing dance videos corresponding to the user intention information "play dance videos" can be searched from the video service process database.

[0120] In some embodiments, based on the above embodiments, if the first preset process and the second preset process can be performed at the same time, for example, playing a song and checking the weather can be performed at the same time, then the two preset processes are performed at the same time. If the first preset process and the second preset process cannot be performed at the same time, the preset processes can be performed separately. For example, playing a song and a dance video in the above embodiments cannot be performed at the same time. Then a pop-up window such as Fig. 9 The dialog box shown prompts the user to select the process to be performed. If there are at least three preset processes, the user can also Fig. 9 In the dialog box shown, the preset processes are sorted. Finally, the terminal device executes the preset processes according to the sorting. In this way, the user only needs to input a voice request once to control the terminal device to execute multiple preset processes. The user can also Fig. 9 In the dialog box shown, select the current process to be performed and the preset process. This can further improve the user experience. You can also set the priority of the preset process in advance, and execute the preset process with the highest priority first.

[0121] In some embodiments, the process database can be a local database or a database set on a server. The preset processes can also be configured on different databases according to categories. Before searching for the preset process based on the preset intention information and the user intention information, the category of the user intention information can be determined, and then the preset process can be searched in the database of the corresponding category. For example, if the category of the user intention information is life, the preset process is searched in the database of the life category. If the category of the user intention information is entertainment, the preset process is searched in the database of the entertainment category. This can shorten the time to find the preset process and improve the efficiency of voice interaction.

[0122] In some embodiments, if the semantic recognition information does not include user slot information, the preset process is executed according to the spare slot information. For example, the user inputs a voice request sentence "query weather", and the semantic recognition information obtained after semantic analysis processing includes user intention information to query weather. However, the semantic recognition information does not include user slot information, specifically time slot information and location slot information. At this time, the spare time slot information "current time" and spare location slot information can be used.

[0123] In some embodiments, if the corresponding preset process is not found on the current smart device, a non-support message may be returned. The non-support message is used to prompt the user that the current user intent information cannot be supported by the current smart device, that is, the current device does not support the function corresponding to the current user intent information. The semantic recognition information can also be sent to other devices with the corresponding preset process. For example, Fig.10 In the framework diagram shown, if the current device is a smart speaker, the user's intention information is to play a video, and the smart speaker does not support the video playback function. The smart speaker further sends the semantic recognition information to other smart devices. For example, it sends it to a smart TV and controls the smart TV to execute a preset process, that is, to perform a video playback operation.

[0124] In some embodiments, a voice processing capability list may also be constructed, for example, all devices in the same local area network store a voice processing list for recording the voice processing capabilities of other devices. Fig.11 A flow chart of another voice interaction method is shown, the method comprising the following steps:

[0125] In step S201, a voice request sentence input by a user is obtained;

[0126] In step S202, semantic analysis is performed on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information, and user slot information;

[0127] In step S203, a preset process is searched in a process database of a business type corresponding to the business type information, wherein the preset intention information of the preset process corresponds to the user intention information, and in the process database, the preset process and the preset intention information have a mapping relationship;

[0128] In step S204, when the corresponding preset process is not found, a voice processing capability list is obtained, wherein the voice processing capability list records the process information of the candidate device, and the candidate device is a device in the same local area network as the current device;

[0129] In step S205, searching the voice processing capability list for the candidate device having the preset process corresponding to the user intention information;

[0130] In step S206, the semantic recognition information is sent to the candidate device, so that when the candidate device has the corresponding preset process, the preset process is executed according to the semantic recognition information.

[0131] The above embodiments can improve the success rate of voice interaction and enhance the user experience.

[0132] The present invention provides a smart device for voice interaction, which is used to perform Figure 2 The corresponding embodiments are as follows: Fig.12 As shown, the smart device provided by this application at least includes:

[0133] The user input unit 1001 is configured to obtain a voice request sentence input by a user;

[0134] The semantic analysis processing unit 1002 is configured to perform semantic analysis on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information at least includes user intention information;

[0135] A process search unit 1003 is configured to search for a corresponding preset process according to the user intention information, wherein the preset intention information of the preset process corresponds to the user intention information, and the preset process is set according to different scenarios;

[0136] The first process execution unit 1004 is configured to execute the preset process according to the semantic recognition information when the corresponding preset process is found.

[0137] In some embodiments, the smart device for voice interaction provided in the embodiments of the present application further includes:

[0138] The second process execution unit 1005 is configured to obtain a voice processing capability list when the corresponding preset process is not found, wherein the voice processing capability list records the process information of the candidate device, and the candidate device is a device in the same local area network as the current device;

[0139] The semantic recognition information is sent to the candidate device, so that the candidate device executes the preset process according to the semantic recognition information.

[0140] What has been described above includes examples of implementations of the present invention. For the purpose of describing the claimed subject matter, it is certainly not possible to describe every conceivable combination of components or methods, but it is to be appreciated that many additional combinations and permutations of the present innovation are possible. Accordingly, the claimed subject matter is intended to include all such changes, modifications, and variations that fall within the spirit and scope of the appended claims. In addition, the above description of the illustrated implementations of the present application, including the things described in the "Abstract", is not intended to be enumerated in detail or to limit the disclosed implementations to the precise forms disclosed. Although specific implementations and examples are described in this application for illustrative purposes, various modifications that are considered to be within the scope of such implementations and examples are possible, as will be appreciated by those skilled in the relevant art.

[0141] In addition, the word "example" or "exemplary" is used in this application to mean "serving as an example, instance, or illustration." Any aspect or design described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs. On the contrary, the use of the word "example" or "exemplary" is intended to present concepts in a concrete way.

Claims

1. A voice interaction method for a smart device, characterized in that: include: Obtain the voice request sentence input by the user; Performing semantic analysis on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information, and user slot information; Determine a process database of a business type corresponding to the business type information, wherein the process database stores preset intent information; Encoding the user intention information and the preset intention information to obtain an encoding of the user intention information and an encoding of the preset intention information; Searching the process database for preset intention information that has a matching relationship with the user intention information, wherein the matching relationship includes that the encoding of the user intention information is the same as the encoding of the preset intention information, or that the encoding of the user intention information includes the encoding of the preset intention information; Searching, according to the preset intention information, for a preset process having a mapping relationship with the preset intention information; When the corresponding preset process is found, the user slot information is filled into the preset process; Execute the preset process after filling the user slot information; When the preset process is not found, a voice processing capability list is obtained, wherein the voice processing capability list records the preset processes possessed by the candidate device, and the candidate device is a device in the same local area network as the smart device; The semantic recognition information is sent to the candidate device, so that when the candidate device has the corresponding preset process, the preset process is executed according to the semantic recognition information.

2. The voice interaction method according to claim 1, characterized in that: After semantic analysis of the voice request sentence, first semantic recognition information and second semantic recognition information are obtained; Searching the process database for a first preset process and a second preset process corresponding to the first semantic identification information and the second semantic identification information, respectively; When the first preset process and the second preset process can be executed simultaneously, the first preset process and the second preset process are executed simultaneously; When the first preset process and the second preset process cannot be executed simultaneously, the first preset process and the second preset process are executed separately.

3. The voice interaction method according to claim 2, characterized in that: When the first preset process and the second preset process cannot be executed simultaneously, the first preset process and the second preset process are executed separately according to the process execution priority.

4. The voice interaction method according to claim 1, characterized in that: The user slot information includes at least one of location slot information, time slot information and character slot information.

5. The voice interaction method according to claim 1, characterized in that: When the semantic identification information does not include user slot information, The preset process is executed according to the spare slot information.

6. The voice interaction method according to claim 1, characterized in that: The method further comprises: When the corresponding preset process is not found, a prompt message is displayed on the display of the smart device, and the prompt message is used to prompt that the user intention information cannot be supported by the smart device.

7. A voice interaction system for a smart device, characterized in that: include: The human-computer interaction interface is configured to receive a user voice request statement; A speech recognition engine is configured to perform semantic analysis on the speech request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information and user slot information; A process database is configured to store preset intention information, preset processes, and a mapping relationship between the preset intention information and the preset processes; A process configuration engine is configured to perform encoding on the user intention information and the preset intention information to obtain the encoding of the user intention information and the encoding of the preset intention information; search the process database for preset intention information that has a matching relationship with the user intention information, and search for the preset process that has a mapping relationship with the preset intention information according to the preset intention information; the matching relationship includes that the encoding of the user intention information is the same as the encoding of the preset intention information, or that the encoding of the user intention information includes the encoding of the preset intention information; The process execution engine is configured to fill the user slot information into the preset process when the corresponding preset process is found; execute the preset process after filling the user slot information; and obtain a voice processing capability list when the corresponding preset process is not found, wherein the voice processing capability list records the preset processes possessed by the alternative device, and the alternative device is a device in the same local area network as the smart device; send the semantic recognition information to the alternative device so that when the alternative device has the corresponding preset process, the preset process is executed according to the semantic recognition information.

8. An intelligent device for voice interaction, characterized in that: include: A user input unit is configured to obtain a voice request sentence input by a user; A semantic analysis processing unit, configured to perform semantic analysis on the voice request sentence to obtain semantic recognition information, wherein the semantic recognition information includes service type information, user intent information and user slot information; A process search unit is configured to determine a process database of a business type corresponding to the business type information, wherein the process database stores preset intention information; Encoding the user intention information and the preset intention information to obtain an encoding of the user intention information and an encoding of the preset intention information; Searching the preset intention information having a matching relationship with the user intention information in the process database, wherein the matching relationship includes that the encoding of the user intention information is the same as the encoding of the preset intention information, or the encoding of the user intention information includes the encoding of the preset intention information; searching the preset process having a mapping relationship with the preset intention information according to the preset intention information; The first process execution unit is configured to execute, when a corresponding preset process is found, filling the user slot information into the preset process; and executing the preset process after filling the user slot information; The second process execution unit is configured to execute, when the corresponding preset process is not found, obtaining a voice processing capability list, wherein the voice processing capability list records the preset processes possessed by the alternative device, and the alternative device is a device in the same local area network as the smart device; sending the semantic recognition information to the alternative device so that when the alternative device possesses the corresponding preset process, the preset process is executed according to the semantic recognition information.

Images