Semantic understanding method and device, equipment and medium
Through the two-layer dialogue manager architecture and thematic dialogue label system, the accuracy of intention recognition in the open domain dialogue system is solved, the user experience is improved, and the accurate semantic understanding in multiple domain themes and sub-theme scenarios are realized. Interaction.
Patent Information
- Application Number
- CN202410902892.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-11-08
AI Technical Summary
In an open-field dialogue system, how to accurately identify the intention of user statements, improve user experience and provide more accurate interaction results, especially in human-computer interaction scenarios of multiple domain topics and sub-topics, the existing technology has voice. Identify errors and semantic understanding problems.
Using a two-layer dialogue manager architecture approach, a framework that combines conversation skills independent of the topic and conversation skills dependent on the topic, defines the theme dialogue tag system, conducts semantic understanding of speech text, and generates dialogue based on the understanding results. Improve the accuracy of intention recognition.
It improves the accuracy of user statement intention recognition, provides more accurate interaction results, improves user experience, solves the problems of speech recognition errors and semantic understanding errors, and is suitable for human-computer interaction scenarios in multiple domain topics and sub-topics.
Smart Images

Figure CN118917324A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a semantic understanding method, device, equipment and medium. Background Art
[0002] In recent years, with the development of artificial intelligence technology, conversational robots have been increasingly used in daily life. At present, the research work on conversational robot technology mainly focuses on task-oriented conversations, such as restaurant reservations and airline ticket reservations. The success of task-based conversational robots depends on their ability to conduct meaningful and useful conversations in a limited domain. Although these limited-domain systems have the advantage of clear optimization criteria, they cannot be used in spontaneous conversation scenarios, which gives rise to the demand for open-domain conversational robots.
[0003] However, open-domain intelligent robots need to understand diverse contexts from any potential domain to decide how to provide feedback in a natural conversational manner and generate human-like responses. In order to better understand the user's expression and then give the user the correct information, natural language understanding plays a particularly important role. The natural semantic understanding function can be divided into three subtasks: topic identification, intent identification, and semantic slot filling. In a dialogue system, only when the user's subject domain is clearly defined can his specific needs be correctly analyzed. Otherwise, it will only lead to incorrect intent recognition results in the subsequent intent recognition task.
[0004] At present, the human-computer interaction system included in smart speakers, like the open-domain dialogue system, usually supports human-computer interaction in multiple domain topics and various sub-topics. For both parties, there is no clear goal, so it is impossible to directly use the task-oriented dialogue system to implement the functions preset by the smart speaker. In addition, in actual application scenarios, semantic understanding errors caused by speech recognition errors, user speech containing sensitive words, and user information requests that exceed the preset functional scope of smart speakers have gradually become a problem that needs to be optimized.
[0005] Therefore, how to accurately identify the intention of user sentences in an open-domain dialogue system is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention
[0006] The present invention provides a semantic understanding method, device, equipment and medium to improve the accuracy of identifying the user's sentence intention, thereby feeding back more accurate interaction results to the user and improving the user experience.
[0007] According to one aspect of the present invention, a semantic understanding method is provided, the method comprising:
[0008] Determine a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to the topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic;
[0009] Using the dialogue manager based on the double-layer dialogue manager architecture, semantic understanding is performed on the speech text to be processed to obtain a semantic understanding result corresponding to the speech text to be processed;
[0010] According to the semantic understanding result, a dialogue corresponding to the speech text to be processed is generated.
[0011] According to another aspect of the present invention, there is provided a semantic understanding device, the device comprising:
[0012] A dialogue manager construction module, used to determine a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to the topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic;
[0013] A semantic understanding module, configured to use the dialogue manager based on the two-layer dialogue manager architecture to perform semantic understanding on the speech text to be processed, and obtain a semantic understanding result corresponding to the speech text to be processed;
[0014] A dialogue generation module is used to generate a dialogue corresponding to the speech text to be processed according to the semantic understanding result.
[0015] According to another aspect of the present invention, a semantic understanding device is provided, the device comprising:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the semantic understanding method described in any embodiment of the present invention.
[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the semantic understanding method described in any embodiment of the present invention when executed.
[0020] The technical solution of the embodiment of the present invention is to determine a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to a topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic; the dialogue manager based on the two-layer dialogue manager architecture is used to perform semantic understanding on the speech text to be processed, and obtain the semantic understanding result corresponding to the speech text to be processed; according to the semantic understanding result, a dialogue corresponding to the speech text to be processed is generated. This technical solution is used to improve the accuracy of recognizing the user's sentence intention, provide users with more accurate feedback on interaction results, and improve user experience.
[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 A flowchart of a semantic understanding method provided in Embodiment 1 of the present invention;
[0024] Figure 2 A schematic diagram of the structure of a double-layer dialog manager framework provided in Embodiment 1 of the present invention;
[0025] Figure 3 A schematic diagram of a topic conversation tag definition system provided in Embodiment 1 of the present invention;
[0026] Figure 4 A flowchart of a semantic understanding method provided in Embodiment 2 of the present invention;
[0027] Figure 5 A flowchart of a semantic understanding method provided in Embodiment 3 of the present invention;
[0028] Figure 6A schematic diagram of the architecture of a dialogue manager provided in Embodiment 3 of the present invention;
[0029] Figure 7 A schematic diagram of the structure of a semantic understanding device provided in Embodiment 4 of the present invention;
[0030] Figure 8 It is a schematic diagram of the structure of an electronic device for implementing the semantic understanding method of an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0033] Embodiment 1
[0034] Figure 1 This is a flowchart of a semantic understanding method provided in the first embodiment of the present invention. This embodiment is applicable to the case of intelligent dialogue in an open domain scenario. The method can be executed by a semantic understanding device. The semantic understanding device can be implemented in the form of hardware and / or software. The semantic understanding device can be configured in a device with data processing. Figure 1 As shown, the method includes:
[0035] S110, determining a dialogue manager based on a two-layer dialogue manager architecture. The dialogue manager is determined according to the topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic.
[0036] The human-computer interaction system currently included in smart speakers, such as open-domain dialogue systems, usually supports human-computer interaction in multiple domain topics and various sub-topics. This does not have a clear goal for both parties in the dialogue, so it is impossible to directly use task-oriented dialogue systems to implement the functions preset by smart speakers.
[0037] In view of the above problems, the present invention proposes a two-layer dialogue manager framework based on hierarchical decomposition, which structures the functional themes of multiple different fields supported by the smart speaker into a dialogue management task on a set of topic dialogues in each different field. The two-layer dialogue manager framework follows a two-layer system architecture, which distinguishes the open-domain dialogue robot from the level of topic-dependent and topic-independent: that is, the topic-independent dialogue skills will handle intentions that are not related to the field, so that users can interact in the core topic and select relevant responses from the topic robot based on the context; while the topic-dependent dialogue skills independently manage the topic fragments of the robot in a specific field through a task-oriented dialogue architecture.
[0038] Figure 2 This is a schematic diagram of the structure of a two-layer dialogue manager framework provided in the first embodiment of the present invention. Figure 2 As shown in the figure, the first layer consists of the main functional modules of the smart speaker (i.e., conversation skills independent of the topic), which at least include the out-of-range module and the speech recognition error module to intervene in advance in scenarios that are not related to the functional services provided by the speaker. In addition, the first layer can also include functional modules in other fields, such as small talk, question-and-answer modules, etc. The second layer is a refinement of the topic conversations in the first layer related to specific fields (i.e., conversation skills dependent on the topic), which can be divided into sub-topics of different dimensions.
[0039] The initial semantic understanding module of the smart speaker is obtained by automatic recognition based on preset rules and keywords. It has many errors, which often lead to the user's semantics not being correctly understood, providing users with unexpected feedback dialogues. In order to solve the above-mentioned problems, this solution reconstructs the included subject functions, and re-labels the user's sentences according to the subject labels defined by the reconstruction to obtain predetermined subject dialogue labels, making it possible to apply machine learning algorithms for automatic recognition. In the process of reconstructing the subject labels, many functional themes are merged because their boundaries are difficult to distinguish or belong to a higher level of semantics. For example, poetry, children's resources and audiobooks, jokes, etc. can be merged into audiobooks, and multiple question-and-answer related topics can be used as secondary tags and merged into a unified question-and-answer topic. In addition, functions can be deleted because they are used too little, such as dream interpretation, movies, etc.
[0040] The topic conversation tag definition system derived from the two-layer conversation manager framework proposed in the present invention can flexibly apply various machine learning classification algorithms and rule-based pattern matching methods to automatically identify tags of various topics / sub-topics, and the smart speaker can navigate the interaction between the user and the smart speaker based on the identified tags.
[0041] Figure 3 Schematic diagram of a topic conversation tag definition system provided by Embodiment 1 of the present invention. Figure 3 As shown in the figure, the first layer of the topic dialogue tag definition system supports 18 topics, including: ASR (Automatic Speech Recognition, speech recognition error), small talk, OOS (Out Of Scope), Q&A, navigation, travel, audiobooks, life services, sensitive words, appointments, calculations, dates, music, radio stations, home control, speaker control, weather and speaker personality. Among them, the first-level topics such as life services, audiobooks, navigation, Q&A and tourist attractions contain more detailed second-level tags.
[0042] In this solution, the two-layer dialogue manager architecture can be seamlessly embedded in the dialogue manager in the dialogue system to form a two-layer dialogue manager architecture for processing user information.
[0043] S120: Using the dialogue manager based on the two-layer dialogue manager architecture, perform semantic understanding on the speech text to be processed, and obtain a semantic understanding result corresponding to the speech text to be processed.
[0044] Specifically, the semantic understanding result may be the semantic understanding content of the speech text to be processed by the dialog manager and the corresponding confidence level.
[0045] In this scheme, optionally, before using the semantic understanding module based on the two-layer dialogue manager framework to perform semantic understanding on the speech text to be processed and obtain a semantic understanding result corresponding to the speech text to be processed, the method also includes: collecting speech and performing speech recognition on the collected speech to obtain the speech text to be processed corresponding to the speech.
[0046] Among them, speech recognition can adopt existing related technologies, which will not be described in detail here. For example, the pattern matching method can be used for speech recognition. In the training stage, the user speaks each word in the vocabulary one by one, and its feature vector is stored as a template in the template library. In the recognition stage, the feature vector of the input speech is compared with each template in the template library in turn for similarity, and the one with the highest similarity is output as the recognition result to obtain the speech text to be processed.
[0047] S130: Generate a dialogue corresponding to the speech text to be processed according to the semantic understanding result.
[0048] Specifically, whether the corresponding semantic understanding result is credible can be judged according to the confidence of each semantic understanding result. If the confidence corresponding to the semantic understanding result is greater than the set threshold, the semantic understanding result is determined to be credible; otherwise, the semantic understanding result is determined to be unreliable. The threshold can be determined according to actual application conditions and / or a large number of experiments and experiences.
[0049] In this solution, a dialogue text is generated based on the credible semantic understanding results, and then the dialogue text is fed back to the user through voice broadcast.
[0050] The embodiment of the present invention provides a semantic understanding method, which determines a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to a topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic; the dialogue manager based on the two-layer dialogue manager architecture is used to perform semantic understanding on the speech text to be processed, and a semantic understanding result corresponding to the speech text to be processed is obtained; according to the semantic understanding result, a dialogue corresponding to the speech text to be processed is generated. This technical solution improves the accuracy of recognizing the user's sentence intention, provides users with more accurate feedback on interaction results, and improves user experience.
[0051] Embodiment 2
[0052] Figure 4 This is a flowchart of a semantic understanding method provided in Embodiment 2 of the present invention. This embodiment is optimized based on the above embodiment. Figure 4 As shown, the method includes:
[0053] S210: Based on the topic dialogue tag definition system, a speech text sample set with topic dialogue tags is obtained. The topic dialogue tag definition system is determined according to a two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic.
[0054] Specifically, an initial speech text sample set can be obtained from historical conversations; the initial speech text sample set is preprocessed to delete repeated, garbled, unlabeled or other erroneous speech text samples to obtain an intermediate speech text sample set; the subject function of the intermediate speech text sample set is reconstructed, and the user's sentences are re-labeled according to the subject labels defined by the reconstruction to obtain a speech text sample set with subject conversation labels.
[0055] S220: Input the speech-text sample set into an initial model, train the initial model, and determine a dialog manager based on a two-layer dialog manager architecture.
[0056] In order to verify the effectiveness of the two-layer semantic understanding framework proposed in this scheme, it is necessary to use machine learning algorithms and / or rule-based pattern matching methods to automatically identify the topic tags defined by the topic dialogue tag definition system to determine the dialogue manager based on the two-layer dialogue manager architecture.
[0057] Among them, appointments, calculations, dates, music, radio stations, speaker personality, home control, speaker control and weather are suitable for identification using rule-based pattern matching methods alone.
[0058] Optionally, the speech-text sample set is input into an initial model, the initial model is trained, and a dialogue manager based on a two-layer dialogue manager architecture is determined, including: for each training round, the speech-text sample set is input into an initial model, the initial model is trained, and a prediction performance index of the initial model for the topic dialogue label is obtained; if the prediction performance index meets a preset threshold, a dialogue manager based on a two-layer dialogue manager architecture is obtained.
[0059] In this scheme, if the confusion matrix is used to represent the speech text sample set, TP represents the number of positive samples correctly predicted as positive samples, TN represents the number of negative samples correctly predicted as negative samples, FP represents the number of negative samples incorrectly predicted as positive samples, and FN represents the number of positive samples incorrectly predicted as negative samples. Then the accuracy, precision, recall and harmonic mean can be defined as prediction performance indicators.
[0060] The accuracy can be determined by the following formula:
[0061]
[0062] In the formula, Accuracy is the correct rate, which is used to indicate the proportion of correctly predicted samples among all predicted samples.
[0063] Among them, the precision rate can be determined by the following formula:
[0064]
[0065] In the formula, Precision is the precision rate, which is used to indicate the proportion of correctly predicted samples among the samples identified as positive samples.
[0066] The recall rate can be determined by the following formula:
[0067]
[0068] In the formula, Recall is the recall rate, which is used to indicate the proportion of samples that can be correctly predicted among all predicted samples.
[0069] The harmonic mean can be determined by the following formula:
[0070]
[0071] In the formula, F1 is the harmonic mean, which is used to represent the harmonic mean of precision and recall.
[0072] In each training round, the above prediction performance index can be compared with the preset threshold. If the prediction performance index meets the preset threshold, the training is stopped. Of course, a certain number of training rounds can also be set to determine the training termination condition of the model.
[0073] Furthermore, based on the above results of the first-layer topic identification, the Bert model can be used to identify the various sub-topics contained in the first-layer topic. It should be noted that as long as each type of topic label contains a sufficient amount of annotated data, the combination of rules and machine learning can effectively identify each topic and sub-topic label, thereby more accurately understanding the true intention expressed by the user's sentence.
[0074] For example, Figure 3 Taking the 18 predefined topics of the smart speaker shown in the figure as an example, 9 topics suitable for identification using machine learning methods are selected, and their topic data distribution is shown in Table 1.
[0075] Table 1
[0076] Training set Validation set Test Set ASR 13389 1913 3826 Small talk 982 140 281 OOS 975 139 278 Questions and Answers 21172 3024 6049 navigation 2461 352 703 travel 40869 5839 11678 Audiobooks 2874 411 821 Lifestyle Services 1614 231 461 Sensitive words 1760 251 503
[0077] The other 9 categories of topics (Appointment, Arithmetic, Date, Music, Radio, Persona, SmartHome, ControlCom, Weather) are suitable for identification using rule-based pattern matching alone, so they are not considered for the time being. Furthermore, the ASR training data has been expanded from 1539 to 13389, a significant increase, while the data of other categories of topics have not changed. Based on the data in Table 3, the Bert model is applied to train and predict the 9 categories of topics, and the results are shown in Table 2.
[0078] Table 2
[0079]
[0080]
[0081] From the results in Table 2, we can see that simply increasing the amount of data covered by the ASR class, while keeping the data of other classes basically unchanged, the ASR evaluation index in the obtained prediction results has been significantly improved, while the chat and OOS topics have not been improved even with the best pre-trained model Bert because the amount of data covered is insufficient. This also shows from another perspective that based on the topic label system defined in the two-layer semantic understanding framework proposed by the present invention, if it can be ensured that each category contains enough data, the application of machine learning models can well distinguish various topics, so as to correctly judge the true intention of the user to express.
[0082] When all 18 categories of subject data predefined in this embodiment are used and divided into training set and test set in a ratio of 8:2, their distribution is shown in Table 3.
[0083] Table 3
[0084]
[0085]
[0086] Using all the data shown in Table 3, and using rules and machine learning methods, the two-layer semantic understanding framework model defined in the present invention is trained, and its performance is predicted. The recognition results can be seen in Table 4. Table 4 shows the prediction results obtained using two machine learning representative models: the Logistic regression model based on traditional machine learning and the Bert model based on deep neural network.
[0087] Table 4
[0088]
[0089] From the results, we can see that for the Logistic regression model, except for the poor values of the OOS and speaker personality themes, the recognition effects of other themes are within an acceptable range; and the Bert model performs much better than the Logistic regression model on all 18 themes, especially for the three themes that are more difficult to distinguish: ASR, small talk and OOS, its performance has been improved to a certain extent. The reason for the poor performance on the OOS theme is that the amount of data it covers is not enough for the model to effectively learn and recognize the various intentions expressed by the internal definition of OOS.
[0090] According to the first-level topic identification results, the Bert model is further used to identify the various sub-topics contained in the first-level topics. The test results are shown in Table 5.
[0091] Table 5
[0092] Precision Recall Harmonic mean Navigation-OOS 0.50 0.47 0.48 Navigation-Check current location 0.78 0.90 0.84 Navigation-Check time 0.93 1.0 0.96 Navigation-Check distance 0.92 0.79 0.85 Navigation-Check traffic conditions 0.94 0.99 0.96 Navigation-Destination 0.92 0.97 0.94 Tourist Attractions-OOS 0.57 0.70 0.63 Tourist attractions-areas 0.88 0.92 0.90 Tourist attractions-surrounding areas 0.57 0.89 0.69 Tourist attractions-attractions 1.0 1.0 1.0 Audiobooks - OOS 0.69 0.69 0.69 Audiobooks-Children's Songs 0.85 0.92 0.98 Audiobooks - Drama 0.86 0.96 0.90 Audiobooks-Stories 0.94 0.98 0.96 Audiobooks - Literature 0.93 0.86 0.89 Audiobooks-Crosstalk 0.89 1.0 0.94 Audiobooks - Jokes 0.25 0.33 0.29 Life Service-OOS 0.71 0.54 0.61 Lifestyle Services-LBS 0.84 0.92 0.88 Lifestyle Services-Designated Locations 0.81 0.76 0.79
[0093] From Table 5, we know that except for a few secondary topics (audiobooks-jokes, navigation-OOS, audiobooks-OOS, life services-OOS, tourist attractions-OOS), the recognition results of most secondary topics are within a good range due to the lack of sufficient annotated data. Combining the prediction results shown in Tables 4 and 5, under the two-layer semantic understanding framework proposed by the present invention, according to the 18 categories of topic tags, as long as each category of topic tags contains a sufficient amount of annotated data, the combination of rules and machine learning can effectively identify each topic and sub-topic tag, so as to more accurately understand the true intention expressed by the user's sentence.
[0094] S230: Using the dialogue manager based on the two-layer dialogue manager architecture, perform semantic understanding on the speech text to be processed, and obtain a semantic understanding result corresponding to the speech text to be processed.
[0095] S240: Generate a dialogue corresponding to the speech text to be processed according to the semantic understanding result.
[0096] The embodiment of the present invention provides a semantic understanding method, which obtains a speech text sample set with a topic dialogue tag based on a topic dialogue tag definition system; wherein the topic dialogue tag definition system is determined according to a two-layer dialogue manager framework and a predetermined topic dialogue tag; the first layer in the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic; the speech text sample set is input into an initial model, the initial model is trained, and a dialogue manager based on a two-layer dialogue manager architecture is determined; a dialogue manager based on a two-layer dialogue manager architecture is used to perform semantic understanding on the speech text to be processed, and a semantic understanding result corresponding to the speech text to be processed is obtained; and a dialogue corresponding to the speech text to be processed is generated according to the semantic understanding result. This technical solution further improves the accuracy of user sentence intention recognition by performing model training on the speech text sample set with a topic dialogue tag obtained based on the topic dialogue tag definition system.
[0097] Embodiment 3
[0098] Figure 5 This is a flowchart of a semantic understanding method provided in Embodiment 2 of the present invention. This embodiment is optimized based on the above embodiment. Figure 5 As shown, the method includes:
[0099] S310, determining a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to the topic dialogue tag definition system. The topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic.
[0100] S320: Determine a target function to be performed by a dialog manager based on a two-layer dialog manager architecture according to the speech text to be processed.
[0101] Among them, the target function can be a conversation skill independent of the conversation topic, which is used to intervene in advance in scenarios that are unrelated to the functional services provided by the smart speaker.
[0102] In this solution, the dialogue manager based on the two-layer dialogue manager architecture can be composed of a master dialogue manager and a slave dialogue manager. The master dialogue manager can be a top-level dialogue manager that determines which functional module to execute; the slave dialogue manager can be a dialogue manager based on the topic layer that further determines which topic dialogue to execute in the structured topic dialogue to obtain the semantic understanding result.
[0103] S330: According to the target function, semantic understanding is performed on the speech text to be processed to obtain a semantic understanding result corresponding to the speech text to be processed.
[0104] After determining the target function, the speech text to be processed may be further recognized to obtain a semantic understanding result corresponding to the speech text to be processed.
[0105] Optionally, the functions include at least one of an out-of-range function, a speech recognition error function, a question-and-answer function, a chat function, a speaker personality function, and a structured topic dialogue function; accordingly, according to the target function, semantic understanding is performed on the speech text to be processed to obtain a semantic understanding result corresponding to the speech text to be processed, including: if the target function is a structured topic dialogue function, determining the target topic of the speech text to be processed.
[0106] Figure 6 This is a schematic diagram of the architecture of a dialog manager provided in Embodiment 3 of the present invention. Figure 6As shown, the speech recognition module of component 1 recognizes the collected speech and outputs the speech text to be processed. The various components of the semantic understanding module of component 2 process the speech text to be processed. After preliminary analysis, it will enter component 3 and be guided by component 6 (main dialogue manager): component 3 determines whether the speech text to be processed is a new dialogue. If it is not a new dialogue, it will continue the previous topic and enter the corresponding structured topic dialogue; if there is no structured topic dialogue that can be run at present, the system will activate other components; if it is a new dialogue, it will enter component 4. Component 4 determines whether the smart speaker understands correctly. Its decision is based on the confidence of the label output by the speech recognition. If the confidence value of the smart speaker output of the speech text to be processed is too low, which is equivalent to a speech recognition error, the user will be asked to repeat the previous words; if it is understood correctly, it will enter component 5. Component 5 determines whether it has passed the sensitive word review. If the detected topic has not passed the review, the user will be asked to pay attention to the wording and select a different topic; if the detected topic passes the review, it will enter component 6. Component 6 main dialogue manager determines which response generation module (such as out of range function, speech recognition error function, question and answer function, small talk function, structured topic dialogue function) generates the dialogue. If the speech text to be processed falls into the structured topic dialogue category, component 7 topic-based dialogue manager will select the corresponding dialogue topic, and each dialogue topic has its own topic-based dialogue manager. Component 7 selects the current state of the dialogue, and the smart speaker selects the corresponding response.
[0107] Among them, since although speech recognition may generate a label with a low confidence value, the structured topic dialogue usually produces a feedback, component 3 can adopt a strategy of directly entering the structured topic dialogue without going through the dialogue manager.
[0108] Among them, the decision of component 5 is determined by the sensitive word filter algorithm.
[0109] Among them, the topic-layer-based dialogue manager can be rule-based or adopt a machine learning method.
[0110] Among them, the structured topic dialogue layer covers the main interactions between users and robots. It is designed to cover the most frequently used topics and provide deeper human-computer interaction to attract users to stay in the conversation.
[0111] In addition, regarding the statistics of OOS topics, if there are new functional modules or topic conversations beyond the pre-defined function modules of the smart speaker, they can be flexibly added to the two-layer semantic understanding framework and the dialogue manager framework designed by the present invention. The advantage of this structure-based topic dialogue design is that it can be maintained and responded to according to the context of the dialogue.
[0112] Among them, the speaker personality function module in the target function mainly deals with issues related to the robot's personality, such as the color and age of the robot's personality. The aforementioned user messages do not need to be processed in a structured topic dialogue. Instead, a manually edited answer list or a rule-based robot written in the form of AIML (Artificial Intelligence Markup Language) language rules can be used to respond.
[0113] The question-answering module in the target function is used to answer factual questions raised by users, and the actual process of generating answers can be solved by an external independent question-answering system. The question-answering system can be a question-answering system based on question-answer pairs, a question-answering system based on knowledge graphs, or a simple query based on a triple knowledge base.
[0114] Among them, the chat function module in the target function can generate responses by a generative neural network based on Sequence-to-Sequence. The chat function module is mainly used in scenarios where the conversation does not involve any specific topic or requires relevant knowledge of a series of entities. When the chat topic is recognized or no other function module generates a response, the chat function module will be triggered.
[0115] S340: Generate a dialogue corresponding to the speech text to be processed according to the semantic understanding result.
[0116] The embodiment of the present invention provides a semantic understanding method, which determines a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to a topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer in the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic; according to the speech text to be processed, the target function to be performed by the dialogue manager based on the two-layer dialogue manager architecture is determined; according to the target function, the speech text to be processed is semantically understood to obtain a semantic understanding result corresponding to the speech text to be processed; according to the semantic understanding result, a dialogue corresponding to the speech text to be processed is generated. This technical solution constructs a dialogue manager based on hierarchical decomposition, improves the accuracy of user sentence intention recognition in the human-computer interaction system, provides users with more accurate interaction results, and improves user experience.
[0117] Embodiment 4
[0118] Figure 7 This is a schematic diagram of the structure of a semantic understanding device provided by Embodiment 4 of the present invention. Figure 7 As shown, the device comprises:
[0119] A dialogue manager construction module 410 is used to determine a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to the topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic;
[0120] A semantic understanding module 420, configured to use the dialogue manager based on the two-layer dialogue manager architecture to perform semantic understanding on the speech text to be processed, and obtain a semantic understanding result corresponding to the speech text to be processed;
[0121] The dialogue generation module 430 is used to generate a dialogue corresponding to the speech text to be processed according to the semantic understanding result.
[0122] The embodiment of the present invention provides a semantic understanding device, which determines a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to a topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic; the dialogue manager based on the two-layer dialogue manager architecture is used to perform semantic understanding on the speech text to be processed, and a semantic understanding result corresponding to the speech text to be processed is obtained; according to the semantic understanding result, a dialogue corresponding to the speech text to be processed is generated. This technical solution improves the accuracy of recognizing the user's sentence intention, provides users with more accurate feedback on interaction results, and improves user experience.
[0123] Optionally, the dialogue manager construction module 410 includes:
[0124] A speech text sample set acquisition unit, used to acquire a speech text sample set with a topic dialogue tag based on the topic dialogue tag definition system;
[0125] The dialog manager construction unit is used to input the speech text sample set into an initial model, train the initial model, and determine a dialog manager based on a two-layer dialog manager architecture.
[0126] Furthermore, the dialog manager constructs a unit, including:
[0127] A model performance determination subunit, for inputting the speech text sample set into an initial model for each training round, training the initial model, and obtaining a prediction performance index of the initial model for the topic dialogue label;
[0128] The dialog manager constructs a subunit, which is used to obtain a dialog manager based on a two-layer dialog manager architecture if the prediction performance indicator meets a preset threshold.
[0129] Furthermore, the semantic understanding module 420 includes:
[0130] A target function determination unit, used to determine a target function to be performed by a dialog manager based on a two-layer dialog manager architecture according to the speech text to be processed;
[0131] The semantic understanding unit is used to perform semantic understanding on the speech text to be processed according to the target function, and obtain a semantic understanding result corresponding to the speech text to be processed.
[0132] Further, the function includes at least one of an out-of-range function, a voice recognition error function, a question-and-answer function, a small talk function, a speaker personality function, and a structured topic conversation function;
[0133] Accordingly, the semantic understanding unit includes:
[0134] The target topic determination subunit is used to determine the target topic of the speech text to be processed if the target function is a structured topic dialogue function.
[0135] Furthermore, the device also includes:
[0136] The speech recognition module is used to collect speech and perform speech recognition on the collected speech to obtain the speech text to be processed corresponding to the speech before using the semantic understanding module based on the two-layer dialogue manager framework to perform semantic understanding on the speech text to be processed and obtain the speech text to be processed corresponding to the speech.
[0137] The semantic understanding device provided in the embodiment of the present invention can execute the semantic understanding method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0138] Embodiment 5
[0139] Figure 8A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0140] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0141] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0142] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the semantic understanding method.
[0143] In some embodiments, the semantic understanding method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the semantic understanding method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the semantic understanding method in any other appropriate manner (e.g., by means of firmware).
[0144] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0145] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0146] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0147] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0148] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0149] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0150] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0151] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A semantic understanding method, characterized in that: The method comprises: Determine a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to a topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of topic-independent dialogue skills, and the second layer is composed of topic-dependent dialogue skills; Using the dialogue manager based on the double-layer dialogue manager architecture, semantic understanding is performed on the speech text to be processed to obtain a semantic understanding result corresponding to the speech text to be processed; According to the semantic understanding result, a dialogue corresponding to the speech text to be processed is generated.
2. The method according to claim 1, characterized in that Determine a dialog manager based on a two-layer dialog manager architecture, including: Based on the topic dialogue tag definition system, obtaining a speech text sample set with topic dialogue tags; The speech-text sample set is input into an initial model, the initial model is trained, and a dialog manager based on a two-layer dialog manager architecture is determined.
3. The method according to claim 2, characterized in that Inputting the speech text sample set into an initial model, training the initial model, and determining a dialog manager based on a two-layer dialog manager architecture, including: For each training round, the speech text sample set is input into the initial model, the initial model is trained, and a prediction performance index of the initial model for the topic dialogue label is obtained; If the prediction performance indicator meets the preset threshold, a dialog manager based on a two-layer dialog manager architecture is obtained.
4. The method according to claim 1, characterized in that: The dialog manager based on the two-layer dialog manager architecture is used to perform semantic understanding on the speech text to be processed, and obtain a semantic understanding result corresponding to the speech text to be processed, including: Determine, according to the speech text to be processed, a target function to be performed by a dialog manager based on a two-layer dialog manager architecture; According to the target function, semantic understanding is performed on the speech text to be processed to obtain a semantic understanding result corresponding to the speech text to be processed.
5. The method according to claim 4, characterized in that The functions include at least one of an out-of-range function, a voice recognition error function, a question-and-answer function, a small talk function, a speaker personality function, and a structured topic conversation function; Accordingly, according to the target function, semantic understanding is performed on the speech text to be processed to obtain a semantic understanding result corresponding to the speech text to be processed, including: If the target function is a structured topic dialogue function, the target topic of the speech text to be processed is determined.
6. The method according to claim 1, characterized in that Before using the semantic understanding module based on the two-layer dialogue manager framework to perform semantic understanding on the speech text to be processed and obtain a semantic understanding result corresponding to the speech text to be processed, the method further includes: Speech is collected and speech recognition is performed on the collected speech to obtain a speech text to be processed corresponding to the speech.
7. A semantic understanding device, characterized in that: include: A dialogue manager construction module, used to determine a dialogue manager based on a two-layer dialogue manager architecture; wherein the dialogue manager is determined according to the topic dialogue tag definition system; the topic dialogue tag definition system is determined according to the two-layer dialogue manager framework and predetermined topic dialogue tags; the first layer of the two-layer dialogue manager framework is composed of dialogue skills independent of the topic, and the second layer is composed of dialogue skills dependent on the topic; A semantic understanding module, configured to use the dialogue manager based on the two-layer dialogue manager architecture to perform semantic understanding on the speech text to be processed, and obtain a semantic understanding result corresponding to the speech text to be processed; A dialogue generation module is used to generate a dialogue corresponding to the speech text to be processed according to the semantic understanding result.
8. The device according to claim 7, characterized in that Semantic understanding module, including: A function determination unit, configured to determine a target function to be performed by a dialog manager based on a two-layer dialog manager architecture according to the speech text to be processed; The semantic understanding unit is used to perform semantic understanding on the speech text to be processed according to the target function, and obtain a semantic understanding result corresponding to the speech text to be processed.
9. A semantic understanding device, characterized in that: The device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the semantic understanding method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the semantic understanding method described in any one of claims 1 to 7 when executed.
