A voice interaction method, system, device and medium based on artificial intelligence

Abstract

The present application provides a voice interaction method, system, device and medium based on artificial intelligence, which obtains interactive voices in different dialogue turns; extracts the context dependency relationships between each dialogue turn from all the interactive voices; determines the semantic loss amount of the interactive voice in each dialogue turn, and determines the dialogue coherence degree in each dialogue turn through all the semantic loss amounts and the context dependency relationships between each dialogue turn; classifies the intention of the interactive voice in the current dialogue scenario to obtain different semantic intentions of the target user in the current dialogue scenario, and further determines the semantic association relationship between each semantic intention; determines the semantic integrity in the current dialogue scenario through the semantic association relationship and the dialogue coherence degree of the interactive voice in each dialogue turn, and further generates a feedback control instruction for the voice interaction device according to the semantic integrity. By adopting the solution of the present application, it is possible to maintain the semantic consistency of the context dialogue in a complex dialogue scenario.

Description

Technical Field

[0001] This application relates to the field of voice interaction technology. More specifically, this application relates to a voice interaction method, system, device, and medium based on artificial intelligence. Background Art

[0002] Voice interaction is a way to realize communication between humans and devices using artificial intelligence technology. Users send instructions to the system through voice input. After processing, the system makes corresponding feedback. Artificial intelligence plays a core role in voice interaction, enabling the system to understand the voice content, recognize the user's intention, and make intelligent responses according to the context. The widely used artificial intelligence technology has improved the accuracy of speech recognition, making the interaction more natural and fluent, and supporting multi-round conversations, emotion recognition, and personalized recommendations. Voice interaction has been widely used in intelligent assistants, in-vehicle systems, smart homes, and other fields, continuously optimizing the user experience and improving the convenience and intelligence level of interaction.

[0003] In the prior art, voice interaction based on artificial intelligence is mainly realized through speech recognition, natural language processing, and generation models. Speech recognition converts voice signals into text. Then, natural language processing analyzes the text semantics to understand the user's intention. Subsequently, the generation model generates an appropriate response according to the intention. Finally, the response is converted into voice output through speech synthesis technology. However, in multi-round conversations, especially in complex voice interaction scenarios, the user's intention and conversation context are dynamically changing. During the voice interaction process, multiple topics, complex background information, and even emotional fluctuations may be involved, making the system unable to continuously understand the user's intention in multi-round conversations, resulting in problems such as the system being unable to maintain semantic consistency in multi-round conversations, thus causing logical confusion and other issues. Therefore, how to achieve semantic consistency in context conversations in complex dialogue scenarios has become a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a voice interaction method, system, device, and medium based on artificial intelligence, which can achieve semantic consistency in context conversations in complex dialogue scenarios.

[0005] In a first aspect, this application provides a voice interaction method based on artificial intelligence, including the following steps:

[0006] Obtain the interaction voice of the target user in different dialogue rounds through a voice interaction device;

[0007] Extract the context dependency relationships between each dialogue round from all the interaction voices based on the attention mechanism;

[0008] Determine the semantic loss amount of the interactive speech in each conversation turn according to the semantic features of the interactive speech in each conversation turn, and determine the conversation coherence degree of the target user in each conversation turn through all the semantic loss amounts and the context dependence relationship between the respective conversation turns;

[0009] Classify the intent of the interactive speech of the target user in the current conversation scenario to obtain different semantic intents of the target user in the current conversation scenario, and further determine the semantic association relationship between the respective semantic intents;

[0010] Determine the semantic integrity of the target user in the current conversation scenario through the semantic association relationship and the conversation coherence degree of the interactive speech in each conversation turn, and further generate a feedback control instruction for the voice interaction device according to the semantic integrity;

[0011] In some embodiments, extracting the context dependence relationship between the respective conversation turns from all the interactive speeches based on the attention mechanism specifically includes:

[0012] Convert all the interactive speeches into a plurality of speech vectors;

[0013] Construct an attention vector space for each speech vector based on a preset attention mechanism;

[0014] Determine the semantic dependence degree of each conversation turn according to the attention vector space of each speech vector;

[0015] Determine the context dependence relationship between the respective conversation turns through the semantic dependence degrees of all the conversation turns.

[0016] In some embodiments, determining the semantic loss amount of the interactive speech in each conversation turn according to the semantic features of the interactive speech in each conversation turn specifically includes:

[0017] Determine the semantic features of the interactive speech in each conversation turn;

[0018] Determine the semantic similarity between the interactive speeches in the respective conversation turns according to all the semantic features;

[0019] Determine the semantic loss amount of the interactive speech in each conversation turn through the semantic similarity between the interactive speeches in the respective conversation turns.

[0020] In some embodiments, determining the conversation coherence degree of the target user in each conversation turn through all the semantic loss amounts and the context dependence relationship between the respective conversation turns specifically includes:

[0021] Determine the conversation state amount in each conversation turn based on a preset sliding window and all the semantic loss amounts;

[0022] Determine the dialogue coherence of the target user in each dialogue turn based on the dialogue state quantity in each dialogue turn and the context dependency relationship between the various dialogue turns.

[0023] In some embodiments, the interactive speech of the target user in the current dialogue scenario is classified by intention to obtain the specific different semantic intentions of the target user in the current dialogue scenario, including:

[0024] Extract multiple dialogue keywords from the interactive speech of the target user in the current dialogue scenario;

[0025] Determine the semantic confidence of each dialogue keyword based on a pre-trained large language model;

[0026] Classify all the dialogue keywords according to the semantic confidence of all the dialogue keywords to obtain different semantic intentions of the target user in the current dialogue scenario.

[0027] In some embodiments, determining the semantic association relationship between the various semantic intentions specifically includes:

[0028] Obtain all the dialogue keywords;

[0029] Perform topic analysis on all the dialogue keywords based on a pre-trained topic analysis model to obtain the topic features of the interactive speech of the target user in the current dialogue scenario;

[0030] Determine the semantic association relationship between the various semantic intentions according to the topic features of the interactive speech of the target user in the current dialogue scenario and the different semantic intentions of the target user in the current dialogue scenario.

[0031] In some embodiments, determining the semantic integrity of the target user in the current dialogue scenario through the semantic association relationship and the dialogue coherence of the interactive speech in each dialogue turn specifically includes:

[0032] Perform linear fitting on the dialogue coherence of the interactive speech in all dialogue turns to obtain the fitting curve of the dialogue coherence;

[0033] Obtain the semantic confidence of each dialogue keyword;

[0034] Determine the semantic association graph of the target user in the current dialogue scenario according to all the semantic confidences and the semantic association relationship;

[0035] Determine the semantic integrity of the target user in the current dialogue scenario through the semantic association graph and the fitting curve of the dialogue coherence.

[0036] In a second aspect, the present application provides a voice interaction system based on artificial intelligence, including:

[0037] An acquisition module, configured to acquire the interactive speech of a target user in different dialogue turns through a voice interaction device;

[0038] A processing module, configured to extract the context dependency relationships between different dialogue turns from all the interactive speech based on an attention mechanism;

[0039] The processing module is further configured to determine the semantic loss amount of the interactive speech in each dialogue turn according to the semantic features of the interactive speech in each dialogue turn, and determine the dialogue coherence degree of the target user in each dialogue turn through all the semantic loss amounts and the context dependency relationships between the different dialogue turns;

[0040] The processing module is further configured to perform intent classification on the interactive speech of the target user in the current dialogue scenario, obtain different semantic intents of the target user in the current dialogue scenario, and further determine the semantic association relationships between the various semantic intents;

[0041] An execution module, configured to determine the semantic integrity degree of the target user in the current dialogue scenario through the semantic association relationships and the dialogue coherence degree of the interactive speech in each dialogue turn, and further generate a feedback control instruction for the voice interaction device according to the semantic integrity degree.

[0042] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to acquire the code and execute the above-mentioned voice interaction method based on artificial intelligence.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned voice interaction method based on artificial intelligence is implemented.

[0044] The technical solutions provided by the disclosed embodiments of the present application have the following beneficial effects:

[0045] In the speech interaction method, system, device and medium based on artificial intelligence provided by this application, the interaction speech of the target user in different dialogue turns is obtained through a speech interaction device; the context dependence between each dialogue turn is extracted from all the interaction speech based on an attention mechanism; the semantic loss amount of the interaction speech in each dialogue turn is determined according to the semantic features of the interaction speech in each dialogue turn, and the dialogue coherence degree of the target user in each dialogue turn is determined through all the semantic loss amounts and the context dependence between each dialogue turn; the interaction speech of the target user in the current dialogue scenario is classified by intention, and different semantic intentions of the target user in the current dialogue scenario are obtained, and then the semantic association relationship between each semantic intention is determined; the semantic integrity of the target user in the current dialogue scenario is determined through the semantic association relationship and the dialogue coherence degree of the interaction speech in each dialogue turn, and then a feedback control instruction for the speech interaction device is generated according to the semantic integrity.

[0046] It can be seen that in this application, the semantic integrity of the target user in the current dialogue scenario can be determined through the semantic association relationship and the dialogue coherence degree of the interaction speech in each dialogue turn; among them, first, the context dependence between each dialogue turn is extracted by using an attention mechanism, and through the context dependence, the system can dynamically capture and associate the semantic context of the user in different turns, and dynamically adjust the semantic weight, so as to enhance the dialogue coherence; second, the semantic loss amount of the interaction speech in each dialogue turn is determined, and the semantic loss amount can effectively improve the system's perception ability of the change of speech features, and then reduce the semantic understanding error caused by speech differences. Furthermore, the dialogue coherence degree of the target user in each dialogue turn is determined. Through the dialogue coherence degree, the stability of the user's speech features and semantic information in multi-turn interaction can be effectively measured, ensuring that the system can accurately understand the context and maintain a logically coherent dialogue process; then, the interaction speech of the target user in the current dialogue scenario is classified by intention, and different semantic intentions of the target user in the current dialogue scenario are obtained. Through intention classification, the system's parsing ability of the user's dialogue content can be effectively improved, ensuring semantic consistency in multi-turn interaction. Furthermore, the semantic association relationship between each semantic intention is determined, where the semantic association relationship can reflect the logical relationship at the semantic level, thus maintaining the overall coherence of the dialogue; further, the semantic integrity of the target user in the current dialogue scenario is determined. Through the semantic integrity, the system can be helped to evaluate the integrity of the dialogue information, ensuring the consistency of the context semantics throughout the interaction process; finally, a feedback control instruction for the speech interaction device is generated according to the semantic integrity; in summary, the solution of this application can achieve semantic consistency in the context dialogue in a complex dialogue scenario. Description of the Drawings

[0047] Figure 1It is an exemplary flowchart of an AI-based voice interaction method shown in some embodiments of the present application;

[0048] Figure 2 It is a schematic flowchart of extracting the context dependency relationship between each conversation turn shown in some embodiments of the present application;

[0049] Figure 3 It is a schematic flowchart of determining the semantic loss amount shown in some embodiments of the present application;

[0050] Figure 4 It is a schematic structural diagram of an AI-based voice interaction system shown in some embodiments of the present application;

[0051] Figure 5 It is a schematic structural diagram of a computer device implementing the AI-based voice interaction method shown in some embodiments of the present application. Detailed implementation manners

[0052] To better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0053] Refer to Figure 1 , this figure is an exemplary flowchart of an AI-based voice interaction method shown in some embodiments of the present application. The AI-based voice interaction method

[0054] In step 101, interaction voices of the target user in different conversation turns are obtained through a voice interaction device.

[0055] Specifically, obtaining the interaction voices of the target user in different conversation turns through the voice interaction device can be implemented in the following manner, that is: obtaining the interaction voices of the target user in different conversation turns through the cloud data transmission interface of the voice interaction device.

[0056] It should be noted that the interaction voice described in the present application refers to the voice information input or output by the target user during the voice interaction process.

[0057] In step 102, the context dependency relationship between each conversation turn is extracted from all the interaction voices based on the attention mechanism.

[0058] In some embodiments, refer to Figure 2 shown, this figure is a schematic flowchart of extracting the context dependency relationship between each conversation turn in some embodiments of the present application. In this embodiment, extracting the context dependency relationship between each conversation turn from all the interaction voices based on the attention mechanism can be implemented in the following steps:

[0059] First, in step 1021, all interactive voices are converted into multiple voice vectors;

[0060] Second, in step 1022, an attention vector space for each voice vector is constructed based on a preset attention mechanism;

[0061] Then, in step 1023, the semantic dependency degree of each dialogue turn is determined according to the attention vector space of each voice vector;

[0062] Finally, in step 1024, the context dependency relationship between each dialogue turn is determined by the semantic dependency degrees of all dialogue turns.

[0063] When specifically implemented, converting all interactive voices into multiple voice vectors can be achieved in the following way, that is: using a pre-trained voice model (such as Wav2Vec 2.0, Whisper, etc.) to convert each interactive voice into a voice vector, thereby obtaining multiple voice vectors. Here, the voice vector represents a numerical feature vector of the interactive voice. In other embodiments, other methods can also be used to achieve this, which will not be elaborated here.

[0064] When specifically implemented, constructing an attention vector space for each voice vector based on a preset attention mechanism can be achieved in the following way, that is: using a preset attention mechanism (such as the self-attention mechanism) to perform a linear transformation on each voice vector, and then obtaining the query vector, key vector, and value vector of each voice vector. Thus, the set composed of the query vector, key vector, and value vector of each voice vector is used as the attention vector space of each voice vector. In other embodiments, other methods can also be used to achieve this, which is not limited here.

[0065] It should be noted that the attention vector space described in this application refers to the vector set formed after converting the voice vector into a query vector, a key vector, and a value vector through the attention mechanism, which reflects the attention distribution of the voice vector under different dialogue turns.

[0066] In specific implementation, determining the semantic dependence degree of each conversation turn according to the attention vector space of each speech vector can be implemented in the following manner, that is: First, select a speech vector as the selected speech vector, calculate the dot product of the query vector and the key vector in the attention vector space of the selected speech vector, and use the obtained dot product as the attention score of the selected speech vector. Then, calculate the cosine similarity between the query vector, the key vector, and the value vector in the attention vector space of the selected speech vector and the query vector, the key vector, and the value vector in the attention vector spaces of other speech vectors, sum all the cosine similarities, multiply the obtained sum value by the attention score of the selected speech vector, and use the obtained multiplication value as the semantic dependence degree of the selected speech vector corresponding to the conversation turn. Then continue to determine the semantic dependence degrees of the remaining speech vectors corresponding to the conversation turns. In other embodiments, other methods can also be used for implementation, which are not limited here.

[0067] It should be noted that the semantic dependence degree described in this application represents the association strength in semantic expression between each conversation turn and the previous and subsequent conversation turns.

[0068] In specific implementation, determining the context dependence relationship between each conversation turn through the semantic dependence degrees of all conversation turns can be implemented in the following manner, that is: First, exclude the maximum semantic dependence degree and the minimum semantic dependence degree from the semantic dependence degrees of all conversation turns. Then, calculate the average value of the remaining semantic dependence degrees after excluding the maximum semantic dependence degree and the minimum semantic dependence degree, and use the obtained average value as the context dependence relationship between each conversation turn. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.

[0069] It should be noted that the context dependence relationship described in this application represents the characteristic parameter of the mutual influence in semantic expression between each conversation turn.

[0070] In step 103, determine the semantic loss amount of the interactive speech in each conversation turn according to the semantic features of the interactive speech in each conversation turn, and determine the conversation coherence degree of the target user in each conversation turn through all the semantic loss amounts and the context dependence relationship between each conversation turn.

[0071] In some embodiments, as shown in Figure 3 This figure is a schematic flowchart of determining the semantic loss amount in some embodiments of this application. In this embodiment, determining the semantic loss amount of the interactive speech in each conversation turn according to the semantic features of the interactive speech in each conversation turn can be implemented through the following steps:

[0072] Determine the semantic features of the interactive speech in each conversation turn;

[0073] Determine the semantic similarity between interactive voices in each dialogue turn according to all semantic features;

[0074] Determine the semantic loss amount of the interactive voice in each dialogue turn according to the semantic similarity between the interactive voices in each dialogue turn.

[0075] When specifically implemented, the semantic features of the interactive voice in each dialogue turn can be determined in the following way, that is: obtain the voice vector of the interactive voice in each dialogue turn, and then use the scikit-learn library in Python to perform principal component analysis on the voice vectors of the interactive voices in each dialogue turn, and use the vectors obtained after the principal component analysis as the semantic features of the interactive voices in the corresponding dialogue turns, so as to obtain the voice vectors of the interactive voices in each dialogue turn. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.

[0076] It should be noted that the semantic features described in this application represent the digital vector representation of the key semantic information of the interactive voice in the dialogue turn.

[0077] When specifically implemented, the semantic similarity between the interactive voices in each dialogue turn can be determined according to all semantic features in the following way, that is: first, calculate the cosine similarity for every two semantic features, then calculate the mean value of all the cosine similarities, and use the obtained mean value as the global cosine similarity. Further, select the maximum cosine similarity and the minimum cosine similarity from all the cosine similarities, and then divide the difference between the maximum cosine similarity and the minimum cosine similarity by the global cosine similarity, and use the obtained value as the semantic similarity between the interactive voices in each dialogue turn. In other embodiments, other methods can also be used to implement this, which is not limited here.

[0078] It should be noted that the semantic similarity described in this application represents the degree of similarity of semantic information between the interactive voices in each dialogue turn. The greater the semantic similarity, the higher the degree of similarity of semantic information between the interactive voices in each dialogue turn.

[0079] In specific implementation, the semantic loss amount of the interactive speech in each conversation turn can be determined by the semantic similarity between the interactive speeches in each conversation turn in the following way, that is: First, obtain the cosine similarity between every two semantic features, then perform a difference operation on all the cosine similarities, then sum all the values obtained after the difference operation, and use the sum value as the similarity difference value. Further, select a semantic feature as the selected semantic feature, and then select the maximum cosine similarity from all the cosine similarities between the selected semantic feature and other semantic features. Divide the maximum cosine similarity by the similarity difference value, and then multiply the obtained value by the semantic similarity between the interactive speeches in each conversation turn, and use the multiplied value as the semantic loss amount of the interactive speech corresponding to the selected semantic feature in the conversation turn. Continue to determine the semantic loss amount of the interactive speech corresponding to the remaining semantic features in the conversation turn. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.

[0080] It should be noted that the semantic loss amount in this application represents an index of the deviation degree of semantic information between the interactive speech in the current conversation turn and the interactive speech in other conversation turns.

[0081] In some embodiments, the conversation coherence degree of the target user in each conversation turn can be determined by all the semantic loss amounts and the context dependence between the conversation turns in the following steps:

[0082] Determine the conversation state amount in each conversation turn based on a preset sliding window and all the semantic loss amounts;

[0083] Determine the conversation coherence degree of the target user in each conversation turn according to the conversation state amount in each conversation turn and the context dependence between the conversation turns.

[0084] It should be noted that the sliding window in this application can be implemented according to the existing sliding window technology. Among them, the size of the sliding window is set to the value obtained by taking the integer part of the square root of the total number of all semantic loss amounts, and the sliding step of the sliding window is set to 1.

[0085] When specifically implemented, determining the dialogue state quantity in each dialogue turn based on a preset sliding window and all semantic loss quantities can be achieved in the following manner, that is: First, sort all the semantic loss quantities in ascending order, and use the obtained sequence after sorting as the semantic loss quantity sequence. Then, align the first semantic loss quantity in the semantic loss quantity sequence with the preset sliding window, select the maximum semantic loss quantity and the minimum semantic loss quantity from the sliding window, and use the difference between the maximum semantic loss quantity and the minimum semantic loss quantity as the dialogue state quantity corresponding to the first semantic loss quantity in the corresponding dialogue turn. Move the sliding window to align the second semantic loss quantity in the semantic loss quantity sequence, and then determine the dialogue state quantity corresponding to the second semantic loss quantity in the corresponding dialogue turn. Then, move the sliding window in sequence until the sliding window aligns with the last semantic loss quantity in the semantic loss quantity sequence, so as to obtain the dialogue state quantities corresponding to the remaining semantic loss quantities in the semantic loss quantity sequence in sequence. In other embodiments, other methods can also be used for implementation, which are not limited here.

[0086] It should be noted that the dialogue state quantity described in this application represents a parameter for measuring the stability of the interaction state in the current dialogue turn compared to the previous dialogue turn.

[0087] When specifically implemented, determining the dialogue coherence degree of the target user in each dialogue turn based on the dialogue state quantity in each dialogue turn and the context dependence relationship between the dialogue turns can be achieved in the following manner, that is: Multiply the context dependence relationship between the dialogue turns by the dialogue state quantity in each dialogue turn, and use the obtained values as the dialogue coherence degree of the target user in each dialogue turn. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.

[0088] It should be noted that the dialogue coherence degree described in this application represents the logical coherence degree of the semantic expression of the target user in this dialogue turn. The greater the dialogue coherence degree, the higher the logical coherence degree of the semantic expression of the target user in this dialogue turn, and vice versa.

[0089] In step 104, classify the interaction speech of the target user in the current dialogue scenario to obtain different semantic intents of the target user in the current dialogue scenario, and then determine the semantic association relationship between the semantic intents.

[0090] In some embodiments, classifying the interaction speech of the target user in the current dialogue scenario to obtain different semantic intents of the target user in the current dialogue scenario can be achieved by the following steps:

[0091] Extract multiple dialogue keywords from the interaction speech of the target user in the current dialogue scenario;

[0092] Determine the semantic confidence of each dialogue keyword based on a pre-trained large language model;

[0093] Classify the features of all dialogue keywords according to the semantic confidence of all dialogue keywords to obtain different semantic intents of the target user in the current dialogue scenario.

[0094] When specifically implemented, extracting multiple dialogue keywords from the interactive speech of the target user in the current dialogue scenario can be achieved in the following way, that is: First, use the Transformer model to convert the interactive speech of the target user in the current dialogue scenario into text information. Then, use a keyword extraction algorithm (such as the TextRank algorithm) to extract multiple keywords from the text information, and use the obtained keywords as dialogue keywords. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.

[0095] It should be noted that the pre-trained large language model in this application uses the GPT-4 model. GPT-4 is a large multi-modal language model developed by OpenAI, which performs excellently in language understanding. GPT-4 can handle complex sentence structures and delicate semantic understandings, support more natural conversations and text generations. GPT-4 evaluates the importance of each word in the sequence through the self-attention mechanism and assigns different weights according to its importance, so as to better understand the language structure and patterns. In addition, GPT-4 can also understand the anaphora resolution in multi-turn conversations and more accurately handle complex semantic relationships such as context, metaphor, and humor in natural language.

[0096] When specifically implemented, determining the semantic confidence of each dialogue keyword based on a pre-trained large language model can be achieved in the following way, that is: First, input each dialogue keyword into the pre-trained large language model, and use the output result of the large language model as the semantic information of each dialogue keyword. Second, use the Word2Vec model to convert each dialogue keyword and the semantic information of each dialogue keyword into vectors, so as to obtain the vector representation of each dialogue keyword and the vector representation of the semantic information of each dialogue keyword. Then, calculate the dot product between the vector representation of each dialogue keyword and the vector representation of the semantic information of each dialogue keyword, and use the obtained dot product as the semantic confidence of each dialogue keyword. In other embodiments, other methods can also be used to implement this, which is not limited here.

[0097] It should be noted that the semantic confidence in this application represents the reliability of measuring the semantic understanding of dialogue keywords.

[0098] In specific implementation, feature classification is performed on all dialogue keywords according to their semantic confidence levels, and different semantic intents of the target user in the current dialogue scenario can be obtained in the following manner: First, a clustering algorithm (such as the Mean Shift algorithm) is used to cluster the semantic confidence levels of all dialogue keywords, thereby obtaining multiple data clusters. Each data cluster is composed of semantic confidence levels. Then, the set of dialogue keywords corresponding to the semantic confidence levels within the same data cluster is used as a semantic intent of the target user in the current dialogue scenario, and thus different semantic intents of the target user in the current dialogue scenario are obtained. In other embodiments, other methods can also be used for implementation, which will not be elaborated here.

[0099] It should be noted that in this application, the semantic intent represents the core demand characteristics of the same type expressed by the target user in the current dialogue scenario, where the semantic intent is a set composed of multiple dialogue keywords.

[0100] In some embodiments, the following steps can be used to determine the semantic association relationship between each semantic intent:

[0101] Obtain all dialogue keywords;

[0102] Based on the pre-trained topic analysis model, perform topic analysis on all dialogue keywords to obtain the topic characteristics of the interactive speech of the target user in the current dialogue scenario;

[0103] According to the topic characteristics of the interactive speech of the target user in the current dialogue scenario and the different semantic intents of the target user in the current dialogue scenario, determine the semantic association relationship between each semantic intent.

[0104] It should be noted that the pre-trained topic analysis model in this application is the BERTopic model. The BERTopic model is a powerful tool that combines the pre-trained model BERT and topic modeling. It captures the semantic information of all dialogue keywords through the BERT model and then uses topic modeling techniques to cluster this semantic information to obtain topics.

[0105] In specific implementation, based on the pre-trained topic analysis model, performing topic analysis on all dialogue keywords to obtain the topic characteristics of the interactive speech of the target user in the current dialogue scenario can be achieved in the following manner: First, input all dialogue keywords into the pre-trained topic analysis model, thereby outputting multiple topic candidate words. Then, the set composed of all topic candidate words is used as the topic characteristics of the interactive speech of the target user in the current dialogue scenario. In other embodiments, other methods can also be used for implementation, which is not limited here.

[0106] It should be noted that the subject features described in this application are a set composed of multiple subject candidate words.

[0107] In specific implementation, the semantic association relationship between each semantic intention can be determined according to the subject features of the interactive speech of the target user in the current conversation scenario and the different semantic intentions of the target user in the current conversation scenario in the following way, that is: calculate the cosine similarity between each subject candidate word in the subject features of the interactive speech of the target user in the current conversation scenario and each conversation keyword in each semantic intention of the target user in the current conversation scenario, and then sum all the cosine similarities, and use the obtained sum value as the semantic association relationship between each semantic intention. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.

[0108] It should be noted that the semantic association relationship in this application represents a measurement parameter for the mutual influence of each semantic intention at the semantic level.

[0109] In step 105, the semantic integrity of the target user in the current conversation scenario is determined through the semantic association relationship and the conversation coherence of the interactive speech in each conversation turn, and then a feedback control instruction for the voice interaction device is generated based on the semantic integrity.

[0110] In some embodiments, the semantic integrity of the target user in the current conversation scenario can be determined through the semantic association relationship and the conversation coherence of the interactive speech in each conversation turn by the following steps:

[0111] Perform linear fitting on the conversation coherence of the interactive speech in all conversation turns to obtain a fitting curve of the conversation coherence;

[0112] Obtain the semantic confidence of each conversation keyword;

[0113] Determine the semantic association graph of the target user in the current conversation scenario according to all the semantic confidences and the semantic association relationship;

[0114] Determine the semantic integrity of the target user in the current conversation scenario through the semantic association graph and the fitting curve of the conversation coherence.

[0115] In specific implementation, the linear fitting of the conversation coherence of the interactive speech in all conversation turns to obtain a fitting curve of the conversation coherence can be achieved in the following way, that is: use an existing linear fitting algorithm (such as the least squares support vector machine algorithm) to perform linear fitting on all conversation coherences, and use the obtained fitting curve as the fitting curve of the conversation coherence. Among them, each value on the fitting curve is used as the fitting value of the conversation coherence, and each fitting value of the conversation coherence corresponds to a conversation coherence. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.

[0116] In specific implementation, to determine the semantic association graph of the target user in the current conversation scenario based on all semantic confidence levels and the semantic association relationship, the following method can be adopted, that is: Based on graph theory, each semantic confidence level is used as a node, and then the absolute value of the difference between every two semantic confidence levels is used as the edge between the corresponding nodes of every two semantic confidence levels, thereby obtaining a graph structure. Further, the semantic association relationship is used to update the weights of all edges, and the graph structure with the weights of all edges updated is used as the semantic association graph of the target user in the current conversation scenario. Specifically, the update method is to multiply the weight of each edge in the graph structure by the semantic association relationship, and replace the weight of each edge in the graph structure with the value obtained by all multiplications. In other embodiments, other methods can also be used for implementation, which is not limited here.

[0117] It should be noted that the semantic association graph in this application represents a graph structure composed of different semantic elements of the target user in the current conversation scenario and their association relationships.

[0118] In specific implementation, to determine the semantic integrity of the target user in the current conversation scenario through the fitting curve of the semantic association graph and the conversation coherence, the following method can be adopted, that is: First, use the Prim algorithm to calculate the minimum spanning tree of the semantic association graph, sum the weights of all edges in the minimum spanning tree, and use the obtained value as the semantic association weight value. Then, subtract the corresponding conversation coherence from each conversation coherence fitting value on the fitting curve, take the absolute value of all the obtained subtracted values, further multiply each obtained absolute value by the semantic association weight value, so as to normalize all the multiplied values, then take the mean of all the normalized values, and use the obtained mean value as the semantic integrity of the target user in the current conversation scenario. In other embodiments, other methods can also be used for implementation, which is not limited here.

[0119] It should be noted that the semantic integrity in this application represents the degree of completeness of the semantic information of the target user in the current conversation scenario. The greater the semantic integrity, the higher the degree of completeness of the semantic information of the target user in the current conversation scenario.

[0120] In specific implementation, the feedback control instruction of the voice interaction device can be generated according to the semantic integrity in the following way: First, the semantic integrity is pre-divided into different levels, and each level corresponds to a feedback control instruction. Usually, it can be divided into two levels, namely: high level and low level. Determine the level corresponding to the semantic integrity. If the level corresponding to the semantic integrity is high level, directly output the feedback control instruction corresponding to the voice content. If the level corresponding to the semantic integrity is low level, first supplement new voice content, and then generate the corresponding feedback control instruction according to the supplemented voice content. For example, when the semantic integrity is greater than 0.6, it is high level. When the semantic integrity is high level, directly output the corresponding feedback control instruction according to the voice content without supplementing new voice content. When the semantic integrity is in the range less than 0.6, the level corresponding to the semantic integrity is low level. When the semantic integrity is low level, new voice content needs to be supplemented before outputting the corresponding feedback control instruction. In specific implementation, for example: when the voice content is "Turn on the lights in the living room and adjust the brightness to 80%", the semantic integrity is determined to be high level (i.e., greater than 0.6). At this time, the voice interaction device directly outputs the corresponding feedback control instruction, that is, sends an instruction to the smart home system to turn on the lights in the living room and adjust the brightness to 80%. When the voice content is "Turn the lights up a bit", the semantic integrity is determined to be low level (i.e., less than 0.6). Because it is not clear which light is specifically referred to and to what extent it needs to be brightened, new voice content needs to be supplemented at this time, such as further asking the user "Do you mean the lights in the living room or the bedroom? To which brightness level do you need to adjust?" etc. After the user's supplementary answer, for example, the user says "It's the lights in the living room, turn them to the brightest", generate the corresponding feedback control instruction according to the supplemented complete voice content, that is, send an instruction to the smart home system to turn the lights in the living room to the brightest. In other embodiments, other methods can also be used to implement this, which will not be elaborated here.

[0121] In addition, on the other hand of the present application, in some embodiments, the present application provides a voice interaction system based on artificial intelligence. Refer to Figure 4 , which is a schematic structural diagram of a voice interaction system based on artificial intelligence shown according to some embodiments of the present application. The voice interaction system 400 based on artificial intelligence includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described as follows:

[0122] The acquisition module 401. In the present application, the acquisition module 401 is mainly used to acquire the interaction voice of the target user in different dialogue turns through a voice interaction device;

[0123] The processing module 402. In the present application, the processing module 402 is used to extract the context dependency between each dialogue turn from all the interaction voices based on the attention mechanism;

[0124] It should be noted that the processing module 402 in the present application is further configured to determine the semantic loss amount of the interactive speech in each conversation turn according to the semantic features of the interactive speech in each conversation turn, and determine the conversation coherence degree of the target user in each conversation turn through all the semantic loss amounts and the context dependence relationship between the respective conversation turns;

[0125] In addition, the processing module 402 in the present application is further configured to classify the intent of the interactive speech of the target user in the current conversation scenario, obtain different semantic intents of the target user in the current conversation scenario, and further determine the semantic association relationship between the respective semantic intents;

[0126] The execution module 403. In the present application, the execution module 403 is mainly configured to determine the semantic integrity of the target user in the current conversation scenario through the semantic association relationship and the conversation coherence degree of the interactive speech in each conversation turn, and further generate a feedback control instruction for the voice interaction device according to the semantic integrity.

[0127] In addition, the present application further provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned artificial intelligence-based voice interaction method.

[0128] In some embodiments, refer to Figure 5 , this figure is a schematic structural diagram of a computer device for implementing the artificial intelligence-based voice interaction method according to some embodiments of the present application. The artificial intelligence-based voice interaction method in the above embodiments can be implemented by Figure 5 the computer device shown. The computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0129] The processor 501 can be a general-purpose central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more for controlling the execution of the artificial intelligence-based voice interaction method in the present application.

[0130] The communication bus 502 can be used to transmit information between the above components.

[0131] The memory 503 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 can exist independently and be connected to the processor 501 through the communication bus 502. The memory 503 can also be integrated with the processor 501.

[0132] Among them, the memory 503 is used to store the program code for executing the solution of this application and is controlled by the processor 501 for execution. The processor 501 is used to execute the program code stored in the memory 503. The program code can include one or more software modules. The methods described in the above method embodiments can be implemented by one or more software modules in the program code in the processor 501 and the memory 503.

[0133] The communication interface 504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0134] In a specific implementation, as an embodiment, the computer device can include multiple processors, and each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0135] The computer device described above can be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0136] In addition, the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-described voice interaction method based on artificial intelligence is implemented.

[0137] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0138] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A voice interaction method based on artificial intelligence, characterized in that, It includes the following steps: Obtain the interaction voices of the target user in different dialogue turns through a voice interaction device; Extract the context dependence relationships between each dialogue turn from all the interaction voices based on an attention mechanism; Determine the semantic loss amount of the interaction voice in each dialogue turn according to the semantic features of the interaction voice in each dialogue turn, where the semantic loss amount is an index representing the degree of semantic information deviation between the interaction voice in the current dialogue turn and the interaction voices in other dialogue turns, and determine the dialogue coherence degree of the target user in each dialogue turn through all the semantic loss amounts and the context dependence relationships between each dialogue turn; Classify the intentions of the interaction voice of the target user in the current dialogue scenario to obtain different semantic intentions of the target user in the current dialogue scenario, and then determine the semantic association relationships between each semantic intention; Determine the semantic integrity of the target user in the current dialogue scenario through the semantic association relationships and the dialogue coherence degree of the interaction voice in each dialogue turn, and then generate a feedback control instruction for the voice interaction device according to the semantic integrity; Among them, determining the semantic integrity of the target user in the current dialogue scenario through the semantic association relationships and the dialogue coherence degree of the interaction voice in each dialogue turn specifically includes: Perform linear fitting on the dialogue coherence degrees of the interaction voices in all dialogue turns to obtain a fitting curve of the dialogue coherence degree; Obtain the semantic confidence of each dialogue keyword; Determine the semantic association graph of the target user in the current dialogue scenario according to all the semantic confidences and the semantic association relationships; Determine the semantic integrity of the target user in the current dialogue scenario through the semantic association graph and the fitting curve of the dialogue coherence degree.

2. The method according to claim 1, wherein Extracting the context dependence relationships between each dialogue turn from all the interaction voices based on an attention mechanism specifically includes: Convert all the interaction voices into multiple voice vectors; Construct an attention vector space for each voice vector based on a preset attention mechanism; Determine the semantic dependence degree of each dialogue turn according to the attention vector space of each voice vector; Determine the context dependence relationships between each dialogue turn through the semantic dependence degrees of all dialogue turns.

3. The method according to claim 1, characterized in that, Determining the semantic loss amount of the interaction voice in each dialogue turn according to the semantic features of the interaction voice in each dialogue turn specifically includes: Determine the semantic features of the interaction voice in each dialogue turn; Determine the semantic similarity between the interaction voices in each dialogue turn according to all the semantic features; Determine the semantic loss amount of the interaction voice in each dialogue turn through the semantic similarity between the interaction voices in each dialogue turn.

4. The method according to claim 1, wherein Determining the dialogue coherence degree of the target user in each dialogue turn through all the semantic loss amounts and the context dependence relationships between each dialogue turn specifically includes: Determine the dialogue state amount in each dialogue turn based on a preset sliding window and all the semantic loss amounts; Determine the dialogue coherence degree of the target user in each dialogue turn according to the dialogue state amount in each dialogue turn and the context dependence relationships between each dialogue turn.

5. The method according to claim 1, wherein Classify the interaction speech of the target user in the current conversation scenario to obtain different semantic intents of the target user in the current conversation scenario, specifically including: Extract multiple conversation keywords from the interaction speech of the target user in the current conversation scenario; Determine the semantic confidence of each conversation keyword based on a pre-trained large language model; Classify the features of all conversation keywords according to the semantic confidence of all conversation keywords to obtain different semantic intents of the target user in the current conversation scenario.

6. The method according to claim 1, characterized in that, Determine the semantic association relationship between each semantic intent, specifically including: Obtain all conversation keywords; Perform topic analysis on all conversation keywords based on a pre-trained topic analysis model to obtain the topic features of the interaction speech of the target user in the current conversation scenario; Determine the semantic association relationship between each semantic intent according to the topic features of the interaction speech of the target user in the current conversation scenario and different semantic intents of the target user in the current conversation scenario.

7. A voice interaction system based on artificial intelligence, which uses the method described in any one of claims 1 to 6 for voice interaction, characterized in that, The system includes: An acquisition module for acquiring the interaction speech of the target user in different conversation rounds through a voice interaction device; A processing module for extracting the context dependence relationship between each conversation round from all the interaction speech based on an attention mechanism; The processing module is further configured to determine the semantic loss amount of the interaction speech in each conversation round according to the semantic features of the interaction speech in each conversation round, and determine the conversation coherence degree of the target user in each conversation round through all the semantic loss amounts and the context dependence relationship between each conversation round; The processing module is further configured to classify the intent of the interaction speech of the target user in the current conversation scenario to obtain different semantic intents of the target user in the current conversation scenario, and then determine the semantic association relationship between each semantic intent; An execution module for determining the semantic integrity of the target user in the current conversation scenario through the semantic association relationship and the conversation coherence degree of the interaction speech in each conversation round, and then generating a feedback control instruction for the voice interaction device according to the semantic integrity.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the artificial intelligence-based voice interaction method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the artificial intelligence-based voice interaction method according to any one of claims 1 to 6.

Images