Intelligent voice interaction method and system

Abstract

The present application relates to the field of voice interaction technology, and in particular to an intelligent voice interaction method and system, the method comprising: obtaining the voice signal of the current round of questioning voice and converting it into voice text; weighted summing of the word vectors of each word according to the semantic importance of each word in the voice text to obtain the question semantics; obtaining the predicted probability of each standard question in the current round according to the prediction model; calculating the similarity between the question semantics and each standard question, and obtaining the question of the current round according to the similarity and the predicted probability; inputting the question and the standard questions and answer texts of each round before the current round into a large text generation model, obtaining the answer text of the current round and converting it into a broadcast voice. Through the technical solution of the present application, it is possible to accurately identify the questions raised by the user, and then obtain the accurate answer text, thereby improving the user experience during voice interaction.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to an intelligent voice interaction method and system. Background Art

[0002] Voice interaction involves voice recognition technology, which identifies questions raised by users, determines the answers based on the questions raised, and converts the answers into voice information, which is then played to the user.

[0003] At present, the patent application document with application publication number CN117555916A discloses a voice interaction method and system based on natural language processing, wherein the method includes: obtaining voice information input by the user, and converting the voice information into natural language text information based on voice recognition technology; performing text preprocessing on the natural language text information to obtain natural language text information after text preprocessing; converting the natural language text information after text preprocessing into a structured query language based on an NLP semantic parsing model; obtaining corresponding target data based on the structured query language; inputting the corresponding target data into an optimized natural language generation model, and converting the corresponding target data into a response text based on the optimized natural language generation model; converting the response text into a response voice based on voice synthesis technology, and outputting the response voice.

[0004] The above method directly converts the voice information input by the user into a structured query language, and inputs the target data corresponding to the structured query language into the optimized natural language generation model to obtain the response text; however, in the process of collecting the user's input voice information, there may be noisy background sounds. These background sounds will cause the voice information to be distorted and the questions raised by the user cannot be accurately identified, resulting in inaccurate response text and a poor user experience during voice interaction. Summary of the Invention

[0005] In order to solve the technical problem of inaccurate response text, the present application provides an intelligent voice interaction method and system that can accurately identify questions raised by users, and then obtain accurate response text, thereby improving the user experience during voice interaction.

[0006] In a first aspect, the present application provides an intelligent voice interaction method, which includes: obtaining the signal-to-noise ratio of each time frame in the user's question voice in the current round, deleting the time frame with a signal-to-noise ratio less than a preset value to obtain a voice signal, and converting the voice signal into voice text; calculating the semantic importance of each word in the voice text, and weightedly summing the word vectors of each word according to the semantic importance to obtain the question semantics, wherein the semantic importance is positively correlated with the TF-IDF value of the word in the standard question set and the energy value of the corresponding voice segment; training a prediction model based on historical interaction information, the prediction model is used to obtain the predicted probability of each standard question in the current round based on the standard questions and answer texts of the user in each round before the current round; calculating the similarity between the question semantics and each standard question, taking the weighted sum of the similarity and the predicted probability as the target probability of each standard question, and selecting the standard question with the maximum target probability as the question in the current round; inputting the question question and the standard questions and answer texts of each round before the current round into a text generation large model, obtaining the answer text of the current round, and converting the answer text into broadcast voice.

[0007] The user's voice question in the current round is collected, and the signal-to-noise ratio of each time frame in the voice question is obtained. A lower signal-to-noise ratio indicates greater background noise in the corresponding time frame. Time frames with signal-to-noise ratios below a preset value are deleted to obtain the voice signal, which is then converted into speech text. Different words in the speech text have different abilities to represent semantic information. Therefore, the semantic importance of each word is accurately measured based on its TF-IDF value in the standard question set and the energy value of the corresponding speech segment. The normalized semantic importance is used to weight the sum of the word vectors of each word to accurately obtain the question semantics of the voice question. The similarity between the question semantics and the standard questions can be used to determine which standard question the user's voice question in the current round belongs to. At the same time, the time series of the standard questions and response texts from the user in previous rounds is input into the prediction model to obtain the predicted probability of each standard question in the current round. The target probability of each standard question is determined by combining the similarity and predicted probability. The standard question with the highest target probability is selected as the question for the current round, achieving accurate recognition of the question in the current round. The question is then input into the text to generate a large model, obtaining accurate response text, and improving the user experience during voice interaction.

[0008] Preferably, before calculating the semantic importance of each word in the voice text, the interaction method further includes: in response to the proportion of the deleted time frame in the total duration of the question voice being greater than a preset proportion, re-collecting the user's question voice in the current round.

[0009] When the deleted time frames account for a greater proportion of the total duration of the question voice than the preset proportion, it indicates that there is a lot of background noise in the question voice, and the user is reminded to ask the question again to re-collect the user's question voice in the current round to ensure the validity of the question voice.

[0010] Preferably, calculating the semantic importance of each word in the speech text includes: calculating the semantic importance of each word in the speech text TF-IDF value in the standard question set; force alignment of speech text and speech information to obtain words The corresponding speech segment; the ratio of the average short-time energy of the speech segment to the average short-time energy of the speech signal is used as the word The energy value of the word The semantic importance of is the product of TF-IDF value and energy value.

[0011] Semantic importance is comprehensively considered from two aspects: the ability to distinguish standard questions and the degree of importance given by users when expressing them, to achieve accurate quantification of the semantic importance of each word in the speech text.

[0012] Preferably, the forced alignment is achieved using the Aeneas tool or the Gentle tool.

[0013] Preferably, obtaining the question semantics includes: taking the ratio of the semantic importance of any word to the sum of the semantic importance of all words as the normalized semantic importance; and weighted summing the word vectors of each word according to the normalized semantic importance to obtain the question semantics.

[0014] Preferably, the historical interaction information includes standard questions and answer texts of all rounds in the historical interaction process; training the prediction model based on the historical interaction information includes: using the question and answer records before any round in the historical interaction information as training samples, and using the standard questions of any round as category labels; inputting the training samples into the prediction model to obtain output results, and performing gradient descent on the prediction model based on the cross-entropy loss between the output results and the category labels to update the prediction model until the cross-entropy loss is less than the preset loss, or the number of updates is greater than the preset number, and the training is completed.

[0015] The prediction model is trained based on the question and answer records of all rounds in the historical interaction information. The trained prediction model can accurately obtain the predicted probability of each standard question in the current round based on the user's standard questions and answer texts in the rounds before the current round.

[0016] Preferably, calculating the similarity between the question semantics and each standard question includes: obtaining the standard semantics of each standard question, wherein the standard semantics is the result of weighted summation of the word vectors of each word in the standard question based on the normalized TF-IDF value; and calculating the similarity between the question semantics and the standard semantics of each standard question.

[0017] Preferably, standard issues The target probability for:

[0018] ; For question semantics and standard issues The similarity of Standard issue The predicted probability of is the adjustment coefficient.

[0019] Similarity can be regarded as the probability value of the standard question obtained from the voice information of the current round of questioning; and the predicted probability can be regarded as the probability value of the standard question obtained from the time series of the standard questions and answer texts of the rounds before the current round; the target probability of each standard question is accurately obtained by combining similarity and predicted probability.

[0020] Preferably, the adjustment coefficient is negatively correlated with the number of the current round and negatively correlated with the proportion of the deleted time frame in the total duration of the question voice.

[0021] The adjustment coefficient is used to control the degree of influence of the similarity between the question semantics and the standard question on the target probability. The larger the adjustment coefficient, the more dependent the target probability is on the question speech in the current round. The adjustment coefficient is determined based on the proportion of deleted time frames in the total duration of the question speech and the number of rounds in the current round to ensure the accurate characterization of the target probability.

[0022] In the second aspect of the present application, an intelligent voice interaction system is also provided, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an intelligent voice interaction method according to the first aspect of the present application is implemented.

[0023] The technical solution of this application has the following beneficial technical effects:

[0024] The user's voice question in the current round is collected, and the signal-to-noise ratio of each time frame in the voice question is obtained. A lower signal-to-noise ratio indicates greater background noise in the corresponding time frame. Time frames with signal-to-noise ratios below a preset value are deleted to obtain the voice signal, which is then converted into speech text. Different words in the speech text have different abilities to represent semantic information. Therefore, the semantic importance of each word is accurately measured based on its TF-IDF value in the standard question set and the energy value of the corresponding speech segment. The normalized semantic importance is used to weight the sum of the word vectors of each word to accurately obtain the question semantics of the voice question. The similarity between the question semantics and the standard questions can be used to determine which standard question the user's voice question in the current round belongs to. At the same time, the time series of the standard questions and response texts from the user in previous rounds is input into the prediction model to obtain the predicted probability of each standard question in the current round. The target probability of each standard question is determined by combining the similarity and predicted probability. The standard question with the highest target probability is selected as the question for the current round, achieving accurate recognition of the question in the current round. The question is then input into the text to generate a large model, obtaining accurate response text, and improving the user experience during voice interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flowchart of an intelligent voice interaction method according to an embodiment of the present application.

[0026] Figure 2 This is a structural block diagram of an intelligent voice interaction system according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0028] According to the first aspect of this application, this application provides an intelligent voice interaction method. In the scenario of intelligent online customer service, a user will ask a question, and the intelligent customer service system will generate a response text based on the question, and convert the response text into a broadcast voice, completing a round of interactive question and answer. The user and the intelligent customer service system will conduct multiple rounds of interactive question and answer until the user no longer asks new questions, and the online customer service process ends.

[0029] Figure 1 This is a flow chart of an intelligent voice interaction method according to an embodiment of the present application. Figure 1 As shown, the intelligent voice interaction method includes steps S101 to S105, which are described in detail below.

[0030] S101, obtaining the signal-to-noise ratio of each time frame in the user's question voice in the current round, deleting the time frame with a signal-to-noise ratio less than a preset value, obtaining a voice signal, and converting the voice signal into voice text.

[0031] In one embodiment, multiple interactive Q&A sessions are conducted during the voice interaction process, each consisting of two phases: a user asking a question and the system responding. During the voice interaction process, the user's questioning voice is collected. Because the collection of questioning voices is affected by ambient sound, some voices may not be accurately captured in noisy environments. Therefore, it is necessary to obtain the signal-to-noise ratio (SNR) of each time frame of the user's questioning voice in the current round.

[0032] The question speech is segmented into multiple time frames, typically ranging from 20 to 40 milliseconds in length. The signal-to-noise ratio (SNR) is a measure of signal quality, evaluating the relative quality of the speech signal in each time frame relative to the background noise. A higher SNR indicates higher speech quality in that time frame.

[0033] The preset value is 10. If the signal-to-noise ratio of any time frame is less than the preset value, it means that the time frame contains a lot of background noise and the question voice cannot be accurately collected. The voice information of the time frame is deleted to obtain the voice signal, and the voice signal is converted into voice text.

[0034] S102: Calculate the semantic importance of each word in the speech text, and perform weighted summation of the word vectors of each word based on the semantic importance to obtain the question semantics. The semantic importance is positively correlated with the TF-IDF value of the word in the standard question set and the energy value of the corresponding speech segment.

[0035] In one embodiment, before calculating the semantic importance of each word in the voice text, the interaction method further includes: in response to the deleted time frame accounting for a greater proportion than a preset proportion in the total duration of the question voice, re-collecting the user's question voice in the current round.

[0036] It can be understood that the preset proportion is 0.5. When the proportion of the deleted time frame in the total duration of the question voice is greater than the preset proportion, it means that there is a lot of background noise in the question voice, and the user is reminded to ask the question again to re-collect the user's question voice in the current round.

[0037] In one embodiment, calculating the semantic importance of each word in the speech text includes: calculating the semantic importance of each word in the speech text TF-IDF value in the standard question set; force alignment of speech text and speech information to obtain words The corresponding speech segment; the ratio of the average short-time energy of the speech segment to the average short-time energy of the speech signal is used as the word The energy value of the word The semantic importance of is the product of TF-IDF value and energy value.

[0038] Among them, Aeneas tool, Gentle tool or other alignment tools can be used to achieve forced alignment of speech text and speech information, and this application does not impose any restrictions.

[0039] word The corresponding speech segment includes at least one time frame, and the short-time energy of each time frame can be calculated. Since the accent frame usually has higher short-time energy, the ratio of the average short-time energy of the speech segment to the average short-time energy of the speech signal can be used as the energy value. If the energy value of a word is larger, it means that the user has a higher tone when expressing the word, and pays more attention to the expression of the word, and the word is more important in the speech text.

[0040] The calculation method of TF-IDF value is a well-known technology for those skilled in the art. Here we briefly introduce it. The TF-IDF value is the product of the word frequency and the inverse document frequency; the word frequency is the word The frequency of occurrence of words in the speech text, reflecting Importance in speech text; the inverse document frequency (IDF) value can measure the importance of words The more common a word is in standard questions, the higher its IDF value is, indicating that its ability to distinguish standard questions is stronger. Therefore, the TF-IDF value can reflect the ability to distinguish words. Ability to distinguish standard issues.

[0041] In this way, semantic importance is comprehensively considered from two aspects: the ability to distinguish standard questions and the degree of importance attached by users when expressing them, so as to achieve accurate quantification of the semantic importance of each word in the speech text.

[0042] In one embodiment, after obtaining the semantic importance of each word, obtaining the question semantics includes: taking the ratio of the semantic importance of any word to the sum of the semantic importance of all words as the normalized semantic importance; and weighted summing the word vectors of each word according to the normalized semantic importance to obtain the question semantics.

[0043] The word vector is obtained through the Word2Vec model or the Bert model.

[0044] S103: training a prediction model based on historical interaction information, wherein the prediction model is used to obtain the prediction probability of each standard question in the current round based on the user's standard questions and answer texts in rounds before the current round.

[0045] In one embodiment, the prediction model includes a time series feature sub-model and a classification sub-model. The time series feature sub-model is used to extract time series features from the question and answer records of each round before any round to obtain the interaction vector of any round, and the interaction vector is input into the classification sub-model to obtain the predicted probability of each standard question in the any round; a question and answer record includes a standard question and the answer text corresponding to the standard question.

[0046] Among them, the time series feature sub-model is an LSTM or GRU model, and the interaction vector of any round is input into the classification sub-model to obtain the prediction probability of each standard question, and the sum of the prediction probabilities of each standard question is 1.

[0047] In one embodiment, the historical interaction information includes question and answer records of all rounds in the historical interaction process, and the question and answer records include standard questions and answer texts; training the prediction model based on the historical interaction information includes: using the question and answer records before any round in the historical interaction information as training samples, and using the standard questions of any round as category labels; inputting the training samples into the prediction model to obtain output results, and performing gradient descent on the prediction model based on the cross-entropy loss between the output results and the category labels to update the prediction model until the cross-entropy loss is less than the preset loss, or the number of updates is greater than the preset number, and the training is completed.

[0048] Among them, the preset loss is 0.01 and the preset number of times is 300.

[0049] In this way, the prediction model is trained based on the question and answer records of all rounds in the historical interaction information. The trained prediction model can accurately obtain the predicted probability of each standard question in the current round based on the user's standard questions and answer texts in the rounds before the current round.

[0050] S104, calculating the similarity between the question semantics and each standard question, taking the weighted sum of the similarity and the predicted probability as the target probability of each standard question, and selecting the standard question with the maximum target probability as the question for the current round.

[0051] In one embodiment, calculating the similarity between the question semantics and each standard question includes: obtaining the standard semantics of each standard question, where the standard semantics is the result of weighted summation of the word vectors of each word in the standard question based on the normalized TF-IDF value; and calculating the similarity between the question semantics and the standard semantics of each standard question.

[0052] It can be understood that the greater the similarity of the standard questions, the greater the probability that the question voice in the current round belongs to a standard question. The similarity can be regarded as the probability value of the standard question obtained from the voice information of the question voice in the current round; and the predicted probability can be regarded as the probability value of the standard question obtained from the time series of the standard questions and answer texts in the rounds before the current round; the target probability of each standard question is obtained by combining the similarity and the predicted probability.

[0053] Specifically, the standard issue The target probability for:

[0054] ; For question semantics and standard issues The similarity of Standard issue The predicted probability of is the adjustment coefficient.

[0055] The adjustment coefficient is used to control the degree of influence of the similarity between the question semantics and the standard question on the target probability. The larger the adjustment coefficient, the more dependent the target probability is on the question speech in the current round. To ensure the accuracy of the target probability, the value of the adjustment coefficient can be determined based on the proportion of deleted time frames in the total question speech duration. The smaller the proportion of deleted time frames in the total question speech duration, the more accurate the question speech can be collected, and the larger the adjustment coefficient should be. At the same time, the adjustment coefficient is also related to the number of rounds in the current round. The larger the number of rounds in the current round, the more standard questions and response texts the prediction model can collect, and the accuracy of the predicted probability will also increase accordingly, and the smaller the adjustment coefficient should be. Therefore, the adjustment coefficient is negatively correlated with the number of rounds in the current round and negatively correlated with the proportion of deleted time frames in the total question speech duration.

[0056] Specifically, the adjustment coefficient ranges from 0 to 1; for:

[0057] , is the proportion of the deleted time frame in the total duration of the question voice, The round number of the current round.

[0058] In one embodiment, after obtaining the target probability of each standard question, the standard question with the maximum target probability is selected as the question for the current round. This can accurately obtain the question for the current round in the case of large background noise. The question is any one in the standard question set.

[0059] S105, input the question and the standard questions and answer texts of the rounds before the current round into a text to generate a large model, obtain the answer text of the current round, and convert the answer text into a broadcast voice.

[0060] In one embodiment, after obtaining the user's question in the current round, the question, standard questions from previous rounds, and the answer text are input into a text generation macro model to obtain the answer text for the current round. The answer text for the current round is then converted into a broadcast voice to achieve intelligent voice interaction. The text generation macro model can be an existing macro model such as DeepSeek or GPT.

[0061] According to the second aspect of the present application, the present application also provides an intelligent voice interaction system. Figure 2 This is a structural block diagram of an intelligent voice interaction system according to an embodiment of the present application. Figure 2 As shown, the system 50 includes a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, an intelligent voice interaction method according to the first aspect of the present application is implemented. The system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. Their configuration and functions are well known in the art and are therefore not described here in detail.

[0062] It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all fall within the scope of protection of the present application.

Claims

1. An intelligent voice interaction method, characterized in that: The interaction method includes: Obtaining the signal-to-noise ratio of each time frame in the user's current round of question speech, including: dividing the question speech into multiple time frames, where the length of the time frame is any one of 20 milliseconds to 40 milliseconds; deleting the time frame with a signal-to-noise ratio less than a preset value to obtain a speech signal, and converting the speech signal into speech text; In response to the deleted time frame accounting for a greater proportion than a preset proportion in the total duration of the question voice, recollecting the user's question voice in the current round; Calculate the semantic importance of each word in the speech text and weight the word vectors of each word according to the semantic importance to obtain the question semantics. The semantic importance is positively correlated with the TF-IDF value of the word in the standard question set and the energy value of the corresponding speech segment. A prediction model is trained based on historical interaction information. The prediction model is used to obtain the predicted probability of each standard question in the current round based on the standard questions and answers of the user in the rounds before the current round. Calculate the similarity between the question semantics and each standard question, take the weighted sum of the similarity and the predicted probability as the target probability of each standard question, and select the standard question with the maximum target probability as the question for the current round; The question and the standard questions and answer texts of the rounds before the current round are input into a text generation model to obtain the answer text of the current round, and the answer text is converted into a broadcast voice.

2. The intelligent voice interaction method according to claim 1, characterized in that: Calculating the semantic importance of each word in the speech text includes: Counting words in speech text TF-IDF value in the standard question set; Force the voice text and voice information to align and obtain words The corresponding speech segment; The ratio of the average short-time energy of the speech segment to the average short-time energy of the speech signal is used as the word The energy value of the word The semantic importance of is the product of TF-IDF value and energy value.

3. The intelligent voice interaction method according to claim 2, characterized in that: Use the Aeneas tool or the Gentle tool to achieve forced alignment.

4. The intelligent voice interaction method according to claim 1, wherein: Obtaining the semantics of the problem includes: The ratio of the semantic importance of any word to the sum of the semantic importance of all words is taken as the normalized semantic importance; The word vectors of each word are weighted and summed according to the normalized semantic importance to obtain the question semantics.

5. The intelligent voice interaction method according to claim 1, characterized in that: The historical interaction information includes standard questions and answer texts of all rounds in the historical interaction process; Training the prediction model based on historical interaction information includes: The question-and-answer records before any round in the historical interaction information are used as training samples, and the standard questions of any round are used as category labels; The training samples are input into the prediction model to obtain the output results, and the prediction model is gradient descended according to the cross entropy loss between the output results and the category labels to update the prediction model until the cross entropy loss is less than the preset loss or the number of updates is greater than the preset number, and the training is completed.

6. The intelligent voice interaction method according to claim 1, characterized in that: Calculating the similarity between the question semantics and each standard question includes: Obtaining the standard semantics of each standard question, where the standard semantics is the result of weighted summation of the word vectors of each word in the standard question based on the normalized TF-IDF value; Calculate the similarity between the question semantics and the standard semantics of each standard question.

7. The intelligent voice interaction method according to claim 1, characterized in that: Standard Issues The target probability for: ; For question semantics and standard issues The similarity of Standard issue The predicted probability of is the adjustment coefficient.

8. The intelligent voice interaction method according to claim 7, characterized in that: The adjustment coefficient is negatively correlated with the number of the current round and negatively correlated with the proportion of the deleted time frame in the total duration of the question voice.

9. An intelligent voice interaction system, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an intelligent voice interaction method according to any one of claims 1 to 8 is implemented.

Images