Method, device, storage medium and computer device for identifying end of dialogue turn

Abstract

The application discloses a method and device for identifying the end of a dialogue turn, a storage medium and computer equipment, and relates to the technical field of artificial intelligence, and mainly aims to improve the identification accuracy of the end of a dialogue turn. The method comprises the following steps: obtaining dialogue text and a voice signal corresponding to a dialogue to be identified; inputting the dialogue text into a first preset dialogue turn identification model to identify the end of a dialogue turn, and obtaining a first identification result corresponding to the dialogue to be identified; inputting the voice signal into a second preset dialogue turn identification model to identify the end of a dialogue turn, and obtaining a second identification result corresponding to the dialogue to be identified; and judging whether the dialogue turn corresponding to the dialogue to be identified is ended based on the first identification result and the second identification result. The application is suitable for identifying the end of a dialogue turn.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, storage medium, and computer device for recognizing the end of a speech round. Background Technology

[0002] In the insurance industry, with the development of artificial intelligence, voice chatbot technology is becoming increasingly mature. Voice chatbots can be used to filter out more interested users and improve online sales efficiency. After a user finishes speaking in any round, the voice chatbot needs to provide an effective response to the user's words. Therefore, in order to enable effective dialogue between the voice chatbot and the user, improve service quality, and increase user satisfaction, determining whether a user's speaking round has ended in multi-round dialogue scenarios has become a pressing issue.

[0003] Currently, the determination of whether a user's speaking turn has ended is usually achieved by analyzing the text content of the user's speech. However, this method cannot determine the user's speaking state. For example, it cannot determine whether the word "um" is affirmative or interrogative. Different speaking tones will result in different judgments on whether the speaking turn has ended, leading to low accuracy in recognizing the end of a speech turn. Summary of the Invention

[0004] This invention provides a method, apparatus, storage medium, and computer device for recognizing the end of a speech round, mainly to improve the accuracy of speech round recognition.

[0005] According to a first aspect of the present invention, a method for identifying the end of a speech round is provided, comprising:

[0006] Acquire the text and voice signal of the speech to be identified;

[0007] The script text is input into the first preset script round recognition model to perform script round end recognition, and the first recognition result corresponding to the script to be recognized is obtained;

[0008] The voice signal is input into the second preset speech round recognition model to perform speech round end recognition, and the second recognition result corresponding to the speech to be recognized is obtained;

[0009] Based on the first recognition result and the second recognition result, it is determined whether the speech round corresponding to the speech to be recognized has ended.

[0010] According to a second aspect of the present invention, a device for recognizing the end of a speech round is provided, comprising:

[0011] The acquisition unit is used to acquire the text and voice signal of the speech to be recognized;

[0012] The first recognition unit is used to input the script text into the first preset script round recognition model to perform script round end recognition, and obtain the first recognition result corresponding to the script to be recognized.

[0013] The second recognition unit is used to input the voice signal into the second preset speech round recognition model to perform speech round end recognition and obtain the second recognition result corresponding to the speech to be recognized.

[0014] The judgment unit is used to determine, based on the first recognition result and the second recognition result, whether the speech round corresponding to the speech to be recognized has ended.

[0015] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0016] Acquire the text and voice signal of the speech to be identified;

[0017] The script text is input into the first preset script round recognition model to perform script round end recognition, and the first recognition result corresponding to the script to be recognized is obtained;

[0018] The voice signal is input into the second preset speech round recognition model to perform speech round end recognition, and the second recognition result corresponding to the speech to be recognized is obtained;

[0019] Based on the first recognition result and the second recognition result, it is determined whether the speech round corresponding to the speech to be recognized has ended.

[0020] According to a fourth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the following steps:

[0021] Acquire the text and voice signal of the speech to be identified;

[0022] The script text is input into the first preset script round recognition model to perform script round end recognition, and the first recognition result corresponding to the script to be recognized is obtained;

[0023] The voice signal is input into the second preset speech round recognition model to perform speech round end recognition, and the second recognition result corresponding to the speech to be recognized is obtained;

[0024] Based on the first recognition result and the second recognition result, it is determined whether the speech round corresponding to the speech to be recognized has ended.

[0025] According to the present invention, a method, apparatus, storage medium, and computer device for identifying the end of a speech turn are provided. Compared with the current method of only analyzing the text content of a user's speech to determine whether the user's speaking turn has ended, the present invention acquires the speech text and voice signal corresponding to the speech to be identified; inputs the speech text into a first preset speech turn recognition model for speech turn end recognition to obtain a first recognition result corresponding to the speech to be identified; then inputs the voice signal into a second preset speech turn recognition model for speech turn end recognition to obtain a second recognition result corresponding to the speech to be identified; finally, based on the first recognition result and the second recognition result, it is determined whether the speech turn corresponding to the speech to be identified has ended. Thus, by performing speech turn recognition on the speech text and voice information corresponding to the speech to be identified respectively to obtain the first recognition result and the second recognition result, and finally determining whether the speech turn corresponding to the speech to be identified has ended based on the first recognition result and the second recognition result, the present invention avoids the situation where analyzing only the text content leads to the inability to judge the tone of the speech, thereby improving the accuracy of speech turn end recognition. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0027] Figure 1 The flowchart of a method for recognizing the end of a speech round provided by an embodiment of the present invention is shown;

[0028] Figure 2 This invention provides a flowchart of another method for recognizing the end of a speech round according to an embodiment of the invention.

[0029] Figure 3 This diagram illustrates the structure of a speech round end recognition device provided in an embodiment of the present invention.

[0030] Figure 4 This invention provides a schematic diagram of the structure of another speech round end recognition device according to an embodiment of the invention;

[0031] Figure 5 A schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0032] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0033] Currently, the method of determining whether a user's speaking turn has ended by analyzing only the text content of the user's speech has low accuracy because it cannot determine the tone of the speech.

[0034] To address the aforementioned problems, embodiments of the present invention provide a method for recognizing the end of a speech round, such as... Figure 1 As shown, the method includes:

[0035] 101. Obtain the text and voice signal of the speech to be recognized.

[0036] Among them, the script to be identified is the customer script obtained by the outbound call robot during its conversation with the customer.

[0037] In this embodiment of the invention, to overcome the problem of low accuracy in recognizing the end of a speech cycle in the prior art, this embodiment of the invention performs speech cycle recognition on the speech text and voice information corresponding to the speech to be recognized separately, obtaining a first recognition result and a second recognition result. Finally, based on the first recognition result and the second recognition result, it is determined whether the speech cycle corresponding to the speech to be recognized has ended. This avoids the situation where analyzing only the text content leads to the inability to determine the tone of the speech, thereby improving the accuracy of recognizing the end of a speech cycle. This embodiment of the invention is mainly applied to scenarios where the end of a speech cycle is to be recognized. The executing entity of this embodiment of the invention is a device or equipment capable of recognizing whether a speech cycle has ended, specifically it can be set on the client or server side.

[0038] Specifically, when the outbound call robot is talking to a customer, the call is recorded using a recording device. Simultaneously, if the customer does not speak again after a preset time has elapsed since the last speech, the customer's speech segment is identified as the speech to be recognized. This speech is then transmitted to a preset ASR (Automatic Speech Recognition) model to obtain the corresponding speech text. At the same time, the semantic data corresponding to the speech to be recognized for that customer is extracted from the recording device to obtain the corresponding voice signal. The speech text and voice signal are then recognized separately to obtain a first recognition result and a second recognition result. Finally, based on the first and second recognition results, it is determined whether the speech round corresponding to the speech to be recognized has ended, thus improving the accuracy of speech round completion recognition.

[0039] 102. Input the script text into the first preset script round recognition model to perform script round end recognition, and obtain the first recognition result corresponding to the script to be recognized.

[0040] The first recognition result refers to the recognition result of whether the dialogue round has ended. In this embodiment of the invention, after obtaining the dialogue text corresponding to the dialogue to be recognized, the semantic information vector corresponding to the dialogue text is first obtained using a pre-trained BERT-Base model. Then, the semantic information vector is input into a preset multilayer perceptron, and the multilayer perceptron outputs the first recognition result corresponding to the dialogue to be recognized. At the same time, the voice signal is input into a second preset dialogue round recognition model to recognize the end of the dialogue round, and a second recognition result corresponding to the dialogue to be recognized is obtained. Finally, based on the first recognition result and the second recognition result, it is determined whether the dialogue round corresponding to the dialogue to be recognized has ended, so as to avoid the robot interrupting the customer's speech and failing to grasp the customer's true intention, thereby improving the robot's intelligence and increasing the fluency of the dialogue.

[0041] 103. Input the voice signal into the second preset speech round recognition model to perform speech round end recognition, and obtain the second recognition result corresponding to the speech to be recognized.

[0042] Specifically, the second preset dialogue round recognition model can be a preset classifier, and the second recognition result refers to the recognition result of whether the dialogue round has ended.

[0043] In this embodiment of the invention, after obtaining the speech signal corresponding to the speech to be identified, the spectrogram corresponding to the speech signal is determined, and the spectrogram is input into the preset classifier for classification to obtain the second recognition result corresponding to the speech to be identified. Finally, based on the first recognition result and the second recognition result, it is determined whether the speech round corresponding to the speech to be identified has ended, thereby avoiding the situation where only the text content is analyzed, which leads to the inability to judge the tone of the speech, thus improving the recognition accuracy of the end of the speech round.

[0044] 104. Based on the first identification result and the second identification result, determine whether the speech round corresponding to the speech to be identified has ended.

[0045] In this invention, the end of a dialogue turn refers to the end of a customer's description of the same content. In this embodiment, after determining the first recognition result corresponding to the dialogue to be identified based on the dialogue text, and the second recognition result corresponding to the dialogue to be identified based on the voice signal, the first and second recognition results are comprehensively considered to determine whether the dialogue turn corresponding to the dialogue to be identified has ended. This process involves performing dialogue turn recognition on the dialogue text and voice information corresponding to the dialogue to be identified, obtaining the first and second recognition results respectively. Finally, based on the first and second recognition results, it is determined whether the dialogue turn corresponding to the dialogue to be identified has ended. This avoids the situation where analyzing only the text content leads to an inability to determine the tone of the dialogue, thereby improving the accuracy of dialogue turn end recognition.

[0046] According to the present invention, a method for recognizing the end of a speech turn is provided. Compared with the current method that only analyzes the text content of a user's speech to determine whether the user's speaking turn has ended, the present invention obtains the speech text and voice signal corresponding to the speech to be recognized; inputs the speech text into a first preset speech turn recognition model for speech turn end recognition to obtain a first recognition result corresponding to the speech to be recognized; then inputs the voice signal into a second preset speech turn recognition model for speech turn end recognition to obtain a second recognition result corresponding to the speech to be recognized; finally, based on the first recognition result and the second recognition result, it is determined whether the speech turn corresponding to the speech to be recognized has ended. Thus, by performing speech turn recognition on the speech text and voice information corresponding to the speech to be recognized respectively to obtain the first recognition result and the second recognition result, and finally determining whether the speech turn corresponding to the speech to be recognized has ended based on the first recognition result and the second recognition result, the present invention avoids the situation where analyzing only the text content leads to the inability to judge the tone of the speech, thereby improving the recognition accuracy of speech turn end.

[0047] Furthermore, to better illustrate the process of recognizing the end of a dialogue turn, as a refinement and extension of the above embodiments, this invention provides another method for recognizing the end of a dialogue turn, such as... Figure 2 As shown, the method includes:

[0048] 201. Obtain the text and voice signal of the speech to be recognized.

[0049] Specifically, during a call between the outbound robot and a customer, the conversation is recorded in real time using a recording device. At the same time, the silence time after the customer finishes speaking is recorded. If the silence time is longer than a preset time, the voice signal corresponding to the customer's speech segment is obtained through the recording device. Meanwhile, the voice data corresponding to the speech segment to be recognized recorded by the recording device is converted into text content, i.e., speech text, using an ASR model.

[0050] 202. Determine each character contained in the speech text and determine the embedding vector corresponding to each character.

[0051] In this embodiment of the invention, in order to determine the first recognition result corresponding to the speech to be recognized, it is first necessary to determine each character contained in the speech text. For example, if the speech text is "medical insurance application requirements", then the characters corresponding to the speech text are "medical / insurance / application / requirement / ". Then, word embedding methods such as Word2Vec are used to convert each character in the speech text into an embedding vector, and the embedding vector corresponding to each character is input into a preset natural language model for semantic information recognition to obtain the semantic information vector corresponding to the speech to be recognized. Finally, based on the semantic information vector, the first recognition result corresponding to the speech to be recognized is determined.

[0052] 203. Input the embedded vector into a preset natural language model for semantic information recognition to obtain the semantic information vector corresponding to the utterance to be recognized.

[0053] The preset natural language model is the BERT model. The BERT model includes multiple encoders, each of which is connected end to end. The output of the previous encoder is used as the input of the next encoder. The encoder specifically includes a self-attention layer and a feedforward neural network layer.

[0054] Specifically, in order to determine the first recognition result corresponding to the utterance to be recognized, it is first necessary to extract the semantic information vector corresponding to the utterance to be recognized. Based on this, step 203 specifically includes: inputting the embedding vector into different attention subspaces in the self-attention layer for feature extraction to obtain the first feature vector of the utterance text in the different attention subspaces; multiplying the first feature vector of the utterance text in the different attention subspaces with the weights corresponding to the different attention subspaces and summing them to obtain the output vector of the self-attention layer corresponding to the utterance text; adding the output vector of the self-attention layer and the embedding vector to obtain the second feature vector corresponding to the utterance text; and inputting the second feature vector into the feedforward neural network layer for feature extraction to obtain the semantic information vector corresponding to the utterance to be recognized.

[0055] The first feature vector is the output vector of the self-attention layer, and the semantic information vector corresponding to the utterance to be recognized is the output vector of the feedforward neural network layer of the last encoder.

[0056] Specifically, in the process of extracting the semantic information vector corresponding to the utterance to be recognized using the BERT model, the embedding vector corresponding to each character is first input into the self-attention layer of the first encoder in the BERT model for feature extraction, and the output vector of the self-attention layer is obtained, which is the first feature vector corresponding to each character. The specific process of feature extraction in the self-attention layer is as follows: based on the embedding vector corresponding to each character, the query vector, key vector and value vector corresponding to each character are determined; the query vector corresponding to the target character in each character is multiplied with the key vector corresponding to each character to obtain the attention score of each character for the target character; the attention score corresponding to each character is multiplied with the value vector and summed to obtain the first feature vector corresponding to the target character.

[0057] In this embodiment of the invention, during the process of obtaining the first feature vector corresponding to each character, the embedding vector corresponding to each character in the text corresponding to the utterance to be identified can be multiplied with the weight matrix corresponding to the self-attention layer in the BERT model to obtain the query vector, key vector, and value vector corresponding to each character. Furthermore, it is necessary to calculate the attention score corresponding to each character. When calculating the attention score corresponding to any character (target character) among the characters, it is necessary to score the target character using each character in the utterance text. Specifically, the query vector corresponding to the target character is multiplied with the key vector corresponding to each character to obtain the score value of each character for the target character, i.e., the attention score. Then, the attention score corresponding to each character and the value vector are multiplied and summed to finally obtain the output vector of the self-attention layer corresponding to the target character, i.e., the first feature vector corresponding to the target character. Thus, the first feature vector corresponding to each character can be determined in the above manner so as to obtain the semantic information vector corresponding to the utterance to be identified using the first feature vector corresponding to each character.

[0058] Furthermore, to obtain the semantic information vector corresponding to the utterance to be identified, after inputting the embedding vectors corresponding to each character in the utterance text to the self-attention layer of the first encoder to extract the first feature vector corresponding to each character, the first feature vector needs to be added to the embedding vector corresponding to each character to obtain the second feature vector corresponding to each character. The second feature vector is then input to the feedforward neural network layer of the first encoder for feature extraction to obtain the output vector of the first encoder. Since the BERT model in this embodiment contains multiple encoders and the multiple encoders are connected in a concatenated manner, the output vector of the first encoder is input to the second encoder for feature extraction to obtain the output vector of the second encoder. In this way, the output vector of the previous encoder is used as the input vector of the next encoder, and finally the output vector of the last encoder is determined as the semantic information vector corresponding to the utterance to be identified.

[0059] 204. Input the semantic information vector into the first preset speech round recognition model to perform speech round end recognition, and obtain the first recognition result corresponding to the speech to be recognized.

[0060] Specifically, the first preset dialogue round recognition model can be a multilayer perceptron model, which is a neural network model that includes an input layer, a hidden layer, and an output layer.

[0061] In this embodiment of the invention, after determining the semantic information vector corresponding to the utterance to be identified, it is necessary to determine the first recognition result corresponding to the utterance to be identified based on the semantic information vector. Based on this, step 204 specifically includes: inputting the semantic information vector into the multilayer perceptron and extracting the features output by the last fully connected layer in the multilayer perceptron; inputting the features output by the last fully connected layer into the softmax layer in the multilayer perceptron to obtain the first probability value of the utterance round ending and the second probability value of the utterance round not ending corresponding to the utterance to be identified.

[0062] Specifically, the embedding vectors corresponding to each character in the text of the speech to be identified are input into the hidden layer through the input layer of the multilayer perceptron model, and the output of the hidden layer is:

[0063] f(W1x+b1)

[0064] Where x is the embedding vector corresponding to each character, w1 is the weight of the hidden layer, which is also the connection coefficient of the multilayer perceptron, b1 is the bias coefficient of the hidden layer, and the f function can usually be the sigmoid function or the tanh function, as shown below:

[0065] sigmoid(x) = 1 / (1+e)-x )

[0066] tanh(x)=(e x -e -x ) / (e 1 +e -x )

[0067] Furthermore, after inputting the embedding vectors corresponding to each character in the speech text into the hidden layer of the multilayer perceptron model through the input layer to obtain the output of the hidden layer, the result is then input into the output layer, namely the softmax layer of the multilayer perceptron. The speech round results are then identified through this output layer, and the resulting recommendation is as follows:

[0068] softmax(W2f(W1x+b1)+b2)

[0069] Where W2 is the weight coefficient of the output layer and b2 is the bias coefficient of the output layer, the output layer of this multilayer perceptron model can output the first recognition result corresponding to the speech to be recognized. The first recognition result is actually the classification probability of whether the speech round corresponding to the speech to be recognized has ended. That is, the first recognition result is the first probability value of the speech round to be recognized ending and the second probability value of the speech round not ending.

[0070] 205. Input the voice signal into the second preset speech round recognition model to perform speech round end recognition, and obtain the second recognition result corresponding to the speech to be recognized.

[0071] Specifically, the second preset dialogue round recognition model can be a preset classifier, which is a preset neural network model. The preset neural network model can be a multi-layer model, such as a CNN-LSTM model.

[0072] In this embodiment of the invention, to improve the recognition accuracy of the second preset speech round recognition model, it is first necessary to determine the spectrogram corresponding to the speech signal. Based on this, step 205 specifically includes: performing cross-framing processing on the speech signal to obtain a framed speech signal; performing windowing processing on the framed speech signal to obtain a windowed speech signal; performing Fourier transform on the windowed speech signal to obtain each spectrum vector corresponding to the windowed speech signal; connecting each spectrum vector in parallel along the time axis in a preset coordinate system to obtain the spectrogram corresponding to the speech signal; and inputting the spectrogram into the second preset speech round recognition model for speech round end recognition to obtain the second recognition result corresponding to the speech to be recognized.

[0073] Specifically, during a call, raw voice data is collected by recording the voice stream bytes. First, the voice signal is processed by cross-framing. Then, the framed voice signal is windowed, with each window being 25ms in size and shifted by 10ms. After that, the voice signal in each window undergoes a short-time Fourier transform to obtain the Mel filter bank features of the voice signal, i.e., the spectrum vector, which has 160 dimensions. A preset coordinate system is established, with time as the horizontal axis. The spectrum vectors are then connected in parallel along the time axis of the preset coordinate system to obtain the spectrogram corresponding to the voice signal. Based on the spectrogram, a preset classifier is used to obtain the second recognition result corresponding to the speech to be recognized. Based on this, the method includes: determining the voice feature vector corresponding to the spectrogram; inputting the voice feature vector into the classifier for classification to obtain a third probability value for the speech round to end and a fourth probability value for the speech round not to end.

[0074] Specifically, local and global features in the spectrogram are extracted using the convolutional layers of a preset CNN (Convolutional Neural Network) model, and the local and global features are fused to obtain the speech feature vector corresponding to the spectrogram. Then, the speech feature vector is input into a preset classifier for classification to obtain the classification probability of whether the speech round corresponding to the speech to be identified has ended. That is, the second recognition result is the third probability value of the speech round of the speech to be identified ending and the fourth probability value of the speech round not ending.

[0075] 206. Based on the first recognition result and the second recognition result, determine whether the speech round corresponding to the speech to be recognized has ended.

[0076] In this embodiment of the invention, after determining the first and second recognition results corresponding to the speech to be identified, it is necessary to determine whether the speech round corresponding to the speech to be identified has ended based on the first and second recognition results. Therefore, step 206 specifically includes: determining a first weighting coefficient corresponding to both the first and second probability values, and determining a second weighting coefficient corresponding to both the third and fourth probability values; based on the first and second weighting coefficients, adding the first and third probability values ​​to obtain a first total probability value indicating the speech round corresponding to the speech to be identified has ended, and adding the second and fourth probability values ​​to obtain a second total probability value indicating the speech round corresponding to the speech to be identified has not ended; if the first total probability value is greater than the second total probability value, it is determined that the speech round corresponding to the speech to be identified has ended; if the first total probability value is less than the second total probability value, it is determined that the speech round corresponding to the speech to be identified has not ended.

[0077] Specifically, a first weighting coefficient is pre-set for the recognition results of the first preset dialogue round recognition model, and a second weighting coefficient is set for the recognition results of the second preset dialogue round recognition model. After determining the first probability value of the dialogue round ending and the second probability value of the dialogue round not ending corresponding to the dialogue to be recognized using the first preset dialogue round recognition model, and determining the third probability value of the dialogue round ending and the fourth probability value of the dialogue round not ending corresponding to the dialogue to be recognized using the second preset dialogue round recognition model, the first weighting coefficient is multiplied by the first probability value to obtain a first product, and the second weighting coefficient is multiplied by the third probability value to obtain a second product. The first product and the second product are added together to obtain the first total probability of the dialogue round ending corresponding to the dialogue to be recognized. Simultaneously, the first weighting coefficient is multiplied by the second probability value to obtain a third product, and the second weighting coefficient is multiplied by the fourth probability value to obtain a fourth product. The third product and the fourth product are added together to obtain a second total probability value for the conversation round corresponding to the speech to be identified not being finished. If the first total probability value is greater than the second total probability value, it is determined that the conversation round corresponding to the speech to be identified has ended. If the first total probability value is less than the second total probability value, it is determined that the conversation round corresponding to the speech to be identified has not ended. If the first total probability value is equal to the second total probability value, the maximum probability value is determined between the first probability value and the second probability value, and the identification result corresponding to the maximum probability value is determined as the identification result of whether the conversation round corresponding to the speech to be identified has ended.

[0078] For example, if the script text is input into the first preset script turn recognition model, the first probability value for the end of the script turn corresponding to the script to be recognized is 0.8, and the second probability value for the not-ending script turn corresponding to the script to be recognized is 0.2. If the voice signal is input into the second preset script turn recognition model, the third probability value for the end of the script turn corresponding to the script to be recognized is 0.2, and the fourth probability value for the not-ending script turn corresponding to the script to be recognized is 0.8. The first weighting coefficient corresponding to the first and second probability values ​​is 0.75. The second weighting coefficient corresponding to the third and fourth probability values ​​is 0.25. Therefore, the first total probability value of the end of the dialogue round is calculated to be 0.75×0.8+0.25×0.2=0.65, and the second total probability value of the not-ending dialogue round is 0.75×0.2+0.25×0.8=0.35. It can be seen that the first total probability value is greater than the second total probability value. Since the dialogue round identification result corresponding to the first total probability value is the end of the dialogue round, it is finally determined that the dialogue round corresponding to the dialogue to be identified is the end.

[0079] Furthermore, if the dialogue sequence corresponding to the dialogue to be identified ends, the outbound call robot will respond according to the customer's intention. If the dialogue sequence corresponding to the dialogue to be identified does not end, it will continue to wait for a preset time. If the user continues to speak within the preset waiting time, the dialogue segment that the user continues to speak will be identified as the customer's description of the same content as the previous dialogue segment. The outbound call robot will then make a corresponding response to the description of the same content.

[0080] According to another method for recognizing the end of a speech turn provided by the present invention, compared with the current method of only analyzing the text content of the user's speech to determine whether the user's speaking turn has ended, the present invention obtains the speech text and voice signal corresponding to the speech to be recognized; inputs the speech text into a first preset speech turn recognition model for speech turn end recognition to obtain a first recognition result corresponding to the speech to be recognized; then inputs the voice signal into a second preset speech turn recognition model for speech turn end recognition to obtain a second recognition result corresponding to the speech to be recognized; finally, based on the first recognition result and the second recognition result, it is determined whether the speech turn corresponding to the speech to be recognized has ended. Thus, by performing speech turn recognition on the speech text and voice information corresponding to the speech to be recognized respectively to obtain the first recognition result and the second recognition result, and finally determining whether the speech turn corresponding to the speech to be recognized has ended based on the first recognition result and the second recognition result, this method avoids the situation where analyzing only the text content leads to the inability to judge the tone of the speech, thereby improving the recognition accuracy of speech turn end.

[0081] Furthermore, as Figure 1 In a specific implementation, this invention provides a device for recognizing the end of a speech round, such as... Figure 3 As shown, the device includes: an acquisition unit 31, a first identification unit 32, a second identification unit 33, and a judgment unit 34.

[0082] The acquisition unit 31 can be used to acquire the text and voice signal of the speech to be identified.

[0083] The first recognition unit 32 can be used to input the script text into a first preset script round recognition model for script round end recognition, and obtain the first recognition result corresponding to the script to be recognized.

[0084] The second recognition unit 33 can be used to input the voice signal into the second preset speech round recognition model to perform speech round end recognition and obtain the second recognition result corresponding to the speech to be recognized.

[0085] The judgment unit 34 can be used to determine whether the speech round corresponding to the speech to be recognized has ended based on the first recognition result and the second recognition result.

[0086] In specific application scenarios, in order to determine the first recognition result corresponding to the utterance to be recognized, such as Figure 4 As shown, the first identification unit 32 includes a first determination module 321, a semantic recognition module 322, and a first identification module 323.

[0087] The first determining module 321 can be used to determine each character contained in the speech text and determine the embedding vector corresponding to each character.

[0088] The semantic recognition module 322 can be used to input the embedded vector into a preset natural language model to perform semantic information recognition, and obtain the semantic information vector corresponding to the utterance to be recognized.

[0089] The first recognition module 323 can be used to input the semantic information vector into the first preset speech round recognition model to perform speech round recognition and obtain the first recognition result corresponding to the speech to be recognized.

[0090] In specific application scenarios, in order to determine the semantic information vector corresponding to the spoken text, the semantic recognition module 322 includes a feature extraction submodule and a summation submodule.

[0091] The feature extraction submodule can be used to input the embedding vector into different attention subspaces in the self-attention layer for feature extraction, so as to obtain the first feature vector of the utterance text under the different attention subspaces.

[0092] The summation submodule can be used to multiply the first feature vector of the speech text under different attention subspaces with the weights corresponding to the different attention subspaces and sum them to obtain the self-attention layer output vector corresponding to the speech text.

[0093] Specifically, the summation submodule can be used to add the output vector of the self-attention layer and the embedding vector to obtain the second feature vector corresponding to the utterance text.

[0094] Specifically, the feature extraction submodule can be used to input the second feature vector into the feedforward neural network layer for feature extraction, so as to obtain the semantic information vector corresponding to the utterance to be recognized.

[0095] In a specific application scenario, in order to determine the first recognition result corresponding to the utterance to be recognized, the first recognition module 323 can be specifically used to input the semantic information vector into the multilayer perceptron, extract the features output by the last fully connected layer in the multilayer perceptron, and input the features output by the last fully connected layer into the softmax layer in the multilayer perceptron to obtain the first probability value of the utterance turn ending and the second probability value of the utterance turn not ending corresponding to the utterance to be recognized.

[0096] In specific application scenarios, in order to determine the second recognition result corresponding to the speech to be recognized, the second recognition unit 33 includes a frame processing module 331, a windowing processing module 332, a transformation module 333, a parallel module 334, and a second recognition module 335.

[0097] The frame-segmentation processing module 331 can be used to perform cross-frame-segmentation processing on the speech signal to obtain the framed speech signal.

[0098] The windowing processing module 332 can be used to perform windowing processing on the framed audio signal to obtain a windowed audio signal.

[0099] The transformation module 333 can be used to perform Fourier transform on the windowed speech signal to obtain the various spectral vectors corresponding to the windowed speech signal.

[0100] The parallel module 334 can be used to connect various spectral vectors in parallel along the time axis in a preset coordinate system to obtain the spectrogram corresponding to the speech signal.

[0101] The second recognition module 335 can be used to input the spectrogram into the second preset speech round recognition model for speech round recognition, and obtain the second recognition result corresponding to the speech to be recognized.

[0102] In a specific application scenario, in order to determine the second recognition result corresponding to the speech to be recognized based on the spectrogram, the second recognition module 335 includes a determination submodule and a classification submodule.

[0103] The determining submodule can be used to determine the speech feature vector corresponding to the spectrogram.

[0104] The classification submodule can be used to input the speech feature vector into the classifier for classification, and obtain a third probability value for the completion of the speech round corresponding to the speech to be identified and a fourth probability value for the speech round not being completed.

[0105] In specific application scenarios, in order to determine whether the speech round corresponding to the speech to be identified has ended, the determination unit 34 includes a second determination module 341 and an addition module 342.

[0106] The second determining module 341 can be used to determine a first weighting coefficient that corresponds to the first probability value and the second probability value, and to determine a second weighting coefficient that corresponds to the third probability value and the fourth probability value.

[0107] The addition module 342 can be used to add the first probability value and the third probability value based on the first weight coefficient and the second weight coefficient to obtain a first total probability value that the speech round corresponding to the speech to be identified has ended, and to add the second probability value and the fourth probability value to obtain a second total probability value that the speech round corresponding to the speech to be identified has not ended.

[0108] The second determining module 341 can be specifically used to determine the end of the speech round corresponding to the speech to be identified if the first total probability value is greater than the second total probability value.

[0109] The second determining module 341 can also be used to determine that the speech round corresponding to the speech to be identified has not ended if the first total probability value is less than the second total probability value.

[0110] It should be noted that other corresponding descriptions of the functional modules involved in the speech round end recognition device provided in this embodiment of the invention can be found in [reference]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0111] Based on the above, Figure 1 Accordingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: acquiring the text and voice signal corresponding to the speech to be identified; inputting the text into a first preset speech round recognition model for speech round end recognition to obtain a first recognition result corresponding to the speech to be identified; inputting the voice signal into a second preset speech round recognition model for speech round end recognition to obtain a second recognition result corresponding to the speech to be identified; and determining, based on the first recognition result and the second recognition result, whether the speech round corresponding to the speech to be identified has ended.

[0112] Based on the above, Figure 1 The method shown and as Figure 3 The embodiment of the device shown in the invention also provides a physical structure diagram of a computer device, such as... Figure 5As shown, the computer device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor. Both the memory 42 and the processor 41 are mounted on a bus 43. When the processor 41 executes the program, it performs the following steps: acquiring the text and voice signal corresponding to the speech to be recognized; inputting the text into a first preset speech round recognition model for speech round end recognition, obtaining a first recognition result corresponding to the speech to be recognized; inputting the voice signal into a second preset speech round recognition model for speech round end recognition, obtaining a second recognition result corresponding to the speech to be recognized; and determining whether the speech round corresponding to the speech to be recognized has ended based on the first recognition result and the second recognition result.

[0113] Through the technical solution of this invention, the invention acquires the text and voice signal corresponding to the speech to be identified; inputs the text into a first preset speech round recognition model for speech round end recognition, and obtains a first recognition result corresponding to the speech to be identified; then inputs the voice signal into a second preset speech round recognition model for speech round end recognition, and obtains a second recognition result corresponding to the speech to be identified; finally, based on the first recognition result and the second recognition result, it is determined whether the speech round corresponding to the speech to be identified has ended. Thus, by performing speech round recognition on the text and voice information corresponding to the speech to be identified respectively, the first recognition result and the second recognition result are obtained, and finally, based on the first recognition result and the second recognition result, it is determined whether the speech round corresponding to the speech to be identified has ended. This avoids the situation where analyzing only the text content leads to the inability to judge the tone of the speech, thereby improving the recognition accuracy of speech round end.

[0114] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for recognizing the end of a conversation round, characterized in that, include: Acquire the text and voice signal of the speech to be identified; The script text is input into the first preset script round recognition model to perform script round end recognition, and the first recognition result corresponding to the script to be recognized is obtained. Determine the spectrogram corresponding to the speech signal, extract local and global features from the spectrogram, fuse the local and global features to obtain a speech feature vector, input the speech feature vector into a second preset speech round recognition model for speech round end recognition, and obtain the second recognition result corresponding to the speech to be recognized; Based on the first identification result and the second identification result, it is determined whether the speech round corresponding to the speech to be identified has ended, wherein the first identification result or the second identification result is the probability value of the speech round ending and the probability value of the speech round not ending, and the second identification result is the first probability value of the speech round ending and the second probability value of the speech round not ending.

2. The method according to claim 1, characterized in that, The step of inputting the script text into a first preset script round recognition model for script round end recognition to obtain the first recognition result corresponding to the script to be recognized includes: Determine each character contained in the speech text, and determine the embedding vector corresponding to each character; The embedded vector is input into a preset natural language model for semantic information recognition to obtain the semantic information vector corresponding to the utterance to be recognized. The semantic information vector is input into the first preset speech round recognition model to perform speech round recognition, and the first recognition result corresponding to the speech to be recognized is obtained.

3. The method according to claim 2, characterized in that, The preset natural language model is a preset encoder, which includes a self-attention layer and a feedforward neural network layer. The step of inputting the embedding vector into the preset natural language model for semantic information recognition to obtain the semantic information vector corresponding to the utterance to be recognized includes: The embedded vector is input into different attention subspaces in the self-attention layer for feature extraction, thereby obtaining the first feature vector of the speech text under the different attention subspaces. The first feature vector of the spoken text under different attention subspaces is multiplied by the weights corresponding to the different attention subspaces and summed to obtain the self-attention layer output vector corresponding to the spoken text; The output vector of the self-attention layer and the embedding vector are added together to obtain the second feature vector corresponding to the utterance text; The second feature vector is input into the feedforward neural network layer for feature extraction to obtain the semantic information vector corresponding to the utterance to be identified.

4. The method according to claim 2, characterized in that, The first preset dialogue round recognition model is a multilayer perceptron. The step of inputting the semantic information vector into the first preset dialogue round recognition model for dialogue round completion recognition, and obtaining the first recognition result corresponding to the dialogue to be recognized, includes: The semantic information vector is input into the multilayer perceptron, and the features output by the last fully connected layer in the multilayer perceptron are extracted. The features output by the last fully connected layer are input into the softmax layer of the multilayer perceptron to obtain the first probability value of the speech round ending and the second probability value of the speech round not ending corresponding to the speech to be identified.

5. The method according to claim 1, characterized in that, Determining the spectrogram corresponding to the speech signal includes: The speech signal is subjected to cross-framing processing to obtain the framed speech signal. The framed audio signal is windowed to obtain a windowed audio signal. Perform a Fourier transform on the windowed speech signal to obtain the corresponding spectral vectors of the windowed speech signal; By connecting the various spectral vectors in parallel along the time axis in the preset coordinate system, the spectrogram corresponding to the speech signal is obtained.

6. The method according to claim 5, characterized in that, The second preset dialogue round recognition model is a classifier. The step of inputting the speech feature vector into the second preset dialogue round recognition model for dialogue round end recognition includes: The speech feature vector is input into the classifier for classification to obtain a third probability value for the completion of the speech round and a fourth probability value for the uncompleted speech round corresponding to the speech to be identified.

7. The method according to claim 6, characterized in that, The step of determining whether the speech round corresponding to the speech to be identified has ended based on the first identification result and the second identification result includes: Determine a first weighting coefficient that corresponds to both the first probability value and the second probability value, and determine a second weighting coefficient that corresponds to both the third probability value and the fourth probability value; Based on the first weighting coefficient and the second weighting coefficient, the first probability value and the third probability value are added together to obtain the first total probability value of the completion of the speech round corresponding to the speech to be identified, and the second probability value and the fourth probability value are added together to obtain the second total probability value of the completion of the speech round corresponding to the speech to be identified. If the first total probability value is greater than the second total probability value, then it is determined that the speech round corresponding to the speech to be identified has ended; If the first total probability value is less than the second total probability value, then it is determined that the speech round corresponding to the speech to be identified has not ended.

8. A device for recognizing the end of a speech round, characterized in that, include: The acquisition unit is used to acquire the text and voice signal of the speech to be recognized. The first recognition unit is used to input the script text into a first preset script round recognition model to perform script round end recognition, and obtain the first recognition result corresponding to the script to be recognized. The second recognition unit is used to determine the spectrogram corresponding to the speech signal, extract local features and global features from the spectrogram, fuse the local features and global features to obtain a speech feature vector, input the speech feature vector into the second preset speech round recognition model to perform speech round end recognition, and obtain the second recognition result corresponding to the speech to be recognized. The judgment unit is used to determine whether the speech round corresponding to the speech to be identified has ended based on the first recognition result and the second recognition result, wherein the first recognition result or the second recognition result is a probability value of the speech round ending and a probability value of the speech round not ending, and the second recognition result is a first probability value of the speech round ending and a second probability value of the speech round not ending.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images