Voice response method, device, computer equipment and computer readable storage medium

Abstract

The present invention relates to the field of speech synthesis technology of artificial intelligence technology. The present invention provides a speech response method, apparatus, computer equipment, and computer-readable storage medium, wherein the method comprises: when interacting with a user's voice, receiving the user's voice data, inputting the voice data into a pre-trained voice extraction model, determining the voice features of the voice data, converting the voice data into speech text, inputting the speech text into a pre-trained intent recognition model, determining the user's intent in the speech text, determining a reply text based on the user's intent, synthesizing a reply speech based on the reply text and the voice features, and sending the reply speech to the user. The present invention synthesizes corresponding reply speech based on the user's voice features, thereby achieving different voice replies for different users' voice data, improving the flexibility of voice question and answer, and thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of speech synthesis technology of artificial intelligence technology, and in particular to a speech response method, apparatus, computer equipment and computer-readable storage medium. Background Art

[0002] With the development of artificial intelligence, robots are playing an increasingly important role. For example, many companies currently use machine customer service to conduct voice Q&A with customers using preset voice Q&A methods, which greatly improves service efficiency.

[0003] However, existing customer service robots all use a unified voice question-and-answer method, and use the same answer method for the same questions raised by different customers, resulting in overly mechanical answers. Therefore, the flexibility of voice question-and-answer is poor, affecting the user experience. Summary of the Invention

[0004] The main purpose of the present invention is to provide a voice answering method, apparatus, computer equipment and computer-readable storage medium to enhance the flexibility of voice question and answer and improve the user experience.

[0005] In order to achieve the above object of the invention, the present invention provides a voice response method, which includes:

[0006] When interacting with the user's voice, receive the user's voice data;

[0007] Inputting the speech data into a pre-trained speech extraction model to determine speech features of the speech data;

[0008] Converting the speech data into speech text, inputting the speech text into a pre-trained intent recognition model, and determining the user intent of the speech text;

[0009] Determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features;

[0010] The response voice is sent to the user.

[0011] Preferably, the voice feature includes a voiceprint feature, and synthesizing the answering voice according to the reply text and the voice feature includes:

[0012] Fusing the reply text with the voiceprint feature to obtain a fused feature;

[0013] The fusion features are synthesized into audio using a vocoder to obtain a response voice.

[0014] Preferably, the voice feature includes an intonation feature, and synthesizing the answering voice according to the reply text and the voice feature includes:

[0015] Querying a target intonation feature corresponding to the intonation feature from a database;

[0016] Convert the reply text into a corresponding text vector;

[0017] Determining a vector corresponding to the target intonation feature to obtain an intonation feature vector;

[0018] splicing the text vector and the intonation feature vector and inputting the concatenated vector into an acoustic model to obtain corresponding acoustic features;

[0019] The acoustic features are converted into playable response speech.

[0020] In one embodiment, converting the voice data into voice text includes:

[0021] Preprocessing the voice data to obtain voice information;

[0022] Extracting content from the voice information to obtain voice content;

[0023] Decoding the speech content using a Viterbi algorithm to obtain an optimal speech path;

[0024] The speech text is determined according to the optimal speech path.

[0025] Preferably, inputting the voice text into a pre-trained intent recognition model to determine the user intent of the voice text includes:

[0026] Inputting the speech text into a pre-trained intent recognition model, calling the intent recognition model to perform sentence processing on the speech text to obtain multiple sentences;

[0027] Query the sentence vector corresponding to each sentence from the pre-built vector table, calculate the average sentence vector corresponding to multiple sentences, and obtain the average sentence vector;

[0028] Obtain multiple standard sentence vectors from a pre-built standard sentence library;

[0029] Calculating the similarity between each of the standard sentence vectors and the average sentence vector;

[0030] The standard sentence vector with the highest similarity is determined as the target sentence vector, and the intention corresponding to the target sentence vector is determined as the user intention of the speech text.

[0031] Furthermore, before inputting the voice data into a pre-trained intent recognition model to determine the user intent of the voice text, the method further includes:

[0032] Acquire training data; wherein the training data includes multiple speech and text samples and target intent recognition results corresponding to each speech and text sample;

[0033] The training data is input into a pre-built neural network model for training to obtain a trained intent recognition model.

[0034] Preferably, inputting the training data into a pre-built neural network model for training to obtain a trained intent recognition model includes:

[0035] Inputting the training data into a pre-built neural network model for training, and calculating the loss value of the trained neural network model using a preset loss function;

[0036] Determining whether the loss value is lower than a preset loss value;

[0037] If so, it is determined that the training result of the neural network model meets the requirements, and the neural network model whose training result meets the requirements is used as the trained intent recognition model.

[0038] The present invention also provides a voice response device, comprising:

[0039] A receiving module, used to receive user voice data when interacting with the user's voice;

[0040] An input module, configured to input the speech data into a pre-trained speech extraction model to determine speech features of the speech data;

[0041] A determination module, configured to convert the voice data into voice text, input the voice text into a pre-trained intent recognition model, and determine the user intent of the voice text;

[0042] A synthesis module, configured to determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features;

[0043] A sending module is used to send the response voice to the user.

[0044] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0045] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0046] The present invention provides a voice response method, device, computer equipment and computer-readable storage medium. When interacting with user voice, the method receives user voice data, inputs the voice data into a pre-trained voice extraction model, determines the voice features of the voice data, converts the voice data into voice text, inputs the voice text into a pre-trained intent recognition model, determines the user intent of the voice text, determines a reply text based on the user intent, synthesizes a reply voice based on the reply text and the voice features, and sends the reply voice to the user. Since the voice features of different users may be different, the present invention combines the user's voice features to synthesize corresponding reply voices, thereby making different voice replies to the voice data of different users, improving the flexibility of voice questions and answers, and thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 1 is a flow chart of a voice response method according to an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of a flow chart of a voice response method according to another embodiment of the present invention;

[0049] Figure 3 Schematic diagram of a flow chart of a voice response method according to another embodiment of the present invention;

[0050] Figure 4 1 is a flow chart of a voice response method according to another embodiment of the present invention;

[0051] Figure 5 Schematic diagram of a flow chart of a voice response method according to another embodiment of the present invention;

[0052] Figure 6 1 is a flow chart of a voice response method according to another embodiment of the present invention;

[0053] Figure 7 Schematic diagram of a flow chart of a voice response method according to another embodiment of the present invention;

[0054] Figure 8 This is a schematic block diagram of the structure of a voice response device according to an embodiment of the present invention;

[0055] Figure 9 The figure is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0056] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] This invention proposes a voice response method. Embodiments of the invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0059] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0060] The present invention proposes a voice response method, which uses a server as the execution body. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0061] This voice response method is used to solve the technical problem that current customer service robots all use a unified voice Q&A method, using the same answer method for the same question raised by different customers, the answer is too mechanical, the flexibility of voice Q&A is poor, and it affects the user experience. Figure 1 In one embodiment, the voice response method includes the following steps S11-S15:

[0062] S11. Receive user voice data during voice interaction with the user;

[0063] S12, inputting the voice data into a pre-trained voice extraction model to determine the voice features of the voice data;

[0064] S13, converting the voice data into voice text, inputting the voice text into a pre-trained intent recognition model, and determining the user intent of the voice text;

[0065] S14, determining a reply text according to the user's intention, and synthesizing a response voice according to the reply text and the voice features;

[0066] S15: Send the response voice to the user.

[0067] As described in step S11 above, during voice interaction processes such as telephone sales, after-sales service, and intelligent question-and-answer, the user's client can send voice data to the server. The voice data may contain questions raised by the user. The server receives the user's voice data, identifies the questions raised by the user in the voice data, and queries the database for the standard answer corresponding to the question, and responds based on the standard answer to complete the voice response.

[0068] As described in step S12 above, this embodiment inputs the voice data into a pre-trained voice extraction model and uses the voice extraction model to determine the voice features of the voice data. The voice extraction model is a pre-trained neural network model that is used to extract the voice features of the voice data. The voice features include the user's voiceprint, intonation, and speaking rate, which are characteristics that reflect the user's pronunciation.

[0069] In one embodiment, when training a speech extraction model, a large number of speech data samples and determined standard speech features for each speech data sample may be pre-acquired. The speech data samples and the corresponding standard speech features are used as training samples. The training samples are input into a pre-built initial neural network model for iterative training. When the initial neural network model meets the iteration conditions, the training results of the initial neural network model are obtained, and it is determined whether the training results meet the requirements. If so, the trained initial neural network model is used as the trained speech extraction model. The iteration conditions may include the training time or the number of training times.

[0070] Furthermore, training samples and trained speech extraction models can be stored on the blockchain, allowing the server to broadcast the data to the blockchain and call the trained intent recognition model. This blockchain represents a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of network transactions, used to verify the validity of the information (for anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product and service layer, and an application service layer.

[0071] As described in step S13 above, this embodiment converts the voice data into voice text. In order to obtain the voice text in the voice data and facilitate subsequent voice text processing, in this embodiment, the voice data needs to be converted into text. Therefore, a text recognition model can be used to perform text recognition processing on the voice data to obtain the voice text in the voice data. In addition, the voice data can be divided into multiple voice segments, and ASR (Automatic Speech Recognition) technology can be used to convert each voice segment in the voice data into text, ultimately forming the voice text.

[0072] Furthermore, in order to ensure the accuracy of text recognition, the voice text can also be corrected to obtain a standard text, wherein the text correction processing corrects text errors of preset error types in the voice text, and the error types include: homophonic words, such as: get a pair of glasses - get a pair of glasses; confusing words.

[0073] After obtaining the voice text, the voice text is input into a pre-trained intention recognition model to determine the user intention of the voice text, wherein the voice extraction model is a pre-trained neural network model for determining the user intention of the voice text.

[0074] As described in steps S14-S15 above, this embodiment can construct a comparison table for querying the corresponding reply text for each user intent. This reply text is generally a standard reply content. When a reply is needed, the corresponding reply text is searched from the comparison table based on the user intent. A reply speech is synthesized based on the reply text and voice features, and the synthesized reply speech is sent to the user to reply to the user's voice data.

[0075] Among them, speech synthesis refers to the artificial generation of human voices through text, that is, allowing the device to emit speech based on the corresponding input text. In recent years, neural speech synthesis systems have developed rapidly, breaking away from the traditional cumbersome synthesis process and no longer relying on highly professional grammar and lexical feature extraction. Instead, the original text is converted into Mel spectrum through a neural network, and then the Mel spectrum is converted into a sound waveform, realizing an end-to-end speech synthesis system. The synthesized speech quality is close to that of human voice, and it has very important applications in the artificial intelligence industry, such as voice assistants and translation in various scenarios, audiobooks, news broadcasts, AI phones and AI anchors.

[0076] This embodiment can use a speech synthesis model to synthesize a response speech. The speech synthesis model (Text To Speech, TTS) refers to a model used to convert text into speech. The speech synthesis model may include: an encoding layer, an attention mechanism, and a fully connected layer. The encoding layer is used to perform vector position encoding on the response text to convert the response text into a text vector and record the position information of the text vector. The attention mechanism is used to extract audio features from the text vector to generate a sound spectrum. The fully connected layer is used to output the sound spectrum to generate the response speech. This embodiment uses speech synthesis technology and neural network technology to analyze the user's voice data, so that the output response speech is closer to the user's own voice characteristics, making it more friendly.

[0077] A voice response method provided by the present invention receives user voice data during voice interaction with the user, inputs the voice data into a pre-trained voice extraction model, determines the voice features of the voice data, converts the voice data into voice text, inputs the voice text into a pre-trained intent recognition model, determines the user intent of the voice text, determines a reply text based on the user intent, synthesizes a reply voice based on the reply text and the voice features, and sends the reply voice to the user. Since the voice features of different users may be different, the present invention synthesizes corresponding reply voices in combination with the user's voice features, thereby making different voice replies to the voice data of different users, improving the flexibility of voice questions and answers, and thus improving the user experience.

[0078] In one embodiment, reference Figure 2 As shown, the voice feature includes a voiceprint feature, and synthesizing the answer voice according to the reply text and the voice feature may specifically include the following steps S21-S22:

[0079] S21, fusing the reply text with the voiceprint feature to obtain a fused feature;

[0080] S22. Perform audio synthesis on the fused features using a vocoder to obtain a response voice.

[0081] In this embodiment, the decoder in the speech conversion model can be used to fuse the reply text and the voiceprint features to obtain fused features, and the vocoder in the speech conversion model can be used to perform audio synthesis on the fused features to obtain the response speech.

[0082] The speech conversion model is used to fuse the reply text with the voiceprint features so that the semantics of the reply text can be expressed through the voiceprint of the user's speech. It includes a vocoder and a decoder. The decoder is used to fuse the reply text with the voiceprint features; the vocoder is used to perform audio synthesis on the fused features to obtain the answering speech.

[0083] In this embodiment, the fusion features are synthesized using a vocoder to obtain a synthesis result of the reply text and the voiceprint features of any user. This allows the synthesized voice of the reply text to be characterized by the timbre of any user's voice, forming a customized user-specific reply voice for the reply text, making the output reply voice closer to the user's own voiceprint features and thus more friendly. The vocoder is constructed using the Griffin-Lim algorithm, which reconstructs the speech waveform signal through the amplitude spectrum and phase spectrum to achieve audio recognition. Therefore, in this embodiment, the amplitude spectrum and phase spectrum are performed based on the fusion features, and the vocoder is used to reconstruct the speech signal of the fusion features to obtain the reply voice.

[0084] In one embodiment, when the reply text and the voiceprint feature are fused, the reply text can be converted into a vector form to obtain a reply text vector, the voiceprint feature can be converted into a vector form to obtain a voiceprint feature vector, and the reply text vector and the voiceprint feature vector can be combined to obtain a fusion feature.

[0085] In one embodiment, reference Figure 3 As shown, the voice feature includes an intonation feature, and synthesizing the answer voice according to the reply text and the voice feature may specifically include the following steps S31-S35:

[0086] S31, querying a database for a target intonation feature corresponding to the intonation feature;

[0087] S32, converting the reply text into a corresponding text vector;

[0088] S33, determining a vector corresponding to the target intonation feature to obtain an intonation feature vector;

[0089] S34, concatenating the text vector and the intonation feature vector and inputting the concatenated vector into an acoustic model to obtain corresponding acoustic features;

[0090] S35: Convert the acoustic features into a playable response voice.

[0091] In this embodiment, the database pre-records the target intonation feature corresponding to each intonation feature. For example, when the user has a higher intonation feature, the corresponding target intonation feature is a lower intonation feature, so as to select the appropriate intonation feature to synthesize the response voice, thereby easing the atmosphere.

[0092] After determining the target intonation feature, this embodiment can use the word2vec model to convert the reply text into a vector form to obtain the text vector corresponding to the reply text, and determine the vector corresponding to the target intonation feature to obtain the intonation feature vector. The text vector and the intonation feature vector are spliced ​​and input into the acoustic model to obtain the corresponding acoustic feature.

[0093] The word2vec model represents words in natural language as dense real-valued vectors. The word2vec model is a three-layer neural network capable of performing word co-occurrence analysis. Training the word2vec model with a large corpus transforms the processing of words in the corpus into operations on vectors in a K-dimensional vector space. The similarity between vectors can be used to represent word similarity, and thus, the semantic similarity of text. Therefore, the vectors obtained from training the word2vec model can be used for a wide range of natural language processing tasks.

[0094] The acoustic model can adopt an encoder-decoder structure, mainly built using CNN, bidirectional LSTM and linear layers. Specifically, it includes an encoding unit and a decoding unit. The encoding unit includes a 2-layer CNN model and a 2-layer bidirectional LSTM model, and a self-attention mechanism is added to learn the intervals between words to better learn the alignment relationship between text and acoustic features. In addition, a ReLU (Rectified Linear Unit) activation function is used for nonlinear activation between each two layers of the network. The decoding unit includes a 1-layer CNN model and a 3-layer bidirectional LSTM model. A fully connected layer is connected after the last layer of bidirectional LSTM to generate acoustic features of a specified dimension, such as generating 160-dimensional acoustic features. Among them, the acoustic features are Mel spectrum features.

[0095] The playable response voice incorporates the user's intonation characteristics, meaning it plays the synthesized response voice in the same sentence structure as the user. For example, if the user's sentence is a question, the response voice will be played in the question's intonation; if the user's sentence is a declarative sentence, the response voice will be played in the declarative intonation.

[0096] In one embodiment, reference Figure 4 As shown, the converting of the voice data into voice text may specifically include the following steps S41-S44:

[0097] S41, preprocessing the voice data to obtain voice information;

[0098] S42, extracting the content of the voice information to obtain voice content;

[0099] S43, using the Viterbi algorithm to decode the voice content to obtain an optimal voice path;

[0100] S44. Determine the speech text according to the optimal speech path.

[0101] This embodiment can remove interference information from the voice data by preprocessing the voice data to obtain voice information, and then use the Viterbi algorithm to decode the voice content in the voice information to obtain the optimal voice path. According to the optimal voice path, the voice content can be accurately converted into the text information.

[0102] The Viterbi algorithm is a dynamic programming algorithm used to find the most likely sequence of hidden states (called the Viterbi path) that led to a sequence of observed events, particularly in the context of Markov information sources and hidden Markov models. The "Viterbi path" and "Viterbi algorithm" are also used to find dynamic programming algorithms that are most likely to explain observations. For example, in statistical parsing, dynamic programming algorithms can be used to find the most likely context-independent derivative (parse) of a string.

[0103] In one embodiment, reference Figure 5 As shown, the step of inputting the voice text into a pre-trained intention recognition model to determine the user intention of the voice text may specifically include the following steps S51-S55:

[0104] S51, inputting the speech text into a pre-trained intention recognition model, calling the intention recognition model to perform sentence processing on the speech text to obtain multiple sentences;

[0105] S52. Query the sentence vector corresponding to each sentence from the pre-built vector table, calculate the average of the sentence vectors corresponding to multiple sentences, and obtain the average sentence vector;

[0106] S53, obtaining multiple standard sentence vectors from a pre-built standard sentence library;

[0107] S54, calculating the similarity between each of the standard sentence vectors and the average sentence vector;

[0108] S55: Determine the standard sentence vector with the highest similarity as the target sentence vector, and determine the intention corresponding to the target sentence vector as the user intention of the speech text.

[0109] When the intention cannot be determined by regular expressions, the speech text can be processed by calling the intention recognition model to obtain multiple sentences, and the sentence vector corresponding to each sentence is queried from the pre-built vector table. The average value of the sentence vectors corresponding to the multiple sentences is calculated to obtain the average sentence vector, and multiple standard sentence vectors are obtained from the pre-built standard sentence library. The similarity between each of the standard sentence vectors and the average sentence vector is calculated, and the standard sentence vector with the highest similarity is determined as the target sentence vector, and the intention corresponding to the target sentence vector is determined as the user intention of the speech text, so that the user intention of the speech text can be accurately determined, ensuring that the intention of the speech data can be recognized, thereby improving the accuracy of intent recognition. Wherein, the vector table includes a mapping relationship between sentences and vectors, and the standard sentence library stores multiple standard sentence vectors and the intention corresponding to each standard sentence vector.

[0110] In one embodiment, reference Figure 6 As shown, before inputting the voice data into a pre-trained intent recognition model to determine the user intent of the voice text, the following steps S61-S62 may be further included:

[0111] S61, obtaining training data; wherein the training data includes a plurality of speech and text samples and a target intent recognition result corresponding to each speech and text sample;

[0112] S62: Input the training data into a pre-built neural network model for training to obtain a trained intent recognition model.

[0113] The training data in this embodiment includes multiple speech and text samples and the target intent recognition results corresponding to each speech and text sample. The training samples are used to train a pre-built neural network model, and the parameters of the neural network model are continuously adjusted during the training process until a qualified neural network model is obtained. The amount of training data must be greater than a preset data volume. The greater the amount of training data, the more convergent the trained intent recognition model and the higher the recognition accuracy.

[0114] When the amount of training data is lower than a preset value, the data enhancement algorithm can be used to amplify the training data to obtain training data with a data amount greater than the preset value. The data enhancement algorithm can be to randomly cut the speech text sample to obtain multiple sub-speech text samples, and then arbitrarily select two segments from the multiple sub-speech text samples for splicing to obtain a segment of speech text sample after amplification, and use it as training data; in addition, the text in the speech text sample can also be randomly replaced to obtain a segment of speech text sample after amplification, and use it as training data. The present invention does not make specific limitations.

[0115] In one embodiment, reference Figure 7 As shown, the inputting of the training data into a pre-built neural network model for training to obtain a trained intent recognition model may specifically include the following steps S71-S73:

[0116] S71, inputting the training data into a pre-built neural network model for training, and calculating the loss value of the trained neural network model using a preset loss function;

[0117] S72, determining whether the loss value is lower than a preset loss value;

[0118] S73. If yes, determine that the training result of the neural network model meets the requirements, and use the neural network model whose training result meets the requirements as the trained intent recognition model.

[0119] In this embodiment, after training the neural network model, a preset loss function can be used to calculate the loss value of the trained neural network model. When the loss value meets a preset threshold or is less than the preset loss value, the training requirement is met, indicating that the neural network model has met the training requirements and the training of the neural network model is completed, thereby improving the recognition effect of the intent recognition model. The loss function is used to evaluate the degree of difference between the predicted value and the true value of the neural network model. The better the loss function, the better the performance of the neural network model.

[0120] When the loss value of the neural network model is not less than the preset loss value, the loss value can be used to perform forward propagation in the neural network structure of the neural network model, adjust the relevant parameters of the neural network model, and retrain the adjusted neural network model based on the reset relevant parameters until the loss value of the neural network model is less than the preset loss value. At this point, the training of the neural network model is completed, and a neural network model whose training results meet the requirements is obtained, and then a trained intent recognition model is obtained.

[0121] Reference Figure 8 , an embodiment of the present invention further provides a voice response device, the device comprising:

[0122] The receiving module 11 is used to receive the user's voice data when interacting with the user's voice;

[0123] An input module 12 is configured to input the speech data into a pre-trained speech extraction model to determine speech features of the speech data;

[0124] A determination module 13 is configured to convert the voice data into voice text, input the voice text into a pre-trained intention recognition model, and determine the user intention of the voice text;

[0125] A synthesis module 14 is configured to determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features;

[0126] The sending module 15 is configured to send the response voice to the user.

[0127] In this embodiment, during voice interaction processes such as telephone sales, after-sales service, and intelligent question-and-answer sessions, the user's client can send voice data to the server. The voice data may contain questions raised by the user. The server receives the user's voice data, identifies the questions raised by the user in the voice data, and queries the database for the standard answer corresponding to the question. It then responds based on the standard answer to complete the voice response.

[0128] In this embodiment, voice data is input into a pre-trained voice extraction model and the voice extraction model is used to determine the voice features of the voice data. The voice extraction model is a pre-trained neural network model that is used to extract voice features of the voice data. The voice features include the user's voiceprint, intonation, and speaking rate, which are characteristics that reflect the user's pronunciation.

[0129] In one embodiment, when training a speech extraction model, a large number of speech data samples and determined standard speech features for each speech data sample may be pre-acquired. The speech data samples and the corresponding standard speech features are used as training samples. The training samples are input into a pre-built initial neural network model for iterative training. When the initial neural network model meets the iteration conditions, the training results of the initial neural network model are obtained, and it is determined whether the training results meet the requirements. If so, the trained initial neural network model is used as the trained speech extraction model. The iteration conditions may include the training time or the number of training times.

[0130] Furthermore, training samples and trained speech extraction models can be stored on the blockchain, allowing the server to broadcast the data to the blockchain and call the trained intent recognition model. This blockchain represents a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of network transactions, used to verify the validity of the information (for anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product and service layer, and an application service layer.

[0131] This embodiment converts speech data into speech text. To obtain the speech text in the speech data and facilitate subsequent speech text processing, this embodiment requires converting the speech data into text. Therefore, a text recognition model can be used to perform text recognition processing on the speech data to obtain the speech text in the speech data. In addition, the speech data can be divided into multiple speech segments, and ASR (Automatic Speech Recognition) technology can be used to convert each speech segment in the speech data into text, ultimately forming the speech text.

[0132] Furthermore, in order to ensure the accuracy of text recognition, the voice text can also be corrected to obtain a standard text, wherein the text correction processing corrects text errors of preset error types in the voice text, and the error types include: homophonic words, such as: get a pair of glasses - get a pair of glasses; confusing words.

[0133] After obtaining the voice text, the voice text is input into a pre-trained intention recognition model to determine the user intention of the voice text, wherein the voice extraction model is a pre-trained neural network model for determining the user intention of the voice text.

[0134] In this embodiment, a comparison table can be constructed to query the corresponding reply text for each user intent. The reply text is generally a standard reply content. When a reply is needed, the corresponding reply text is searched from the comparison table based on the user intent. A reply voice is synthesized based on the reply text and the voice features, and the synthesized reply voice is sent to the user to reply to the user's voice data.

[0135] Among them, speech synthesis refers to the artificial generation of human voices through text, that is, allowing the device to emit speech based on the corresponding input text. In recent years, neural speech synthesis systems have developed rapidly, breaking away from the traditional cumbersome synthesis process and no longer relying on highly professional grammar and lexical feature extraction. Instead, the original text is converted into Mel spectrum through a neural network, and then the Mel spectrum is converted into a sound waveform, realizing an end-to-end speech synthesis system. The synthesized speech quality is close to that of human voice, and it has very important applications in the artificial intelligence industry, such as voice assistants and translation in various scenarios, audiobooks, news broadcasts, AI phones and AI anchors.

[0136] This embodiment can use a speech synthesis model to synthesize a response speech. The speech synthesis model (Text To Speech, TTS) refers to a model used to convert text into speech. The speech synthesis model may include: an encoding layer, an attention mechanism, and a fully connected layer. The encoding layer is used to perform vector position encoding on the response text to convert the response text into a text vector and record the position information of the text vector. The attention mechanism is used to extract audio features from the text vector to generate a sound spectrum. The fully connected layer is used to output the sound spectrum to generate the response speech. This embodiment uses speech synthesis technology and neural network technology to analyze the user's voice data, so that the output response speech is closer to the user's own voice characteristics, making it more friendly.

[0137] As described above, it can be understood that the various components of the voice response device proposed in the present invention can realize the functions of any of the voice response methods described above, and the specific structure will not be repeated here.

[0138] Reference Figure 9 The present invention also provides a computer device in an embodiment, the internal structure of which can be as follows Figure 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The computer-designed processor is used to provide computing and control capabilities. The memory of the computer device includes a computer-readable storage medium and an internal memory. The computer-readable storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the computer-readable storage medium. The database of the computer device is used to store relevant data of the voice response method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a voice response method is implemented.

[0139] The processor executes the voice response method, including:

[0140] When interacting with the user's voice, receive the user's voice data;

[0141] Inputting the speech data into a pre-trained speech extraction model to determine speech features of the speech data;

[0142] Converting the speech data into speech text, inputting the speech text into a pre-trained intent recognition model, and determining the user intent of the speech text;

[0143] Determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features;

[0144] The response voice is sent to the user.

[0145] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a voice response method is implemented, comprising the steps of:

[0146] When interacting with the user's voice, receive the user's voice data;

[0147] Inputting the speech data into a pre-trained speech extraction model to determine speech features of the speech data;

[0148] Converting the speech data into speech text, inputting the speech text into a pre-trained intent recognition model, and determining the user intent of the speech text;

[0149] Determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features;

[0150] The response voice is sent to the user.

[0151] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the computer program can include the processes of the above-described embodiment methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0152] In summary, the greatest beneficial effects of the present invention are:

[0153] The present invention provides a voice response method, device, computer equipment and computer-readable storage medium. When interacting with user voice, the method receives user voice data, inputs the voice data into a pre-trained voice extraction model, determines the voice features of the voice data, converts the voice data into voice text, inputs the voice text into a pre-trained intent recognition model, determines the user intent of the voice text, determines a reply text based on the user intent, synthesizes a reply voice based on the reply text and the voice features, and sends the reply voice to the user. Since the voice features of different users may be different, the present invention combines the user's voice features to synthesize corresponding reply voices, thereby making different voice replies to the voice data of different users, improving the flexibility of voice questions and answers, and thus improving the user experience.

[0154] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0155] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A voice response method, characterized in that: The method comprises: When interacting with the user's voice, receive the user's voice data; Inputting the speech data into a pre-trained speech extraction model to determine speech features of the speech data; Converting the speech data into speech text, inputting the speech text into a pre-trained intent recognition model, and determining the user intent of the speech text; Determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features; Sending the response voice to the user; The step of inputting the voice text into a pre-trained intent recognition model to determine the user intent of the voice text includes: Inputting the speech text into a pre-trained intent recognition model, calling the intent recognition model to perform sentence processing on the speech text to obtain multiple sentences; Query the sentence vector corresponding to each sentence from the pre-built vector table, calculate the average sentence vector corresponding to multiple sentences, and obtain the average sentence vector; Obtain multiple standard sentence vectors from a pre-built standard sentence library; Calculating the similarity between each of the standard sentence vectors and the average sentence vector; Determining the standard sentence vector with the highest similarity as a target sentence vector, and determining the intent corresponding to the target sentence vector as the user intent of the speech text; wherein the vector table includes a mapping relationship between sentences and vectors, and the standard sentence library stores multiple standard sentence vectors and the intent corresponding to each standard sentence vector; Before inputting the voice data into a pre-trained intent recognition model to determine the user intent of the voice text, the method further includes: Acquire training data; wherein the training data includes multiple speech and text samples and target intent recognition results corresponding to each speech and text sample; The training data is input into a pre-built neural network model for training to obtain a trained intent recognition model.

2. The method according to claim 1, characterized in that The voice feature includes a voiceprint feature, and synthesizing the answering voice according to the reply text and the voice feature includes: Fusing the reply text with the voiceprint feature to obtain a fused feature; The fusion features are synthesized into audio using a vocoder to obtain a response voice.

3. The method according to claim 1, characterized in that The voice feature includes an intonation feature, and synthesizing the answering voice according to the reply text and the voice feature includes: Querying a target intonation feature corresponding to the intonation feature from a database; Convert the reply text into a corresponding text vector; Determining a vector corresponding to the target intonation feature to obtain an intonation feature vector; splicing the text vector and the intonation feature vector and inputting the concatenated vector into an acoustic model to obtain corresponding acoustic features; The acoustic features are converted into playable response speech.

4. The method according to claim 1, wherein The converting of the voice data into voice text comprises: Preprocessing the voice data to obtain voice information; Extracting content from the voice information to obtain voice content; Decoding the speech content using a Viterbi algorithm to obtain an optimal speech path; The speech text is determined according to the optimal speech path.

5. The method according to claim 1, wherein Inputting the training data into a pre-built neural network model for training to obtain a trained intent recognition model includes: Inputting the training data into a pre-built neural network model for training, and calculating the loss value of the trained neural network model using a preset loss function; Determining whether the loss value is lower than a preset loss value; If so, it is determined that the training result of the neural network model meets the requirements, and the neural network model whose training result meets the requirements is used as the trained intent recognition model.

6. A voice response device, characterized in that: The device comprises: A receiving module, used to receive user voice data when interacting with the user's voice; An input module, configured to input the speech data into a pre-trained speech extraction model to determine speech features of the speech data; A determination module, configured to convert the voice data into voice text, input the voice text into a pre-trained intent recognition model, and determine the user intent of the voice text; A synthesis module, configured to determine a reply text according to the user's intention, and synthesize a response voice according to the reply text and the voice features; A sending module, configured to send the response voice to the user; The step of inputting the voice text into a pre-trained intent recognition model to determine the user intent of the voice text includes: Inputting the speech text into a pre-trained intent recognition model, calling the intent recognition model to perform sentence processing on the speech text to obtain multiple sentences; Query the sentence vector corresponding to each sentence from the pre-built vector table, calculate the average sentence vector corresponding to multiple sentences, and obtain the average sentence vector; Obtain multiple standard sentence vectors from a pre-built standard sentence library; Calculating the similarity between each of the standard sentence vectors and the average sentence vector; Determining the standard sentence vector with the highest similarity as a target sentence vector, and determining the intent corresponding to the target sentence vector as the user intent of the speech text; wherein the vector table includes a mapping relationship between sentences and vectors, and the standard sentence library stores multiple standard sentence vectors and the intent corresponding to each standard sentence vector; Before inputting the voice data into a pre-trained intent recognition model to determine the user intent of the voice text, the method further includes: Acquire training data; wherein the training data includes multiple speech and text samples and target intent recognition results corresponding to each speech and text sample; The training data is input into a pre-built neural network model for training to obtain a trained intent recognition model.

7. A computer device, characterized in that: include: processor; Memory; The memory stores a computer program, and the processor implements the voice response method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the voice response method according to any one of claims 1 to 5.

Images