Intelligent question and answer method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides an intelligent question and answer method, which comprises the following steps: performing voice recognition on user voice to obtain a recognition text; performing semantic recognition on the recognition text to obtain a semantic recognition result; carrying out fingering word replacement on the recognition text to obtain text information; performing style recognition on the user voice and the text information to obtain a user style; performing style text generation on the user voice, the user style and the information to obtain a reply style and a reply text; and performing voice generation on the reply style and the reply text to obtain a reply voice. The method is applied to an intelligent customer service or assistant scene. According to the invention, voice recognition is carried out through the preset question and answer model, and the accuracy and efficiency of voice recognition are improved. According to the embodiment of the invention, the accuracy of voice content and style recognition is ensured through anonym replacement, so that the accuracy and efficiency of the generated style and text are improved, and the generated reply voice is ensured to accord with the style of the user.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent question-answering method, device, computer equipment and storage medium. Background Art

[0002] At present, intelligent customer service or intelligent assistant is one of the application scenarios of natural language processing technology, which is usually based on retrieval-based question-answering technology, that is, business experts first define some question-answering knowledge bases in advance, a series of question-answering pairs, each standard question corresponds to an answer, and then use semantic matching model technology to find synonymous questions for user questions, and then reply with corresponding answers. However, the question-answering effect of the above solution depends on semantic matching, and is usually presented in text, resulting in poor question-answering effect. Therefore, a technical solution is urgently needed to solve the above problem. Summary of the invention

[0003] Embodiments of the present invention provide an intelligent question-answering method, apparatus, computer device, and storage medium to solve the problem in the prior art that the reply process relies on semantic matching and the reply is presented in text, resulting in poor answering results.

[0004] An intelligent question-answering method, comprising: Acquire the user voice of the target user, and perform voice recognition on the user voice through a voice recognition module in a preset question-answering model to obtain a recognized text; Performing semantic recognition on the recognition text by using the semantic recognition module in the preset question-answering model to obtain a semantic recognition result; Performing pronoun replacement on the recognized text according to the semantic recognition result to obtain text information; Performing style recognition on the user voice and the text information through a style recognition module in the preset question-answering model to obtain a user style; Generating style text from the user voice, the user style and the text information through the generation module in the preset question-answering model to obtain a reply style and a reply text; The reply style and the reply text are voice-synthesized through a voice conversion model to obtain a reply voice corresponding to the user voice.

[0005] An intelligent question-answering device, comprising: A speech conversion module is used to obtain the user speech of the target user, and perform speech recognition on the user speech through the speech recognition module in the preset question-answering model to obtain a recognition text; A semantic recognition module, used to perform semantic recognition on the recognition text through the semantic recognition module in the preset question-answering model to obtain a semantic recognition result; A pronoun replacement module, used for performing pronoun replacement on the recognized text according to the semantic recognition result to obtain text information; A style recognition module, used to perform style recognition on the user voice and the text information through the style recognition module in the preset question-answering model to obtain the user style; An information reply module, used to generate style text from the user voice, the user style and the text information through the generation module in the preset question-answering model to obtain a reply style and a reply text; The text conversion module is used to perform speech synthesis on the reply style and the reply text through a speech conversion model to obtain a reply speech corresponding to the user speech.

[0006] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the intelligent question-answering method when executing the computer program.

[0007] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intelligent question-answering method is implemented.

[0008] The present invention provides an intelligent question-answering method, device, equipment and medium. The method performs semantic recognition and pronoun replacement on the recognized text through a semantic recognition module in a preset question-answering model, thereby realizing recognition of text semantics, thereby realizing replacement of pronouns in the recognized text, ensuring the accuracy of voice content and style recognition, and thus improving the accuracy and efficiency of the generated style and text. The style recognition module in the preset question-answering model performs style recognition on user voice and text information, thereby realizing recognition of user style. The generation module in the preset question-answering model performs style text generation on user style and text information, thereby realizing generation of reply style and generation of reply text, ensuring continuity in voice style, thereby realizing generation of reply voice, reducing computing resources, and thus ensuring that the generated reply voice conforms to the user style. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0010] Figure 1 is a schematic diagram of an application environment of an intelligent question-answering method in one embodiment of the present invention; Figure 2is a flow chart of an intelligent question-answering method in one embodiment of the present invention; Figure 3 is a flow chart of an intelligent question-answering method in another embodiment of the present invention; Figure 4 is a functional block diagram of an intelligent question-answering device in one embodiment of the present invention; Figure 5 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] The intelligent question answering method provided by the embodiment of the present invention can be applied as follows: Figure 1 Specifically, the intelligent question-answering method is applied in an intelligent question-answering device, which includes Figure 1 The client and server shown in the figure communicate with each other through the network to solve the problem in the prior art that local stylization of images cannot achieve the expected effect. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The client, also known as the user end, refers to a program that corresponds to the server and provides classification services to customers. The client can be installed on, but not limited to, various computers, laptops, smart phones, tablets, and portable wearable devices.

[0013] In one embodiment, if Figure 2 As shown, an intelligent question answering method is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps: S10, obtaining the user voice of the target user, and performing voice recognition on the user voice through a voice recognition module in a preset question-answering model to obtain a recognized text.

[0014] Understandably, user voice refers to the sound data of the user after preprocessing, for example, the voice of the user asking questions, or the voice of the user consulting. The preset question-and-answer model refers to the question-and-answer model set in advance, which is obtained by training with a large amount of sample data. The recognized text refers to the text content corresponding to the user voice. The speech recognition module is used to convert the voice into text. The target user refers to the user who asks the question.

[0015] Specifically, the user voice of the target user is obtained, and the user voice is input into the preset question-answer model. The user voice is recognized by the voice recognition module in the preset question-answer model, that is, the user voice is framed, that is, the user voice is divided into fixed time periods (such as 25 milliseconds) to obtain multiple frame units. In order to avoid excessive changes in adjacent frame units, there is an overlapping area between two adjacent frame units. Each frame unit is multiplied by a window function so that the left and right ends of each frame unit are continuous, thereby obtaining a continuous time window. All windowed frame units are subjected to short-time Fourier transform to obtain the spectrum corresponding to each frame unit, that is, to obtain the spectrum distributed in different time windows on the time axis. The spectrum is processed by a Mel filter to obtain a Mel spectrum corresponding to the speech spectrogram, that is, the linear natural spectrum is converted into a Mel spectrum reflecting the characteristics of human hearing. The logarithm of the Mel spectrum is obtained to obtain the logarithmic energy of the Mel spectrum, and the logarithmic energy is inversely transformed according to the discrete cosine transform, and the second to thirteenth coefficients after the discrete cosine transform are taken as the Mel frequency cepstrum coefficients, and the Mel frequency cepstrum coefficients are determined as MFCC features. The MFCC features are pattern matched using the acoustic model, that is, by calculating the distance between the MFCC features and each pronunciation template, the most matching pronunciation template is found to obtain the speech text. The speech text is then corrected and detected by the language model, that is, the recognition results are further judged and corrected according to the grammatical structure, semantic rules, etc., and the recognized text can be obtained. For example, in the financial field, the user asks the smart assistant through the microphone, I want to buy a car insurance, what are your recommendations? After collecting the user's voice and recognizing it, the text content can be obtained, that is, "I want to buy a car insurance, what are your recommendations?".

[0016] S20, performing semantic recognition on the recognition text through the semantic recognition module in the preset question-answering model to obtain a semantic recognition result.

[0017] It can be understood that the semantic recognition module refers to a model used to identify the meaning of sentences in a text. The semantic recognition result refers to the meaning represented by the content in the recognized text identified by the model.

[0018] Specifically, semantic recognition is performed on the recognized text through the semantic recognition module in the preset question-answering model, that is, the recognized text is input into the semantic recognition model, and semantic recognition is performed on the recognized text through the semantic recognition ability learned during training, so as to obtain the semantics represented by the recognized text, and the recognized semantics is determined as the semantic recognition result.

[0019] In a specific embodiment, the recognition text is vectorized, that is, the recognition text is segmented, and the segmented words are vectorized, and the position code of each word is added during the vectorization process to increase the recognition accuracy. Then, the vectorized text is feature extracted through the extraction layer, that is, the features of the vectorized text are extracted according to the semantic relationship and grammatical structure, and the long-distance dependencies in the recognition text are captured through the attention mechanism. Finally, the extracted features and the captured relationships are semantically recognized through the fully connected layer to obtain the semantic recognition result. For example, in the financial field, "I want to buy a car insurance, what do you recommend?", through semantic recognition, it can be known that the user wants to recommend suitable vehicle insurance to him.

[0020] S30, performing pronoun replacement on the recognized text according to the semantic recognition result to obtain text information.

[0021] Understandably, the text information refers to the recognized text after replacing the pronoun, with coherent context information.

[0022] Specifically, the pronouns in the recognized text are replaced by semantic recognition results, that is, the pronouns in the recognized text are recognized, that is, the pronouns in the recognized text are recognized by the preset bag of words model, so as to obtain the pronoun recognition results, and the position of each pronoun in the recognized text is determined according to the pronoun recognition results. Then, based on the context information corresponding to the pronoun and the semantic recognition results corresponding to the context information, the content referred to by the pronoun is replaced, that is, the pronoun is replaced with the specific content referred to. In this way, all pronouns in the recognized text are replaced based on the semantic recognition results to obtain text information. For example, in a financial scenario, after recommending multiple insurances to the user, the user asks about different insurances. When asking about the third insurance, what is the difference between this insurance and the previous one? Among them, this insurance refers to the insurance currently being asked, and the previous one refers to the insurance previously asked. After the replacement, it is the current insurance. What is the difference between it and the previous one? S40, performing style recognition on the user voice and the text information through the style recognition module in the preset question-answering model to obtain the user style.

[0023] It can be understood that the user style refers to the style of the user when speaking. The style recognition module refers to a model used to recognize the speaking style.

[0024] Specifically, the style recognition of the user's voice and text information is performed through the style recognition module in the preset question-answering model, that is, the style recognition of the user's voice is performed through the voice style recognition unit in the style recognition module, that is, the voice style features that can reflect the style are extracted from the user's voice through the voice style recognition unit. At the same time, the style recognition of the text information is performed through the text style recognition unit in the style recognition module, that is, the text style features that can reflect the style are extracted from the text information through the text style recognition unit. Then, the extracted voice style features and text style features are feature fused to obtain the style features, and the style recognition of the user's voice is performed based on the style features, that is, the style features are style predicted to obtain the user's style.

[0025] S50, generating a style text from the user voice, the user style and the text information through a generation module in the preset question-answering model to obtain a reply style and a reply text.

[0026] Understandably, the reply style refers to the style of the reply to the user's voice. The reply text refers to the content of the reply to the user's question or consultation.

[0027] Specifically, the generation module in the preset question-answering model generates style text for the user's voice, user style and text information, that is, the text generation module generates text for the user's voice and text information, that is, the user's question is identified based on the user's voice and text information, and then, the text generation module uses the generation ability learned during training to respond to the identified user's question, that is, to generate a response text corresponding to the user's voice. Similarly, the style generation module generates style for the user's voice and user style, that is, the text generation module uses the generation ability learned during training to respond to the identified user's style, and a response style corresponding to the user's style can be generated.

[0028] S60, performing speech synthesis on the reply style and the reply text through a speech conversion model to obtain a reply speech corresponding to the user speech.

[0029] Understandably, the reply speech refers to the speech synthesized based on the reply text and reply style. The speech conversion model refers to the model used for speech synthesis, that is, converting text into speech.

[0030] Specifically, the reply text and reply style are input into the speech conversion model, and the reply style and reply text are synthesized by the speech conversion model, that is, the reply text is first segmented, that is, the text is divided into words, phrases or sentences, and punctuation marks are added by semantics to determine the sentence structure and pause position, and the reply text is analyzed grammatically and semantically to capture the deep meaning and tone characteristics in the text. Then, the processed text is converted into phonemes to obtain all phonemes corresponding to the reply text, and then the rhythmic features are determined by the reply style, including the stress, rhythm, intonation and other information of the syllables, and all the phonemes and rhythmic features are input into the acoustic model, and the acoustic feature parameters corresponding to the reply text and reply style are generated by the acoustic model. Finally, waveform conversion is performed based on the acoustic feature parameters, that is, the acoustic feature parameters are converted into corresponding audio waveforms by the vocoder, and the audio waveform is converted into speech based on the decoder, so that the reply speech can be obtained.

[0031] In an embodiment of the present invention, an intelligent question-answering method is provided. The method performs semantic recognition and pronoun replacement on the recognized text through a semantic recognition module in a preset question-answering model, thereby realizing recognition of text semantics, thereby realizing replacement of pronouns in the recognized text, ensuring the accuracy of speech content and style recognition, and thereby improving the accuracy and efficiency of the generated style and text. The style recognition module in the preset question-answering model performs style recognition on user speech and text information, thereby realizing recognition of user style. The generation module in the preset question-answering model performs style text generation on user style and text information, thereby realizing generation of reply style and generation of reply text, ensuring continuity in speech style, thereby realizing generation of reply speech, reducing computing resources, and thereby ensuring that the generated reply speech conforms to the user style.

[0032] In one embodiment, if Figure 3 As shown, after step S60, that is, after performing speech synthesis on the reply style and the reply text through the speech conversion model to obtain a reply speech corresponding to the user speech, the method further includes: S70, performing text conversion on the reply voice to obtain a parsed text, and performing text encoding on the parsed text to obtain a parsed text encoding result.

[0033] S80, performing style analysis on the reply speech to obtain a analyzed style, and performing style coding on the analyzed style to obtain a analyzed style coding result.

[0034] S90, after verifying that the parsed text and the parsed style are correct, adding the parsed text encoding result and the parsed style encoding result to the historical text information of the target user.

[0035] Understandably, parsing text refers to the result obtained by performing text recognition on the reply voice. Parsing style refers to the result obtained by performing style recognition on the reply voice. Parsing text encoding result refers to the result of text encoding after reply voice recognition. Parsing style encoding result refers to the result of style encoding of reply voice.

[0036] Specifically, after obtaining the reply voice corresponding to the user voice, a preset recognition model is obtained, and the reply voice is input into the preset recognition model, and the reply voice is voice recognized by the preset recognition model, so as to obtain the parsed text. Alternatively, the reply voice is input into the preset recognition model, and the reply voice is voice recognized by the voice recognition module in the preset recognition model, so as to obtain the parsed text. Then, the parsed text is encoded by the text encoding module in the preset recognition model, that is, the parsed text is segmented, and all the segmentation results are vectorized, and the position encoding of each segmentation result is added to each word vector, so as to obtain the parsed text encoding result. Similarly, a style recognition model is obtained, and the reply voice is input into the style recognition model, and the reply voice is style recognized by the style recognition model, so as to obtain the parsed style. Alternatively, the reply voice is input into the preset recognition model, and the reply voice is style recognized by the style recognition module in the preset recognition model, so as to obtain the parsed style. Then, the parsed style is style encoded by the style encoding module in the preset recognition module, that is, the parsed style is processed by the ability learned during training, so as to obtain the parsed style encoding result. Furthermore, the reply text and reply style output by the preset question-answering model are obtained, and the text style verification of the parsed text and the parsed style is performed based on the reply text and the reply style, that is, the parsed text is verified by the reply text, that is, by calculating the text similarity, it is determined whether the two text contents are exactly the same. If they are not the same, the reply text is determined as the parsed text. At the same time, the parsed style is verified by the reply style to determine whether the reply style and the parsed style are exactly the same. If they are not the same, the reply style is determined as the parsed style. Finally, after verifying that the parsed text and the parsed style are correct, the parsed text encoding result and the parsed style encoding result are added to the historical text information of the target user.

[0037] In the embodiment of the present invention, by recognizing the reply voice, the acquisition of the parsed text and the parsed style is realized, so that the parsed text and the parsed style are added to the text information, thereby ensuring the continuity and solving the problem of voice style disconnection.

[0038] In one embodiment, in step S40, that is, performing style recognition on the user voice and the text information by the style recognition module in the preset question-answering model to obtain the user style includes: S401, encoding the user's voice by a pitch encoding unit in the style recognition module to obtain a pitch encoding sequence.

[0039] S402: Encode the user's voice by using the timbre encoding unit in the style recognition module to obtain a timbre encoding sequence.

[0040] S403: encoding the user speech by the prosody encoding unit in the style recognition module to obtain a prosody encoding sequence.

[0041] S404: Encode the text information by the text encoding unit in the style recognition module to obtain a text encoding sequence.

[0042] S405 , performing style recognition on the user voice based on the pitch coding sequence, the timbre coding sequence, the rhythm coding sequence and the text coding sequence to obtain a user style.

[0043] Understandably, the pitch coding sequence refers to the coding result of the pitch of the user's voice. The timbre coding sequence refers to the coding result of the sound quality and timbre of the user's voice. The rhythm coding sequence refers to the coding result of the rhythm, intonation, speaking speed and other rhythmic features in the user's voice. The text coding sequence refers to the result of coding the text information.

[0044] Specifically, the user's voice is encoded by the pitch encoding unit in the style recognition module, that is, the user's voice is subjected to pre-processing steps such as denoising, filtering, and framing. Framing is to divide the continuous voice signal into shorter frames for frame-by-frame analysis, and the fundamental frequency is extracted from each frame of data using the fundamental frequency algorithm, and then the envelope of the fundamental frequency is extracted to obtain the pitch encoding sequence. Then, the user's voice is encoded by the timbre encoding unit in the style recognition module, that is, the timbre features in the user's voice are extracted, and the extracted timbre features are encoded to obtain the timbre encoding sequence.

[0045] Next, the user's voice is encoded by the prosodic encoding unit in the style recognition module, that is, the prosodic features of the user's voice are extracted, and the extracted prosodic features are encoded to obtain a prosodic encoding sequence. The text information is encoded by the text encoding unit in the style recognition module, that is, the text information is segmented, and then the segmented text is vectorized to obtain a word vector corresponding to the text information, and the position vector of each word is added to the word vector to obtain a text encoding sequence. Finally, the user's voice is styled based on the pitch coding sequence, timbre coding sequence, prosodic coding sequence and text coding sequence, that is, the style recognition ability is learned during training, that is, the pitch coding sequence, timbre coding sequence, text coding sequence and prosodic coding sequence are feature fused or spliced ​​to form a comprehensive feature vector, and then the comprehensive feature vector is style recognized by the style recognition ability to obtain the user's style.

[0046] In the embodiment of the present invention, the encoding of the pitch coding sequence is implemented, the encoding of the timbre coding sequence is implemented, and the encoding of the rhythm coding sequence is implemented, thereby realizing the recognition of the user style, ensuring the accuracy of the user style recognition, and further improving the accuracy of the subsequent replies.

[0047] In one embodiment, in step S50, that is, generating a style text from the user voice, the user style, and the text information by a generation module in the preset question-answering model to obtain a reply style and a reply text, including: S501, performing speech encoding on the user speech through the speech encoding module in the preset question-answering model to obtain a speech embedding sequence.

[0048] S502, performing text encoding on the text information and the historical text information through the text encoding module in the preset question-answering model to obtain a text embedding sequence.

[0049] S503, style encoding the user style through the style encoding module in the preset question-answering model to obtain a style embedding sequence.

[0050] S504, generating and processing the speech embedding sequence, the text embedding sequence and the style embedding sequence through the generation module in the preset question-answering model to obtain a reply style and a reply text.

[0051] Understandably, speech embedding sequence refers to converting speech into a serialized data form that can capture its semantic and temporal features. Text embedding sequence refers to converting words or phrases in the text into corresponding embedding vectors and arranging these vectors in the order of the text to form a serialized representation. Style embedding sequence refers to representing user style features as a series of ordered elements or vectors.

[0052] Specifically, after obtaining the user style, the user voice is voice-encoded by the voice encoding module in the preset question-answering model, that is, the user voice is sampled at a preset sampling frequency to obtain a series of discrete sample values, and then the collected sample values ​​are quantized, and the quantized sample values ​​are encoded to obtain a voice embedding sequence. Then, the text information and historical text information are text-encoded by the text encoding module in the preset question-answering model, that is, the text information is cleaned, segmented, and stop words are removed, so as to improve the accuracy and efficiency of subsequent encoding, and then each word or character in the text is mapped to a high-dimensional vector space, so that the word embedding can capture the semantic relationship between words, so that similar words have a similar distance in the vector space, and finally, all word embeddings are encoded, and the encoding results of the historical text information and the encoding of the text information are spliced ​​to obtain a text embedding sequence. Further, the user style is style-encoded by the style encoding module in the preset question-answering model, that is, the user style is feature-extracted to obtain style features, and then the style features are encoded by the encoding network to obtain a style embedding sequence. Next, the speech embedding sequence, text embedding sequence and style embedding sequence are generated and processed through the generation module in the preset question-answering model. That is, the generation module generates a response style corresponding to the user style based on the speech embedding sequence and the style embedding sequence by using the style generation ability learned during training; at the same time, the generation module generates a response text corresponding to the user's text information based on the speech embedding sequence and the text embedding sequence by using the text generation ability learned during training, and determines the generated response style and response text as the reply style and reply text.

[0053] In the embodiment of the present invention, the encoding of the speech embedding sequence, the encoding of the text embedding sequence, and the encoding of the user style are implemented, thereby realizing the generation of the reply style and reply text, thereby improving the accuracy and efficiency of the generated style and text, and ensuring that the generated reply speech conforms to the user style.

[0054] In one embodiment, step S10, that is, obtaining the user voice of the target user, includes: S101, obtaining original speech, and performing frame processing on the original speech to obtain at least one frame data.

[0055] S102, performing endpoint detection on all the framed data to obtain a starting point and an ending point corresponding to each framed data.

[0056] S103, performing denoising processing on the original speech according to the starting point and the ending point of all the frame data to obtain the user speech.

[0057] Understandably, the original voice refers to the voice of the user's question, or the voice of the user's question and the voice of the reply. The frame data refers to the voice of a fixed segment. The starting point and the ending point refer to the position where each frame data starts and ends.

[0058] Specifically, the original speech is obtained, and all the original speech is cut and processed, that is, the original speech can be divided into segments of speech data by a fixed frequency band (such as 10 seconds, etc.), for example, a segment of 3 minutes of original speech is divided into 18 segments of speech data. Each segmentation unit in the speech data contains the same number of signal sampling points, and the segmentation units are determined as frame data. Then, the energy value of the signal in each frame data is calculated. If the energy value of several consecutive frame data in the front end of each segment of speech data is lower than the preset energy value threshold (the preset energy value threshold can be set according to demand), and the energy value of the next several consecutive frame data is greater than or equal to the preset energy value threshold, then the place where the signal energy value increases is the starting point of the segment of speech data. Similarly, if the energy value of the speech in several consecutive frame data is large, and then the energy value of the speech in several frame data becomes smaller, and it lasts for a certain period of time, it can be considered that the place where the energy value decreases is the end point of the segment of speech data. Thus, the starting point and end point in each segment of frame data are determined. The voice data between the starting point and the ending point in each segment of the frame data is retained, the voice data between each segment of the frame data (between the ending point of the first segment of the frame data and the starting point of the second segment of the frame data) is deleted, and all non-voice data is deleted in sequence. All the retained segmented data are spliced ​​in the order of segmentation to obtain the user voice. For example, among the 18 segments of voice data, the 5th, 9th, and 14th segments of voice are noise or blank voices. They are deleted in order, and the remaining voice data are spliced ​​to obtain the user voice. For example, in the intelligent customer service scenario, after the intelligent customer service obtains the original voice of the user, it frames the voice data and determines the starting point and the ending point in each frame of data, thereby deleting the noise or blank voice in the original voice of the user to obtain the user voice.

[0059] In the embodiment of the present invention, by calculating the energy value of the signal in each segment of framed data and comparing the energy value of the framed data with a preset energy value threshold, the starting point and / or the ending point in each segment of framed data is determined. The voice data is deleted according to the starting point and / or the ending point in each segment of framed data, so as to extract the user's voice, thereby reducing the redundancy of the voice data and improving the accuracy of subsequent style recognition.

[0060] In one embodiment, after step S102, that is, after performing endpoint detection on all the framed data to obtain the starting point and the ending point corresponding to each framed data, the method further includes: S104, determining the interval time between the starting point and the ending point in two consecutive frame data, and comparing the interval time with a preset interval threshold to obtain a time comparison result.

[0061] S105: When the time comparison result indicates that the interval time is greater than the preset interval threshold, determine that the user voice ends.

[0062] It can be understood that the interval time refers to the time interval between the end point and the start point in different frame data.

[0063] Specifically, after obtaining the starting point and the ending point corresponding to each frame data, the time between the starting point and the ending point in two consecutive different frame data is counted to obtain the interval time, so as to determine the interval time between the starting point and the ending point in each two consecutive different frame data. Then, the preset interval threshold is obtained, and each interval time is compared with the preset interval threshold, so as to obtain the time comparison result corresponding to each interval time. When the time comparison result indicates that the interval time is less than or equal to the preset interval threshold, the original voice is denoised to obtain the user voice. When the time comparison result indicates that the interval time is greater than the preset interval threshold, it is determined that the user voice ends. In this way, the meaning represented by all time comparison results is determined, so as to determine the position where the user voice ends. For example, in the financial field, when asking about a certain insurance, the silence time will be long because of considering or recalling the name of the insurance. At this time, the last voice will be retained and the subsequent voice will be recorded.

[0064] In the embodiment of the present invention, by determining the interval time between the starting point and the ending point in two consecutive frame data, the time between the starting point and the ending point is counted. By comparing the interval time and the preset interval threshold, the time comparison result is obtained, and then the end of the speech is determined.

[0065] In one embodiment, before step S10, that is, before performing speech recognition on the user speech by using a preset question-answer model to obtain text information corresponding to the user speech, the following steps are included: S106: Acquire a sample data set, where the sample data set includes at least one sample training data.

[0066] Understandably, the sample training data may be audio information corresponding to each user, or multiple audio information of a user. The sample training data may be collected from different databases, or may be pre-prepared data sent from the client to the database, and then a sample training data set may be constructed based on all the acquired sample training data.

[0067] S107, identifying and predicting all the sample training data using a preset training model to obtain a prediction result corresponding to each of the sample training data.

[0068] Understandably, the prediction results are used to characterize the response style and response information obtained by identifying and predicting the sample training data.

[0069] Specifically, all sample training data are input into a preset training model, and all sample training data are identified and predicted by the preset training model. First, the sample training data is speech recognized by the speech recognition module in the preset training model to obtain sample recognition texts corresponding to each sample training data. Then, semantic recognition is performed on the sample recognition texts, and the pronouns in the sample recognition texts are replaced by the semantic recognition results to obtain sample text information. The style recognition module in the preset training model is used to perform style recognition on each sample training data to obtain a sample recognition style. Next, the generation module in the preset training model is used to generate style texts for the sample training data, the sample recognition style, and the sample text information, that is, a style corresponding to the sample recognition style and a text corresponding to the sample text information are generated, and the generated style and text are determined as prediction results. In this way, prediction results corresponding to each of the sample training data can be obtained.

[0070] S108, performing loss processing on all the prediction results using a preset loss function to obtain a predicted loss value.

[0071] Understandably, the prediction loss value is generated during the prediction process of the sample training data.

[0072] Specifically, after obtaining the prediction label, the preset loss function is obtained, and the loss calculation is performed on all prediction results through the preset loss function to obtain the loss value corresponding to each sample training data. Then, based on all loss values, the total loss value generated by the preset training model during the prediction process is calculated, that is, all loss values ​​can be directly summed up, or all loss values ​​can be weighted to obtain the prediction loss value corresponding to the preset training model.

[0073] S109: When the predicted loss value reaches a preset convergence condition, the preset training model after convergence is determined as a preset question-answering model.

[0074] Understandably, the convergence condition may be that the predicted loss value is less than a set threshold, or that the predicted loss value is very small and will not decrease after 50,000 calculations, and the training is stopped.

[0075] Specifically, after obtaining the predicted loss value, when the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted according to the predicted loss value, and all sample training data are re-input into the preset training model with the adjusted initial parameters, and the preset training model with the adjusted initial parameters is iteratively trained to obtain the predicted loss value corresponding to the preset training model with the adjusted initial parameters. Furthermore, when the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted again according to the predicted loss value, so that the predicted loss value of the preset training model with the adjusted initial parameters reaches the preset convergence condition. In this way, the accuracy of the preset training model becomes higher and higher, and the quality inspection results obtained are constantly approaching the correct results, until the predicted loss value of the preset training model reaches the preset convergence condition, and the preset training model after convergence is determined as the preset question-and-answer model.

[0076] In the embodiment of the present invention, the preset training model is iteratively trained through a large amount of sample training data, and the overall loss value of the preset training model is calculated by comparing the loss function, thereby realizing the determination of the predicted loss value of the preset training model. The initial parameters of the preset training model are adjusted according to the predicted loss value until the model converges, thereby realizing the training of the preset question-answering model, thereby ensuring that the preset question-answering model has a higher expected effect.

[0077] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0078] In one embodiment, an intelligent question-answering device is provided, which corresponds one-to-one to the intelligent question-answering method in the above embodiment. Figure 4As shown, the intelligent question-answering device includes a speech conversion module 10, a semantic recognition module 20, a pronoun replacement module 30, a style recognition module 40, an information reply module 50 and a text conversion module 60. The functional modules are described in detail as follows: The speech conversion module 10 is used to obtain the user speech of the target user, and perform speech recognition on the user speech through the speech recognition module in the preset question-answer model to obtain a recognition text; A semantic recognition module 20, configured to perform semantic recognition on the recognition text through the semantic recognition module in the preset question-answering model to obtain a semantic recognition result; A pronoun replacement module 30, configured to replace the recognized text with pronouns according to the semantic recognition result to obtain text information; A style recognition module 40, configured to perform style recognition on the user voice and the text information through the style recognition module in the preset question-answering model to obtain the user style; An information reply module 50 is used to generate a style text from the user voice, the user style and the text information through the generation module in the preset question-answering model to obtain a reply style and a reply text; The text conversion module 60 is used to perform speech synthesis on the reply style and the reply text through a speech conversion model to obtain a reply speech corresponding to the user speech.

[0079] In one embodiment, the device further comprises: A text parsing module is used to perform text conversion on the reply voice to obtain a parsed text, and to perform text encoding on the parsed text to obtain a parsed text encoding result; A style analysis module, used for performing style analysis on the reply speech to obtain a analysis style, and performing style encoding on the analysis style to obtain a analysis style encoding result; The information adding module is used to add the parsed text encoding result and the parsed style encoding result to the historical text information of the target user after verifying that the parsed text and the parsed style are correct.

[0080] In one embodiment, the information reply module 50 includes: A speech encoding unit, used for performing speech encoding on the user speech through the speech encoding module in the preset question-answering model to obtain a speech embedding sequence; A text encoding unit, used to perform text encoding on the text information and the historical text information through a text encoding module in the preset question-answering model to obtain a text embedding sequence; A style encoding unit, used to perform style encoding on the user style through a style encoding module in the preset question-answering model to obtain a style embedding sequence; An information processing unit is used to generate and process the speech embedding sequence, the text embedding sequence and the style embedding sequence through a generation module in the preset question-answering model to obtain a reply style and a reply text.

[0081] In one embodiment, the style identification module 40 includes: A pitch encoding unit, used for encoding the user's voice through the pitch encoding unit in the style recognition module to obtain a pitch encoding sequence; A timbre encoding unit, used for encoding the user's voice through the timbre encoding unit in the style recognition module to obtain a timbre encoding sequence; A prosody coding unit, configured to encode the user's speech through the prosody coding unit in the style recognition module to obtain a prosody coding sequence; A text encoding unit, used for encoding the text information through the text encoding unit in the style recognition module to obtain a text encoding sequence; The style recognition unit is used to perform style recognition on the user speech based on the pitch coding sequence, the timbre coding sequence, the rhythm coding sequence and the text coding sequence to obtain the user style.

[0082] In one embodiment, the voice conversion module 10 includes: A frame processing unit, used for acquiring original speech, and performing frame processing on the original speech to obtain at least one frame data; An endpoint detection unit, used to perform endpoint detection on all the framed data to obtain a starting point and an end point corresponding to each of the framed data; The denoising processing unit is used to perform denoising processing on the original voice according to the starting point and the ending point of all the frame data to obtain the user voice.

[0083] In one embodiment, the voice conversion module 10 further includes: A threshold comparison unit, used to determine the interval time between the starting point and the ending point in two consecutive frame data, and compare the interval time with a preset interval threshold to obtain a time comparison result; The speech ending unit is used to determine that the user speech ends when the time comparison result indicates that the interval time is greater than the preset interval threshold.

[0084] In one embodiment, the device further comprises: A sample acquisition unit, used to acquire a sample data set, wherein the sample data set includes at least one sample training data; An identification and prediction unit, used to identify and predict all the sample training data through a preset training model to obtain a prediction result corresponding to each of the sample training data; A loss prediction unit, used to perform loss processing on all the prediction results through a preset loss function to obtain a predicted loss value; The model convergence unit is used to determine the preset training model after convergence as the preset question-answering model when the predicted loss value reaches the preset convergence condition.

[0085] For the specific definition of the intelligent question-answering device, please refer to the definition of the intelligent question-answering method above, which will not be repeated here. Each module in the above-mentioned intelligent question-answering device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0086] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used by the intelligent question and answer method in the above embodiment. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an intelligent question and answer method is implemented.

[0087] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned intelligent question-answering method when executing the computer program.

[0088] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned intelligent question-answering method when executed by a processor.

[0089] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0090] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0091] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. An intelligent question-answering method, characterized in that: include: Acquire the user voice of the target user, and perform voice recognition on the user voice through a voice recognition module in a preset question-answering model to obtain a recognized text; Performing semantic recognition on the recognition text by using the semantic recognition module in the preset question-answering model to obtain a semantic recognition result; Performing pronoun replacement on the recognized text according to the semantic recognition result to obtain text information; Performing style recognition on the user voice and the text information through a style recognition module in the preset question-answering model to obtain a user style; Generating style text from the user voice, the user style and the text information through the generation module in the preset question-answering model to obtain a reply style and a reply text; The reply style and the reply text are voice-synthesized through a voice conversion model to obtain a reply voice corresponding to the user voice.

2. The intelligent question-answering method according to claim 1, characterized in that: After performing speech synthesis on the reply style and the reply text through the speech conversion model to obtain a reply speech corresponding to the user speech, the method further includes: Performing text conversion on the reply voice to obtain a parsed text, and performing text encoding on the parsed text to obtain a parsed text encoding result; Performing style analysis on the reply speech to obtain a parsed style, and performing style encoding on the parsed style to obtain a parsed style encoding result; After verifying that the parsed text and the parsed style are correct, the parsed text encoding result and the parsed style encoding result are added to the historical text information of the target user.

3. The intelligent question-answering method according to claim 1, characterized in that: The generating module in the preset question-answering model generates style text from the user voice, the user style and the text information to obtain a reply style and a reply text, including: Performing speech encoding on the user's speech through the speech encoding module in the preset question-answering model to obtain a speech embedding sequence; Performing text encoding on the text information and the historical text information through a text encoding module in the preset question-answering model to obtain a text embedding sequence; Style encoding the user style is performed by a style encoding module in the preset question-answering model to obtain a style embedding sequence; The speech embedding sequence, the text embedding sequence and the style embedding sequence are generated and processed by the generation module in the preset question-answering model to obtain a reply style and a reply text.

4. The intelligent question-answering method according to claim 1, wherein: The performing style recognition on the user voice and the text information by the style recognition module in the preset question-answering model to obtain the user style includes: The pitch encoding unit in the style recognition module encodes the user's voice to obtain a pitch encoding sequence; The user's voice is encoded by the timbre encoding unit in the style recognition module to obtain a timbre encoding sequence; The user's speech is encoded by the prosody encoding unit in the style recognition module to obtain a prosody encoding sequence; The text information is encoded by the text encoding unit in the style recognition module to obtain a text encoding sequence; The user's voice is style-recognized based on the pitch coding sequence, the timbre coding sequence, the rhythm coding sequence and the text coding sequence to obtain a user style.

5. The intelligent question-answering method according to claim 1, wherein: The step of obtaining the user voice of the target user includes: Acquire original speech, and perform frame processing on the original speech to obtain at least one frame data; Performing endpoint detection on all the framed data to obtain a starting point and an ending point corresponding to each of the framed data; The original speech is subjected to denoising processing according to the starting point and the ending point of all the frame data to obtain the user speech.

6. The intelligent question-answering method according to claim 5, characterized in that: After performing endpoint detection on all the framed data to obtain the starting point and the ending point corresponding to each of the framed data, the method further includes: Determine the interval time between the starting point and the ending point in two consecutive frame data, and compare the interval time with a preset interval threshold to obtain a time comparison result; When the time comparison result indicates that the interval time is greater than the preset interval threshold, it is determined that the user voice ends.

7. The intelligent question-answering method according to claim 1, wherein: Before performing speech recognition on the user speech by using a preset question-answer model to obtain text information corresponding to the user speech, the method includes: Acquire a sample data set, wherein the sample data set includes at least one sample training data; Identify and predict all the sample training data using a preset training model to obtain a prediction result corresponding to each of the sample training data; Perform loss processing on all the prediction results using a preset loss function to obtain a predicted loss value; When the predicted loss value reaches a preset convergence condition, the preset training model after convergence is determined as the preset question-answering model.

8. An intelligent question-answering device, characterized in that: include: A speech conversion module is used to obtain the user speech of the target user, and perform speech recognition on the user speech through the speech recognition module in the preset question-answering model to obtain a recognition text; A semantic recognition module, used to perform semantic recognition on the recognition text through the semantic recognition module in the preset question-answering model to obtain a semantic recognition result; A pronoun replacement module, used for performing pronoun replacement on the recognized text according to the semantic recognition result to obtain text information; A style recognition module, used to perform style recognition on the user voice and the text information through the style recognition module in the preset question-answering model to obtain the user style; An information reply module, used to generate style text from the user voice, the user style and the text information through the generation module in the preset question-answering model to obtain a reply style and a reply text; The text conversion module is used to perform speech synthesis on the reply style and the reply text through a speech conversion model to obtain a reply speech corresponding to the user speech.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the intelligent question-answering method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the intelligent question-answering method according to any one of claims 1 to 7 is implemented.

Images