A voice processing method, device, equipment, medium, and program product

Abstract

The present application provides a speech processing method, which includes: a transmitting end of a speech communication system collects a user's speech, then recognizes the user's speech, obtains a recognition result, and the recognition result includes at least pronunciation prompt information, and then sends the recognition result to a receiving end of the speech communication system, so that the receiving end performs speech synthesis according to the recognition result, thereby obtaining a speech that does not include background noise. In this way, the method achieves 100% elimination of background noise, does not lose speech signals, and ensures speech quality.

Description

Technical Field

[0001] The present application relates to the field of voice communication technology, and in particular to a voice processing method, apparatus, device, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of communication technology, voice communication has gradually become a mainstream communication method. Voice communication specifically refers to a communication method in which a user sends voice through an electronic device to communicate with the other end user. At present, there is usually background noise in the voice communication process, which interferes with the user's voice, reduces the user's communication efficiency, and affects the user experience.

[0003] The industry has proposed speech noise reduction algorithms to reduce the interference of background noise on speech. One speech noise reduction algorithm is based on multiple microphones to determine the spatial information of noise, such as the amplitude and phase of noise, and then suppress noise based on the spatial information; another speech noise reduction algorithm is based on deep learning methods, which conducts a large amount of learning on various noises to establish noise samples, and then subtracts noise from the original sound signal to achieve the purpose of noise reduction and improve speech quality.

[0004] However, the above-mentioned speech noise reduction algorithm still cannot eliminate the noise component 100%. Moreover, the above-mentioned noise reduction algorithm will damage the speech signal and reduce the speech quality. Summary of the invention

[0005] The present application provides a speech processing method, in which a transmitting end in a speech communication system sends a recognition result excluding background noise to a receiving end, and the receiving end performs speech synthesis according to the pronunciation prompt information in the recognition result, and the synthesized speech is free of any background noise, thereby achieving 100% elimination of background noise, and the method does not damage the speech signal, thereby ensuring the speech quality. The present application also provides a device, an apparatus, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0006] In a first aspect, the present application provides a speech processing method, which is applied to a speech communication system, wherein the system includes a transmitting end and a receiving end, and the method includes:

[0007] The sending end collects the user's voice;

[0008] The sending end recognizes the user's voice to obtain a recognition result, wherein the recognition result at least includes pronunciation prompt information;

[0009] The sending end sends the recognition result to the receiving end, so that the receiving end performs speech synthesis according to the recognition result.

[0010] In some possible implementations, the sending end sending the recognition result to the receiving end includes:

[0011] The sending end sends the recognition result and the voiceprint information of the user to the receiving end.

[0012] In some possible implementations, the pronunciation prompt information includes phonemes;

[0013] The sending end recognizes the user's voice to obtain a recognition result, including:

[0014] The sending end recognizes the user's voice through an acoustic model to obtain the phonemes corresponding to the voice.

[0015] In some possible implementations, the sending end also obtains the pronunciation time or the pronunciation interval time of the phoneme corresponding to the speech through the acoustic model, and the recognition result also includes the pronunciation time or the pronunciation interval time.

[0016] In some possible implementations, the pronunciation prompt information includes text information having the same pronunciation as the speech.

[0017] In some possible implementations, the text information is obtained by the sending end identifying phonemes through an acoustic model and then decoding the phonemes through a language model.

[0018] In some possible implementations, the method further includes:

[0019] encrypting the recognition result;

[0020] The sending end sends the recognition result to the receiving end, including:

[0021] The sending end sends the encrypted recognition result to the receiving end.

[0022] In some possible implementations, the method further includes:

[0023] The user's voice is filtered according to the user's voiceprint information.

[0024] In some possible implementations, the sending end sending the recognition result to the receiving end includes:

[0025] The sending end generates a character stream according to the recognition result;

[0026] The sending end transmits the character stream to the receiving end in a streaming sequential transmission manner.

[0027] In a second aspect, the present application provides a speech processing method, which is applied to a speech communication system, wherein the system includes a transmitting end and a receiving end, and the method includes:

[0028] The receiving end obtains a recognition result of the user's voice, wherein the recognition result at least includes pronunciation prompt information;

[0029] The receiving end performs speech synthesis according to the pronunciation prompt information to obtain synthesized speech;

[0030] The receiving end plays the synthesized voice to achieve real-time voice communication.

[0031] In some possible implementations, the method further includes:

[0032] Get the user's voiceprint information;

[0033] The receiving end performs speech synthesis according to the pronunciation prompt information to obtain synthesized speech, including:

[0034] The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain synthesized speech.

[0035] In some possible implementations, the pronunciation prompt information includes phonemes;

[0036] The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain synthesized speech, including:

[0037] The receiving end synthesizes the initial speech according to the pronunciation prompt information;

[0038] The receiving end obtains a synthesized speech corresponding to the user according to the initial speech and the voiceprint information of the user.

[0039] In some possible implementations, the recognition result further includes the pronunciation time or pronunciation interval time of the phoneme;

[0040] The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user, including:

[0041] The receiving end performs speech synthesis according to one of the pronunciation time of the phoneme or the pronunciation interval time and the phoneme and the voiceprint information of the user.

[0042] In some possible implementations, the pronunciation prompt information includes text information having the same pronunciation as the speech.

[0043] In some possible implementations, the text information is obtained by the sending end identifying phonemes through an acoustic model and then decoding the phonemes through a language model.

[0044] In some possible implementations, the receiving end obtains the recognition result of the user's voice, including:

[0045] The receiving end receives the encrypted recognition result sent by the sending end;

[0046] The receiving end decrypts the encrypted recognition result to obtain the recognition result.

[0047] In some possible implementations, the recognition result is transmitted in the form of a character stream;

[0048] The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user, including:

[0049] The receiving end performs speech synthesis according to the order of the character stream, the pronunciation prompt information and the voiceprint information of the user.

[0050] In a third aspect, the present application provides a speech processing device, comprising:

[0051] A collection unit, used for collecting the user's voice;

[0052] A recognition unit, configured to recognize the user's voice and obtain a recognition result, wherein the recognition result at least includes pronunciation prompt information;

[0053] The sending unit is used to send the recognition result to the receiving end, so that the receiving end performs speech synthesis according to the recognition result.

[0054] In some possible implementations, the sending unit is used for the sending end to send the recognition result and the voiceprint information of the user to the receiving end.

[0055] In some possible implementations, the pronunciation prompt information includes phonemes;

[0056] The recognition unit is used to recognize the user's voice through an acoustic model to obtain the phonemes corresponding to the voice.

[0057] In some possible implementations, the recognition unit further obtains the pronunciation time or the pronunciation interval time of the phoneme corresponding to the speech through the acoustic model, and the recognition result also includes the pronunciation time or the pronunciation interval time.

[0058] In some possible implementations, the pronunciation prompt information includes text information having the same pronunciation as the speech.

[0059] In some possible implementations, the recognition unit is used to recognize phonemes through an acoustic model, and then decode the phonemes through a language model to obtain the text information.

[0060] In some possible implementations, the device further includes:

[0061] An encryption unit, used for encrypting the recognition result;

[0062] The sending unit is used to send the encrypted recognition result to the receiving end.

[0063] In some possible implementations, the device further includes:

[0064] A filtering unit is used to filter the user's voice according to the user's voiceprint information.

[0065] In some possible implementations, the sending unit is configured to generate a character stream according to the recognition result, and transmit the character stream to the receiving end in a streaming sequential transmission manner.

[0066] In a fourth aspect, the present application provides a speech processing device, comprising:

[0067] An acquisition unit, used to acquire a recognition result of the user's voice, wherein the recognition result at least includes pronunciation prompt information;

[0068] A synthesis unit, used for performing speech synthesis according to the pronunciation prompt information to obtain synthesized speech;

[0069] The playing unit is used to play the synthesized speech to realize real-time voice communication.

[0070] In some possible implementations, the acquisition unit is used to acquire the voiceprint information of the user;

[0071] The synthesis unit is used to perform speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain synthesized speech.

[0072] In some possible implementations, the pronunciation prompt information includes phonemes;

[0073] The synthesis unit is used to synthesize an initial voice according to the pronunciation prompt information, and obtain a synthesized voice corresponding to the user according to the initial voice and the voiceprint information of the user.

[0074] In some possible implementations, the recognition result further includes the pronunciation time or pronunciation interval time of the phoneme;

[0075] The synthesis unit is used to perform speech synthesis according to one of the pronunciation time of the phoneme or the pronunciation interval time and the phoneme and the voiceprint information of the user.

[0076] In some possible implementations, the pronunciation prompt information includes text information having the same pronunciation as the speech.

[0077] In some possible implementations, the text information is obtained by the sending end identifying phonemes through an acoustic model and then decoding the phonemes through a language model.

[0078] In some possible implementations, the device further includes a decryption unit;

[0079] The acquiring unit is used to receive the encrypted recognition result sent by the sending end;

[0080] The decryption unit is used to decrypt the encrypted recognition result to obtain the recognition result.

[0081] In some possible implementations, the recognition result is transmitted in the form of a character stream;

[0082] The synthesis unit is used to perform speech synthesis according to the pronunciation prompt information and the voiceprint information of the user in the order of the character stream.

[0083] In a fifth aspect, the present application provides a device, comprising a processor and a memory. The processor and the memory communicate with each other. The processor is used to execute instructions stored in the memory so that the device performs a speech processing method in any implementation of the first aspect or the second aspect.

[0084] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions instruct a device to execute the speech processing method described in any one of the implementations of the first aspect or the second aspect.

[0085] In a seventh aspect, the present application provides a computer program product comprising instructions, which, when executed on a device, enables the device to execute the speech processing method described in any one of the implementations of the first or second aspects above.

[0086] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.

[0088] Figure 1 An architectural diagram of a voice communication system provided in an embodiment of the present application;

[0089] Figure 2 A command interaction diagram of a voice processing method provided in an embodiment of the present application;

[0090] Figure 3 A command interaction diagram of a voice processing method provided in an embodiment of the present application;

[0091] Figure 4 A command interaction diagram of a voice processing method provided in an embodiment of the present application;

[0092] Figure 5 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application;

[0093] Figure 6 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application;

[0094] Figure 7 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0095] Figure 8 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0096] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0097] First, some technical terms involved in the embodiments of the present application are introduced.

[0098] Voice communication refers to a communication method in which two parties communicate through voice, for example, a user sends voice through an electronic device to communicate with the other party. A typical application scenario of voice communication is real-time voice communication. Real-time voice communication is a method of transmitting voice over a network to achieve a scene like a face-to-face conversation. Real-time voice communication has high requirements for voice transmission delay. Specifically, real-time voice communication requires both parties to transmit voice with low delay.

[0099] Real-time voice communication can be a voice conversation between user 1 and user 2 after the electronic device of user 1 is connected to the electronic device of user 2. For example, user 1 and user 2 perform real-time voice communication based on a call service provided by a telecommunications operator, or based on a call service provided by an Internet application service provider.

[0100] When a user uses an electronic device for voice communication, the user's voice collected by the electronic device is often mixed with some background noise. For example, when a user uses an electronic device for voice communication on a busy street, the electronic device will collect the sound of vehicles driving on the street. For another example, when a user uses an electronic device for voice communication in a crowd, the electronic device can collect the voices of other users. The above background noise will interfere with the user's voice, thereby reducing the user's communication efficiency and affecting the user's voice communication experience.

[0101] At present, the industry uses voice noise reduction algorithms to process voice signals collected by electronic devices to reduce the interference of background noise on the user's voice. For example, in some voice noise reduction algorithms, the voice noise reduction algorithm is based on multiple microphones to determine the spatial information of the noise, such as the amplitude and phase of the noise, and then suppress the noise based on the spatial information. For another example, in other voice noise reduction algorithms, the voice noise reduction algorithm is based on deep learning methods, and a large amount of learning is done on various noises to establish noise samples, and then the noise is subtracted from the original sound signal, thereby achieving the purpose of noise reduction and improving voice quality.

[0102] However, none of the above-mentioned speech noise reduction algorithms can eliminate 100% of the noise components, and after the speech signal collected by the electronic device is processed by the above-mentioned speech noise reduction algorithm, the speech signal will be damaged, thereby reducing the speech quality and affecting the user's voice communication experience.

[0103] In view of this, an embodiment of the present application provides a speech processing method. The speech processing method can be applied to a speech communication system, which includes a transmitting end and a receiving end. Specifically, the transmitting end collects the user's speech, first recognizes the user's speech, obtains a recognition result including pronunciation prompt information, and then sends the recognition result and the user's voiceprint information to the receiving end. After the receiving end receives the recognition result and the user's voiceprint information, it can perform speech synthesis according to the pronunciation prompt information in the recognition result and the user's voiceprint information, thereby obtaining speech that does not include background noise.

[0104] On the one hand, the sending end recognizes the user's voice in advance to obtain a recognition result including pronunciation prompt information, and then sends the recognition result and the user's voiceprint information to the receiving end. Since the recognition result sent by the sending end to the receiving end does not include any noise components such as background noise, the receiving end performs speech synthesis based on the pronunciation prompt information in the recognition result and the user's voiceprint information, so that the synthesized speech is free of any noise components, and theoretically can achieve 100% elimination of noise components. In this way, the method reduces the interference of background noise on the user's voice, improves the user's communication efficiency, and thus improves the user's voice communication experience.

[0105] Furthermore, the receiving end restores the user's voice by voice synthesis, for example, the receiving end performs voice synthesis based on the pronunciation prompt information in the recognition result and the user's voiceprint information. This method does not damage the voice signal, and through the voice synthesis method, the receiving end can restore high-quality voice signals, thereby enabling the user to hear clearer voice, thereby improving the user's voice communication experience.

[0106] On the other hand, the sender sends the recognition result with a smaller amount of data and the user's voiceprint information to the receiver instead of sending the voice data with a larger amount of data to the receiver. This can reduce the amount of data sent from the sender to the receiver, thereby reducing the requirements for the network environment of the sender and the receiver. When the sender and the receiver are in a poor network environment, the user can also have high-quality voice communication.

[0107] The voice processing method provided in the embodiment of the present application can be applied to real-time voice communication scenarios (such as voice calls) and can also be applied to non-real-time voice communication scenarios (such as voice messages). In real-time voice communication scenarios, low-delay voice transmission is required between the sender and the receiver, such as a voice call (such as a phone call) between a user at the sender and a user at the receiver. In non-real-time voice communication scenarios, the communication parties have lower requirements for the delay of voice transmission, such as a user at the sender sending a voice message to a user at the receiver. In some implementations, the user at the sender can use a communication application (such as an instant messaging application) installed on the sender to send a voice message to the user at the receiver. After receiving the voice message, the user at the receiver can choose the time to play the voice message as needed.

[0108] It should be noted that the embodiments of the present application do not limit the application scenarios of the voice processing method. The following is only an introduction using the application of the voice processing method to a real-time voice communication scenario as an example.

[0109] In order to make the technical solution of the present application clearer and easier to understand, the architecture of the voice communication system provided in the embodiment of the present application is introduced below with reference to the accompanying drawings.

[0110] like Figure 1 As shown in the architecture diagram of the voice communication system, the voice communication system 100 includes a terminal 10 and a terminal 20. Figure 1 It is only schematically illustrated that the terminal 10 or the terminal 20 may be a mobile phone. Of course, the terminal 10 or the terminal 20 includes but is not limited to a tablet computer, a laptop computer, a personal digital assistant (PDA) or a smart wearable device, etc. Among them, the smart wearable device may be a smart watch, a smart bracelet, smart glasses, etc.

[0111] In some embodiments, one terminal is a transmitting terminal and the other terminal is a receiving terminal. For example, terminal 10 is a transmitting terminal and terminal 20 is a receiving terminal. Figure 1 As shown, when user 1 speaks, terminal 10 can collect the voice of user 1, then recognize the voice of user 1, obtain a recognition result including the pronunciation prompt information of user 1, and then send the recognition result and the voiceprint information of user 1 to terminal 20. Terminal 20 performs speech synthesis according to the pronunciation prompt information of user 1 and the voiceprint information of user 1 in the recognition result to obtain synthesized speech, and then terminal 20 plays the synthesized speech. In this way, user 2 can hear the voice of user 1 without background noise.

[0112] Correspondingly, terminal 10 can also be a receiving end, and terminal 20 can be a sending end. After user 2 hears the voice of user 1, user 2 can make a voice response to user 1. When user 2 makes a voice response, terminal 20 can collect the voice of user 2, and then recognize the voice of user 2 to obtain a recognition result including the pronunciation prompt information of user 2, and then send the recognition result and the voiceprint information of user 2 to terminal 10. Terminal 10 performs speech synthesis based on the pronunciation prompt information of user 2 in the recognition result and the voiceprint information of user 2 to obtain a synthesized speech. In this way, user 1 can hear the voice of user 2 without background noise.

[0113] In the voice communication system 100, taking the terminal 10 as the transmitting end and the terminal 20 as the receiving end as an example, the terminal 10 recognizes the user's voice, and the terminal 20 synthesizes the voice according to the recognition result and the user's voiceprint information. The terminal 10 can adopt a noise reduction solution with lower requirements. For example, the voice input device of the terminal 10 can be a single microphone, and then based on the spectral subtraction principle, the steady-state noise in the surrounding environment is filtered out, so that the terminal 10 can accurately recognize the collected voice of the user 1 and can collect the voiceprint information of the user 1.

[0114] It should be noted that the speech processing method provided in the embodiment of the present application can be provided to the user in the form of a computer program. The computer program can be a separate application (application, APP), or a functional module, plug-in or applet of an existing application. The terminal 10 and the terminal 20 implement the speech processing method of the present application by running the above-mentioned computer program.

[0115] In order to make the technical solution of the present application clearer and easier to understand, the voice processing method provided in the embodiment of the present application is described in detail below, taking the voice communication between user 1 on the terminal 10 side and user 2 on the terminal 20 side as an example.

[0116] like Figure 2As shown, the figure shows a command interaction diagram of a voice processing method, the method comprising the following steps:

[0117] S202: Terminal 10 collects the voice of user 1.

[0118] The terminal 10 can collect the voice of the user 1 through the microphone, and the microphone can be a built-in microphone of the terminal 10, or a microphone external to the terminal 10, such as a headset with a microphone. In some embodiments, when the user 1 speaks, the terminal 10 can collect the voice of the user 1 through the microphone, that is, the terminal 10 can collect the content of the user 1 speaking.

[0119] S204: Terminal 10 recognizes the voice of user 1 and obtains a recognition result.

[0120] The recognition result includes pronunciation prompt information, which refers to prompt information for the user's pronunciation. For example, the pronunciation prompt information may include phonemes, or text information with the same pronunciation as the user's voice. The following describes the process of the terminal 10 recognizing the user's 1 voice for the above two types of pronunciation prompt information.

[0121] The first type: the pronunciation prompt information includes phonemes.

[0122] Phonemes are the basic units of pronunciation. Phonemes can be divided into vowel phonemes and consonant phonemes according to whether the airflow is obstructed during pronunciation. Vowel phonemes refer to the sounds formed when the airflow is not obstructed in the oral cavity and pharynx during pronunciation, and are referred to as vowels; consonant phonemes refer to the sounds formed when the airflow is obstructed to a certain extent in the oral cavity and pharynx during pronunciation, and are referred to as consonants. It should be noted that vowels and consonants can be different in different languages.

[0123] The pronunciation prompt information may include multiple phonemes. Among them, multiple phonemes may be independent or may be associated with each other to form at least one syllable. A syllable is a relatively natural structural unit in speech. In Chinese, a syllable refers to the sound of a Chinese character. For example, the syllables of "Mandarin" include "pǔ", "tōng" and "huà", which have 8 phonemes, namely "p", "u", "t", "o", "ng", "h", "u" and "a". Based on this, when the user utters the voice "Mandarin", the pronunciation prompt information may be 8 phonemes or 3 syllables formed by 8 phonemes. In English, a syllable may be a word or part of a word. A syllable in a monosyllabic word is the word, and a syllable in a polysyllabic word is a part of the word. For example, the word "water" includes "wa" and "ter" Two syllables. When the user pronounces the word "water", the pronunciation prompt information may include the phonemes of the word, or a syllable formed by multiple phonemes, such as "wa". "ter" The above is merely an example description of phonemes and syllables and does not constitute a limitation of the present application.

[0124] In some embodiments, the terminal 10 can recognize the voice of the user 1 through an acoustic model, and then obtain the phonemes corresponding to the voice of the user 1. The acoustic model refers to a model that converts the voice into the phonemes corresponding to the voice. For example, the acoustic model can be a deep neural networks-hidden Markov model (DNN-HMM).

[0125] Specifically, a trained acoustic model may be pre-stored on terminal 10. After terminal 10 collects the voice of user 1, it extracts acoustic features from the sound waveform of the voice of user 1, and then inputs the acoustic features into the acoustic model to obtain the phonemes corresponding to the voice of user 1.

[0126] In some embodiments, the terminal 10 first frames the sound waveform of the user 1's voice, so that the terminal 10 divides the voice into multiple segments, and then extracts acoustic features for each segment. After the terminal 10 inputs multiple acoustic features into the acoustic model, it can obtain a phoneme sequence. Among them, the acoustic feature extraction method includes but is not limited to linear predictive coding (LPC), Mel-frequency cepstral coefficients (MFCC) calculation, etc.

[0127] In some embodiments, the terminal 10 can also obtain the pronunciation time of the phoneme corresponding to the voice of user 1 through the acoustic model. The pronunciation time refers to the time period in which user 1 actually pronounces in the time period corresponding to the voice of user 1. For example, the time period corresponding to the voice of user 1 is 5 seconds, and user 1 does not pronounce in the time interval of 0 to 3 seconds, and pronounces in the time interval of 3 to 5 seconds, then the pronunciation time of the phoneme corresponding to the voice of user 1 is 2 seconds. In actual situations, the pronunciation time of the phoneme corresponding to the voice of user 1 may be much less than 2 seconds, such as 10 milliseconds, 15 milliseconds or 20 milliseconds. In this way, the terminal 10 can obtain the actual pronunciation time of user 1 in the time period corresponding to the voice of user 1, and then put the pronunciation time into the recognition result, so that the terminal 20 can synthesize a voice consistent with the speaking speed of user 1 according to the recognition result including the pronunciation time.

[0128] In other embodiments, the terminal 10 can also obtain the pronunciation interval time of the phonemes corresponding to the voice of user 1 through the acoustic model. Among them, the pronunciation interval time refers to the interval time between the phonemes of user 1 in the time period corresponding to the voice of user 1. For example, the time period corresponding to the voice of user 1 is 3 seconds, and user 1 emits a phoneme (or a syllable composed of multiple phonemes) in the time interval of 0 to 1 second, does not pronounce in the time interval of 1 to 2 seconds, and emits a phoneme (or a syllable composed of multiple phonemes) in the time interval of 2 to 3 seconds, then the pronunciation interval time of the factor corresponding to the voice of user 1 is 1 second. In actual situations, the pronunciation interval time of the factor corresponding to the voice of user 1 may be less than 1 second, such as 50 milliseconds, 60 milliseconds or 100 milliseconds. In this way, the terminal 10 can obtain the pronunciation interval time of user 1 in the time period corresponding to the voice of user 1, and then put the pronunciation interval time into the recognition result, so that the terminal 20 synthesizes the voice consistent with the sentence punctuation when user 1 speaks according to the recognition result including the pronunciation interval time.

[0129] In other implementations, the terminal 10 may simultaneously put the pronunciation time and the pronunciation interval time into the recognition result, so that the terminal 20 can synthesize a speech with the same speaking speed and sentence punctuation as the user 1 according to the recognition result.

[0130] The first type of pronunciation prompt information is introduced above, and the second type of pronunciation prompt information is introduced below.

[0131] The second type: the pronunciation prompt information includes text information that has the same pronunciation as the user's voice.

[0132] Text information refers to text information, and the text can be Chinese, English, or other languages. In some implementations, the text can also be composed of multiple languages. For example, the text information can be "I want to drink Coke", and the text information with the same pronunciation is the text information with the same pronunciation as the aforementioned "I want to drink Coke", such as "I am thirsty". For another example, the text information can be "alter", and the text information with the same pronunciation is the text information with the same pronunciation as the aforementioned "alter", such as "altar". The above-mentioned "I want to drink Coke" and "I am thirsty" as well as "alter" and "altar" are merely schematic introductions of text information with the same pronunciation, and do not constitute a limitation of the present application.

[0133] In some embodiments, the terminal 10 may first recognize the voice of the user 1 through the acoustic model to obtain the phonemes corresponding to the voice of the user 1, and then decode the phonemes through the language model to obtain the text information of the voice of the user 1. The process of the terminal 10 recognizing the voice of the user 1 through the acoustic model can be referred to the description in the first method above, and will not be repeated here.

[0134] A language model refers to a model that converts phonemes into text information. After the terminal 10 obtains the phoneme sequence corresponding to the voice of the user 1 in the above manner, it converts the phoneme sequence into text information through the voice model. In some embodiments, the text information may be text information with the same pronunciation, and the terminal 10 does not need to accurately perform semantic recognition. For example, the terminal 10 does not need to convert the various phoneme sequences output by the acoustic model into text information through the voice model respectively, and then score the different text information to determine the final text information. In this way, the terminal 10 can recognize the voice of the user 1 more quickly, obtain the text information corresponding to the voice of the user 1, reduce the recognition time of the terminal 10, improve the recognition efficiency of the terminal 10, and thereby reduce the communication delay between the terminal 10 and the terminal 20.

[0135] S206 : Terminal 10 sends the recognition result to terminal 20 .

[0136] In some implementations, terminal 10 may send the recognition result and the voiceprint information of user 1 to terminal 20. In other implementations, terminal 10 may also send the recognition result directly to terminal 20. The above two implementations are described below.

[0137] The first implementation method: Terminal 10 sends the recognition result and the voiceprint information of user 1 to terminal 20.

[0138] Voiceprint information refers to information that can reflect the physiological and behavioral characteristics of the speaker. Since different speakers' vocal organs (such as tongue, teeth, larynx, lungs, and nasal cavity) are different in size and shape, different people's voiceprint information is also different. Therefore, the identity of the speaker can be determined by the voiceprint information.

[0139] In some embodiments, user 1 may pre-enter the voiceprint information of user 1 on terminal 10. For example, the voiceprint information of user 1 may be entered in a text-related manner. Specifically, terminal 10 collects the voice of user 1 when reading a preset text, and then processes the voice based on linear predictive coding (LPC) technology to obtain the voiceprint information of user 1, which is then stored locally on terminal 10. After terminal 10 pre-enters the voiceprint information of user 1, terminal 10 does not need to obtain the voiceprint information of user 1 in real time during voice communication with terminal 20, which reduces the computing pressure of terminal 10. Furthermore, by collecting the voice of user 1 when reading a preset text, terminal 10 obtains more accurate voiceprint information of user 1.

[0140] In other embodiments, the terminal 10 may also obtain the voiceprint information of the user 1 in real time. For example, after the terminal 10 collects the voice of the user 1, the terminal 10 processes the voice through the linear prediction coding technology to obtain the voiceprint information of the user 1, and then stores it locally in the terminal 10. In this embodiment, the user 1 does not need to read the preset text, and the terminal 10 obtains the voiceprint information of the user 1 in real time through the collected voice of the user 1.

[0141] The second implementation manner: the terminal 10 sends the recognition result to the terminal 20.

[0142] Specifically, terminal 10 directly sends the recognition result to terminal 20, so that the receiving end performs speech synthesis according to the pronunciation prompt information in the recognition result to obtain the synthesized speech. The synthesized speech can express the content that user 1 wants to express. The synthesized speech does not include the voiceprint information of user 1. For example, the synthesized speech can be a speech with a preset timbre. In some examples, terminal 20 can present the identifier of user 1 to user 2, so that user 2 can determine the identity information of user 1 based on the identifier of user 1, and then know that the synthesized speech is the voice of user 1. Among them, the identifier of user 1 can be user 1's phone number, user name, account number, etc.

[0143] In some embodiments, when the terminal 10 communicates with the terminal 20 for the first time, the terminal 10 may send the voiceprint information of the user 1 to the terminal 20. After receiving the voiceprint information of the user 1 sent by the terminal 10, the terminal 20 stores the voiceprint information. In this way, the terminal 10 may directly send the recognition result to the terminal 20 without sending the voiceprint information of the user 1 at the same time. When the user of the terminal 10 subsequently communicates with the user of the terminal 20, the terminal 10 and the terminal 20 may directly perform voice processing based on the stored voiceprint information of the opposite user to achieve real-time voice communication.

[0144] In some examples, a mapping relationship between the voiceprint information of user 1 and the identifier of user 1 is pre-stored in terminal 20. After receiving the recognition result sent by terminal 10, terminal 20 determines the voiceprint information of user 1 from the mapping relationship according to the identifier of user 1, and then performs speech synthesis according to the voiceprint information of user 1 and the recognition result.

[0145] In some scenarios, the terminal 20 may store the voiceprint information of the user 1 until the end of the current voice communication. For example, if the user 1 speaks twice in succession, after the user 1 speaks the first time, the terminal 10 sends the recognition result of the voice and the voiceprint information of the user 1 to the terminal 20. After the user 1 speaks the second time, the terminal 10 only sends the recognition result of the voice to the terminal 20, without sending the voiceprint information of the user 1 to the terminal 20 again.

[0146] In other scenarios, the terminal 20 may store the voiceprint information of the user 1 until a preset time point. For example, the user 1 and the user 2 have two voice communications. When the user 1 has the first voice communication with the user 2, the terminal 10 sends the recognition result of the user 1's voice and the voiceprint information of the user 1 to the terminal 20. When the user 1 has the second voice communication with the user 2, the terminal 10 only sends the recognition result of the user 1's voice to the terminal 20, and there is no need to send the voiceprint information of the user 1 to the terminal 20 again.

[0147] When the terminal 10 and the terminal 20 perform voice communication, the terminal 10 does not need to send the voiceprint information of the user 1 to the terminal 20 multiple times, which further reduces the amount of data transmitted from the terminal 10 to the terminal 20.

[0148] In this embodiment, the terminal 10 sends the recognition result with a small amount of data and the voiceprint information of the user 1 to the terminal 20. Compared with directly sending the voice data with a large amount of data to the terminal 20, the terminal 10 can transmit the recognition result with a small amount of data and the voiceprint information of the user 1 to the terminal 20 in a short time, thereby reducing the time required for the terminal 10 to transmit data to the terminal 20. Taking the narrowband voice communication with low voice quality as an example, the original bit rate is 128kbps, and the bit rate after compression is 4.75kbps to 12.2kbps. In this embodiment, the pronunciation prompt information in the recognition result sent by the terminal 10 to the terminal 20 is represented by characters, for example, by characters to represent text information or by characters to represent phonemes (or syllables composed of multiple phonemes). Taking the representation of text information by characters, the terminal takes the Chinese character standard GB-2313 as an example, each character occupies 2 bytes, i.e. 16 bits, and the normal speaking speed is 120 characters per minute, which is converted into a bit rate of 32bps.

[0149] It can be seen that the recognition result sent by terminal 10 to terminal 20 can reduce the amount of data transmitted by nearly a thousand times compared with directly sending voice data to terminal 20. The data volume of voiceprint information is about 10kb, and terminal 10 only transmits voiceprint information to terminal 20 once. After terminal 20 saves the voiceprint information, terminal 10 only needs to transmit the recognition result to terminal 20 later. Furthermore, even if the network environment where terminal 10 is located is poor or the network environment where terminal 10 is located deteriorates (for example, user 1 carries terminal 10 into an area with a poor network environment such as an elevator), terminal 10 can also transmit a recognition result with a smaller amount of data to terminal 20, thereby realizing voice communication between user 1 and user 2. This improves the availability of real-time voice communication.

[0150] In some implementations, the terminal 10 may generate a character stream based on the recognition result, and then transmit the character stream to the terminal 20 in a streaming sequential transmission manner. For example, in the process of transmitting the recognition result from the terminal 10 to the terminal 20, each time the terminal 10 recognizes the voice of the user 1 and obtains a syllable, the terminal 10 directly encodes the syllable and transmits it to the terminal 20 in a streaming sequential transmission manner. The terminal 10 does not need to wait for all the syllables corresponding to the voice of the user 1 to be obtained, and then uniformly encode the syllables and transmit them to the terminal 20. In this way, the terminal 10 can generate a character stream based on the recognition result and transmit the character stream to the terminal 20 while recognizing the voice of the user 1.

[0151] When the terminal 20 receives the character stream transmitted by the terminal 10, each time the terminal 20 receives the character stream transmitted by the terminal 10, the terminal 20 can perform speech synthesis according to the received character stream and the voiceprint information of the user 1. In some embodiments, the terminal 20 can perform speech synthesis according to the pronunciation prompt information and the voiceprint information of the user 1 in the order of the character stream. In this way, the terminal 20 can perform speech synthesis while receiving the character stream transmitted by the terminal 10.

[0152] In the above manner, the terminal 10 can complete the transmission of the character stream corresponding to the recognition result to the terminal 20 in a relatively short time, thereby reducing the communication delay between the terminal 10 and the terminal 20 .

[0153] S208: The terminal 20 performs speech synthesis according to the recognition result to obtain synthesized speech.

[0154] In some embodiments, when the voiceprint information of user 1 is not stored in terminal 20 and terminal 10 does not send the voiceprint information of user 1 to terminal 20, terminal 20 directly performs speech synthesis according to the recognition result to obtain synthesized speech, and the synthesized speech does not include background noise. Terminal 20 can play the synthesized speech, and user 2 can know what user 1 wants to express.

[0155] In other embodiments, when the voiceprint information of user 1 is stored in terminal 20 or terminal 10 sends the voiceprint information of user 1 to terminal 20, terminal 20 performs speech synthesis based on the recognition result sent by terminal 10 and the voiceprint information of user 1 to obtain the voice of user 1 without background noise. Specifically, terminal 20 synthesizes the initial speech according to the pronunciation prompt information, and the initial speech can be a standard speech, such as the speech of a robot. The voiceprint information of user 1 is not included in the initial speech. Then, terminal 20 obtains the synthesized speech corresponding to user 1 based on the initial speech and the voiceprint information of user 1. The synthesized speech includes the voiceprint information of user 1. After terminal 20 plays the synthesized speech, user 2 can determine that the synthesized speech is the voice of user 1.

[0156] In some implementations, the terminal 20 may process the pronunciation prompt information in the recognition result and the voiceprint information of the user 1 through a resonance peak model to synthesize the speech of the user 1 without background noise.

[0157] In some embodiments, the recognition result also includes the pronunciation time of the phoneme corresponding to the voice of user 1 obtained by terminal 10 through the acoustic model. After receiving the recognition result sent by terminal 10, terminal 20 obtains the pronunciation time, and terminal 20 can determine the speaking speed of user 1 according to the pronunciation time. After terminal 20 performs speech synthesis according to the recognition result including the pronunciation time of the phoneme corresponding to the voice of user 1, the speaking speed of the synthesized speech can be made consistent with the speaking speed of the speech of user 1 when speaking, thereby reducing the situation where the synthesized speech is too rigid and improving the user experience of voice communication.

[0158] In other embodiments, the recognition result also includes the pronunciation interval time of the phonemes corresponding to the voice of user 1 obtained by the terminal 20 through the acoustic model. The terminal 20 can determine the sentence segmentation of the user 1 when speaking according to the pronunciation interval time. After the terminal 20 performs speech synthesis according to the recognition result including the pronunciation interval time of the phonemes corresponding to the voice of user 1, the synthesized speech can be consistent with the sentence segmentation of the speech of user 1 when speaking.

[0159] In other embodiments, the terminal 20 may also perform speech synthesis based on the recognition results including both the pronunciation time and the pronunciation interval time, so that the terminal 20 can synthesize speech with the same speaking speed and sentence punctuation as the user 1.

[0160] It should be noted that the speed at which terminal 10 recognizes speech is much higher than the speed at which terminal 10 encodes speech in a traditional manner. Correspondingly, the speed at which terminal 20 synthesizes speech is also much higher than the speed at which terminal 20 decodes speech in a traditional manner. This method further reduces the communication delay between terminal 10 and terminal 20.

[0161] S210: Terminal 20 plays the synthesized speech.

[0162] The terminal 20 plays the synthesized speech so that the user 2 at the terminal 20 side can hear the synthesized speech, and the synthesized speech does not include any background noise component, thereby realizing real-time voice communication without any noise.

[0163] Based on the above description, the embodiment of the present application provides a method for voice processing. On the one hand, the recognition result sent by terminal 10 to terminal 20 does not include any noise components such as background noise. Terminal 20 performs voice synthesis based on the recognition result that does not include background noise and the voiceprint information of user 1, so that the synthesized voice does not contain any noise components, and theoretically can achieve 100% elimination of noise components. In this way, the method reduces the interference of background noise on the voice of user 1, improves the communication efficiency of user 1, and thus improves the voice communication experience of user 1.

[0164] Furthermore, terminal 20 restores the voice of user 1 through voice synthesis, which does not damage the voice signal, and terminal 20 can restore high-quality voice signals, so that user 2 can hear clearer voice, thereby improving the voice communication experience of user 1 and user 2.

[0165] On the other hand, the terminal 10 sends the recognition result with a smaller data volume and the voiceprint information of the user 1 to the terminal 20, instead of sending the voice data of the user 1 with a larger data volume to the terminal 20. This method can reduce the amount of data transmitted from the terminal 10 to the terminal 20, thereby reducing the requirements for the network environment in which the terminal 10 and the terminal 20 are located, so that the terminal 10 and the terminal 20 can also perform high-quality voice communication in a poor network environment.

[0166] In some embodiments, after the terminal 20 performs speech synthesis, the user 2 can respond to the synthesized speech after hearing the synthesized speech played by the terminal 20. For example, the user 2 responds to the user 1 on the terminal 10 side through the terminal 20. Figure 3 The present application also provides a method for speech processing. Figure 2 In addition to the steps shown, the following steps are also included:

[0167] S302: Terminal 20 collects the voice of user 2.

[0168] S304: Terminal 20 recognizes the voice of user 2 and obtains a recognition result.

[0169] S306: Terminal 20 sends the recognition result to terminal 10.

[0170] S308: The terminal 10 performs speech synthesis according to the recognition result to obtain synthesized speech.

[0171] S310: The terminal 10 plays the synthesized speech.

[0172] In this embodiment, terminal 10 can be used as both a transmitter and a receiver, that is, terminal 10 has both the functions of a transmitter and a receiver. Similarly, terminal 20 can be used as both a transmitter and a receiver, that is, terminal 20 has both the functions of a transmitter and a receiver. Terminal 10 and terminal 20 can implement different functions in different scenarios to achieve voice communication between user 1 on the terminal 10 side and user 2 on the terminal 20 side.

[0173] It should be noted that S302 to S310 are similar to the above Figure 2 The steps S202 to S210 are similar and will not be described in detail here.

[0174] In some embodiments, the terminal may also encrypt the transmitted data such as the recognition result and the voiceprint information to improve the security of the voice communication between user 1 and user 2. For ease of understanding, the following is an example of terminal 10 encrypting the data transmitted to terminal 20. Figure 4 As shown, the embodiment of the present application also provides a speech processing method, which includes the following steps:

[0175] S402: Terminal 10 collects the voice of user 1.

[0176] S404: Terminal 10 filters the voice of user 1 using the voiceprint information.

[0177] In some embodiments, the terminal 10 may pre-store the voiceprint information of the user 1. After the terminal 10 collects the voice of the user 1, the terminal 10 may first reduce the noise of the user 1's voice according to the voiceprint information of the user 1 to remove the voices of other users. Specifically, the terminal 10 compares the voiceprint information of the collected voice with the voiceprint information of the user 1. If the comparison result shows that the voice is the voice of the user 1, the terminal 10 performs subsequent processing on the voice, such as recognition; if the comparison result shows that the voice is not the voice of the user 1, but the voice of other users, the terminal 10 determines that the voice is noise, and the terminal 10 can filter the noise. In this way, the terminal 10 can filter out the voices of other users included in the voice of the user 1, and further reduce the noise in the voice of the user 1 collected by the terminal 10.

[0178] S406: The terminal 10 recognizes the filtered speech and obtains a recognition result.

[0179] The way in which the terminal 10 recognizes the filtered voice is similar to the way in which the terminal 10 directly recognizes the voice of the user 1 , and details may be found in S204 in the above embodiment, which will not be described again here.

[0180] After the terminal 10 performs noise reduction processing on the collected voice of the user 1 in advance, the accuracy of the terminal 10 in recognizing the voice can be improved, and thus the terminal 10 can obtain a more accurate recognition result.

[0181] S408: The terminal 10 encrypts the recognition result.

[0182] In some embodiments, the terminal 10 may also encrypt the recognition result to obtain a ciphertext. When the data transmitted by the terminal 10 to the terminal 20 includes the voiceprint information of the user 1, the terminal 10 may also encrypt the voiceprint information of the user 1 together with the recognition result to obtain a ciphertext. For example, a preset mapping relationship between plain text and ciphertext is stored in the terminal 10, and then the recognition result and the voiceprint information in plain text are converted into ciphertext according to the mapping relationship between plain text and ciphertext. The preset mapping relationship between plain text and ciphertext may be a mapping relationship between plain text and ciphertext pre-set between the terminal 10 and the terminal 20.

[0183] In some embodiments, the terminal 10 may set up encryption information libraries for the recognition result and the voiceprint information respectively. For example, a first encryption information library storing a first mapping relationship between a first plaintext and a first ciphertext and a second encryption information library storing a second mapping relationship between a second plaintext and a second ciphertext. The terminal 10 encrypts the recognition result according to the first mapping relationship stored in the first encryption information library, and converts the recognition result in plaintext into ciphertext. The terminal 10 encrypts the voiceprint information according to the second mapping relationship stored in the second encryption information library, and converts the voiceprint information in plaintext into ciphertext. In some implementations, the terminal 10 may also add an identifier to the ciphertext of the recognition result so that the terminal 20 obtains the recognition result in plaintext from the corresponding encryption information library according to the identifier. Similarly, the terminal 10 may also add an identifier to the ciphertext of the voiceprint information so that the terminal 20 obtains the voiceprint information in plaintext from the corresponding encryption information library according to the identifier.

[0184] In other embodiments, the terminal 10 only needs to send the recognition result to the terminal 20, and there is no need to repeatedly send the voiceprint information of the user 1 to the terminal 20. The terminal 10 only needs to encrypt the recognition result, thus reducing the time required for the terminal 10 to encrypt the transmitted data, and further reducing the delay of the terminal 10 transmitting data to the terminal 20.

[0185] S410: Terminal 10 sends a ciphertext to terminal 20.

[0186] Terminal 10 sends the above ciphertext to terminal 20. Even if the ciphertext is intercepted by criminals such as hackers, the criminals cannot know the data actually sent by terminal 10 to terminal 20 through the ciphertext, thereby improving the security of data transmitted between terminal 10 and terminal 20.

[0187] S412: Terminal 20 decrypts the ciphertext to obtain a recognition result.

[0188] Correspondingly, the mapping relationship between the above-mentioned plaintext and ciphertext is also stored in terminal 20. After terminal 20 receives the ciphertext sent by terminal 10, it determines the plaintext corresponding to the ciphertext according to the above-mentioned mapping relationship between the plaintext and the ciphertext to obtain the recognition result in plaintext form. When the data transmitted by terminal 10 to terminal 20 includes the voiceprint information of user 1, terminal 20 can also obtain the voiceprint information in plaintext form, thereby realizing decryption.

[0189] In some embodiments, after receiving the ciphertext sent by the terminal 10, the terminal 20 may also determine the corresponding encryption information library through the identifier carried by the ciphertext, and then obtain the plaintext corresponding to the ciphertext by querying the corresponding encryption information library to achieve decryption. For example, the terminal 20 determines that the ciphertext is the ciphertext of the recognition result according to the identifier carried by the ciphertext, and then the terminal 20 queries the first mapping relationship in the first encryption information library to decrypt the ciphertext, and then obtains the recognition result in plain text.

[0190] S414: The terminal 20 performs speech synthesis according to the recognition result to obtain synthesized speech.

[0191] The manner in which the terminal 20 performs speech synthesis according to the recognition result may be specifically referred to S208 in the above embodiment, which will not be described in detail here.

[0192] S416: Terminal 20 plays the synthesized speech.

[0193] The terminal 20 plays the synthesized voice so that the user 2 on the terminal 20 side can hear the voice of the user 1 without any background noise, thereby achieving real-time voice communication without any noise.

[0194] Based on the above description, an embodiment of the present application provides a voice processing method. In the method, the terminal 10 encrypts the data transmitted to the terminal 20, such as the recognition result and the voiceprint information, to obtain a ciphertext, and the terminal 10 transmits the ciphertext to the terminal 20, thereby improving the security of the transmitted data. Even if the criminals intercept the ciphertext, they cannot decrypt the ciphertext to obtain the data actually sent by the terminal 10 to the terminal 20. In this way, the method can improve the security of real-time voice communication.

[0195] Combination of the above Figures 1 to 4 The speech processing method provided in the embodiment of the present application is introduced in detail. The apparatus and device provided in the embodiment of the present application will be introduced in conjunction with the accompanying drawings.

[0196] See also Figure 5 The structure diagram of the speech processing device shown in FIG. 5 includes:

[0197] A collection unit 502 is used to collect the user's voice;

[0198] The recognition unit 504 is used to recognize the user's voice and obtain a recognition result, wherein the recognition result at least includes pronunciation prompt information;

[0199] The sending unit 506 is used to send the recognition result to the receiving end, so that the receiving end performs speech synthesis according to the recognition result.

[0200] In some possible implementations, the sending unit 506 is configured to enable the sending end to send the recognition result and the voiceprint information of the user to the receiving end.

[0201] In some possible implementations, the pronunciation prompt information includes phonemes;

[0202] The recognition unit 504 is used to recognize the user's voice through an acoustic model to obtain the phonemes corresponding to the voice.

[0203] In some possible implementations, the recognition unit 504 further obtains the pronunciation time or the pronunciation interval time of the phoneme corresponding to the speech through the acoustic model, and the recognition result also includes the pronunciation time or the pronunciation interval time.

[0204] In some possible implementations, the pronunciation prompt information includes text information having the same pronunciation as the speech.

[0205] In some possible implementations, the recognition unit 504 is configured to recognize phonemes through an acoustic model, and then decode the phonemes through a language model to obtain the text information.

[0206] In some possible implementations, the device further includes:

[0207] An encryption unit 508, used to encrypt the recognition result;

[0208] The sending unit 506 is used to send the encrypted recognition result to the receiving end.

[0209] In some possible implementations, the device further includes:

[0210] The filtering unit 510 is configured to filter the user's voice according to the user's voiceprint information.

[0211] In some possible implementations, the sending unit 506 is configured to generate a character stream according to the recognition result, and transmit the character stream to the receiving end in a streaming sequential transmission manner.

[0212] According to the embodiment of the present application, the speech processing device 500 can correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the speech processing device 500 are for realizing the process of the method in the above method embodiment, and for the sake of brevity, they are not repeated here.

[0213] See also Figure 6 The structure diagram of the speech processing device shown in FIG. 600 includes:

[0214] An acquisition unit 602 is used to acquire a recognition result of a user's speech, wherein the recognition result at least includes pronunciation prompt information;

[0215] A synthesis unit 604, configured to perform speech synthesis according to the pronunciation prompt information to obtain synthesized speech;

[0216] The playing unit 606 is used to play the synthesized speech to achieve real-time voice communication.

[0217] In some possible implementations, the acquisition unit 602 is used to acquire the voiceprint information of the user;

[0218] The synthesis unit 604 is used to perform speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain synthesized speech.

[0219] In some possible implementations, the pronunciation prompt information includes phonemes;

[0220] The synthesis unit 604 is used to synthesize an initial speech according to the pronunciation prompt information, and obtain a synthesized speech corresponding to the user according to the initial speech and the voiceprint information of the user.

[0221] In some possible implementations, the recognition result further includes the pronunciation time or pronunciation interval time of the phoneme;

[0222] The synthesis unit 604 is used to perform speech synthesis according to one of the pronunciation time of the phoneme or the pronunciation interval time, the phoneme, and the voiceprint information of the user.

[0223] In some possible implementations, the pronunciation prompt information includes text information having the same pronunciation as the speech.

[0224] In some possible implementations, the text information is obtained by the sending end identifying phonemes through an acoustic model and then decoding the phonemes through a language model.

[0225] In some possible implementations, the apparatus further includes a decryption unit 608;

[0226] The acquiring unit 602 is configured to receive the encrypted recognition result sent by the sending end;

[0227] The decryption unit 608 is used to decrypt the encrypted recognition result to obtain the recognition result.

[0228] In some possible implementations, the recognition result is transmitted in the form of a character stream;

[0229] The synthesis unit 604 is used to perform speech synthesis according to the order of the character stream, the pronunciation prompt information and the voiceprint information of the user.

[0230] According to the embodiment of the present application, the speech processing device 600 can correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the speech processing device 600 are for realizing the process of the method in the above method embodiment, and for the sake of brevity, they are not repeated here.

[0231] The present application also provides a voice processing device. The voice processing device may be a terminal 10, and the voice processing device is used to implement the following Figure 5 Functions of the speech processing device 500 in the illustrated embodiment. The following takes the terminal 10 as an example to describe the hardware architecture of the speech processing device.

[0232] Figure 7 A schematic diagram of the structure of a terminal 10 is provided, such as Figure 7 As shown, the terminal 10 includes a bus 701, a processor 702, a communication interface 703 and a memory 704. The processor 702, the memory 704 and the communication interface 703 communicate with each other via the bus 701.

[0233] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0234] The processor 702 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0235] The communication interface 703 is used for communicating with the outside. For example, the communication interface 703 can be used for communicating with the terminal 20. The communication interface is used to send the recognition result to the terminal 20 so that the terminal 20 performs speech synthesis according to the recognition result.

[0236] The memory 704 may include a volatile memory, such as a random access memory (RAM). The memory 704 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0237] The memory 704 stores executable codes, and the processor 702 executes the executable codes to perform the speech processing method in the aforementioned embodiment.

[0238] Specifically, in implementing Figure 5 In the case of the embodiment shown, and Figure 5 When each module or unit of the speech processing device 500 described in the embodiment is implemented by software, the execution Figure 5 The software or program code required for the functions of each module / unit in the embodiment may be partially or completely stored in the memory 704. The processor 702 executes the program code corresponding to each unit stored in the memory 704 to perform the speech processing method in the above-mentioned embodiment.

[0239] The present application also provides a voice processing device. The voice processing device may be a terminal 20, and the voice processing device is used to implement the following Figure 6 Functions of the speech processing device 600 in the illustrated embodiment. The following takes the terminal 20 as an example to describe the hardware architecture of the speech processing device.

[0240] Figure 8 A structural diagram of a first speech processing device is provided, such as Figure 8As shown, the terminal 20 includes a bus 801, a processor 802, a communication interface 803 and a memory 804. The processor 802, the memory 804 and the communication interface 803 communicate with each other via the bus 801.

[0241] The communication interface 803 is used for communicating with the outside. For example, the communication interface 803 can be used for communicating with the terminal 10. The communication interface is used to receive the recognition result sent by the terminal 10 so that the terminal 20 performs speech synthesis according to the recognition result.

[0242] The memory 804 stores executable codes, and the processor 802 executes the executable codes to perform the speech processing method in the aforementioned embodiment.

[0243] Specifically, in implementing Figure 6 In the case of the embodiment shown, and Figure 6 When each module or unit of the speech processing device 600 described in the embodiment is implemented by software, the execution Figure 6 The software or program code required for the functions of each module / unit in the embodiment may be partially or completely stored in the memory 804. The processor 802 executes the program code corresponding to each unit stored in the memory 804 to perform the speech processing method in the above-mentioned embodiment.

[0244] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned speech processing method applied to the speech processing device 500 or the speech processing device 600.

[0245] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.

[0246] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0247] When the computer program product is executed by a computer, the computer executes any of the speech processing methods in the embodiments of the present application. The computer program product may be a software installation package, and when any of the speech processing methods in the embodiments of the present application needs to be used, the computer program product may be downloaded and executed on a computer.

[0248] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

Claims

1. A speech processing method, characterized in that: Applied to a voice communication system, the system includes a transmitting end and a receiving end, and the method includes: The sending end collects the user's voice; The sending end recognizes the user's voice to obtain a recognition result, wherein the recognition result at least includes pronunciation prompt information; the pronunciation prompt information includes a phoneme, a pronunciation time of the phoneme, and a pronunciation interval time; The sending end sends the recognition result to the receiving end, so that the receiving end performs speech synthesis according to the recognition result and the voiceprint information of the user to obtain synthesized speech; the synthesized speech is the voice of the user, and the user's voice is consistent with the speaking speed and sentence punctuation of the user when speaking; the voiceprint information of the user is pre-entered by the sending end or obtained in real time and sent to the receiving end, or is pre-stored by the receiving end.

2. The method according to claim 1, characterized in that The sending end sends the recognition result to the receiving end, and the method includes: The sending end sends the voiceprint information of the user to the receiving end, so that the receiving end performs speech synthesis according to the recognition result and the voiceprint information of the user.

3. The method according to claim 1, characterized in that The sending end recognizes the user's voice to obtain a recognition result, including: The sending end recognizes the user's voice through an acoustic model to obtain the phonemes corresponding to the voice.

4. The method according to claim 3, characterized in that The sending end also obtains the pronunciation time and pronunciation interval time of the phonemes corresponding to the speech through the acoustic model.

5. The method according to claim 1, characterized in that: The pronunciation prompt information includes text information having the same pronunciation as the voice.

6. The method according to claim 5, characterized in that The text information is obtained by the sending end identifying phonemes through an acoustic model and then decoding the phonemes through a language model.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: encrypting the recognition result; The sending end sends the recognition result to the receiving end, including: The sending end sends the encrypted recognition result to the receiving end.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The user's voice is filtered according to the user's voiceprint information.

9. The method according to any one of claims 1 to 6, characterized in that: The sending end sends the recognition result to the receiving end, including: The sending end generates a character stream according to the recognition result; The sending end transmits the character stream to the receiving end in a streaming sequential transmission manner.

10. A speech processing method, characterized in that: Applied to a voice communication system, the system includes a transmitting end and a receiving end, and the method includes: The receiving end obtains a recognition result of the user's voice, wherein the recognition result at least includes pronunciation prompt information; the pronunciation prompt information includes a phoneme, a pronunciation time of the phoneme, and a pronunciation interval time; The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain a synthesized speech; the synthesized speech is the speech of the user, and the speech speed and sentence punctuation of the user are consistent with those of the user when speaking; the voiceprint information of the user is pre-entered by the sending end or obtained in real time and sent to the receiving end, or is pre-stored by the receiving end; The receiving end plays the synthesized voice to achieve real-time voice communication.

11. The method according to claim 10, characterized in that The method further comprises: Acquire the voiceprint information of the user.

12. The method according to claim 10, characterized in that The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain synthesized speech, including: The receiving end synthesizes the initial speech according to the pronunciation prompt information; The receiving end obtains a synthesized speech corresponding to the user according to the initial speech and the voiceprint information of the user.

13. The method according to claim 10, characterized in that The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user, including; The receiving end performs speech synthesis according to the pronunciation time of the phoneme, the pronunciation interval time, the phoneme, and the voiceprint information of the user.

14. The method according to claim 10, characterized in that The pronunciation prompt information includes text information having the same pronunciation as the voice.

15. The method according to claim 14, characterized in that The text information is obtained by the sending end identifying phonemes through an acoustic model and then decoding the phonemes through a language model.

16. The method according to any one of claims 10 to 15, characterized in that The receiving end obtains the recognition result of the user's voice, including: The receiving end receives the encrypted recognition result sent by the sending end; The receiving end decrypts the encrypted recognition result to obtain the recognition result.

17. The method according to any one of claims 11 to 15, characterized in that The recognition result is transmitted in the form of a character stream; The receiving end performs speech synthesis according to the pronunciation prompt information and the voiceprint information of the user, including: The receiving end performs speech synthesis according to the order of the character stream, the pronunciation prompt information and the voiceprint information of the user.

18. A speech processing device, characterized in that: include: A collection unit, used for collecting the user's voice; A recognition unit, configured to recognize the user's voice and obtain a recognition result, wherein the recognition result at least includes pronunciation prompt information; The pronunciation prompt information includes phonemes, pronunciation time of the phonemes and pronunciation interval time; A sending unit, configured to send the recognition result to a receiving end, so that the receiving end performs speech synthesis according to the recognition result and the voiceprint information of the user to obtain synthesized speech; The synthesized speech is the user's speech, and the user's speech is consistent with the user's speaking speed and sentence punctuation; the user's voiceprint information is pre-recorded by the sending end or acquired in real time and sent to the receiving end, or is pre-stored by the receiving end.

19. A speech processing device, characterized in that: include: An acquisition unit, which acquires a recognition result of the user's voice, wherein the recognition result at least includes pronunciation prompt information; The pronunciation prompt information includes phonemes, pronunciation time of the phonemes and pronunciation interval time; a synthesis unit, configured to perform speech synthesis according to the pronunciation prompt information and the voiceprint information of the user to obtain a synthesized speech; the synthesized speech is the speech of the user, and the speech speed and sentence punctuation of the user are consistent with those of the user when speaking; the voiceprint information of the user is pre-entered by the sending end or obtained in real time and sent to the receiving end, or is pre-stored by the receiving end; The playing unit is used to play the synthesized speech to realize real-time voice communication.

20. A device, characterized in that The device comprises a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the device performs the method according to any one of claims 1 to 17.

21. A computer-readable storage medium, characterized in that: The method comprises instructions for instructing a device to execute the method according to any one of claims 1 to 17.

22. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 17.

Images