Narrowband private network voice transmission method and system based on ASR and TTS

Abstract

The invention relates to the technical field of voice, and relates to a narrowband private network voice transmission method and system based on ASR and TTS, and the method comprises the steps: collecting a voice signal, and carrying out the preprocessing of the voice signal; inputting the voice signal into an acoustic model to obtain a first text sequence output by the acoustic model; performing natural language processing on the first text sequence to obtain a target text sequence, and transmitting the target text sequence to a receiving end through wireless communication; obtaining a second text sequence output by the receiving end through wireless communication; and inputting the second text sequence into the speech synthesis model to obtain a second speech signal. According to the invention, the bit error rate of wireless receiving and transmitting can be reduced, and the voice is played after text synthesis at the receiving end, so that the played sound quality is stable without the background noise problem.

Description

Technical Field

[0001] The present invention generally relates to the field of speech technology, and more specifically, to a narrowband private network speech transmission method and system based on ASR and TTS. Background Art

[0002] Narrowband private network communication is a communication technology specifically used for voice or data transmission. Through low bit rate transmission, narrowband private network communication technology is suitable for walkie-talkies, radios and other equipment, and can achieve effective communication under resource-constrained conditions. In the fields of public safety, traffic management, etc., narrowband private network communication technology can provide reliable communication services even under unsatisfactory network conditions.

[0003] At present, the implementation of narrowband private network voice communication is mostly based on audio codecs, which convert the analog voice signal into a digital signal, modulate it, and then use related wireless technology to send it to the receiving end for demodulation to achieve voice communication. However, due to bit errors in the transmission process of wireless digital signals, voice communication (such as walkie-talkie communication) may cause problems such as dropped words, unclear sound, and high background noise, which affects the user experience. Summary of the invention

[0004] In order to solve the technical problems of dropped words, unclear voice and loud background noise in the above-mentioned voice communication, the present invention provides solutions in the following aspects.

[0005] In a first aspect, a narrowband private network voice sending method based on ASR and TTS includes: acquiring a voice signal and preprocessing the voice signal; inputting the voice signal into an acoustic model to obtain a first text sequence output by the acoustic model, wherein the acoustic model is used to output a text sequence according to the voice signal; performing natural language processing on the first text sequence to obtain a target text sequence, and transmitting the target text sequence to a receiving end via wireless communication.

[0006] Preferably, a narrowband private network voice sending method based on ASR and TTS also includes: calculating the semantic similarity between the first text sequence and the target text sequence; in response to the semantic similarity being less than a threshold of a preset size, issuing an alarm, and causing the first text sequence to replace the target text sequence and be transmitted to the receiving end via wireless communication.

[0007] Preferably, the calculating of the semantic similarity between the first text sequence and the target text sequence comprises: calculating the Levenshtein distance between the first text sequence and the target text sequence as L ; Get the absolute value of the difference in the number of stop words between the first text sequence and the target text sequence n 1; Obtain the absolute value of the difference in the number of keywords between the first text sequence and the target text sequence n 2 ; Calculate the semantic similarity S ,in , η 1 is the first weight of the preset size, η 2 The second weight is a preset size.

[0008] Preferably, the preprocessing of the speech signal includes: performing noise reduction, frame division and windowing processing on the speech signal.

[0009] Preferably, constructing the acoustic model includes: collecting m groups of acoustic training data, wherein each group of acoustic training data includes a speech sample signal, and the speech sample signal label is a transcribed text of the speech sample signal, m is a positive integer of a preset size; constructing a first neural network model; by m The first neural network model is trained using the acoustic training data of the group, so that the trained first neural network model is used as the acoustic model.

[0010] Preferably, training the first neural network model includes: constructing a first cross entropy loss function; for any set of acoustic training data, inputting it into the first neural network model to obtain the predicted text output by the first neural network model; constructing one-hot encoding; obtaining the probability distribution of each character in the predicted text according to the one-hot encoding, and recording it as the first probability distribution, obtaining the probability distribution of each character in the transcribed text and recording it as the second probability distribution; obtaining the difference between the first probability distribution and the second probability distribution through the cross entropy function, and correcting the parameters of the first neural network model according to the difference.

[0011] Preferably, performing natural language processing on the first text sequence includes: inputting the first text sequence into a trained NLP model, and determining that the text sequence output by the NLP model is the target text sequence.

[0012] A narrowband private network voice receiving method based on ASR and TTS, the method comprising: obtaining a second text sequence output by a receiving end through wireless communication; inputting the second text sequence into a speech synthesis model to obtain a second speech signal, wherein the speech synthesis model is used to generate a speech signal according to the text sequence.

[0013] Preferably, constructing the speech synthesis model includes: collecting k groups of speech synthesis training data, wherein each group of speech synthesis training data includes a text sample, and the label of the text sample is the speech recording data of the text sample,k is a positive integer of a preset size; construct a second neural network model; through k The speech synthesis neural network model is trained using the speech synthesis training data of the group, so that the trained speech synthesis neural network model is used as the speech synthesis model.

[0014] In the second aspect, a narrowband private network voice transmission system based on ASR and TTS, the system comprising: a transceiver antenna, which is used to receive or send a text sequence; an audio circuit, which is used to convert a voice signal into voice, or receive voice and convert the voice into a voice signal; a processor, which is connected to the audio circuit and is used to convert the received voice signal into a text sequence, or input the voice signal into the audio circuit; the processor is also connected to the transceiver antenna, the processor receives or sends a text sequence through the transceiver antenna, and the processor is used to convert the received text sequence into a voice signal; the processor is used to implement a narrowband private network voice sending method based on ASR and TTS as described in any one of the above invention contents, or to implement a narrowband private network voice receiving method based on ASR and TTS as described in any one of the above invention contents.

[0015] The beneficial effects of the present invention are: The present invention wirelessly transmits text / text encoding information, and only requires a small transmission bandwidth and data volume to achieve information transmission. Since text information only requires a simple encoding method to achieve transmission, the present invention can greatly reduce the bit error rate of wireless transmission and reception. The present invention allows the voice to be played after being synthesized by the text at the receiving end, so that the sound quality is stable and there is no background noise problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 is a schematic diagram showing the steps of a method for sending narrowband private network voice based on ASR and TTS according to an embodiment of the present invention; Figure 2 is a schematic diagram showing the steps of a method for receiving narrowband private network voice based on ASR and TTS according to an embodiment of the present invention; Figure 3 It is a structural block diagram schematically showing a narrowband private network voice transmission system based on ASR and TTS according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0018] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0019] Figure 1 is a schematic diagram showing the steps of a method for sending narrowband private network voice based on ASR and TTS according to an embodiment of the present invention; like Figure 1 The narrowband private network voice transmission method based on ASR and TTS includes steps S101 to S103.

[0020] Step S101: Obtain a speech signal and pre-process the speech signal.

[0021] In one embodiment, the preprocessing of the speech signal includes: performing noise reduction, framing and windowing on the speech signal. It should be noted that performing noise reduction on the speech signal can reduce background noise in the speech signal and improve the quality of the speech signal; framing the speech signal and processing each frame separately to facilitate the acoustic model to extract features of the speech signal (the acoustic model outputs a text sequence after extracting features from the speech signal); windowing the speech signal can reduce the edge effect introduced during framing.

[0022] Step S102: input the speech signal into an acoustic model to obtain a first text sequence output by the acoustic model.

[0023] The acoustic model is used to output a text sequence according to the speech signal. In one embodiment, constructing the acoustic model includes: collecting m groups of acoustic training data, wherein each group of acoustic training data includes a speech sample signal, and the speech sample signal label is a transcribed text of the speech sample signal, m is a positive integer of a preset size; constructing a first neural network model; by m The first neural network model is trained using the acoustic training data of the group, so that the trained first neural network model is used as the acoustic model.

[0024] Furthermore, training the first neural network model includes: constructing a first cross entropy loss function; for any set of acoustic training data, inputting it into the first neural network model to obtain the predicted text output by the first neural network model; constructing one-hot encoding; obtaining the probability distribution of each character in the predicted text according to the one-hot encoding, and recording it as the first probability distribution, obtaining the probability distribution of each character in the transcribed text and recording it as the second probability distribution; obtaining the difference between the first probability distribution and the second probability distribution through the cross entropy function, and correcting the parameters of the first neural network model according to the difference.

[0025] It should be noted that the cross entropy loss function is a commonly used function for evaluating the difference between two probability distributions.

[0026] Step S103: performing natural language processing on the first text sequence to obtain a target text sequence, and transmitting the target text sequence to a receiving end via wireless communication.

[0027] In one embodiment, performing natural language processing on the first text sequence includes: inputting the first text sequence into a trained NLP model, and determining that the text sequence output by the NLP model is a target text sequence.

[0028] In one embodiment, transmitting the target text sequence to the receiving end via wireless communication includes: transmitting the target text sequence to the receiving end via a LoRa network.

[0029] Furthermore, in order to ensure that the semantic changes of the first text sequence are not large after natural language processing, the present invention also includes: calculating the semantic similarity between the first text sequence and the target text sequence; in response to the semantic similarity being less than a threshold of a preset size, issuing an alarm, and causing the first text sequence to replace the target text sequence and be transmitted to the receiving end via wireless communication.

[0030] It should be noted that semantic similarity is a measure used to evaluate the degree of similarity between two text sequences at the semantic level. The greater the semantic similarity between two text sequences, the more similar the two text sequences are at the semantic level. In order to avoid a large difference between the target text sequence obtained after natural language processing of the first text sequence and the target text sequence, that is, the natural language processing modifies the first text sequence more, causing the message that the user wants to convey through voice to be wrong, the present invention issues an alarm when the semantic similarity is less than a preset threshold value (that is, the semantic similarity is small), and causes the first text sequence to replace the target text sequence and be transmitted to the receiving end through wireless communication.

[0031] Further, calculating the semantic similarity between the first text sequence and the target text sequence includes: calculating the Levenshtein distance between the first text sequence and the target text sequence, denoted as L ; obtaining the absolute value of the difference in the number of stop words between the first text sequence and the target text sequence n 1 ; obtaining the absolute value of the difference in the number of keywords between the first text sequence and the target text sequence n 2 ; calculating the semantic similarity S where η 1 is the first weight of a preset size, and η 2 is the second weight of a preset size.

[0032] It should be noted that the Levenshtein distance is usually calculated using the dynamic programming algorithm. The Levenshtein distance (also known as the edit distance) is a metric used to measure the difference between two strings. It represents the minimum number of operations required to convert one string into another, and these operations include: Insert: Inserting a character into the string; Delete: Deleting a character from the string; Replace: Replacing one character with another. That is, the Levenshtein distance between the first text sequence and the target text sequence is the minimum number of operations to convert the first text sequence into the target text sequence. Based on this, the greater the Levenshtein distance between the first text sequence and the target text sequence, the smaller the text similarity between the two.

[0033] Furthermore, stop words refer to words that frequently appear in the text but contribute little to the overall meaning or theme of the text. Common stop words include "de", "shi", "zai", "he", "you", etc. The number of stop words usually reflects the complexity of the text. Based on this, the absolute value of the difference in the number of stop words between the first text sequence and the target text sequence n 1 represents the gap between the complexities of the two, and n 1 the larger it is, the greater the complexity gap between the two, and the smaller the text similarity between the two.

[0034] Keywords refer to words or phrases that have important meanings in the text and usually can represent the theme or core content of the text. Keywords can be nouns, verbs, or phrases and have strong information content. Based on this, the difference in the number of keywords between the first text sequence and the target text sequence n 2 ​reflects the gap between the information density of the two, and n 2 The larger it is, the greater the information density gap between the two is, and the smaller the text similarity between the two is.

[0035] Figure 2 is a schematic diagram showing the steps of a method for receiving narrowband private network voice based on ASR and TTS according to an embodiment of the present invention; like Figure 2 As shown, a narrowband private network voice receiving method based on ASR and TTS includes steps S201 to S202.

[0036] Step S201: obtaining a second text sequence output by a receiving end through wireless communication.

[0037] In one embodiment, the second text sequence output by the receiving end is received through the LoRa network.

[0038] It should be noted that the LoRa (Long Range) network is a low-power wide area network (LPWAN) technology designed for long-distance communications and low data rate applications. The core of the LoRa network is the LoRa modulation technology, which uses spread spectrum modulation to achieve long-distance, low-power wireless communications. Unlike traditional wireless communications, LoRa extends the signal to a wider frequency band so that it can still be received at low signal strength.

[0039] Step S202: inputting the second text sequence into the speech synthesis model to obtain a second speech signal.

[0040] Among them, the speech synthesis model is used to generate speech signals based on the text sequence. It should be noted that the speech synthesis model is a technology that generates natural language speech through computers, and is widely used in voice assistants, navigation systems, barrier-free technologies and other fields. The working flow of the speech synthesis model usually includes the following steps: Text analysis: Analyze the input text sequence, including word segmentation, syntactic analysis, part-of-speech tagging, etc., to understand the semantics of the text sequence; Rhythm generation: Generate appropriate rhythmic features based on the text content, such as tone, stress, rhythm, etc., to make the generated speech more natural; Vocoder generates audio: Input the analyzed and generated features into the vocoder to synthesize the final speech signal (speech signal).

[0041] In one embodiment, constructing the speech synthesis model includes: collecting k groups of speech synthesis training data, wherein each group of speech synthesis training data includes a text sample, and the label of the text sample is the speech recording data of the text sample, kis a positive integer of a preset size; construct a second neural network model; through k The speech synthesis neural network model is trained using the speech synthesis training data of the group, so that the trained speech synthesis neural network model is used as the speech synthesis model.

[0042] Figure 3 It is a structural block diagram schematically showing a narrowband private network voice transmission system based on ASR and TTS according to an embodiment of the present invention.

[0043] like Figure 3 As shown, a narrowband private network voice transmission system based on ASR and TTS includes: a transceiver antenna, an audio circuit, and a processor.

[0044] The transceiver antenna is used to receive or send a text sequence. The audio circuit is used to convert a voice signal into speech, or to receive speech and convert the speech into a voice signal. In one embodiment, the audio circuit is configured with a speaker and a microphone.

[0045] The processor is connected to the audio circuit and is used to convert the received voice signal into a text sequence, or input the voice signal into the audio circuit; the processor is also connected to the transceiver antenna, and the processor receives or sends the text sequence through the transceiver antenna, and the processor is used to convert the received text sequence into a voice signal.

[0046] In one embodiment, the processor is configured with an ASR speech recognition unit and a TTSA speech synthesis unit. The ASR speech recognition unit is used to convert a speech signal into a text sequence; and the TTSA speech synthesis unit is used to convert a text sequence into a speech signal.

[0047] In one embodiment, the processor is connected to the transceiver antenna via a LoRa gateway.

[0048] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A method for sending narrowband private network voice based on ASR and TTS, characterized in that: include: Acquire a speech signal and pre-process the speech signal; Inputting the speech signal into an acoustic model to obtain a first text sequence output by the acoustic model, wherein the acoustic model is used to output a text sequence according to the speech signal; Natural language processing is performed on the first text sequence to obtain a target text sequence, and the target text sequence is transmitted to a receiving end via wireless communication.

2. According to the method for sending narrowband private network voice based on ASR and TTS in claim 1, it is characterized in that: Also includes: Calculating semantic similarity between the first text sequence and the target text sequence; In response to the semantic similarity being less than a preset threshold, an alarm is issued, and the first text sequence replaces the target text sequence and is transmitted to a receiving end through wireless communication.

3. The method for sending narrowband private network voice based on ASR and TTS according to claim 2, characterized in that: Calculating the semantic similarity between the first text sequence and the target text sequence includes: Calculate the Levenshtein distance between the first text sequence and the target text sequence and record it as L ; Get the absolute value of the difference in the number of stop words between the first text sequence and the target text sequence n 1; Obtain the absolute value of the difference in the number of keywords between the first text sequence and the target text sequence n 2; Calculate the semantic similarity S ,in , η 1 is the first weight of the preset size, η 2 is the second weight of the preset size.

4. The method for sending narrowband private network voice based on ASR and TTS according to claim 1, characterized in that: The preprocessing of the speech signal includes: performing noise reduction, frame division and windowing processing on the speech signal.

5. The method for sending narrowband private network voice based on ASR and TTS according to claim 1, characterized in that: Constructing the acoustic model includes: collection m groups of acoustic training data, wherein each group of acoustic training data includes a speech sample signal, and the speech sample signal label is a transcribed text of the speech sample signal, m is a positive integer of the preset size; Constructing the first neural network model; pass m The first neural network model is trained using the acoustic training data of the group, so that the trained first neural network model is used as the acoustic model.

6. A narrowband private network voice sending method based on ASR and TTS according to claim 5, characterized in that: Training the first neural network model includes: Construct the first cross entropy loss function; For any set of acoustic training data, input it into the first neural network model to obtain the predicted text output by the first neural network model; Construct a one-hot code; obtain the probability distribution of each character in the predicted text according to the one-hot code and record it as the first probability distribution, and obtain the probability distribution of each character in the transcribed text and record it as the second probability distribution; The difference between the first probability distribution and the second probability distribution is obtained through a cross entropy function, and the parameters of the first neural network model are modified according to the difference.

7. The method for sending narrowband private network voice based on ASR and TTS according to claim 1, characterized in that: The natural language processing of the first text sequence includes: inputting the first text sequence into a trained NLP model, and determining that the text sequence output by the NLP model is a target text sequence.

8. A narrowband private network voice receiving method based on ASR and TTS, characterized in that: include: Obtaining a second text sequence output by a receiving end through wireless communication; The second text sequence is input into a speech synthesis model to obtain a second speech signal, wherein the speech synthesis model is used to generate a speech signal according to the text sequence.

9. A narrowband private network voice receiving method based on ASR and TTS according to claim 8, characterized in that: Constructing the speech synthesis model includes: collection k groups of speech synthesis training data, wherein each group of speech synthesis training data includes a text sample, and the label of the text sample is the speech recording data of the text sample, k is a positive integer of the preset size; Constructing a second neural network model; pass k The speech synthesis neural network model is trained using the speech synthesis training data of the group, so that the trained speech synthesis neural network model is used as the speech synthesis model.

10. A narrowband private network voice transmission system based on ASR and TTS, characterized in that: include: a transceiver antenna for receiving or sending a text sequence; An audio circuit for converting a voice signal into voice, or receiving voice and converting the voice into a voice signal; A processor, which is connected to the audio circuit and is used to convert a received voice signal into a text sequence, or input a voice signal into the audio circuit; the processor is also connected to the transceiver antenna, the processor receives or sends a text sequence through the transceiver antenna, and the processor is used to convert the received text sequence into a voice signal; the processor is used to implement a narrowband private network voice sending method based on ASR and TTS as described in any one of claims 1-7, or implement a narrowband private network voice receiving method based on ASR and TTS as described in any one of claims 8-9.

Images