Real-time translation system, method and earphone

Abstract

The invention discloses a real-time translation system, and the system comprises a collection unit which is used for collecting an audio signal in a surrounding environment; the endpoint detection unit is used for identifying a starting point and an ending point of the collected audio signal and segmenting the audio signal into independent audio clips; the voice recognition unit is used for performing recognition processing on the audio clips and converting the audio clips into text data; the translation unit is used for translating the text data into a target language text; the text-to-speech unit is used for converting the translated target language text into a smooth audio signal; and the output unit is used for playing the converted audio signal by using a target language, by adopting the method, the end-to-end processing from voice capture to translation output is realized by utilizing the structural space advantage of the earphone cabin and being assisted by reasonable calculation hardware capability, a convenient and efficient cross-language communication solution is provided for a user, and the user experience is improved. And compared with the real-time translation of the terminal, the cost is lower, and the threshold is smaller.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to a real-time translation system, method and earphone. Background Art

[0002] Under normal circumstances, both parties on a call will communicate in the same language, but if they encounter a language they are not familiar with, translation is required. There are two existing translation methods. One is the online translation function that comes with the terminal. This method is that the manufacturer implants its own translation function in the terminal when it leaves the factory to obtain the function of real-time translation during the call, but the price is relatively high and it depends on the network signal. The translation capability is based on remote processing; the second is the terminal translation APP or translation device. This translation method is only applicable to offline, and it can no longer translate in real time during a call, and its applicability to scenarios is limited.

[0003] Therefore, there is an urgent need to design a real-time translation system, method and headset to overcome one or more deficiencies of the above-mentioned prior arts. Summary of the invention

[0004] The technical solution adopted by the present invention to solve the above technical problems is: in the first aspect, a real-time translation system is provided, characterized in that it includes: a collection unit, which is used to collect audio signals in the surrounding environment; an endpoint detection unit, which is used to identify the starting point and the ending point of the collected audio signal, and divide the audio signal into independent audio segments; a speech recognition unit, which is used to recognize and process the audio segments, and convert the audio segments into text data; a translation unit, which is used to translate the text data into a target language text; a text-to-speech unit, which is used to convert the translated target language text into a fluent audio signal; and an output unit, which is used to play the converted audio signal in the target language.

[0005] In a preferred embodiment, the endpoint detection unit includes: an audio processing module, and the audio processing module is used to perform echo cancellation and noise reduction processing on the collected audio signal.

[0006] In a preferred embodiment, the speech recognition unit comprises: a language recognition module, wherein the language recognition module is used to recognize the language of the audio segment.

[0007] In a preferred embodiment, the speech recognition unit further includes: a voiceprint feature collection module, which is used to collect voiceprint features in the audio clip and divide the voiceprint features into the first target and the second target according to the voiceprint features.

[0008] In a preferred embodiment, the features collected by the voiceprint feature collection module also include: timbre and pitch.

[0009] In a preferred embodiment, the translation unit can translate online or offline.

[0010] In a second aspect, a real-time translation method is provided, characterized in that the translation method comprises the following steps:

[0011] S1, collect audio, use the audio collector to collect audio signals in the surrounding environment in real time;

[0012] S2, analyzing the audio signal collected by the audio collector, analyzing the voice end and the non-voice segment, and identifying the starting point and the ending point of the voice segment, then performing echo and noise reduction processing on the audio signal of the voice segment, and cutting the processed audio signal into independent audio segments;

[0013] S3, processing the segmented audio segments using a speech recognition engine, converting the audio segments into text data, identifying the language of the audio segments while converting, and extracting the voiceprint, timbre, and pitch features in the audio segments, and setting them as the first target or the second target;

[0014] S4, translating the text data obtained in step S3 using the translation model to convert the source language text into the target language text;

[0015] S5, converting the target language text information in S4 into a fluent audio signal, and playing the audio in the target language.

[0016] A third aspect provides a translation headset, comprising the real-time translation system described in any one of the above embodiments, characterized in that it comprises: an earphone compartment, to which the earphone is communicatively connected, and the earphone is provided with a collection unit and an output unit; an endpoint detection unit, a speech recognition unit, and a translation unit are provided in the earphone compartment; and a terminal, which is wirelessly connected to the earphone compartment.

[0017] The beneficial effects of the present invention are as follows: the present application realizes end-to-end processing from voice capture to translation output by utilizing the structural space advantages of the earphone compartment and supplemented by reasonable computing hardware capabilities, thereby providing users with a convenient and efficient cross-language communication solution. Compared with the real-time translation provided by the terminal, the cost is lower and the threshold is lower. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a system block diagram of the present invention.

[0019] In the figure:

[0020] 10. Collection unit; 11. Endpoint detection unit; 111. Audio processing module; 12. Speech recognition unit; 121. Language recognition module; 122. Voiceprint feature collection module; 13. Translation unit; 14. Text-to-speech unit; 15. Output unit. DETAILED DESCRIPTION

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0022] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms, "connection" and "installation" should be understood in a broad sense. For example, "connection" can be a detachable connection or a non-detachable connection; it can be a direct connection or an indirect connection through an intermediate medium. In addition, "connection" can be a direct connection or an indirect connection through an intermediate medium. Among them, "fixed" means that the two are connected to each other and the relative position relationship after connection remains unchanged. The directional terms mentioned in the embodiments of the present invention, such as "inside", "outside", "top", "bottom", etc., are only reference to the directions of the accompanying drawings. Therefore, the directional terms used are for better and clearer explanation and understanding of the embodiments of the present invention, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0023] In the embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0024] In the embodiments of the present invention, "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0025] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0026] like Figure 1As shown, in order to solve the problems in the prior art that the real-time translation cost is high and the translation software cannot translate in real time, the present invention provides a translation system, including a collection unit 10, which is arranged in the earphone. During a call, the collection unit 10 collects relevant audio signals and sends the audio signals to an endpoint detection unit 11 in the earphone compartment. The endpoint detection unit 11 uses Voice activity detection, VAD (voice activity detection) to recognize the audio signal sent by the earphone, and divides the audio signal into a speech segment and a non-speech segment. The non-speech segment refers to silence or noise. In order to improve the accuracy of the translation, the endpoint detection unit 11 is also provided with an audio processing module 111. The audio processing module 111 will eliminate echoes and perform noise reduction processing on the audio information of the speech segment to reduce impurity information in the audio information. After the echo and noise reduction processing, the processed audio information is analyzed to obtain the starting point and the ending point, and the audio information is cut into independent audio segments. The endpoint detection unit 11 then sends the independent audio segments to the speech recognition unit 12. The speech recognition unit 12 uses Automatic Speech Recognition (speech recognition technology) and ka ld i2 converts the audio clip and converts the audio signal into text data. During the conversion process, the speech recognition unit 12 will also identify the language of the current audio clip, record it, and then send it to the translation unit 13. The translation unit 13 will translate the source language text data into the target language text. The translation unit 13 is provided with a memory, and the memory stores the translation data. The memory will be updated when the network conditions are met to ensure the accuracy of the text translated by the offline database. When the network signal is good, the latest updated translation data is used first. When there is no network signal, the last updated translation data is used. After the conversion is completed, the translation unit 13 will send the target language text to the text-to-speech unit 14. The text-to-speech unit 14 uses Text-To-Speech, TTS (text-to-speech technology) to convert the target language text into a fluent audio signal, and then broadcasts the audio signal through the output unit 15. The output unit 15 is also arranged in the earphone, so as to realize real-time translation during the call and improve the smoothness of communication.

[0027] The specific process is that in a call environment, the translation system will collect timbre, tone, voiceprint and other features according to the voiceprint feature collection module 122 to mark the first target and the second target; taking Chinese and English as an example, the party is the first target and the language is Chinese, and the other party is the second target and the language is English. The first target can pre-set the translation language to Chinese through the terminal APP; when the English audio signal of the second target is transmitted to the earphone, the earphone transmits the audio signal to the earphone compartment, wherein the communication method between the earphone and the earphone compartment can be wireless or wired, and the endpoint detection unit 11, speech recognition unit 12, and translation unit 13 in the earphone compartment perform the aforementioned processing on the audio signal, and then output it to the first target in Chinese through the output unit 15 to achieve one-way translation.

[0028] To further enhance the user experience, you can also use the terminal APP to display the audio of the second target in Chinese, so that the first target can understand it more intuitively.

[0029] If the first target does not speak English, the translation system can also use the language recognition module 121 in the speech recognition unit 12 to obtain the language category of the second target according to the needs of the first target, and convert the Chinese audio signal into an English audio signal for output; specifically, the acquisition unit 10 will obtain the audio signal of the second target and send it to the earphone compartment, the earphone compartment will convert the audio signal of the second target into text data, and convert the text data into an English audio signal through the language information recognized by the language recognition module 121, and send the audio signal to the terminal, which outputs it to the second target, thereby realizing two-way translation, wherein the communication method between the terminal and the earphone compartment is preferably wireless communication, and the options are Bluetooth, Wi-Fi Direct or mobile data network.

[0030] It should be pointed out that when the language of the first target needs to be converted into the language of the second target for output, the translation system can be put into interchange mode by touching the earphones, etc. At this time, the audio signal of the first target will be converted into the audio signal of the language of the second target and output to the second target.

[0031] To sum up, this application realizes end-to-end processing from voice capture to translation output by utilizing the structural space advantages of the earphone pod and supplemented by reasonable computing hardware capabilities, providing users with a convenient and efficient cross-language communication solution. Compared with the real-time translation that comes with the terminal, it has lower costs and lower barriers.

[0032] The present invention is not limited to what is described in the specification and implementation modes, and therefore additional advantages and modifications can be easily realized by those skilled in the art. Therefore, without departing from the spirit and scope of the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein.

Claims

1. A real-time translation system, characterized in that: include: A collection unit, used for collecting audio signals in the surrounding environment; The endpoint detection unit is used to identify the starting point and the ending point of the collected audio signal and divide the audio signal into independent audio segments; the speech recognition unit is used to recognize and process the audio segments and convert the audio segments into text data; the translation unit is used to translate the text data into the target language text; the text-to-speech unit is used to convert the translated target language text into a fluent audio signal; The output unit is used to play the converted audio signal in the target language.

2. The real-time translation system according to claim 1, characterized in that: The endpoint detection unit includes: an audio processing module, and the audio processing module is used to perform echo cancellation and noise reduction processing on the collected audio signal.

3. The real-time translation system according to claim 2, characterized in that: The speech recognition unit includes: a language recognition module, and the language recognition module is used to recognize the language of the audio segment.

4. The real-time translation system according to claim 3, characterized in that: The speech recognition unit further includes: a voiceprint feature collection module, which is used to collect voiceprint features in the audio clip and divide the voiceprint features into a first target and a second target according to the voiceprint features.

5. The real-time translation system according to claim 4, characterized in that: The features collected by the voiceprint feature collection module also include: timbre and pitch.

6. The real-time translation system according to claim 5, characterized in that: The translation unit can translate online or offline.

7. A real-time translation method, characterized in that: The translation method comprises the following steps: S1, collect audio, use the audio collector to collect audio signals in the surrounding environment in real time; S2, analyzing the audio signal collected by the audio collector, analyzing the voice end and the non-voice segment, and identifying the starting point and the ending point of the voice segment, then performing echo and noise reduction processing on the audio signal of the voice segment, and cutting the processed audio signal into independent audio segments; S3, processing the segmented audio segments using a speech recognition engine, converting the audio segments into text data, identifying the language of the audio segments while converting, and extracting the voiceprint, timbre, and pitch features in the audio segments, and setting them as the first target or the second target; S4, translating the text data obtained in step S3 using the translation model to convert the source language text into the target language text; S5, converting the target language text information in S4 into a fluent audio signal, and playing the audio in the target language.

8. A translation headset, comprising the real-time translation system according to any one of claims 1 to 6, characterized in that include: An earphone compartment, wherein the earphone compartment is communicatively connected to an earphone, and the earphone is provided with a collection unit and an output unit; The earphone compartment is provided with an endpoint detection unit, a speech recognition unit, and a translation unit; A terminal is wirelessly connected to the earphone compartment.

Images