Bluetooth earphone-based speech translation system and method

Abstract

The present invention provides a Bluetooth earphone-based speech translation system, comprising: a first Bluetooth translation earphone, a second Bluetooth translation earphone, an audio signal processing center, and a translation module. The audio signal processing center comprises a Fourier transform module, a signal cross-correlation processing module, a determination module, and a gain module. Time-frequency signal processing and signal cross-correlation processing are respectively performed on a first audio signal and a second audio signal acquired respectively by the first Bluetooth translation earphone and the second Bluetooth translation earphone, so as to determine a sound source position and perform gain processing on the first audio signal and the second audio signal; and finally, the translation module performs recognition and translation. The present invention further provides a Bluetooth earphone-based speech translation method. The speech translation system of the present invention solves the problem of confusion of speech recognition and translation, achieving more effective communication; and unnecessary speech recognition and translation are greatly reduced.

Description

A voice translation system and method based on Bluetooth headset Technical Field

[0001] The present invention relates to the field of speech translation, and in particular to a speech translation system and method based on a Bluetooth headset. Background Art

[0002] With economic globalization, exchanges between countries in business and daily life are becoming increasingly frequent. International academic presentations are often held between countries. Due to the diversity of languages, communication generally requires the use of translation equipment. To ensure the confidentiality and convenience of the translated content, translation equipment is often used in conjunction with a pair of Bluetooth translation headsets. Each party in the communication wears a Bluetooth headset, and each headset can collect voice signals and send them to a translator or cloud-based translation engine for translation. Finally, the translated voice signal is sent to the other headset for playback. Although this method can effectively achieve effective communication between two parties speaking different languages ​​in most cases, this type of translation Bluetooth headset has certain limitations. The two parties we usually communicate with are face to face and the distance between them is relatively small. For example, a pair of Bluetooth headsets are headset A and headset B, and the two people communicating are A and B. A wears headset A and B wears headset B. When A is speaking, due to the close distance between the two, both headset A and headset B can collect A's audio signal. Since the translator or cloud translation engine cannot identify whether the audio signal comes from A or B, the translator or cloud translation engine will recognize and translate any audio signal from headset A and headset B, that is, the translator or cloud translation engine will recognize and translate A's audio signal collected from headset A, and will also recognize and translate A's audio signal collected from headset B. Similarly, this situation will also occur when only B is speaking. This can easily cause confusion in the recognition and translation of the translator or translation engine, resulting in inaccurate recognition and translation, affecting normal communication and exchanges, while also increasing a large amount of unnecessary translation work and causing a waste of translation traffic resources. Contents of the invention

[0003] The purpose of the present invention is to overcome the defects and shortcomings of the existing technology and provide a voice translation system based on Bluetooth headsets. The system can identify which party of the communication the audio signals collected by two Bluetooth headsets come from, and only send the audio signal from the person wearing the headset to the translation machine or cloud translation engine for voice recognition and translation. This solves the confusion problem of voice recognition and translation, making communication and exchange more effective; at the same time, it also greatly reduces unnecessary voice recognition and translation.

[0004] To achieve the above object, the present invention provides a speech translation system based on a Bluetooth headset, the system comprising:

[0005] a first translation Bluetooth headset and a second translation Bluetooth headset, wherein the first translation Bluetooth headset and the second translation Bluetooth headset are respectively worn by users communicating with each other;

[0006] An audio signal processing center includes a Fourier transform module, a signal cross-correlation processing module, a judgment module and a gain module. The first audio signal and the second audio signal collected by the first translation Bluetooth headset and the second translation Bluetooth headset respectively are sent to the Fourier transform module for time-frequency signal processing, and then the first audio signal and the second audio signal are subjected to signal cross-correlation processing by the signal cross-correlation processing module; and the judgment module determines whether the first audio signal and the second audio signal are derived from the same sound source according to the magnitude of the signal cross-correlation value of the first audio signal and the second audio signal; when the first audio signal and the second audio signal are derived from the same sound source, the sound source position is determined according to the time delay relationship between the first audio signal and the second audio signal; the gain module sets the gain factor G1 of the first audio signal and the gain factor G2 of the second audio signal according to the obtained sound source position information;

[0007] The translation module recognizes and translates the first audio signal and the second audio signal after being processed by the gain module, and sends the translated audio signals to the first translation Bluetooth headset or the second translation Bluetooth headset accordingly.

[0008] According to an embodiment of the present invention, when the signal cross-correlation value between the first audio signal and the second audio signal ranges from (0.7, 1), the first audio signal and the second audio signal are signals from the same sound source.

[0009] According to one embodiment of the present invention, when the first audio signal and the second audio signal both come from the user wearing the first translation Bluetooth headset, the first gain factor G1=1, and the second gain factor G2=0; when the first audio signal and the second audio signal both come from the user wearing the second translation Bluetooth headset, the first gain factor G1=0, and the second gain factor G2=1.

[0010] According to one embodiment of the present invention, the audio signal processing center also includes a signal amplitude detection module for detecting the signal amplitudes of the first audio signal and the second audio signal; when the first audio signal and the second audio signal are from the same sound source, if the signal amplitude of the first audio signal is greater than the signal amplitude of the second audio signal, the first audio signal and the second audio signal are both from the user wearing the first translation Bluetooth headset; otherwise, the first audio signal and the second audio signal are both from the user wearing the second translation Bluetooth headset.

[0011] Another object of the present invention is to provide a speech translation method based on a Bluetooth headset, the method comprising the following steps: first, performing Fourier transform on the first audio signal and the second audio signal collected by the first translation Bluetooth headset and the second translation Bluetooth headset respectively, and performing time-frequency signal processing; then, performing signal cross-correlation processing on the first audio signal and the second audio signal after the time-frequency signal processing to obtain the signal cross-correlation value of the first audio signal and the second audio signal, and judging whether the first audio signal and the second audio signal come from the same sound source according to the magnitude of the signal cross-correlation value; when the first audio signal and the second audio signal come from the same sound source, judging the sound source position according to the time delay relationship between the first audio signal and the second audio signal; and setting the gain factor G1 of the first audio signal and the gain factor G2 of the second audio signal according to the sound source position information; finally, after gaining the first audio signal and the second audio signal, transmitting them to a translation machine or a cloud translation engine for recognition and translation, and sending the translated audio signals to the first translation Bluetooth headset or the second translation Bluetooth headset accordingly.

[0012] According to an embodiment of the present invention, determining whether the first audio signal and the second audio signal originate from the same sound source includes:

[0013] The cross-correlation function between the first audio signal and the second audio signal is expressed as follows:

[0014]

[0015] x1(t) is the signal propagation model of the first audio signal, x2(t) is the signal propagation model of the second audio signal,

[0016] x1(t)=α*s(t-τ1)+n1(t) (2)

[0017] x2(t)=β*s(t-τ2)+n2(t) (3)

[0018] Wherein, t represents time, s(·) represents the sound source model, n1(·) and n2(·) represent the noise model, x1(·) and x2(·) represent the signal models received at the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, τ1 and τ2 represent the time taken for the sound source to propagate to the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, α and β represent the energy attenuation factors of the sound source propagating to the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, and τ represents the signal propagation delay;

[0019] And assuming that the noise signal is uncorrelated with the speech signal and the noise signals are uncorrelated with each other, the cross-correlation function of the first audio signal and the second audio signal becomes,

[0020] From formula (4), we can calculate The value of When the value range of is (0.7, 1), the first audio signal and the second audio signal originate from the same sound source.

[0021] According to one embodiment of the present invention, the time delay relationship between the first audio signal and the second audio signal is used to determine the location of the sound source:

[0022] Formula (4) is transformed into,

[0023] According to the characteristics of the correlation function, R ss (τ1-τ2)≤R ss (0), from which the time delay τ=τ1-τ2 between the first audio signal and the second audio signal can be calculated. When the time delay τ is positive, it indicates that the first audio signal and the second audio signal both come from the user wearing the second translation Bluetooth headset. Conversely, when the time delay τ is negative, it indicates that the first audio signal and the second audio signal both come from the user wearing the first translation Bluetooth headset.

[0024] Compared with the prior art, the present invention has the following beneficial effects: the present invention provides a speech translation system and a speech translation method based on Bluetooth headsets, which can identify whether the first audio signal and the second audio signal originate from the same sound source, and determine which sound source the first audio signal and the second audio signal originate from, thereby setting corresponding gain factors for the above-mentioned first audio signal and the second audio signal, that is, when the first audio signal and the second audio signal both come from a user wearing a first translation Bluetooth headset, the first gain factor G1=1, and the second gain factor G2=0; when the first audio signal and the second audio signal both come from a user wearing a second translation Bluetooth headset, the first gain factor G1=0, and the second gain factor G2=1, and the gained first audio signal and the second audio signal are sent to a translation machine or a cloud translation engine for translation, thereby solving the confusion problem of recognition and translation, making communication and exchange more effective; at the same time, it also greatly reduces unnecessary speech recognition and translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of an application scenario model according to an embodiment of the present invention;

[0026] FIG2 is a functional block diagram of a speech translation system based on a Bluetooth headset according to the present invention;

[0027] FIG3 is a flow chart of a method for voice translation based on a Bluetooth headset according to the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention will be further described in detail below in conjunction with specific implementation methods.

[0029] As shown in Figure 1, an example diagram of the application scenario model of the present invention is shown. Usually, a pair of Bluetooth headsets has two left and right headsets, and the left and right Bluetooth headsets are worn by two people communicating with each other. For example, a pair of translation Bluetooth headsets includes a first translation Bluetooth headset (the left Bluetooth headset) and a second translation Bluetooth headset (the right Bluetooth headset). After the two headsets are worn, the first translation Bluetooth headset is located at A and the second translation Bluetooth headset is located at B. The two people communicating are A and B, where A wears the first translation Bluetooth headset and B wears the second translation Bluetooth headset. The content of the present invention is described in detail below using this scenario model. As shown in Figure 2, a functional block diagram of the speech translation system based on Bluetooth headsets of the present invention is shown. The system includes: a first translation Bluetooth headset and a second translation Bluetooth headset, and the first translation Bluetooth headset and the second translation Bluetooth headset are worn by two people communicating with each other. In an embodiment of the present invention, the first translation Bluetooth headset and the second translation Bluetooth headset can be the left and right headsets of a pair of Bluetooth headsets, or they can be two unpaired wireless Bluetooth headsets.

[0030] The speech translation system of the present invention further includes an audio signal processing center, which also includes a storage module, a processor, a power module, etc. for realizing the functions of the translation system. The audio signal processing center serves as the core of signal processing and mainly includes a Fourier transform module, a signal cross-correlation processing module, a judgment module, and a gain module. The signal collected by the first translation Bluetooth headset is a first audio signal, and the signal collected by the second translation Bluetooth headset is a second audio signal. The first audio signal and the second audio signal are sent to the Fourier transform module for time-frequency signal processing, and then the signal cross-correlation processing module performs signal cross-correlation processing on the first audio signal and the second audio signal; thereby obtaining a signal cross-correlation value, and judging whether the first audio signal and the second audio signal originate from the same sound source according to the magnitude of the signal cross-correlation value; when it is judged that the first audio signal and the second audio signal originate from the same sound source, the sound source position is judged according to the time delay relationship between the first audio signal and the second audio signal; when it is judged that the first audio signal and the second audio signal do not originate from the same sound source, the audio signal processing center does not process the two audio signals and only maintains the original signal collection function. The gain module sets a gain factor G1 of the first audio signal and a gain factor G2 of the second audio signal according to the obtained sound source position information;

[0031] In an embodiment of the present invention, the speech translation system further includes a translation module. After the first and second audio signals are processed by the gain module, the translation module recognizes and translates them. The translated audio signals are then sent to the first or second translation Bluetooth headset, respectively. Specifically, the first audio signal collected by the first translation Bluetooth headset is recognized and translated, then sent to the second translation Bluetooth headset for playback; and the second audio signal collected by the second translation Bluetooth headset is recognized and translated, then sent to the first translation Bluetooth headset for playback. This achieves accurate speech translation and avoids translation errors and confusion.

[0032] According to one embodiment of the present invention, whether the first audio signal and the second audio signal are from the same sound source is determined based on a signal cross-correlation value of the first audio signal and the second audio signal. When the signal cross-correlation value of the first audio signal and the second audio signal is (0.7, 1), it can be determined that the first audio signal and the second audio signal are from the same sound source; otherwise, the first audio signal and the second audio signal are from different sound sources.

[0033] According to one embodiment of the present invention, if the first audio signal and the second audio signal are from the same sound source, further signal processing is required. For example, when the first audio signal and the second audio signal both come from the user wearing the first translation Bluetooth headset, the first gain factor G1 is set to 1 and the second gain factor G2 is set to 0; when the first audio signal and the second audio signal both come from the user wearing the second translation Bluetooth headset, the first gain factor G1 is set to 0 and the second gain factor G2 is set to 1. The purpose of setting the gain factors in this way is to obtain the required target speech signal for accurate recognition and translation.

[0034] According to one embodiment of the present invention, the audio signal processing center also includes a signal amplitude detection module for detecting the signal amplitude of the first audio signal and the second audio signal; when the first audio signal and the second audio signal are from the same sound source, if the signal amplitude of the first audio signal is greater than the signal amplitude of the second audio signal, the first audio signal and the second audio signal are both from the user wearing the first translation Bluetooth headset; otherwise, the first audio signal and the second audio signal are both from the user wearing the second translation Bluetooth headset.

[0035] Another object of the present invention is to provide a speech translation method based on a Bluetooth headset, the method comprising the following steps: first, performing Fourier transform on the first audio signal and the second audio signal collected by the first translation Bluetooth headset and the second translation Bluetooth headset respectively, and performing time-frequency signal processing; then, performing signal cross-correlation processing on the first audio signal and the second audio signal after the time-frequency signal processing to obtain the signal cross-correlation value of the first audio signal and the second audio signal, and judging whether the first audio signal and the second audio signal come from the same sound source according to the magnitude of the signal cross-correlation value; when the first audio signal and the second audio signal come from the same sound source, judging the sound source position according to the time delay relationship between the first audio signal and the second audio signal; and setting the gain factor G1 of the first audio signal and the gain factor G2 of the second audio signal according to the sound source position information; finally, after gaining the first audio signal and the second audio signal, transmitting them to the translation machine or the cloud translation engine for recognition and translation, and sending the translated audio signal to the first translation Bluetooth headset or the second translation Bluetooth headset accordingly.

[0036] According to one embodiment of the present invention, determining whether the first audio signal and the second audio signal originate from the same sound source includes:

[0037] The cross-correlation function between the first audio signal and the second audio signal is expressed as follows:

[0038]

[0039] x1(t) is the signal propagation model of the first audio signal, x2(t) is the signal propagation model of the second audio signal,

[0040] x1(t)=α*s(t-τ1)+n1(t) (2)

[0041] x2(t)=β*s(t-τ2)+n2(t) (3)

[0042] Wherein, t represents time, s(·) represents the sound source model, n1(·) and n2(·) represent the noise model, x1(·) and x2(·) represent the signal models received at the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, τ1 and τ2 represent the time taken for the sound source to propagate to the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, α and β represent the energy attenuation factors of the sound source propagating to the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, and τ represents the signal propagation delay;

[0043] And assuming that the noise signal is uncorrelated with the speech signal and the noise signals are uncorrelated with each other, the cross-correlation function of the first audio signal and the second audio signal becomes,

[0044] From formula (4), we can calculate The value of When the value of is (0.7, 1), the first audio signal and the second audio signal come from the same sound source. In the embodiment of the present invention, when When the value of is (0, 0.7), it indicates that the first audio signal and the second audio signal do not come from the same sound source. In this case, the first audio signal and the second audio signal only need to be saved without being identified and translated.

[0045] According to one embodiment of the present invention, when the first audio signal and the second audio signal both come from the same sound source, the location of the sound source can be determined based on the time delay relationship between the first audio signal and the second audio signal:

[0046] Formula (4) is transformed into,

[0047] According to the characteristics of the correlation function, R ss (τ1-τ2)≤R ss(0), from which the time delay τ=τ1-τ2 between the first audio signal and the second audio signal can be calculated. When the time delay τ is positive, it indicates that the first audio signal and the second audio signal are both from the user wearing the second translation Bluetooth headset. On the contrary, when the time delay τ is negative, it indicates that the first audio signal and the second audio signal are both from the user wearing the first translation Bluetooth headset. In the embodiment of the present invention, the location of the signal source can also be determined based on the amplitude or strength of the collected signal, that is, E1(*) represents the energy of signal x1, and E2(*) represents the energy of signal x2, which indicates that both the first audio signal and the second audio signal come from the user wearing the first translation Bluetooth headset; otherwise, This indicates that the first audio signal and the second audio signal both come from the user wearing the second translation Bluetooth headset.

[0048] In summary, the present invention provides a speech translation system and a speech translation method based on Bluetooth headsets. The system can identify whether a first audio signal and a second audio signal originate from the same sound source signal, and determine which sound source the first audio signal and the second audio signal originate from the same sound source signal, thereby setting corresponding gain factors for the above-mentioned first audio signal and the second audio signal. That is, when the first audio signal and the second audio signal both come from a user wearing a first translation Bluetooth headset, the first gain factor G1=1, and the second gain factor G2=0; when the first audio signal and the second audio signal both come from a user wearing a second translation Bluetooth headset, the first gain factor G1=0, and the second gain factor G2=1. The gained first audio signal and the second audio signal are sent to a translation machine or a cloud translation engine for translation, which solves the confusion problem of recognition and translation, making communication and exchange more effective; at the same time, it also greatly reduces unnecessary speech recognition and translation.

[0049] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A voice translation system based on Bluetooth headset, characterized in that: The system comprises: A first translation Bluetooth headset and a second translation Bluetooth headset, wherein the first translation Bluetooth headset and the second translation Bluetooth headset are respectively worn by users communicating with each other; An audio signal processing center, the audio signal processing center includes a Fourier transform module, a signal cross-correlation processing module, a judgment module and a gain module, the first audio signal and the second audio signal respectively collected by the first translation Bluetooth headset and the second translation Bluetooth headset are sent to the Fourier transform module for time-frequency signal processing, and then the first audio signal and the second audio signal are subjected to signal cross-correlation processing by the signal cross-correlation processing module; and the judgment module determines whether the first audio signal and the second audio signal are derived from the same sound source according to the magnitude of the signal cross-correlation value of the first audio signal and the second audio signal; when the first audio signal and the second audio signal are derived from the same sound source, the sound source position is determined according to the time delay relationship between the first audio signal and the second audio signal; the gain module sets the gain factor G1 of the first audio signal and the gain factor G2 of the second audio signal according to the obtained sound source position information; The translation module recognizes and translates the first audio signal and the second audio signal after being processed by the gain module, and sends the translated audio signals to the first translation Bluetooth headset or the second translation Bluetooth headset accordingly.

2. The voice translation system based on Bluetooth headset according to claim 1, characterized in that: When the signal cross-correlation value between the first audio signal and the second audio signal is in the range of (0.7, 1), the first audio signal and the second audio signal are signals of the same sound source.

3. The voice translation system based on Bluetooth headset according to claim 1, characterized in that: When both the first audio signal and the second audio signal come from the user wearing the first translation Bluetooth headset, the first gain factor G1=1, and the second gain factor G2=0; when both the first audio signal and the second audio signal come from the user wearing the second translation Bluetooth headset, the first gain factor G1=0, and the second gain factor G2=1.

4. The voice translation system based on Bluetooth headset according to claim 1, characterized in that: The audio signal processing center also includes a signal amplitude detection module for detecting the signal amplitudes of the first audio signal and the second audio signal; when the first audio signal and the second audio signal are derived from the same sound source, if the signal amplitude of the first audio signal is greater than the signal amplitude of the second audio signal, the first audio signal and the second audio signal are both from a user wearing the first translation Bluetooth headset; otherwise, the first audio signal and the second audio signal are both from a user wearing the second translation Bluetooth headset.

5. A speech translation method based on the speech translation system based on a Bluetooth headset according to any one of claims 1 to 4, characterized in that: The method comprises the following steps: firstly, performing Fourier transformation on the first audio signal and the second audio signal respectively collected by the first translation Bluetooth headset and the second translation Bluetooth headset to perform time-frequency signal processing; then performing signal cross-correlation processing on the first audio signal and the second audio signal after the time-frequency signal processing to obtain the signal cross-correlation value of the first audio signal and the second audio signal, and judging whether the first audio signal and the second audio signal are derived from the same sound source according to the magnitude of the signal cross-correlation value; when the first audio signal and the second audio signal are derived from the same sound source, judging the position of the sound source according to the time delay relationship between the first audio signal and the second audio signal; and setting the gain factor G1 of the first audio signal and the gain factor G2 of the second audio signal according to the sound source position information; finally, transmitting the first audio signal and the second audio signal to a translation machine or a cloud translation engine for recognition and translation after gaining them, and sending the translated audio signals to the first translation Bluetooth headset or the second translation Bluetooth headset accordingly.

6. The method for voice translation based on Bluetooth headset according to claim 5, characterized in that: The determining whether the first audio signal and the second audio signal are from the same sound source comprises: The cross-correlation function between the first audio signal and the second audio signal is expressed as follows: x1(t) is the signal propagation model of the first audio signal, x2(t) is the signal propagation model of the second audio signal, x1(t)=α*s(t-τ1)+n1(t) (2) x2(t)=β*s(t-τ2)+n2(t) (3) Wherein, t represents time, s(·) is the sound source model, n1(·) and n2(·) represent noise models, x1(·) and x2(·) represent the signal models received at the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, τ1 and τ2 represent the time taken for the sound source to propagate to the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, α and β represent the energy attenuation factors of the sound source propagating to the first translation Bluetooth headset and the second translation Bluetooth headset, respectively, and τ represents the signal propagation delay; And assuming that the noise signal is uncorrelated with the speech signal and the noise signals are uncorrelated with each other, the cross-correlation function of the first audio signal and the second audio signal becomes, From formula (4), we can calculate The value of When the value range of is (0.7, 1), the first audio signal and the second audio signal originate from the same sound source.

7. The method for voice translation based on Bluetooth headset according to claim 6, characterized in that: The time delay relationship between the first audio signal and the second audio signal is used to determine the location of the sound source: Formula (4) is transformed into: According to the characteristics of the correlation function, R ss (τ1-τ2)≤R ss (0), from which the time delay τ=τ1-τ2 between the first audio signal and the second audio signal can be calculated. When the time delay τ is positive, it indicates that the first audio signal and the second audio signal are both from the user wearing the second translation Bluetooth headset. Conversely, when the time delay τ is negative, it indicates that the first audio signal and the second audio signal are both from the user wearing the first translation Bluetooth headset.

Images