Audio translation method and system

Abstract

According to the audio translation method and system, a translation machine is connected with other intelligent terminals through a USB, after the USB connection is established, the intelligent terminal recognizes the translation machine as an earphone, on the basis, when the intelligent terminal has a translation demand, original audio data to be translated can be sent to the translation machine through the USB, and the translation machine can translate the original audio data through the USB. The method comprises the following steps: in a state that a translation function of a translation machine is started, translating original audio data to obtain a translated text of a target language, synthesizing the translated text into first translated audio data, obtaining to-be-output first target audio data based on the first translated audio data, and outputting and playing the first target audio data. By the adoption of the scheme, when the intelligent terminal has the audio translation requirement, the functions of automatic translation, audio synthesis and output playing can be achieved by means of the external translation machine, the audio translation requirement of a user on the intelligent terminal is met, and the use scene of the translation machine is expanded.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and more specifically, to an audio translation method and system. Background Art

[0002] Existing translation machines are more focused on the function of translation, such as using translation machines to achieve simultaneous interpretation functions, as a device for face-to-face communication, etc.

[0003] Although some translation machines are equipped with some application APPs, such as taking photos, they cannot install more abundant application APPs like computers and mobile phones, such as multimedia APPs, social APPs, online conference APPs, etc. In some usage scenarios, users may need to use multimedia APPs, social APPs or online conference APPs installed on computers, mobile phones and other terminals. At this time, users may have the need to translate audio, such as in online conference scenarios, they need to translate the speeches of other participants. Since the above APPs are installed on computers, mobile phones and other terminals, the translation function of the translation machine cannot be effectively utilized, resulting in limited usage scenarios of the translation machine and failure to meet the translation needs of users. Summary of the invention

[0004] In view of the above problems, this application is proposed to provide an audio translation method and system to solve the problem that mobile phones, computers and other terminals need to translate voice and expand the use scenarios of translation machines. The specific solution is as follows:

[0005] In a first aspect, an audio translation method is provided, which is applied to a translation machine, wherein the translation machine is connected to an intelligent terminal via a universal serial bus (USB), and after the USB connection is established, the intelligent terminal recognizes the translation machine as a headset, and the audio translation method comprises:

[0006] Receiving original audio data transmitted by the intelligent terminal via USB;

[0007] When the translation machine is in a first state, translating the original audio data to obtain a translated text in a target language, the first state being a state in which the translation function is turned on;

[0008] synthesizing the translated text in the target language into first translated audio data;

[0009] First target audio data to be output is obtained based on the first translated audio data, and the first target audio data is output and played.

[0010] In a possible design, in another implementation of the first aspect of the embodiment of the present application, a process of obtaining first target audio data to be output based on the first translated audio data includes:

[0011] using the first translated audio data as the first target audio data;

[0012] or,

[0013] The first translated audio data and the original audio data are spliced ​​together to obtain spliced ​​first target audio data.

[0014] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the original audio data is undecoded audio stream data; then before translating the original audio data, the method further includes:

[0015] The digital signal processing unit DSP in the translator is called to perform hard decoding on the original audio data to obtain decoded original audio data.

[0016] In a possible design, in another implementation of the first aspect of the embodiments of the present application, before translating the original audio data, the method further includes:

[0017] The original audio data is preprocessed, and the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

[0018] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0019] Obtaining ambient audio data collected by the sound receiving unit;

[0020] When the translation machine is in the first state, translating the environmental audio data to obtain a translated text in the source language;

[0021] synthesizing the translated text in the source language into second translated audio data;

[0022] Second target audio data is obtained based on the second translated audio data, and the second target audio data is output to the smart terminal through the USB connection.

[0023] In one possible design, in another implementation of the first aspect of the embodiments of the present application, a process of obtaining second target audio data based on the second translated audio data includes:

[0024] using the second translated audio data as the second target audio data;

[0025] or,

[0026] The second translated audio data and the environmental audio data are spliced ​​to obtain spliced ​​second target audio data.

[0027] In a possible design, in another implementation of the first aspect of the embodiment of the present application, before translating the environmental audio data, the method further includes:

[0028] The environmental audio data is preprocessed, and the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

[0029] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the process of outputting and playing the first target audio data includes:

[0030] Playing the first target audio data through the speaker of the translation machine;

[0031] or,

[0032] When the translation machine is connected to the earphone, the first target audio data is output to the earphone so as to play the first target audio data through the earphone.

[0033] In a second aspect, an audio translation system is provided, comprising:

[0034] Smart terminals and translation machines;

[0035] The intelligent terminal is connected to the translation machine via a universal serial bus (USB), and after the USB connection is established, the intelligent terminal identifies the translation machine as a headset;

[0036] The intelligent terminal is used to transmit the original audio data to the translation machine via a USB connection;

[0037] The translation machine is used to implement the audio translation method described in any one of the first aspects of the present application.

[0038] In one possible design, in another implementation of the second aspect of the embodiment of the present application, the translation machine is provided with a headset chip and a main control chip, and the USB connection adopts an interface protocol supported by the headset chip;

[0039] The headphone chip is used to implement data forwarding processing between the smart terminal and the main control chip, and the main control chip is used to implement the audio translation method described in any one of the first aspects of the present application.

[0040] In one possible design, in another implementation of the second aspect of the embodiment of the present application, the USB connection between the smart terminal and the translation machine is specifically: the translation machine realizes USB connection with the smart terminal through a configured analog interface.

[0041] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the USB connection uses the UAC protocol for data transmission.

[0042] In a third aspect, an audio translation device is provided, which is applied to a translation machine, wherein the translation machine is connected to an intelligent terminal via a universal serial bus (USB), and after the USB connection is established, the intelligent terminal recognizes the translation machine as a headset, and the audio translation device method comprises:

[0043] The original data receiving module is used to receive the original audio data transmitted by the intelligent terminal via USB;

[0044] A translation module, used for translating the original audio data to obtain a translated text in a target language when the translation machine is in a first state, wherein the first state is a state in which a translation function is enabled;

[0045] a speech synthesis module, configured to synthesize the translated text in the target language into first translated audio data;

[0046] The audio output module is used to obtain first target audio data to be output based on the first translated audio data, and output and play the first target audio data.

[0047] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the audio translation method described in any one of the first aspects of the present application is implemented.

[0048] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the audio translation method described in any one of the first aspects of the present application.

[0049] By means of the above technical scheme, the present application proposes a design scheme that connects the translator to other smart terminals via USB as an external device, that is, the translator is connected to other smart terminals via USB, and after the USB connection is established, the smart terminal identifies the translator as a headset. On this basis, when there is a translation demand, the smart terminal can send the original audio data to be translated to the translator via USB, and when the translator turns on the translation function, the original audio data is translated to obtain the translated text in the target language, and synthesize it into the first translated audio data, and obtain the first target audio data to be output based on the first translated audio data, and output and play the first target audio data. Obviously, by adopting the scheme of the present application, when there is a demand for audio translation in the smart terminal, the functions of automatic translation, audio synthesis and output playback can be realized with the help of the external translator, so as to meet the audio translation needs of users on the smart terminal and expand the use scenarios of the translator. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0051] Figure 1 A framework diagram of an audio translation system provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of various schemes for realizing USB connection of a translation machine provided in an embodiment of the present application;

[0053] Figure 3 A flowchart of an audio translation method provided in an embodiment of the present application;

[0054] Figure 4 Another audio translation system framework diagram provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of an audio translation device provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of the structure of a translation machine provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] Existing translation machines focus more on translation functions. Even if they are equipped with some apps, such as camera functions, they cannot be equipped with some multimedia apps, social apps, online conference apps, etc. like computers, mobile phones and other smart terminals. When users use multimedia apps, social apps or online conference apps through smart terminals, they sometimes face the need to translate audio, such as translating the audio of foreign language movies played by video players, translating the audio of other participants in online meetings, etc. Traditional solutions cannot use translation machines quickly and directly, and users need to perform tedious operations such as recording, exporting, and importing the audio of smart terminals into translation machines, which is extremely inconvenient to use.

[0059] In this regard, the present application proposes a design scheme in which a translation machine uses a USB connection to other smart terminal devices as a peripheral, which solves the problem that smart terminals such as mobile phones and computers need to translate audio, expands the use scenarios of the translation machine, and facilitates user use.

[0060] Combination Figure 1 , Figure 1 A framework diagram of an audio translation system is provided, wherein the audio translation system includes a smart terminal 100 and a translation machine 200.

[0061] The intelligent terminal 100 is connected to the translation machine 200 via a universal serial bus USB. After the USB connection is established, the intelligent terminal 100 recognizes the translation machine 200 as a headset.

[0062] The intelligent terminal 100 can be a terminal device such as a mobile phone, a PC, a tablet computer, an IPAD, etc., on which various types of application APPs can be installed, such as social APPs, online conference APPs, etc. When using these APPs, users may have the need to translate the audio. For example, when using a chat APP to chat with foreigners, the audio sent by the other party needs to be translated into a language that the user can understand. Or, when using an online conference APP, the audio sent by other participants needs to be translated into a language that the user can understand, etc.

[0063] The intelligent terminal 100 can transmit the original audio data to be translated to the translation machine 200 via a USB connection.

[0064] The translation machine 200 is connected as a peripheral device of the intelligent terminal 100 via a USB to implement each step of the audio translation method.

[0065] The process of the intelligent terminal 100 identifying the translation machine 200 as a headset through the USB connection can be implemented in a variety of ways. Figure 2 , this embodiment provides three optional implementation methods.

[0066] The first one: headphone chip solution

[0067] Specifically, a headset chip and a main control chip are provided in the translation machine, and the USB connection adopts an interface protocol supported by the headset chip.

[0068] The earphone chip is used to realize data forwarding processing between the intelligent terminal and the main control chip, and the main control chip is used to realize each step of the audio translation method.

[0069] Furthermore, the headphone chip can also realize the analog / digital conversion function, that is, converting audio data between analog signals and digital signals.

[0070] The second type: analog interface solution

[0071] Specifically, an analog interface (analog switch) is configured by software on the translation machine side, and a USB connection is achieved with the smart terminal through the analog interface to receive the audio data sent by the smart terminal and convert it into an analog signal. In addition, the audio synthesized by the translation machine can also be sent to the smart terminal.

[0072] The third type: digital interface solution

[0073] Specifically, the USB connection between the smart terminal and the translator uses the UAC protocol (full name: USB Audio Class protocol) for data transmission. The UAC protocol defines the communication method between the translator and the smart terminal and the transmission specifications of audio data.

[0074] After the smart terminal is connected to the translator via USB, it needs to obtain the configuration and parameter information of the translator through the UAC protocol (corresponding to obtaining the configuration parameters of the translator), such as sampling rate, bit depth, number of channels, volume control mute settings, etc.

[0075] Further, the USB configuration descriptor is scanned. The USB configuration descriptor is a part of the USB device descriptor, which contains the configuration information of the device, such as the number of interfaces, the number of endpoints, power requirements, etc. After the smart terminal is connected to the translator via USB, the smart terminal scans the USB configuration descriptor to obtain the configuration information of the translator.

[0076] Furthermore, the port is opened after matching. That is, after the smart terminal scans the USB configuration descriptor and parses the configuration information of the translator, the smart terminal matches with the translator for a long time. After the match is successful, the smart terminal will open the ports required for communication with the translator, and these ports will be used for subsequent reception / transmission of audio data.

[0077] After the port is opened and the communication link is established, the terminal can begin to receive or send digital audio data.

[0078] The embodiments of the present application illustrate three optional solutions for using a translation machine as an external headset of a smart terminal via a USB connection, and any one of the solutions can be selected.

[0079] After the translator 200 is connected to the smart terminal 100 via USB, the smart terminal 100 will detect the headphone device. The smart terminal 100 sends a request to the translator 200 via the USB interface to obtain relevant configuration information of the translator, such as configuration items, sampling rate, bit rate, etc.

[0080] The translator 200 returns the relevant configuration information to the smart terminal 100, and the smart terminal 100 sets the corresponding audio input and output parameters according to the configuration information. Afterwards, when the smart terminal 100 needs to use the headphone function, the translator 200 can be identified as a headphone, and the original audio data can be sent to the translator 200 through the USB interface. The data can be used as uplink data and processed on the uplink. After processing, the playback unit of the translator 200 is driven to play.

[0081] Further optionally, the sound receiving unit of the translation machine 200 can also collect environmental audio data, which is used as downlink data, processed via the downlink link, and sent to the smart terminal 100 through the USB interface after processing.

[0082] In some possible implementations, in order to reduce the power consumption of the translator, after the translator establishes a USB connection with the smart terminal, if the translator detects that no audio stream is received (such as no audio stream is received within a set duration), the translator can be controlled to enter a low-power state with the screen off. Here, the low-power state with the screen off refers to the state where the translator turns off the screen. When it is detected that the smart terminal starts to transmit the audio stream, the translator is awakened to light up the screen for subsequent translation work.

[0083] The present application embodiment provides an audio translation method, which is applied to a translation machine 200, and is combined with Figure 3 , the audio translation method may include the following steps:

[0084] Step S100: receiving original audio data transmitted by the smart terminal via USB.

[0085] Specifically, when the smart terminal faces the need for audio translation, it establishes a USB connection with the translation machine, and the smart terminal identifies the translation machine as a headset, and then transmits the original audio data to be translated to the translation machine via USB.

[0086] The original audio data can be the audio stream data that needs to be translated provided by the application APP installed on the smart terminal, such as the audio data of a movie played by a video player, the audio data sent by the chat partner received by a social APP, and the audio data received by an online conference APP.

[0087] In some possible implementations, the intelligent terminal may first decode the audio to be translated, obtain the decoded original audio data, and transmit it to the translator. At this time, the translator receives the decoded original audio data, which can be processed later.

[0088] In some other possible implementations, the intelligent terminal may not decode the audio that needs to be translated, and directly transmit the undecoded original audio data to the translator. Since the amount of undecoded original audio data is smaller than the decoded audio data, the transmission delay via USB is lower. At this time, the translator receives the undecoded original audio data. At this time, the translator can offload the original audio data to the digital signal processing unit DSP, and hard-decode the original audio data through the DSP to obtain the decoded original audio data for subsequent processing.

[0089] By calling DSP for hard decoding, the decoding speed can be further accelerated and the decoding time can be reduced compared to decoding through the intelligent terminal software layer. In addition, the DSP itself consumes relatively low power when processing audio formats, which can further reduce the overall power consumption.

[0090] Further optionally, after obtaining the original audio data, the original audio data may be preprocessed, including but not limited to: noise reduction processing, echo cancellation processing, adaptive gain processing, etc. It is understandable that if the original audio data received by the translator is decoded audio stream data, the original audio data may be directly preprocessed here. If the translator receives undecoded audio stream data, it may further perform preprocessing operations on the original audio data after completing the decoding process.

[0091] Step S110: When the translation machine is in a first state, the original audio data is translated to obtain a translated text in a target language. The first state is a state in which the translation function is turned on.

[0092] Specifically, the translation machine can turn on or off the translation function. For example, a control for whether the translation function is turned on is provided on the human-computer interaction interface, and the user can turn on or off the translation function by triggering the control.

[0093] When the translation function of the translation machine is turned on, the received original audio data is automatically translated, and the original audio data in the source language is translated into the translated text in the target language.

[0094] The target language may be a default language, or a language manually set by the user or automatically set according to the user's daily usage habits.

[0095] In the process of translating the original audio data, you can choose online translation or offline translation. Online translation means that the original audio data can be sent to the translation server for translation, and the translated text can be sent to the translation machine. Offline translation means calling the local translation engine of the translation machine to translate the original audio data offline and obtain the translated text.

[0096] Whether to select online translation or offline translation can be based on the system default settings, such as automatically selecting online translation when connected to the Internet and selecting offline translation when disconnected from the Internet. In addition, users can also manually set online translation or offline translation.

[0097] Step S120: synthesize the translated text in the target language into first translated audio data.

[0098] After obtaining the translated text in the target language, a speech synthesis algorithm may be further called to synthesize the translated text into audio data, which is defined herein as first translated audio data.

[0099] Step S130: obtaining first target audio data to be output based on the first translated audio data, and outputting and playing the first target audio data.

[0100] Specifically, in this step, the first target audio data to be output can be obtained based on the first translated audio data, and the first target audio data can be output and played.

[0101] According to the needs of the business scenario, the first translated audio data can be directly used as the first target audio data in this step. In some other possible implementations, the first translated audio data and the original audio data can also be spliced ​​in this step to obtain the spliced ​​first target audio data.

[0102] In an exemplary scenario, a user attends a meeting through an online conference APP on a smart terminal, and the smart terminal receives original audio data from other participants. The original audio data is not in a language familiar to the user, so the original audio data can be translated through a translator connected to a USB to obtain first translated audio data. At the same time, the user only wants to hear the translated audio, not the original audio, so the first translated audio data can be directly output and played as the first target audio data.

[0103] Another exemplary scenario is that a user attends a meeting through an online conference APP on a smart terminal, and the smart terminal receives original audio data from other participants. The original audio data is not in a language familiar to the user, so the original audio data can be translated through a USB-connected translator to obtain first translated audio data. The user wants to hear both the original audio and the translated audio, so the first translated audio data and the original audio data can be spliced ​​and output as the first target audio data for playback.

[0104] When the first translated audio data and the original audio data are spliced, the splicing order of the two can be freely set, for example, the original audio data is placed first and the first translated audio data is placed later, so that the user can first hear the original audio data and then the first translated audio data. Of course, the splicing order of the two can also be reversed.

[0105] When the first target audio data is output and played, the first target audio data may be played through a speaker of the translation machine.

[0106] In another optional case, the translation machine can also be connected to an external headset, such as a wired headset or a Bluetooth headset. In this case, the first target audio data can be output to the headset and played through the headset.

[0107] The audio translation method provided in the embodiment of the present application is that when there is a translation demand, the smart terminal can send the original audio data to be translated to the translator via USB, and when the translator turns on the translation function, the original audio data is translated to obtain the translated text in the target language, and synthesize it into the first translated audio data, and obtain the first target audio data to be output based on the first translated audio data, and output and play the first target audio data. Obviously, by adopting the method of the present application, when there is a demand for audio translation in the smart terminal, the functions of automatic translation, audio synthesis and output playback can be realized with the help of the external translator, so as to meet the audio translation demand of the user on the smart terminal and expand the use scenario of the translator.

[0108] The above embodiment introduces the process of uploading the original audio data from the intelligent terminal to the translation machine for translation processing and outputting for playback, that is, the processing process of the data uplink. On this basis, the audio translation method of the present application can further include the processing process of the data downlink, which specifically includes the following steps:

[0109] S1. Obtaining environmental audio data collected by a sound receiving unit.

[0110] Specifically, when the translator is connected to an earphone, the ambient audio data collected by the sound receiving unit of the earphone can be obtained. When the translator is not connected to an earphone, the ambient audio data collected by the sound receiving unit of the translator itself can be obtained.

[0111] Taking the scenario where a user chats with other users through a chat software on a smart terminal as an example, the smart terminal can transmit the audio sent by the other party to the translation machine for translation and speech synthesis, and output the synthesized audio in the target language for the user to listen to. On this basis, the translation machine can also collect the voice emitted by the user as the environmental audio data in this step.

[0112] Further optionally, after obtaining the ambient audio data, the ambient audio data may be pre-processed, including but not limited to: noise reduction processing, echo cancellation processing, adaptive gain processing, etc.

[0113] S2. When the translation machine is in the first state, the environmental audio data is translated to obtain a translated text in the source language.

[0114] Specifically, when the translation function is turned on, the translation machine can translate the collected ambient audio data, that is, translate the ambient audio data in the target language into the translated text in the source language. The source language can be understood as the language used by the original audio data sent by the smart terminal in the aforementioned step S100.

[0115] Of course, the source language can also be set by the user, such as setting the translated source language through the human-computer interaction interface of the translator.

[0116] S3. Synthesize the translated text of the source language into second translated audio data.

[0117] Specifically, a speech synthesis algorithm is called to synthesize the translated text of the source language into audio data, which is defined as second translated audio data in this step.

[0118] S4. Obtain second target audio data based on the second translated audio data, and output the second target audio data to the smart terminal through a USB connection.

[0119] According to the needs of the business scenario, the second translated audio data can be directly used as the second target audio data in this step. In some other possible implementations, the second translated audio data and the ambient audio data can also be spliced ​​in this step to obtain the spliced ​​second target audio data.

[0120] An exemplary scenario is that user A chats with user B through a chat software on a smart terminal. User A and user B use different languages. In order for user B to understand what user A says, the translation machine can collect the audio of user A as environmental audio data. And translate it into the language used by user B to obtain the second translated audio data. If user B only wants to hear the second translated audio data and does not want to hear the original audio of user A, the second translated audio data can be used as the second target audio data and sent to the smart terminal via a USB connection to be sent to user B via the chat software.

[0121] Another exemplary scenario is that user A chats with user B through the chat software on the smart terminal. User A and user B use different languages. In order for user B to understand what user A said, the translation machine can collect the audio of user A as environmental audio data. And translate it into the language used by user B to obtain the second translated audio data. If user B wants to hear both the second translated audio data and the original audio of user A, the second translated audio data and the environmental audio data can be spliced ​​as the second target audio data, which can be sent to the smart terminal via a USB connection to be sent to user B via the chat software.

[0122] When the second translated audio data and the ambient audio data are spliced, the splicing order of the two can be freely set, for example, the ambient audio data is placed first and the second translated audio data is placed later, so that the user can hear the ambient audio data first and then the second translated audio data. Of course, the splicing order of the two can also be reversed.

[0123] The audio translation method provided in this embodiment further adds a data downlink processing process, that is, the environmental audio data is collected by the translation machine, and translation and speech synthesis are performed, and the obtained second target audio data is sent to the smart terminal via USB. In this way, the smart terminal can obtain the audio data after the environmental audio data is translated, thereby meeting the smart terminal's needs for translating local environmental audio data.

[0124] Reference Figure 4 , another audio translation system framework diagram is provided.

[0125] The headphone interface of the smart terminal 100 is connected to the translator 200 via USB. The translator 200 can provide a variety of different interface solutions, such as a headphone chip solution, an analog interface solution, and a digital interface solution. For details, please refer to the relevant introduction above.

[0126] The translator 200 may include a main control unit (main control chip), in which a hardware abstraction layer (HAL) is provided. The HAL is a programming layer that allows a computer operating system to interact with hardware devices at a logical layer rather than a hardware layer. It abstracts the specific implementation details of the hardware from the operating system and application software by providing a set of standard interfaces and functions.

[0127] The translator 200 may also include a digital signal processing unit DSP, which may be used as a separate digital signal processing chip or integrated in a main control chip. The digital signal processing unit operates under the control of the main control unit. The digital signal processing unit may include a data selection unit and an audio algorithm processing unit. Among them, the data selection unit is used to realize the splicing and fusion of two audio data, such as splicing the original audio data sent by the smart terminal and the first translated audio data; or splicing the environmental audio data collected by the translator and the second translated audio data. The audio algorithm processing unit is used to perform relevant algorithm processing on the audio, for example, performing pre-processing such as noise reduction, echo cancellation, adaptive gain, etc. on the audio, and performing speech synthesis and other processing on the translated text.

[0128] The translator 200 may also include an interactive unit, a translation engine, and a communication unit. The translation engine provides a local translation function. The interactive unit may provide a user interaction portal for the user to select online translation or offline translation. When offline translation is selected, the translation engine may be used to perform local translation on the audio. When online translation is selected, the audio to be translated may be sent to a remote translation server through the communication unit, and the translated text returned by the remote translation server may be received.

[0129] The translator 200 may further include a playback unit and a sound receiving unit. The first target audio data may be output and played through the playback unit, or may be output and played through an external earphone.

[0130] The sound receiving unit can collect the ambient audio data, or collect the ambient audio data through an external headset.

[0131] The original audio data sent by the intelligent terminal to the translation machine is defined as uplink data, which is processed through the uplink data link and finally output and played through the playback unit as the first target audio data.

[0132] The sound receiving unit collects the ambient audio data, obtains the second target audio data after translation and other processing, and sends the second target audio data to the smart terminal through the USB connection. This process is defined as the downlink data link processing process.

[0133] For uplink data link processing:

[0134] After the translator obtains the uplink data (original audio data), it can directly enter the audio algorithm processing unit of the digital signal processing unit, pre-process the uplink data, and then send it to the hardware abstraction layer of the main control unit. At the same time, the main control unit can set whether to turn on the translation function and select online translation or offline translation through the interactive unit.

[0135] If the translation function is turned on after the uplink data passes through the hardware abstraction layer, it is sent to the translation engine for translation, and the translated text is output. The translated text is sent to the audio algorithm processing unit again for speech synthesis to generate the first translated audio data.

[0136] The first translated audio data can be used as first target audio data to be output and played through the playing unit.

[0137] In another case, the first translated audio data can also be sent to a data selection unit, where the first translated audio data and the original audio data are spliced ​​to obtain spliced ​​first target audio data, which is output and played through a playback unit.

[0138] For downlink data link processing:

[0139] The sound receiving unit collects the downlink data (ambient audio data) and sends it to the audio algorithm processing unit for preprocessing. The preprocessed downlink data is sent to the hardware abstraction layer. At the same time, the main control unit sets whether to enable the translation function and selects online translation or offline translation through the interactive unit.

[0140] If the translation function is turned on after the downlink data passes through the hardware abstraction layer, it is sent to the translation engine for translation, and the translated text is output. The translated text is sent to the audio algorithm processing unit again for speech synthesis to generate second translated audio data.

[0141] The second translated audio data can be used as the second target audio data to be output and sent to the smart terminal via a USB connection.

[0142] In another case, the second translated audio data can also be sent to a data selection unit, where the second translated audio data and the ambient audio data are spliced ​​to obtain spliced ​​second target audio data, which are sent to the smart terminal via a USB connection.

[0143] The audio translation device provided in an embodiment of the present application is described below. The audio translation device described below and the audio translation method described above can refer to each other.

[0144] Audio translation devices are used in translation machines, see Figure 5 , Figure 5 A schematic diagram of the structure of an audio translation device disclosed in an embodiment of the present application.

[0145] like Figure 5 As shown, the device may include:

[0146] The original data receiving module 11 is used to receive the original audio data transmitted by the intelligent terminal via USB;

[0147] A translation module 12, configured to translate the original audio data to obtain a translated text in a target language when the translation machine is in a first state, wherein the first state is a state in which the translation function is enabled;

[0148] A speech synthesis module 13, used for synthesizing the translated text in the target language into first translated audio data;

[0149] The audio output module 14 is used to obtain first target audio data to be output based on the first translated audio data, and output and play the first target audio data.

[0150] In a possible implementation, the process of the audio output module obtaining the first target audio data to be output based on the first translated audio data includes:

[0151] using the first translated audio data as the first target audio data;

[0152] or,

[0153] The first translated audio data and the original audio data are spliced ​​together to obtain spliced ​​first target audio data.

[0154] In a possible implementation, the device of the present application may further include:

[0155] The audio preprocessing module is used to preprocess the original audio data before translating the original audio data, and the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

[0156] In a possible implementation, the device of the present application may further include: an environment data acquisition module and an audio sending module;

[0157] The environmental data acquisition unit is used to acquire environmental audio data collected by the sound receiving unit. On this basis, the translation module is also used to translate the environmental audio data when the translation machine is in the first state to obtain a translated text in the source language; the speech synthesis module is also used to synthesize the translated text in the source language into the second translated audio data;

[0158] The audio sending module is used to obtain second target audio data based on the second translated audio data, and output the second target audio data to the smart terminal through the USB connection.

[0159] In a possible implementation, the process of the audio sending module obtaining the second target audio data based on the second translated audio data includes:

[0160] using the second translated audio data as the second target audio data;

[0161] or,

[0162] The second translated audio data and the environmental audio data are spliced ​​to obtain spliced ​​second target audio data.

[0163] In a possible implementation, the audio preprocessing module is further used to preprocess the environmental audio data, and the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

[0164] The present application also provides a translation machine. Figure 6 As shown, it shows a structural schematic diagram of a translation machine suitable for implementing the embodiment of the present application. Figure 6 The translation machine shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0165] like Figure 6 As shown, the translator may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the audio translation method of the aforementioned embodiment of the present application. When the translator is powered on, various programs and data required for the operation of the translator are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0166] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a memory card, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the translator to communicate with other devices wirelessly or by wire to exchange data. Figure 6 The translator with various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0167] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any audio translation method provided in the embodiment of the present application.

[0168] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any audio translation method provided in the embodiment of the present application.

[0169] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0170] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0171] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0172] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0173] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. An audio translation method, characterized in that: Applied to a translation machine, the translation machine is connected to an intelligent terminal via a universal serial bus USB, and after the USB connection is established, the intelligent terminal recognizes the translation machine as a headset, and the audio translation method includes: Receiving original audio data transmitted by the intelligent terminal via USB; When the translation machine is in a first state, translating the original audio data to obtain a translated text in a target language, the first state being a state in which the translation function is turned on; synthesizing the translated text in the target language into first translated audio data; First target audio data to be output is obtained based on the first translated audio data, and the first target audio data is output and played.

2. The method according to claim 1, characterized in that The process of obtaining first target audio data to be output based on the first translated audio data includes: using the first translated audio data as the first target audio data; or, The first translated audio data and the original audio data are spliced ​​together to obtain spliced ​​first target audio data.

3. The method according to claim 1, characterized in that: The original audio data is undecoded audio stream data; before translating the original audio data, the method further includes: The digital signal processing unit DSP in the translator is called to perform hard decoding on the original audio data to obtain decoded original audio data.

4. The method according to claim 1, characterized in that Before translating the original audio data, the method further includes: The original audio data is preprocessed, and the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

5. The method according to claim 1, characterized in that Also includes: Obtaining ambient audio data collected by the sound receiving unit; When the translation machine is in the first state, translating the environmental audio data to obtain a translated text in the source language; synthesizing the translated text in the source language into second translated audio data; Second target audio data is obtained based on the second translated audio data, and the second target audio data is output to the smart terminal through the USB connection.

6. The method according to claim 5, characterized in that The process of obtaining the second target audio data based on the second translated audio data includes: using the second translated audio data as the second target audio data; or, The second translated audio data and the environmental audio data are spliced ​​to obtain spliced ​​second target audio data.

7. The method according to claim 5, characterized in that Before translating the environmental audio data, the method further includes: The environmental audio data is preprocessed, and the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

8. The method according to claim 1, characterized in that The process of outputting and playing the first target audio data includes: Playing the first target audio data through the speaker of the translation machine; or, When the translation machine is connected to the earphone, the first target audio data is output to the earphone so as to play the first target audio data through the earphone.

9. An audio translation system, characterized in that: include: Smart terminals and translation machines; The intelligent terminal is connected to the translation machine via a universal serial bus (USB), and after the USB connection is established, the intelligent terminal identifies the translation machine as a headset; The intelligent terminal is used to transmit the original audio data to the translation machine via a USB connection; The translation machine is used to implement each step of the audio translation method of any one of claims 1-8.

10. The system according to claim 9, characterized in that The translation machine is provided with an earphone chip and a main control chip, and the USB connection adopts an interface protocol supported by the earphone chip; The earphone chip is used to implement data forwarding processing between the smart terminal and the main control chip, and the main control chip is used to implement each step of the audio translation method of any one of claims 1-8.

11. The system according to claim 9, characterized in that The USB connection between the smart terminal and the translation machine is specifically as follows: the translation machine realizes USB connection with the smart terminal through a configured analog interface.

12. The system according to claim 9, characterized in that The USB connection uses the UAC protocol for data transmission.

Images