Translation machine and audio translation system

Abstract

The invention discloses a translating machine and an audio translation system, the translating machine simultaneously adopts a plurality of different processors to execute different tasks respectively, for example, audio transceiving processing is realized through a mobile communication module, audio related algorithm processing is executed through a digital signal processor, and an offline translation engine is carried by a neural network processor. High-efficiency translation of audio is realized, and other processors are controlled through the main control processor. According to the invention, the control logic of the audio link and the specific algorithm processing logic are separately carried, and the time delay of call translation is effectively reduced through coordination of multiple processors. In the audio data flow path design, far-end audio received by the mobile communication module is directly sent to the digital signal processor to be processed without being transferred by the main control processor, and the digital signal processor directly sends the first target audio to the playing module to be output and played after obtaining the first target audio without being transferred by the main control processor. The processing speed is further improved; and the time delay is reduced.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and more specifically, to a translation machine and an audio translation system. Background Art

[0002] Existing translation machines mostly focus on the function of translation, such as using translation machines to realize simultaneous interpretation functions, as a device for face-to-face communication, etc.

[0003] Some translation machines are equipped with a call translation system. When an incoming call or dialing is detected, the call translation process is started, which can translate the remote voice into the local language and then output it, and translate the local voice into the remote language and then send it out. However, the above call translation process involves the processing of voice data. The audio data processing link of the current translation machine is not optimized enough, resulting in the translation machine equipped with a call translation system being unable to meet the low-latency call translation needs. Summary of the invention

[0004] In view of the above problems, this application is proposed to provide an audio translation method and system to solve the problem that mobile phones, computers and other terminals need to translate voice and expand the use scenarios of translation machines. The specific solution is as follows:

[0005] In a first aspect, a translation machine is provided, comprising:

[0006] Mobile communication module, main control processor, digital signal processor, neural network processor, radio module and playback module;

[0007] The mobile communication module is used to establish a communication link with a remote mobile phone to receive and send audio, and the received remote audio is directly sent to the digital signal processor;

[0008] The main control processor is used to control the work of other processors;

[0009] The digital signal processor is used to pre-process the remote audio under the control of the main control processor, and perform speech synthesis on the translated text corresponding to the remote audio, obtain the first target audio based on the synthesized speech, and send the first target audio to the playback module for output;

[0010] The neural network processor is used to carry an offline translation engine. When the main control processor selects offline translation, the pre-processed remote audio is translated by the offline translation engine to obtain a translated text in the target language, and the translated text is sent to the digital signal processor for speech synthesis.

[0011] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the digital signal processor is further used to pre-process the local audio collected by the sound receiving module under the control of the main control processor, and perform speech synthesis on the translated text corresponding to the local audio, obtain the second target audio based on the synthesized speech, and send the second target audio to the remote mobile phone through the mobile communication module;

[0012] The neural network processor is also used to translate the pre-processed local audio through the offline translation engine when the main control processor selects offline translation, obtain the translated text in the same language as the remote audio, and send the translated text to the digital signal processor for speech synthesis.

[0013] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the digital signal processor performs speech synthesis on the translated text corresponding to the remote audio, and obtains the first target audio based on the synthesized speech, including:

[0014] The digital signal processor performs speech synthesis on the translated text corresponding to the remote audio, and uses the synthesized speech as the first target audio;

[0015] or,

[0016] The digital signal processor performs speech synthesis on the translated text corresponding to the remote audio, and concatenates the synthesized speech with the remote audio to obtain a first target audio.

[0017] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the digital signal processor performs speech synthesis on the translated text corresponding to the local audio, and obtains the second target audio based on the synthesized speech, including:

[0018] The digital signal processor performs speech synthesis on the translated text corresponding to the local audio, and uses the synthesized speech as the second target audio;

[0019] or,

[0020] The digital signal processor performs speech synthesis on the translated text corresponding to the local audio, and concatenates the synthesized speech with the local audio to obtain a second target audio.

[0021] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the neural network processor is also used to call the translation server to translate the preprocessed remote audio when the main control processor selects online translation, obtain the translated text in the target language, and send the translated text to the digital signal processor for speech synthesis.

[0022] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the neural network processor is also used to call the translation server to translate the preprocessed local audio when the main control processor selects online translation, to obtain a translated text in the same language as the remote audio, and to send the translated text to the digital signal processor for speech synthesis.

[0023] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

[0024] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0025] The application processor is used to carry the translation machine application and work under the control of the main control processor.

[0026] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0027] The graphics processor is used to perform tasks related to graphics and image processing under the control of the main control processor.

[0028] In a second aspect, an audio translation system is provided, comprising: a remote mobile phone, and a translation machine provided in any one of the first aspects of the present application.

[0029] By means of the above technical scheme, the present application improves the audio processing link of a translation machine with a call translation function. The translation machine simultaneously uses multiple different processors to perform different task processing respectively, such as realizing audio receiving and sending processing through a mobile communication module, and specifically performing audio-related algorithm processing through a digital signal processor, such as audio preprocessing, speech synthesis, etc., and carrying an offline translation engine through a neural network processor. With the help of the performance of the neural network processor, the translation efficiency of the translation engine can be improved, and efficient translation processing of audio can be achieved. In addition, a main control processor is also designed. The main control processor is used to coordinate and control other processors, that is, the control logic of the audio link and the specific algorithm processing logic are separated and mounted, and the audio translation processing speed is accelerated through multi-processor coordination, effectively reducing the delay of call translation.

[0030] In addition, in the design of the audio data flow path, the remote audio received by the mobile communication module is directly sent to the digital signal processor for processing without the need for transfer through the main control processor. Also, after obtaining the first target audio, the digital signal processor directly sends it to the playback module for output and playback, without the need for transfer through the main control processor, which can further improve the processing speed and reduce the delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0032] Figure 1 A schematic diagram of the structure of a translation machine provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of a process of performing audio processing by a mobile communication module of a translation machine provided in an embodiment of the present application;

[0034] Figure 3a-3c A schematic diagram of a data processing link in three modes of a translation machine provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of the structure of an audio translation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] Currently, some translation machines are equipped with a call translation system. When an incoming call or dialing is detected, the call translation process is started, which can translate the remote voice into the local language and then output it, and translate the local voice into the remote language and then send it out. However, the above call translation process involves the processing of voice data. The audio data processing link of the current translation machine is not optimized enough, resulting in the translation machine equipped with the call translation system being unable to meet the low-latency call translation needs.

[0038] To this end, an embodiment of the present application provides a translation machine with a call voice translation function. On the basis of having the translation function of a traditional translation machine, it also supports the call function and can translate the call voice at the same time. The overall call translation delay is lower, which is convenient for users to use.

[0039] Combination Figure 1 , Figure 1 A structural schematic diagram of a translation machine is provided. The translation machine 100 includes a mobile communication module, a main control processor (Micro Controller Unit, MCU), a digital signal processor (Digital Signal Processor, DSP), a neural network processor (Neural-network Processing Unit, NPU), a sound receiving module and a playback module.

[0040] Further optionally, the translation machine may further include any one or both of an application processor (AP) and a graphics processor (GPU).

[0041] It should be noted that the above-mentioned different processing units may be independent devices or may be integrated into one or more processors.

[0042] The main control processor is used to coordinate and control the work of other processing units.

[0043] The application processor is used to carry the applications on the translator and works under the control of the main processor. The application processor is mainly responsible for handling high-level tasks in the translator, such as running operating systems such as Android or iOS, managing user interfaces, and executing user-installed applications.

[0044] The graphics processor is used to perform tasks related to graphics and image processing under the control of the main processor, such as graphics rendering, generating and displaying images, videos and 3D graphics; image processing, such as image enhancement, denoising, compression, etc.

[0045] The mobile communication module is used to establish a communication link with a remote mobile phone to achieve audio reception and transmission. In this embodiment, the remote mobile phone refers to an electronic device with call capability located at the other end of the network relative to the translator.

[0046] In one possible implementation, Figure 2 As shown, the mobile communication module may include a baseband processor and a modem (composed of a modulator and a demodulator). The translator may adopt two modes: a sim card slot or a virtual sim card module.

[0047] For the convenience of the following description, the link in which the remote mobile phone sends remote audio to the translator and the translator processes the remote audio and finally outputs it is defined as the uplink audio link. Conversely, the link in which the translator collects local audio and processes it to obtain the audio to be sent is defined as the downlink audio link.

[0048] The process of the mobile communication module realizing communication may include:

[0049] First, the translator identifies the sim card. Check the chip information on the sim card, such as IMEI number, ICCID, etc. If the card is invalid, the translator will not be able to connect to the network. A virtual sim card is a software program that simulates a sim card.

[0050] The translator establishes a connection with the operator through the baseband processor. When establishing a connection, the baseband processor sends a request to the operator and obtains relevant authentication information.

[0051] The audio to be sent output by the downlink audio link is processed by the baseband processor to output a low-frequency baseband signal, and then the low-frequency baseband signal to be sent is modulated into a medium-high frequency signal by a modulator, and the medium-high frequency signal is sent to the remote mobile phone.

[0052] The translator demodulates the received electromagnetic wave signal into a low-frequency baseband signal through a demodulator. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, the processed audio is obtained and sent to the uplink audio link.

[0053] The digital signal processor is used to carry out audio processing related algorithms. Under the control of the main control processor, it pre-processes the remote audio received by the mobile communication module and performs speech synthesis on the translated text corresponding to the remote audio, obtains the first target audio based on the synthesized speech, and sends the first target audio to the playback module for output.

[0054] It should be noted here that the mobile communication module can send the remote audio directly to the digital signal processor without being transferred through the main control processor, thereby reducing the data processing delay.

[0055] The above-mentioned pre-processing operation on the far-end audio includes, but is not limited to: noise reduction processing, echo cancellation processing, adaptive gain processing, etc.

[0056] After obtaining the synthesized speech, the digital signal processor obtains the first target audio to be output based on the synthesized speech.

[0057] According to the needs of the business scenario, the digital signal processor can directly use the synthesized speech as the first target audio. In some other possible implementations, the digital signal processor can also splice the synthesized speech and the remote audio to obtain the spliced ​​first target audio.

[0058] For example, when a user uses a translation machine to make a call, if he only wants to hear the translated voice and does not want to hear the original remote audio, he can choose to directly output the synthesized voice as the first target audio. If the user wants to hear both the original remote audio and the translated voice, he can splice the synthesized voice and the remote audio to obtain the first target audio.

[0059] When splicing the synthesized voice and the remote audio, the splicing order of the two can be freely set. For example, the remote audio is placed first and the synthesized voice is placed later, so that the user can hear the remote audio first and then the synthesized voice. Of course, the splicing order of the two can also be reversed.

[0060] When the first target audio is output and played, the first target audio may be played through a speaker of the translation machine.

[0061] In another optional case, the translation machine can also be connected to an external headset, such as a wired headset or a Bluetooth headset. In this case, the first target audio can be output to the headset and played through the headset.

[0062] The neural network processor is used to carry the offline translation engine. Since the neural network processor is designed to accelerate neural network calculations, the speed and effect of offline translation can be improved by carrying the offline translation engine on the neural network processor.

[0063] The translation machine can turn on or off the translation function. For example, a control to indicate whether the translation function is turned on is provided on the human-computer interaction interface, and the user can turn on or off the translation function by triggering the control.

[0064] When the translation function of the translator is turned on and the main control processor selects offline translation, the pre-processed remote audio is translated by the offline translation engine to obtain the translated text in the target language.

[0065] Here, the target language may be a default language, or a language manually set by the user or automatically set according to the user's daily usage habits.

[0066] Optionally, when the main control processor selects online translation, the neural network processor can also call a remote translation server to translate the pre-processed remote audio to obtain the translated text in the target language.

[0067] Whether to choose online translation or offline translation can be based on the system default settings, such as automatically selecting online translation when connected to the Internet and selecting offline translation when disconnected from the Internet. In addition, users can also manually set it to online translation or offline translation.

[0068] When the neural network processor obtains the translated text in the target language, it can send the translated text to the digital signal processor for speech synthesis.

[0069] The translation machine provided in the embodiment of the present application improves the audio processing link. The translation machine simultaneously uses multiple different processors to perform different task processing respectively, such as realizing audio receiving and sending processing through a mobile communication module, and specifically executing audio-related algorithm processing, such as audio preprocessing, speech synthesis, etc., through a digital signal processor. An offline translation engine is carried out through a neural network processor. With the help of the performance of the neural network processor, the translation efficiency of the translation engine can be improved to achieve efficient translation processing of audio. In addition, a main control processor is also designed. The main control processor is used to coordinate and control other processors, that is, the control logic of the audio link and the specific algorithm processing logic are separated and carried, and the audio translation processing speed is accelerated through multi-processor coordination, effectively reducing the delay of call translation.

[0070] In addition, in the design of the audio data flow path, the remote audio received by the mobile communication module is directly sent to the digital signal processor for processing without the need for transfer through the main control processor. Also, after obtaining the first target audio, the digital signal processor directly sends it to the playback module for output and playback, without the need for transfer through the main control processor, which can further improve the processing speed and reduce the delay.

[0071] In some possible implementations, the digital signal processor can also be used to, under the control of the main control processor, pre-process the local audio collected by the sound receiving module, and perform speech synthesis on the translated text corresponding to the local audio, obtain the second target audio based on the synthesized speech, and send the second target audio to the remote mobile phone through the mobile communication module.

[0072] The pre-processing operations on the local audio include but are not limited to: noise reduction processing, echo cancellation processing, adaptive gain processing, etc.

[0073] The digital signal processor performs speech synthesis on the translated text corresponding to the local audio to obtain the synthesized speech, and then obtains the second target audio to be sent based on the synthesized speech.

[0074] According to the needs of the business scenario, the digital signal processor can directly use the synthesized speech as the second target audio. In some other possible implementations, the digital signal processor can also splice the synthesized speech and the local audio to obtain the spliced ​​second target audio.

[0075] When the synthesized voice and local audio are spliced, the splicing order of the two can be freely set. For example, the local audio is placed first and the synthesized voice is placed later, so that the user of the remote mobile phone can hear the local audio first and then the synthesized voice. Of course, the splicing order of the two can also be reversed.

[0076] After obtaining the second target audio, the digital signal processor can send it directly to the mobile communication module without being transferred through the main control processor, which can effectively reduce the delay.

[0077] Furthermore, the neural network processor can also be used to translate the local audio pre-processed by the digital signal processor through the offline translation engine when the main control processor selects offline translation, obtain the translated text in the same language as the remote audio, and send the translated text to the digital signal processor for speech synthesis.

[0078] Optionally, when the main control processor selects online translation, the neural network processor can also call a remote translation server to translate the preprocessed local audio to obtain a translated text in the same language as the remote audio.

[0079] In some embodiments of the present application, the data processing links of the translator of the present application in three optional modes are introduced. The translator of the present application can have three modes: low latency mode, high performance mode and balanced mode, respectively. Figure 3a-3c shown.

[0080] Figure 3a The example shows the data processing link in low-latency mode. The remote audio directly enters the digital signal processor of the translator, which processes the remote audio through the built-in algorithm unit and finally outputs it for playback. The local audio collected by the translator also directly enters the digital signal processor for processing, and the processed audio is finally sent to the remote mobile phone. In this process, since the audio data does not need to pass through the main control processor, the transmission delay can be minimized.

[0081] Figure 3b The data processing link in high-performance mode is illustrated. In actual product design, considering the cost issue, the memory of a general digital signal processor is much smaller than that of the main control processor. Therefore, when the performance requirements are high, such as higher translation accuracy requirements, the digital signal processor may not be able to process it. Therefore, the audio processing can be placed on the main control processor, that is, the received remote audio is directly sent to the main control processor for processing and output for playback. The collected local audio is also directly sent to the main control processor for processing, and the processed audio is finally sent to the remote mobile phone. In this way, with the help of the larger memory of the main control processor, some more complex algorithm processing can be completed to ensure that the performance requirements are met.

[0082] Figure 3c The data processing link in the equalization mode is illustrated. In this mode, the audio-related processing can be completed by the main control processor and the digital signal processor in collaboration, that is, part of the audio algorithm processing is completed by the main control processor, and the other part of the audio algorithm processing is completed by the digital signal processor. For example, simple audio algorithm processing can be completed by the digital signal processor, and complex audio algorithm processing can be completed by the main control processor. The above audio-related processing includes the process of processing remote audio and the process of processing local audio.

[0083] It should be noted that the three modes in the above examples can be switched freely. For example, in response to the user's instruction to switch the mode on the translation machine, it switches to the target mode selected by the user, and then processes data according to the processing link in this mode.

[0084] Combination Figure 4 , which illustrates an audio translation system, including a remote mobile phone 200 and a translation machine 100. Optionally, an earphone 300 may also be included.

[0085] The specific structure of the translation machine 100 is as described above, and may include a main control processor, a digital signal processor, a neural network processor, a radio module, a playback module, etc. Figure 4 Not all are shown.

[0086] The main control processor is equipped with a hardware abstraction layer (HAL). The hardware abstraction layer is a programming layer that allows the computer operating system to interact with hardware devices at the logical level rather than the hardware level. It abstracts the specific implementation details of the hardware from the operating system and application software by providing a set of standard interfaces and functions.

[0087] A data selection unit and an audio algorithm processing unit may be configured in the digital signal processor. The data selection unit is used to realize the splicing and fusion of two audio data, such as splicing the synthesized speech obtained by performing speech synthesis on the translated text corresponding to the remote audio with the remote audio; or splicing the synthesized speech obtained by performing speech synthesis on the translated text corresponding to the local audio with the local audio. The audio algorithm processing unit is used to perform relevant algorithm processing on the audio, for example, performing pre-processing such as noise reduction, echo cancellation, adaptive gain, etc. on the audio, and performing speech synthesis and other processing on the translated text.

[0088] The translation machine 100 may also include an interaction unit and a communication unit. The interaction unit may provide a user interaction portal for the user to select online translation or offline translation. When offline translation is selected, the offline translation engine carried in the neural network processor may be used to perform local translation on the audio. When online translation is selected, the audio to be translated may be sent to a remote translation server through the communication unit, and the translated text returned by the remote translation server may be received.

[0089] The translator 100 can output audio through a local playback module and collect local audio through a local sound receiving module. In addition, the translator 100 can also be connected to a headset 300, so that audio can be output through the headset and local audio can be collected through the microphone of the headset.

[0090] Combination Figure 4 , introduce the audio link of the translator for call translation:

[0091] The translator can be used as both a voice calling terminal and a voice receiving terminal through the mobile communication module.

[0092] After the remote audio is received by the translator, it is processed through the uplink audio link and then output for playback.

[0093] After the translator's audio receiving module receives the local call voice (local audio), it is processed through the downlink audio link and output to the remote mobile phone through the mobile communication module.

[0094] The uplink audio link and the downlink audio link work simultaneously in the system.

[0095] For the uplink audio link processing:

[0096] After the translator obtains the uplink data (remote audio), it can directly enter the audio algorithm processing unit of the digital signal processor, pre-process the uplink data and send it to the hardware abstraction layer of the main control processor. At the same time, the main control processor can set whether to enable the translation function and select online translation or offline translation through the interactive unit.

[0097] After the uplink data passes through the hardware abstraction layer, if the translation function is turned on, it is sent to the translation engine in the neural network processor for translation, and the translated text is output. The translated text is sent to the audio algorithm processing unit again for speech synthesis to generate synthesized speech.

[0098] The synthesized speech can be used as the first target audio to be output and played through the playing module.

[0099] In another case, the synthesized speech can also be sent to the data selection unit, where the synthesized speech and the uplink data are spliced ​​to obtain the spliced ​​first target audio, which is output and played through the playback module.

[0100] For the downlink audio link processing:

[0101] The radio module collects the downlink data (local audio) and sends it to the audio algorithm processing unit for preprocessing. The preprocessed downlink data is sent to the hardware abstraction layer. At the same time, the main control unit sets whether to enable the translation function and selects online translation or offline translation through the interactive unit.

[0102] After the downlink data passes through the hardware abstraction layer, if the translation function is turned on, it is sent to the translation engine for translation and the translated text is output. The translated text is sent to the audio algorithm processing unit again for speech synthesis to generate synthesized speech.

[0103] The synthesized speech can be used as the second target audio to be sent to the remote mobile phone through the mobile communication module.

[0104] In another case, the synthesized speech can also be sent to the data selection unit, where the synthesized speech and the downlink data are spliced ​​to obtain the spliced ​​second target audio, which is sent to the remote mobile phone through the mobile communication module.

[0105] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0106] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0107] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0108] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A translation machine, characterized in that: include: Mobile communication module, main control processor, digital signal processor, neural network processor, radio module and playback module; The mobile communication module is used to establish a communication link with a remote mobile phone to receive and send audio, and the received remote audio is directly sent to the digital signal processor; The main control processor is used to control the work of other processors; The digital signal processor is used to pre-process the remote audio under the control of the main control processor, and perform speech synthesis on the translated text corresponding to the remote audio, obtain the first target audio based on the synthesized speech, and send the first target audio to the playback module for output; The neural network processor is used to carry an offline translation engine. When the main control processor selects offline translation, the pre-processed remote audio is translated by the offline translation engine to obtain a translated text in the target language, and the translated text is sent to the digital signal processor for speech synthesis.

2. The translation machine according to claim 1, characterized in that: The digital signal processor is further used to pre-process the local audio collected by the sound receiving module under the control of the main control processor, and perform speech synthesis on the translated text corresponding to the local audio, obtain the second target audio based on the synthesized speech, and send the second target audio to the remote mobile phone through the mobile communication module; The neural network processor is also used to translate the pre-processed local audio through the offline translation engine when the main control processor selects offline translation, obtain the translated text in the same language as the remote audio, and send the translated text to the digital signal processor for speech synthesis.

3. The translation machine according to claim 1, characterized in that: The process of performing speech synthesis by the digital signal processor on the translated text corresponding to the remote audio and obtaining the first target audio based on the synthesized speech includes: The digital signal processor performs speech synthesis on the translated text corresponding to the remote audio, and uses the synthesized speech as the first target audio; or, The digital signal processor performs speech synthesis on the translated text corresponding to the remote audio, and concatenates the synthesized speech with the remote audio to obtain a first target audio.

4. The translation machine according to claim 2, characterized in that: The digital signal processor performs speech synthesis on the translated text corresponding to the local audio, and obtains the second target audio based on the synthesized speech, including: The digital signal processor performs speech synthesis on the translated text corresponding to the local audio, and uses the synthesized speech as the second target audio; or, The digital signal processor performs speech synthesis on the translated text corresponding to the local audio, and concatenates the synthesized speech with the local audio to obtain a second target audio.

5. The translation machine according to claim 1, characterized in that: The neural network processor is also used to call the translation server to translate the pre-processed remote audio when the main control processor selects online translation, obtain the translated text in the target language, and send the translated text to the digital signal processor for speech synthesis.

6. The translation machine according to claim 2, characterized in that: The neural network processor is also used to call the translation server to translate the pre-processed local audio when the main control processor selects online translation, obtain the translated text in the same language as the remote audio, and send the translated text to the digital signal processor for speech synthesis.

7. The translation machine according to claim 1 or 2, characterized in that: The preprocessing includes at least one of the following: noise reduction processing, echo cancellation processing, and adaptive gain processing.

8. The translation machine according to claim 1, characterized in that: Also includes: The application processor is used to carry the translation machine application and work under the control of the main control processor.

9. The translation machine according to claim 1, characterized in that: Also includes: The graphics processor is used to perform tasks related to graphics and image processing under the control of the main control processor.

10. An audio translation system, characterized in that: include: A remote mobile phone and a translation machine according to any one of claims 1 to 9.

Images