Bidirectional real-time translation method, device and equipment based on terminal information interaction

Abstract

The present invention discloses a two-way real-time translation method, device and equipment based on terminal information interaction, the method includes: starting a simultaneous interpretation service according to a startup instruction, creating a virtual microphone and configuring an audio link for the virtual microphone according to the current active application, obtaining a first voice through an audio link and translating it to obtain a second voice and / or text information, collecting the second voice through the virtual microphone and outputting it through the current active application, obtaining a third voice through an audio link and translating it to obtain a fourth voice for playback, and if text information is obtained, inputting it into the current active application for synchronous output. The above method, by generating a virtual microphone and configuring an audio link with the current active application, thereby realizing audio transmission between the virtual microphone and the current active application, can realize the combined application of simultaneous interpretation services and other third-party services, greatly expanding the flexibility of simultaneous interpretation service applications.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a two-way real-time translation method, device and equipment based on terminal information interaction. Background Art

[0002] When using a terminal for a real-time conference or a real-time call, simultaneous interpretation is required between two or more parties in the call. However, the prior art methods are mainly divided into several key technical links: speech recognition, machine translation, and speech synthesis. After the above key technologies are integrated, they are input through a microphone or text, and after being processed by several core modules, the translated text or audio is obtained in a customized terminal or software. The existing application software installed on the terminal completes the entire translation in its own customized application, and is not compatible with third-party services, that is, it is impossible to combine third-party services to realize the simultaneous interpretation function, and it is impossible to realize linkage with other devices or application software to realize flexible application. Therefore, there are many limitations in the actual application process. Therefore, the simultaneous interpretation application software in the prior art method has the problem of poor application flexibility. Summary of the invention

[0003] The embodiments of the present invention provide a two-way real-time translation method, device and equipment based on terminal information interaction, aiming to solve the problem of poor application flexibility of simultaneous interpretation application software in the prior art methods.

[0004] In a first aspect, an embodiment of the present invention provides a two-way real-time translation method based on terminal information interaction, wherein the method is applied in a terminal device, and the method includes:

[0005] If the input start instruction is received, start the simultaneous interpretation service according to the start instruction;

[0006] Create a virtual microphone and configure the audio link of the virtual microphone according to the currently active application;

[0007] Acquire a first voice through the configured audio link, and translate the first voice through the simultaneous interpretation service to obtain a corresponding second voice and / or text information; the first voice is a user voice collected by the audio collection module of the terminal device or a playback voice played by the terminal device;

[0008] Collecting the second voice through the virtual microphone and outputting it through the currently active application;

[0009] Acquire a third voice through the configured audio link, translate the third voice through the simultaneous interpretation service to obtain a corresponding fourth voice and play the fourth voice; the third voice is voice information obtained by collecting the voice received and played by the current active application through the virtual microphone;

[0010] The text information is input into the currently active application for synchronous output.

[0011] In a second aspect, an embodiment of the present invention further provides a two-way real-time translation device based on terminal information interaction, wherein the device is configured in a terminal device, and the device is used to execute the two-way real-time translation method based on terminal information interaction as described in the first aspect above, and the device includes:

[0012] A starting unit, configured to start the simultaneous interpretation service according to the starting instruction if the starting instruction is received;

[0013] An audio link configuration unit, used to create a virtual microphone and configure the audio link of the virtual microphone according to the currently active application;

[0014] A translation unit, configured to obtain a first voice through a configured audio link, and translate the first voice through the simultaneous interpretation service to obtain a corresponding second voice and / or text information; the first voice is a user voice collected by an audio collection module of the terminal device or a playback voice played by the terminal device;

[0015] An audio output unit, configured to collect the second voice through the virtual microphone and output it through the currently active application;

[0016] an audio playback unit, configured to obtain a third voice through the configured audio link, translate the third voice through the simultaneous interpretation service to obtain a corresponding fourth voice and play the fourth voice; the third voice is voice information obtained by collecting the voice received and played by the currently active application through the virtual microphone;

[0017] The text information output unit is used to input the text information into the currently active application for synchronous output.

[0018] In a third aspect, an embodiment of the present invention further provides a computer device, wherein the device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0019] Memory, used to store computer programs;

[0020] The processor is used to implement the steps of the two-way real-time translation method based on terminal information interaction described in the first aspect when executing the program stored in the memory.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the two-way real-time translation method based on terminal information interaction as described in the first aspect above are implemented.

[0022] The embodiment of the present invention provides a two-way real-time translation method, device and equipment based on terminal information interaction, the method includes: starting the simultaneous interpretation service according to the startup instruction, creating a virtual microphone and configuring the audio link of the virtual microphone according to the current active application, obtaining the first voice through the audio link and translating it to obtain the second voice and / or text information, collecting the second voice through the virtual microphone and outputting it through the current active application, obtaining the third voice through the audio link and translating it to obtain the fourth voice for playback, and if the text information is obtained, inputting it into the current active application for synchronous output. The above method, by generating a virtual microphone and configuring the audio link with the current active application, thereby realizing audio transmission between the virtual microphone and the current active application, can realize the combined application of the simultaneous interpretation service with other third-party services, greatly expanding the flexibility of the simultaneous interpretation service application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0024] Figure 1 A method flow chart of a two-way real-time translation method based on terminal information interaction provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of an application scenario of a two-way real-time translation method based on terminal information interaction provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of another application scenario of the two-way real-time translation method based on terminal information interaction provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of another application scenario of the two-way real-time translation method based on terminal information interaction provided by an embodiment of the present invention;

[0028] Figure 5 A schematic diagram of another application scenario of the two-way real-time translation method based on terminal information interaction provided by an embodiment of the present invention;

[0029] Figure 6A schematic block diagram of a two-way real-time translation device based on terminal information interaction provided by an embodiment of the present invention;

[0030] Figure 7 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In this application, an application (Application, APP) is an application program installed on a terminal device and running based on a system program in the terminal device. The application program can run on the terminal device and implement corresponding functions, such as live broadcast, conference, voice conversation or multimedia playback.

[0032] In the present application, the terminal device 10 may be any device with computing and processing capabilities. Of course, the terminal device may also have audio and video playback and interface display functions. For example, the terminal device may be a mobile phone, a computer, a smart TV, a vehicle-mounted device, a wearable device, an industrial device, etc.

[0033] OCR (Optical Character Recognition) technology converts text on paper documents or screens into text format through electronic devices. It takes a screenshot and then uses an algorithm to recognize the text in the image, which is applicable to all scenarios.

[0034] UI XML is a description language for Android application interface layout. By parsing the XML file, you can get the text information on the application interface. This method is suitable for scenarios where you need to get text from a specific application interface and where there is text in Android APP, such as live comments, video comments, etc.

[0035] Web Spider is a program that automatically browses web pages and extracts text content by analyzing the HTML structure of web pages. It requires users to actively fill in the web address or scan the code for authorization, which is more suitable for live broadcast scenarios.

[0036] Audio source selection: The specified audio source is realized through the cooperation of AudioRecord and AudioManager. The audio source can be a system microphone, an external microphone, or a player. The system microphone is the microphone configured by the terminal device itself, and the player can be a player configured by the system itself or an external player. The external microphone and player can communicate with the terminal device through a wired connection or a wireless connection to realize the transmission of data information. For example, a Bluetooth headset is equipped with a microphone. When the terminal device establishes a Bluetooth connection with the Bluetooth headset, the microphone in the Bluetooth headset can be used as an external microphone. The player can be a Bluetooth speaker. When the terminal device establishes a Bluetooth connection with the Bluetooth speaker, the Bluetooth speaker can be used as an external player.

[0037] Audio focus: Request focus, handle focus changes, manage focus, and abandon audio focus through AudioManager to avoid application audio conflicts.

[0038] Audio storage: In Android's Web Audio API, AudioBuffer is used to store audio data. To improve scenario compatibility, both original audio and synthesized audio are stored here and used for virtual microphone and speaker routing, respectively, according to the front-end strategy.

[0039] Virtual microphone: Through the use of AudioRecord and AudioTrack, the specified audio buffer is used as a microphone. You can set it to be effective globally or for a specified application UID. The specified buffer can be used to switch between the original sound and the synthesized sound of the communication object. The setting of the effective range is a reserved capability.

[0040] Speaker routing: mainly relies on AudioManager and AudioTrack to achieve, here mainly takes into account the user's usage, such as monitoring headphones and implementing speaker routing function.

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0043] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0044] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0045] See also Figure 1 As shown in the figure, the embodiment of the present invention provides a two-way real-time translation method based on terminal information interaction, which is applied to a terminal device and is executed by an application software installed in the terminal device. Figures 2 to 4 As shown, a network connection is established between the terminal device 10 and the docking terminal 20. The terminal device 10 is a device for running an application for two-way real-time translation. The terminal device 10 can be a terminal device such as a laptop computer, a desktop computer, a tablet computer or a mobile phone. The docking terminal 20 is a terminal device that establishes a network connection with the terminal device 10 through a wired LAN or a wireless LAN. The docking terminal 20 can be a terminal device such as a laptop computer, a desktop computer, a tablet computer or a mobile phone. The local user is the user of the terminal device 10, and the online user is the user of the docking terminal 20. Figure 5 In the example, the terminal device 10 is only used by local users for multimedia playback. Figure 1 As shown, the method includes steps S110 to S160.

[0046] S110: If the input start instruction is received, start the simultaneous interpretation service according to the start instruction.

[0047] If the input start instruction is received, the simultaneous interpretation service is started according to the start instruction. The local user can input the start instruction in the terminal device, and the terminal device can start the simultaneous interpretation service according to the start instruction. The simultaneous interpretation service is also an application service, which provides the simultaneous interpretation service by running the program code inside the terminal device.

[0048] In a specific embodiment, step S110 includes sub-steps: creating an independent first process and allocating corresponding system resources respectively; starting a simultaneous interpretation service matching the startup instruction in the first process; obtaining the system configuration language of the terminal device and the target translation language in the startup instruction to configure the language parameters of the simultaneous interpretation service.

[0049] Specifically, an independent first process can be created in the terminal device, and corresponding system resources can be allocated to the first process, and each process can run an application service separately. Specifically, a simultaneous interpretation service matching the start instruction can be started in the first process, and the start instruction contains pointing information pointing to the corresponding interpretation program code. Then, according to the pointing information in the start instruction, the corresponding interpretation program code can be obtained to run and start the simultaneous interpretation service.

[0050] Furthermore, the system configuration language of the terminal device can be obtained, for example, the system configuration language of the terminal device is "Simplified Chinese". The startup instruction also includes a target translation language, which is the language type corresponding to another language that needs to be translated from the local user's language. The target translation language is obtained from the startup instruction and the language parameters of the simultaneous interpretation service are configured together with the system configuration language. The simultaneous interpretation service can perform simultaneous interpretation between the two language types according to the language parameter configuration. Furthermore, the transmission interface parameters in the simultaneous interpretation service can also be configured, and the transmission interface parameters of the simultaneous interpretation service can be configured to specify the sound source head. The sound source can be a microphone in the terminal device, an external microphone or a player.

[0051] S120: Create a virtual microphone and configure an audio link for the virtual microphone according to the currently active application.

[0052] A virtual microphone is created and an audio link is configured for the virtual microphone according to the current active application. Furthermore, a virtual microphone can be created in a terminal device, which is a functional module obtained virtually inside the terminal device. The virtual microphone can realize virtual voice collection. Another process independent of the first process can be created, and a virtual microphone can be created in the newly created process.

[0053] In a specific embodiment, step S120 includes sub-steps: obtaining a scene type matching the current active application; and configuring a unidirectional audio link or a bidirectional audio link between the current active application and the virtual microphone according to the scene type.

[0054] Specifically, the scene type corresponding to the current active application can be obtained, and the scene type is a one-way transmission type or a two-way transmission type. Each active application corresponds to a scene type, and the matching scene type can be obtained according to the current active application. The embodiment of the present application includes four types of applications, namely a live broadcast application, a call application, a conference application, and a video playback application. The live broadcast application and the video playback application only involve one-way voice transmission, so they are both one-way transmission types. The call application and the conference application involve two-way voice transmission, so they are two-way transmission types.

[0055] According to the determined scenario type, the audio link is configured for the current active application and the virtual microphone. Specifically, if the scenario type is a one-way transmission type, the virtual microphone is configured with a one-way audio link, and only one audio link is configured for the virtual microphone, which is output from the simultaneous interpretation service to the current active application via the virtual microphone; if the scenario type is a two-way transmission type, the virtual microphone is configured with a two-way audio link, and in addition to configuring the audio link output from the simultaneous interpretation service to the current active application via the virtual microphone, an audio link is also configured from the current active application to the simultaneous interpretation service via the virtual microphone. Each audio link is configured with a corresponding audio source for input voice information, and the virtual microphone configures a corresponding focus switching strategy for the audio link. The focus switching of the voice information transmitted in the audio link is controlled by the focus switching strategy based on AudioManager, thereby avoiding audio conflicts.

[0056] Furthermore, an audio storage space may be configured for each audio link, such as an audio storage space for a virtual microphone configured for each audio link, which is used to store voice information collected by the virtual microphone; an audio storage space for a speaker route may be configured for each audio link, so that the speaker route can obtain the voice information output by the currently active application or the voice information translated by the simultaneous interpretation service and route it to the speaker for playback.

[0057] S130: Acquire a first voice through the configured audio link, and translate the first voice through the simultaneous interpretation service to obtain a corresponding second voice and / or text information.

[0058] The first voice is acquired through the configured audio link, and the first voice is translated through the simultaneous interpretation service to obtain a corresponding second voice and / or text information. The first voice is a user voice acquired by the audio acquisition module of the terminal device or a playback voice played by the terminal device.

[0059] The first voice is obtained through the configured audio link. Figure 2 , Figure 3 and Figure 4 In the corresponding application scenario, the first voice is the user voice collected by the audio collection module of the terminal device (system microphone or external microphone connected to the terminal device for communication), wherein Figure 2 For the application scenario of live broadcast application, Figure 3 For the application scenario of the call application, Figure 4 For conference application scenarios; Figure 5 In the corresponding application scenario, the first voice is the playback voice played by the terminal device; Figure 5 This is the application scenario of the video playback application.

[0060] In the scenario of live broadcast application, the first voice can be input into the simultaneous interpretation service for translation. The simultaneous interpretation service performs voice recognition on the first voice to obtain the text content corresponding to the first voice information, and translates the text content to obtain text information. The second voice corresponding to the text information can be resynthesized through voice synthesis technology, and the second voice and the text information corresponding to the second voice can be synchronously obtained through the simultaneous interpretation service. The second voice is output to the docking terminal through the current active application for playback; further, the text information corresponding to the second voice can be synchronously sent to the docking terminal for display, that is, the docking terminal plays the second voice and synchronously displays the text information. This application method is also applicable to the application scenarios of call applications and conference applications.

[0061] In the scenario of video playback application, the terminal device plays multimedia files (audio files or video files), the audio link obtains the first voice and translates it into corresponding text information through the simultaneous interpretation service. At this time, the simultaneous interpretation service no longer continues to synthesize the second voice, and directly outputs the obtained text information to the current active application for display. Then the local user can simultaneously obtain the text information obtained by translating the first voice while listening to music or watching movies.

[0062] S140: Collect the second voice through the virtual microphone and output it through the currently active application.

[0063] The second voice is collected through the virtual microphone and outputted through the currently active application. The second voice is collected through the virtual microphone, and further outputted to the currently active application, and the second voice is outputted to the docking terminal through the currently active application.

[0064] In a specific embodiment, the method also includes: starting a corresponding text translation service according to the input text translation instruction and the current active application; translating the text to be translated by the text translation service to obtain corresponding text information; the text to be translated is the text content played by the current active application or the image text content acquired by the image acquisition module of the terminal device.

[0065] The corresponding text translation service is started according to the input text translation instruction and the current active application, wherein the text translation service can be used to translate text content recorded in a certain language into text content recorded in another language. Figure 2 In the live broadcast application scenario shown, the live broadcast screen contains live comments, bullet screens, and other content entered by the audience. In order to facilitate local users to view the live comments and bullet screen content on the terminal device, the live comments, bullet screen, and other content need to be translated. The live comments and bullet screen content are also the text content played by the current active application. Figure 4 In the conference application scenario shown, local users can display PPT and other content on the conference screen or at the conference site. In order to facilitate online users of the docked terminal to view the conference projection content displayed in the currently active application, the conference projection content needs to be translated. The conference projection content is the image text content captured by the image acquisition module of the terminal device.

[0066] Specifically, the user can input a text translation instruction in the terminal device, and the text translation instruction can be configured with the application interface address (applicable to parsing the XML file corresponding to the application interface address through UI XML and obtaining the text content), OCR recognition parameters (applicable to extracting image text content from images with OCR), web address or code scanning authorization (applicable to extracting corresponding text content through WebSpider). The text translation service can be started according to the input text translation instruction and the currently active application, and the corresponding text content input source can be configured for the text translation service. The text to be translated is translated in real time through the text translation service to obtain text information corresponding to the text to be translated. The text to be translated can be as follows: Figure 2 The text content played by the current active application shown in the application scenario in the text translation can be realized by translating the live comments and barrage content recorded by other voices in the live broadcast screen into text information that can be directly viewed by local users; the text to be translated can also be Figure 4 The image acquisition module of the terminal device shown in the application scenario captures the image text content, thereby realizing real-time translation of the conference projection content, obtaining text information and sending it to the docking terminal for display. At this time, online users can directly view the text information.

[0067] In a specific embodiment, starting a corresponding text translation service according to an input text translation instruction and a currently active application includes: creating an independent second process and allocating corresponding system resources respectively; starting a text translation service matching the text translation instruction in the second process; and configuring text translation parameters of the text translation service according to text routing and the text output type of the currently active application.

[0068] Specifically, an independent second process can be created and corresponding system resources can be allocated. A text translation service matching the text translation instruction can be started in the second process, and the text content input source of the text translation service can be configured according to the application interface address, OCR recognition parameters, web address or code scanning authorization configured in the text translation instruction. The target translation language of the text translation service can be configured according to the text output type of the currently active application. The output address of the text translation service can be configured according to the routing address of the text routing. By configuring the text content input source, target translation language and output address, the text translation parameters of the text translation service can be configured.

[0069] S150: Acquire a third voice through the configured audio link, translate the third voice through the simultaneous interpretation service to obtain a corresponding fourth voice, and play the fourth voice.

[0070] The third voice is obtained through the configured audio link, and the third voice is translated through the simultaneous interpretation service to obtain the corresponding fourth voice and play it. The third voice is voice information obtained by collecting the voice received and played by the current active application through the virtual microphone. Furthermore, the third voice is obtained through the audio link, and the third voice can be translated through the simultaneous interpretation service to obtain the fourth voice. The online user of the docking terminal can speak and send the third voice to the current active application of the terminal device through the docking terminal. For example, Figure 3 In the voice conversation scenario shown, the online user of the docked terminal can send a voice message to the terminal device to achieve voice communication between the two parties. Figure 4 In the conference scenario shown in , the online user of the docking terminal can send voice information to the terminal device to enable real-time conference communication between the two parties. The current active application receives the third voice for playback, and the third voice can be acquired through the virtual microphone. The virtual microphone sends the acquired third voice to the simultaneous interpretation service for translation into the fourth voice. The fourth voice is output to the system microphone, external microphone or player based on the speaker routing for playback.

[0071] S160: Input the text information into the currently active application for synchronous output.

[0072] The text information is input into the currently active application for synchronous output. The acquired text information is input into the currently active application for synchronous output. After acquiring the text information corresponding to the second voice and outputting it to the currently active application, it is further output to the docking terminal for display. Figure 5 In the application scenario of the video playback application shown, the corresponding text information can be obtained and output to the currently active application for display, so the local user can simultaneously obtain the text information obtained by translating the first voice while listening to music or watching a movie.

[0073] In the two-way real-time translation method based on terminal information interaction disclosed in the above embodiment, the method includes: starting the simultaneous interpretation service according to the startup instruction, creating a virtual microphone and configuring the audio link of the virtual microphone according to the current active application, obtaining the first voice through the audio link and translating it to obtain the second voice and / or text information, collecting the second voice through the virtual microphone and outputting it through the current active application, obtaining the third voice through the audio link and translating it to obtain the fourth voice for playback, and if the text information is obtained, inputting it into the current active application for synchronous output. The above method, by generating a virtual microphone and configuring the audio link with the current active application, thereby realizing audio transmission between the virtual microphone and the current active application, can realize the combined application of the simultaneous interpretation service with other third-party services, greatly expanding the flexibility of the simultaneous interpretation service application.

[0074] The embodiment of the present invention also provides a two-way real-time translation device based on terminal information interaction, which can be configured in a terminal device and is used to execute any embodiment of the two-way real-time translation method based on terminal information interaction. Figure 6 , Figure 6 A schematic block diagram of a two-way real-time translation device based on terminal information interaction provided by an embodiment of the present invention.

[0075] like Figure 6 As shown, the two-way real-time translation device 100 based on terminal information interaction includes a starting unit 110 , an audio link configuration unit 120 , a translation unit 130 , an audio output unit 140 , an audio playback unit 150 and a text information output unit 160 .

[0076] The starting unit 110 is configured to start the simultaneous interpretation service according to the starting instruction if the starting instruction is received.

[0077] In a specific embodiment, the startup unit 110 includes: a process creation unit, which is used to create an independent first process and allocate corresponding system resources respectively; a service startup unit, which is used to start a simultaneous interpretation service matching the startup instruction in the first process; and a language parameter configuration unit, which is used to obtain the system configuration language of the terminal device and the target translation language in the startup instruction to configure the language parameters of the simultaneous interpretation service.

[0078] The audio link configuration unit 120 is used to create a virtual microphone and perform audio link configuration on the virtual microphone according to the currently active application.

[0079] In a specific embodiment, the audio link configuration unit 120 includes: a scene type acquisition unit, used to obtain a scene type matching the currently active application; and a configuration unit, used to perform a one-way audio link configuration or a two-way audio link configuration between the currently active application and the virtual microphone according to the scene type.

[0080] The translation unit 130 is used to obtain a first voice through the configured audio link, and translate the first voice through the simultaneous interpretation service to obtain a corresponding second voice and / or text information; the first voice is a user voice collected by the audio collection module of the terminal device or a playback voice played by the terminal device.

[0081] The audio output unit 140 is configured to collect the second voice through the virtual microphone and output the second voice through the currently active application.

[0082] The audio playback unit 150 is used to obtain a third voice through the configured audio link, translate the third voice through the simultaneous interpretation service to obtain a corresponding fourth voice and play it; the third voice is voice information obtained by collecting the voice received and played by the currently active application through the virtual microphone.

[0083] The text information output unit 160 is used to input the text information into the currently active application for synchronous output.

[0084] The two-way real-time translation device based on terminal information interaction provided in the embodiment of the present invention applies the above-mentioned two-way real-time translation method based on terminal information interaction, starts the simultaneous interpretation service according to the startup instruction, creates a virtual microphone and configures the audio link for the virtual microphone according to the current active application, obtains the first voice through the audio link and translates it to obtain the second voice and / or text information, collects the second voice through the virtual microphone and outputs it through the current active application, obtains the third voice through the audio link and translates it to obtain the fourth voice for playback, and if text information is obtained, it is input into the current active application for synchronous output. The above-mentioned method, by generating a virtual microphone and configuring the audio link with the current active application, thereby realizing audio transmission between the virtual microphone and the current active application, can realize the combined application of simultaneous interpretation service and other third-party services, greatly expanding the flexibility of simultaneous interpretation service application.

[0085] The above-mentioned two-way real-time translation device based on terminal information interaction can be implemented in the form of a computer program, which can be used in Figure 7 Runs on the computer device shown.

[0086] See also Figure 7 , Figure 71 is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device may be a terminal device for executing a two-way real-time translation method based on terminal information interaction to implement a two-way real-time translation process based on terminal information interaction.

[0087] See also Figure 7 The computer device 500 includes a processor 502 , a memory and a communication interface 505 connected via a communication bus 501 , wherein the memory may include a storage medium 503 and an internal memory 504 .

[0088] The storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute a two-way real-time translation method based on terminal information interaction, wherein the storage medium 503 can be a volatile storage medium or a non-volatile storage medium.

[0089] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0090] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a two-way real-time translation method based on terminal information interaction.

[0091] The communication interface 505 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will appreciate that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device 500 to which the solution of the present invention is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0092] The processor 502 is used to run the computer program 5032 stored in the memory to implement the corresponding functions in the above-mentioned two-way real-time translation method based on terminal information interaction.

[0093] Those skilled in the art will understand that Figure 7 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 7 The embodiments shown are consistent and will not be described again here.

[0094] It should be understood that in the embodiment of the present invention, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0095] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps included in the above-mentioned two-way real-time translation method based on terminal information interaction are implemented.

[0096] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0097] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0099] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a computer-readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk.

[0101] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A two-way real-time translation method based on terminal information interaction, characterized in that: The method is applied in a terminal device, and the method includes: If the input start instruction is received, start the simultaneous interpretation service according to the start instruction; Creating a virtual microphone and configuring an audio link for the virtual microphone according to a currently active application, including: creating another process independent of a first process, and creating a virtual microphone in the newly created process; the first process is a process running for the simultaneous interpretation service; Acquire a first voice through the configured audio link, and translate the first voice through the simultaneous interpretation service to obtain a corresponding second voice and / or text information; the first voice is a user voice collected by the audio collection module of the terminal device or a playback voice played by the terminal device; Collecting the second voice through the virtual microphone and outputting it through the currently active application; Acquire a third voice through the configured audio link, translate the third voice through the simultaneous interpretation service to obtain a corresponding fourth voice and play the fourth voice; the third voice is voice information obtained by collecting the voice received and played by the current active application through the virtual microphone; The text information is input into the currently active application for synchronous output.

2. The two-way real-time translation method based on terminal information interaction according to claim 1, characterized in that: The step of creating a virtual microphone and configuring an audio link for the virtual microphone according to a currently active application includes: Obtaining a scene type that matches the currently active application; A unidirectional audio link configuration or a bidirectional audio link configuration is performed between the current active application and the virtual microphone according to the scene type.

3. The two-way real-time translation method based on terminal information interaction according to claim 1 or 2, characterized in that: The starting the simultaneous interpretation service according to the starting instruction comprises: Create an independent first process and allocate corresponding system resources respectively; Initiating a simultaneous interpretation service matching the initiation instruction in the first process; The system configuration language of the terminal device and the target translation language in the startup instruction are obtained to configure language parameters of the simultaneous interpretation service.

4. The two-way real-time translation method based on terminal information interaction according to claim 1 or 2, characterized in that: The method further comprises: Initiate a corresponding text translation service according to the input text translation instruction and the current active application; The text to be translated is translated by the text translation service to obtain corresponding text information; the text to be translated is the text content played by the current active application or the image text content acquired by the image acquisition module of the terminal device.

5. The two-way real-time translation method based on terminal information interaction according to claim 1, characterized in that: The starting of the corresponding text translation service according to the input text translation instruction and the current active application includes: Create an independent second process and allocate corresponding system resources respectively; Initiating, in the second process, a text translation service matching the text translation instruction; The text translation parameters of the text translation service are configured according to the text route and the text output type of the currently active application.

6. A two-way real-time translation device based on terminal information interaction, characterized in that: The device is configured in a terminal device, and the two-way real-time translation device based on terminal information interaction is used to execute the two-way real-time translation method based on terminal information interaction according to any one of claims 1 to 5, and the device includes: A starting unit, configured to start the simultaneous interpretation service according to the starting instruction if the starting instruction is received; An audio link configuration unit, used to create a virtual microphone and configure the audio link of the virtual microphone according to the currently active application; A translation unit, configured to obtain a first voice through a configured audio link, and translate the first voice through the simultaneous interpretation service to obtain a corresponding second voice and / or text information; the first voice is a user voice collected by an audio collection module of the terminal device or a playback voice played by the terminal device; An audio output unit, configured to collect the second voice through the virtual microphone and output it through the currently active application; an audio playback unit, configured to obtain a third voice through the configured audio link, translate the third voice through the simultaneous interpretation service to obtain a corresponding fourth voice and play the fourth voice; the third voice is voice information obtained by collecting the voice received and played by the currently active application through the virtual microphone; The text information output unit is used to input the text information into the currently active application for synchronous output.

7. The two-way real-time translation device based on terminal information interaction according to claim 6 is characterized in that: The audio link configuration unit comprises: A scene type acquisition unit, used to acquire a scene type that matches the currently active application; A configuration unit is used to configure a unidirectional audio link or a bidirectional audio link between the current active application and the virtual microphone according to the scene type.

8. The two-way real-time translation device based on terminal information interaction according to claim 6 or 7, characterized in that: The startup unit comprises: A process creation unit, used to create independent first processes and allocate corresponding system resources respectively; A service starting unit, configured to start a simultaneous interpretation service matching the starting instruction in the first process; The language parameter configuration unit is used to obtain the system configuration language of the terminal device and the target translation language in the startup instruction to perform language parameter configuration on the simultaneous interpretation service.

9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the two-way real-time translation method based on terminal information interaction described in any one of claims 1 to 5 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the two-way real-time translation method based on terminal information interaction as described in any one of claims 1 to 5 are implemented.

Images