Simultaneous interpretation method and device, electronic equipment and storage medium

Abstract

The invention provides a simultaneous interpretation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining target information of an area where a first target user is located, and building a target virtual scene in an area where a second target user is located, and the target information comprises at least one of the following items: image information of a target object in the area, the target virtual scene comprises a virtual image of the target object, and the target object comprises the first target user; under the condition that construction of the target virtual scene is completed, collecting first voice information of the first target user; converting the first voice information into first target text information, and translating the first target text information into second target text information; and generating second voice information according to the second target text information, outputting the second voice information, and controlling a virtual image of the first target user in the target virtual scene to display an action matched with the second voice information.

Description

Technical Field

[0001] The present invention relates to the technical field of simultaneous interpretation, and in particular, to a method, device, electronic device and storage medium for simultaneous interpretation. Background Art

[0002] With the development of science and technology, simultaneous interpretation has become a hot topic of concern. Among them, simultaneous interpretation can be understood as translating the speaker's language into different languages while the speaker is speaking.

[0003] In the prior art, on the one hand, most simultaneous interpretations are achieved manually, and the vocabulary mastery of interpreters varies, so the accuracy of translation will be affected. On the other hand, translation devices such as translation machines are used for simultaneous interpretation to obtain the output of machine translation language.

[0004] Therefore, there is a problem of poor translation effect in simultaneous interpretation in the prior art. Summary of the Invention

[0005] Embodiments of the present invention provide a method, device, electronic device and storage medium for simultaneous interpretation to solve the problem of poor translation effect in simultaneous interpretation in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a method for simultaneous interpretation, including:

[0007] Obtain target information in the area where the first target user is located, and build a target virtual scene in the area where the second target user is located. The target information includes at least one of the following: image information of a target object in the area. The target virtual scene includes a virtual image of the target object, and the target object includes the first target user;

[0008] When the target virtual scene is built, collect first voice information of the first target user;

[0009] Convert the first voice information into first target text information, and translate the first target text information into second target text information;

[0010] Generate second voice information according to the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display an action matching the second voice information.

[0011] Optionally, the building a target virtual scene in the area where the second target user is located includes:

[0012] Obtain the attribute features of multiple target objects in the target virtual scene to be built, where the attribute features include at least two of the following: field of view range feature, three-dimensional feature, spatial interaction feature, movement duration feature, movement logic feature, and sound feature;

[0013] Based on the attribute features of the target objects, determine the target way to create the target objects, where the target way includes one of panoramic image and panoramic video;

[0014] Create the target objects based on the target way to build the target virtual scene.

[0015] Optionally, the collecting the first voice information of the first target user includes:

[0016] Collect the first voice information of the first target user, where the first voice information includes the speech content of the first target user;

[0017] Decompose the speech content of the first target user to obtain phoneme information, where the phoneme information includes at least one word phoneme and / or at least one pronunciation feature phoneme;

[0018] Determine the speech feature of the first target user according to the phoneme information.

[0019] Optionally, the converting the first voice information into first target text information and translating the first target text information into second target text information includes:

[0020] Convert the first voice information into the first target text information and perform semantic calibration on the first target text information;

[0021] Obtain the current translation method;

[0022] Translate the first target text information according to the preset language type and the translation method to obtain the second target text information.

[0023] Optionally, the generating second voice information according to the second target text information, outputting the second voice information, and controlling the virtual image of the first target user in the target virtual scene to display actions matching the second voice information includes:

[0024] Generate second voice information according to the second target text and the speech feature;

[0025] Set the actions of the virtual character in the target virtual scene according to the second voice information and output the second voice information as the output audio, where the actions of the virtual character include at least one of the following: body actions and facial expressions;

[0026] Wherein, the virtual character is a virtual image of the first target user in the target virtual scene, and the language type of the first target user is different from the preset language type.

[0027] Optionally, after generating the second voice information according to the second target text information, outputting the second voice information, and controlling the virtual image of the first target user in the target virtual scene to display an action matching the second voice information, the method further includes:

[0028] Obtaining an adjustment instruction sent by the second target user through a first target device, where the adjustment instruction includes at least one of the following: volume adjustment, preset language adjustment, and translation mode adjustment;

[0029] Adjusting a second target device according to the adjustment instruction;

[0030] Wherein, the first target device is an operation device of the user side, and the second target device is an audio output device and / or a translation device.

[0031] Optionally, the target object further includes at least one of the following:

[0032] The object in the area, other users in the area except the first target user.

[0033] In a second aspect, an embodiment of the present invention further provides a simultaneous interpretation device, including:

[0034] A first acquisition module, configured to acquire target information of the area where the first target user is located, and build a target virtual scene in the area where the second target user is located, where the target information includes at least one of the following: image information of a target object in the area, the target virtual scene includes a virtual image of the target object, and the target object includes the first target user;

[0035] An acquisition module, configured to acquire first voice information of the first target user when the target virtual scene is built;

[0036] A first processing module, configured to convert the first voice information into first target text information, and translate the first target text information into second target text information;

[0037] A second processing module, configured to generate second voice information according to the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display an action matching the second voice information.

[0038] Optionally, the first acquisition module includes:

[0039] A first acquisition unit, configured to acquire the attribute features of multiple target objects in the target virtual scene to be built, where the attribute features include at least two of the following: field of view feature, three-dimensional feature, spatial interaction feature, movement duration feature, movement logic feature, and sound feature;

[0040] A first determination unit, configured to determine a target manner for creating the target object based on the attribute features of the target object, where the target manner includes one of a panoramic image and a panoramic video;

[0041] A building unit, configured to create the target object based on the target manner to build the target virtual scene.

[0042] Optionally, the acquisition module includes:

[0043] An acquisition unit, configured to acquire the first voice information of the first target user, where the first voice information includes the speech content of the first target user;

[0044] A disassembling unit, configured to disassemble the speech content of the first target user to obtain phoneme information, where the phoneme information includes at least one word phoneme and / or at least one pronunciation feature phoneme;

[0045] A second determination unit, configured to determine the speech feature of the first target user according to the phoneme information.

[0046] Optionally, the first processing module includes:

[0047] A conversion unit, configured to convert the first voice information into the first target text information and perform semantic calibration on the first target text information;

[0048] A second acquisition unit, configured to acquire the current translation method;

[0049] A translation unit, configured to translate the first target text information according to the preset language type and the translation method to obtain the second target text information.

[0050] Optionally, the second processing module includes:

[0051] Generate second voice information according to the second target text and the speech feature;

[0052] Set the actions of the virtual character in the target virtual scene according to the second voice information and output the second voice information as the output audio, where the actions of the virtual character include at least one of the following: body actions and facial expressions;

[0053] Wherein, the virtual character is a virtual image of the first target user in the target virtual scene, and the language type of the first target user is different from the preset language type.

[0054] Optionally, the device further includes:

[0055] A second acquisition module, configured to acquire an adjustment instruction sent by the second target user through a first target device, where the adjustment instruction includes at least one of the following: volume adjustment, preset language adjustment, and translation mode adjustment;

[0056] An adjustment module, configured to adjust a second target device according to the adjustment instruction;

[0057] Wherein, the first target device is an operation device of the user side, and the second target device is an audio output device and / or a translation device.

[0058] Optionally, the target object further includes at least one of the following:

[0059] An object in the area, other users in the area except the first target user.

[0060] In a third aspect, an embodiment of the present invention further provides an electronic device, including a transceiver and a processor,

[0061] The transceiver is configured to acquire target information of the area where the first target user is located, and build a target virtual scene in the area where the second target user is located. The target information includes at least one of the following: image information of a target object in the area. The target virtual scene includes a virtual image of the target object, and the target object includes the first target user;

[0062] The transceiver is further configured to collect first voice information of the first target user when the target virtual scene is built;

[0063] The processor is configured to convert the first voice information into first target text information, and translate the first target text information into second target text information;

[0064] The processor is further configured to generate second voice information according to the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display an action matching the second voice information.

[0065] Fourthly, an embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a program stored on the memory and executable on the processor. When the program is executed by the processor, the steps of the simultaneous interpretation method described in the first aspect are implemented.

[0066] Fifthly, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the simultaneous interpretation method described in the first aspect are implemented.

[0067] Sixthly, an embodiment of the present invention further provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the steps of the simultaneous interpretation method described in the first aspect are implemented.

[0068] In the embodiment of the present invention, first, target information in the area where the first target user is located is obtained, and a target virtual scene is built in the area where the second target user is located. The target virtual scene includes a virtual image of a target object, and the target object includes the first target user. Then, when the building of the target virtual scene is completed, first voice information of the first target user is collected. Then, the first voice information is converted into first target text information, and the first target text information is translated into second target text information. Finally, second voice information is generated based on the second target text information, the second voice information is output, and the virtual image of the first target user in the target virtual scene is controlled to display an action matching the second voice information. Through this method, the voice information of the first target user can be translated into second voice information and input. At the same time, according to the second voice information, the virtual image of the first target user in the target virtual scene can be controlled to display an action matching the second voice information, providing an immersive cross-language communication experience for the second target user and improving the overall translation effect of simultaneous interpretation. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0070] Figure 1 is a flowchart of a simultaneous interpretation method provided by an embodiment of the present invention;

[0071] Figure 2 is a structural diagram of a simultaneous interpretation device provided by an embodiment of the present invention;

[0072] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0073] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0074] The terms "first", "second", etc. in the embodiments of the present invention are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0075] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a simultaneous interpretation method provided by an embodiment of the present invention. As Figure 1 shown, the method includes the following steps:

[0076] 101. Obtain target information in the area where the first target user is located, and build a target virtual scene in the area where the second target user is located. The target information includes at least one of the following: image information of a target object in the area. The target virtual scene includes a virtual image of the target object, and the target object includes the first target user.

[0077] 102. When the construction of the target virtual scene is completed, collect the first voice information of the first target user.

[0078] 103. Convert the first voice information into first target text information, and translate the first target text information into second target text information.

[0079] 104. Generate second voice information according to the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display actions matching the second voice information.

[0080] It should be noted that steps 101, 102, 103, and 104 included in the simultaneous interpretation method provided by the embodiments of the present invention can be executed by an electronic device or other devices, such as a computer, etc., and the embodiments of the present invention do not limit this.

[0081] In step 101, the above-mentioned target information may include image information of target objects within the area where the first target user is located, and the above-mentioned target virtual scene is built based on the image information. Of course, in some other feasible embodiments, the above-mentioned target information may also include video information of target objects within the area where the first target user is located, so that the above-mentioned target virtual scene can be built based on the image information (such as a panoramic image) and the video information (a panoramic video).

[0082] In addition, the above-mentioned target objects may be items, backgrounds, and people within the area where the first target user is located. For example, if the area where the first target user is located is a meeting room, then the tables, chairs, microphones, background boards, and meeting host in the meeting room can all be considered as the above-mentioned target objects.

[0083] It should be noted that the above-mentioned target object may only include the first target user, that is, a virtual image of the first target user is created, or a virtual scene of the environment where the first target user is located can be created. This implementation mode can be set according to user needs, and the embodiments of the present invention do not limit this.

[0084] It should be understood that in the embodiments of the present invention, the first target user and the second target user can be considered as personnel participating in the meeting simultaneously, but the first target user and the second target user are not in the same area. Therefore, through the solution of the embodiments of the present invention, the participation experience of the second target user is improved.

[0085] In step 102, the first voice information of the first target user is collected, which can be understood as collecting the speech content of the first target user. Among them, when it is detected that the first target user starts speaking, it enters the voice recognition state, and the speech content of the first target user is collected through a sound collection device to obtain audio data matching the first target user, that is, the above-mentioned first voice information.

[0086] Among them, the above-mentioned sound collection device can be selected according to user needs and actual working conditions, including but not limited to microphones, recording pens, and computers equipped with sound cards, etc., and the embodiments of the present invention do not limit this.

[0087] In step 103, first, the above-mentioned first voice information is converted into the above-mentioned first target text information. The implementation of speech-to-text can mainly rely on Automatic Speech Recognition (ASR) technology, including but not limited to the following ways to achieve speech recognition: random model method, neural network method, probabilistic syntax analysis method, etc.

[0088] In addition, in the process of translating the above-mentioned first target text information into the second target text information, a translation method can be selected to transform the above-mentioned first text information. The above-mentioned translation method can include strategies such as literal translation or free translation, and the translation method is selected according to the needs of the above-mentioned second target user. The embodiments of the present invention do not limit this.

[0089] In step 104, after generating the above-mentioned second voice information based on the above-mentioned second target text information, it is necessary to output the above-mentioned second voice information and control the virtual image of the first target user in the above-mentioned target virtual scene to display actions matching the above-mentioned second voice information. For example, when the above-mentioned second voice information reflects that the first target user is excited, the virtual image of the first target user in the above-mentioned target virtual scene can be controlled to display matching body movements or facial expressions, that is, it can be understood that the above-mentioned second voice information is expressed in the form of a virtual image, and the virtual image can show corresponding facial expressions and body movements in combination with voice features, enabling participants to intuitively feel the meeting atmosphere.

[0090] In this implementation scheme, first, the target information of the area where the first target user is located is obtained, and a target virtual scene is built in the area where the second target user is located. The target virtual scene includes a virtual image of the target object, and the target object includes the first target user. Then, when the target virtual scene is built, the first voice information of the first target user is collected, and then the first voice information is converted into the first target text information, and the first target text information is translated into the second target text information. Finally, the second voice information is generated based on the second target text information, the second voice information is output, and the virtual image of the first target user in the target virtual scene is controlled to display actions matching the second voice information. Through this method, the voice information of the first target user can be translated into the second voice information and input. At the same time, according to the second voice information, the virtual image of the first target user in the target virtual scene can be controlled to display actions matching the second voice information, providing an immersive cross-language communication experience for the second target user and improving the overall translation effect of simultaneous interpretation.

[0091] Optionally, building the target virtual scene in the area where the second target user is located includes:

[0092] Obtain the attribute features of multiple target objects in the target virtual scene to be built, where the attribute features include at least two of the following: field of view range feature, three-dimensional feature, spatial interaction feature, movement duration feature, movement logic feature, and sound feature;

[0093] Based on the attribute features of the target object, determine the target method for creating the target object, where the target method includes one of panoramic image and panoramic video;

[0094] Create the target object based on the target method to build the target virtual scene.

[0095] In this implementation, first obtain the attribute features of multiple target objects in the above-mentioned target virtual scene to be built, and then determine the target method for constructing the above-mentioned target object according to the above-mentioned attribute features. Finally, complete the construction of the above-mentioned target virtual scene. Through this method, by selecting the creation method of the target object according to the attribute features of different target objects, it is possible to reasonably select the target method to create the target object on the basis of ensuring the authenticity and detail restoration degree of the target object in the virtual reality scene, reduce the workload in the scene construction process, shorten the scene construction time, improve the construction efficiency, and further improve the overall display effect of simultaneous interpretation.

[0096] It should be understood that there can be multiple above-mentioned target objects, that is, there can be multiple different types of target objects in the above-mentioned target virtual scene. The above-mentioned target object can be a static background object or an object that can undergo overall spatial changes. The target object can also be an object that undergoes partial spatial changes. The embodiments of the present invention do not make limitations in this regard.

[0097] In addition, the above-mentioned attribute features are used to represent the features of the target object in dimensions such as spatial state, movement state, and sound state in the virtual reality scene. The attribute features of the above-mentioned target object can include spatial attribute features, movement attribute features, and sound features, etc.

[0098] Similarly, the spatial attribute features can also include field of view range feature, three-dimensional feature, spatial interaction feature, and spatial change feature, etc. The movement attribute features can also include movement duration feature and movement logic feature, etc. The sound feature can include the presence of sound and the absence of sound.

[0099] In some optional implementation manners, the attribute features of the target object can include one or more of field of view range feature, three-dimensional feature, spatial interaction feature, movement duration feature, movement logic feature, sound feature, and spatial change feature. The attribute features of the target object can reflect the effects that the target object needs to present in the virtual reality scene.

[0100] It should be noted that the field of view range feature is used to represent the viewing distance of the target object in the virtual reality scene. The target object in the virtual reality scene can be located in different scenes and positions, and its field of view range feature can include a close view and a distant view.

[0101] Optionally, the collecting of the first voice information of the first target user includes:

[0102] Collecting the first voice information of the first target user, where the first voice information includes the speech content of the first target user;

[0103] Decomposing the speech content of the first target user to obtain phoneme information, where the phoneme information includes at least one word phoneme and / or at least one pronunciation feature phoneme;

[0104] Determining the speech feature of the first target user according to the phoneme information.

[0105] In this implementation, first, the speech content of the above-mentioned first target user is collected, and then the speech content is decomposed to obtain the above-mentioned phoneme information. Based on the above-mentioned phoneme information, the speech feature of the above-mentioned first target user is determined. Through this method, the speech audio of the above-mentioned first target user can be decomposed into multiple word phonemes, and / or multiple pronunciation feature phonemes, and / or a combination of word phonemes and pronunciation feature phonemes. Through word phonemes, pronunciation feature phonemes or a combination of word phonemes and pronunciation feature phonemes, the speech theme or speech context of the above-mentioned first target user can be accurately recognized, which is also convenient for the subsequent translation process, and then the second voice information with the speech feature of the above-mentioned first target user is obtained, improving the overall translation effect of simultaneous interpretation.

[0106] It should be understood that as the speech content of the above-mentioned first target user increases, the more word factors and pronunciation feature phonemes are decomposed, thus adding more different word factors and pronunciation feature phonemes, making the data corresponding to the above-mentioned first target user more abundant and accurate. With the more speech of the above-mentioned first target user, it is possible to search in the data according to the decomposed multiple word phonemes and / or multiple pronunciation feature phonemes, and thus the speech feature of the target person can be determined.

[0107] Optionally, the converting the first voice information into first target text information and translating the first target text information into second target text information includes:

[0108] Converting the first voice information into the first target text information and performing semantic calibration on the first target text information;

[0109] Obtaining the current translation method;

[0110] Translate the first target text information according to the preset language type and the translation method to obtain the second target text information.

[0111] In this embodiment, the conversion process from the first voice information to the second target text information is mainly reflected. First, convert the first voice information into the first target text information, and perform semantic calibration on the first target text information to improve the accuracy of translation. Then, translate the first target text information according to the translation method and language type required by the user. Finally, obtain the second target text information. Through this method, the accuracy of translation in the simultaneous interpretation process is improved. On the other hand, translating according to the user's needs improves the user experience in the simultaneous interpretation process.

[0112] It should be understood that the above translation methods include but are not limited to the following:

[0113] (1) Literal translation method: Translate relying on the literal meaning of the original text, and strive to faithfully convey the meaning and expression of the original text. This method is suitable for situations where the sentence is simple and the language is straightforward, and is more suitable for fields such as official documents and scientific explanations. However, the literal translation method also has limitations and may not be able to fully convey the cultural connotations and expressions of the original text.

[0114] (2) Free translation method: Adjust the sentence structure and reorganize the language to ensure that the translation is more in line with the expression habits and cultural backgrounds of the target language. This method is suitable for scenarios such as art works and advertising that require flexible use of language. However, when performing free translation, attention should be paid to maintaining the core ideas and styles of the original text.

[0115] (3) Adjustment method: The adjustment method makes the translated text easier to understand and accept by introducing cultural elements of the target language. This method is often used in fields full of cultural characteristics such as literary works and historical books.

[0116] (4) Supplementary explanation method: To ensure that readers can correctly understand, the translator can add explanations or notes to supplement information. This method is usually applied to documents in professional fields, scientific papers and other documents that require accuracy.

[0117] (5) Merging and modification method: According to the specific text and requirements, flexibly use different methods for translation. Through the merging and modification method, the accuracy and fluency of the translation can be guaranteed to the greatest extent.

[0118] Optionally, generating second voice information based on the second target text information, outputting the second voice information, and controlling the virtual image of the first target user in the target virtual scene to display actions matching the second voice information includes:

[0119] Generate second voice information based on the second target text and the speech features;

[0120] Set the actions of the virtual character in the target virtual scene according to the second voice information, and output the second voice information as the output audio. The actions of the virtual character include at least one of the following: body actions and facial expressions;

[0121] Wherein, the virtual character is a virtual image of the first target user in the target virtual scene, and the language type of the first target user is different from the preset language type.

[0122] In this implementation, it is necessary to generate second voice information according to the above-mentioned second target text and the above-mentioned speech features. Among them, the above-mentioned speech features are the speech features of the above-mentioned first target user. Then, the above-mentioned second voice information is expressed in the form of a virtual image. The virtual image can show corresponding facial expressions and body actions in combination with the voice features. Through this method, the participants can intuitively feel the meeting atmosphere, thereby improving the translation effect of the overall simultaneous interpretation process.

[0123] Optionally, after generating the second voice information according to the second target text information, outputting the second voice information, and controlling the virtual image of the first target user in the target virtual scene to display actions matching the second voice information, the method further includes:

[0124] Obtain an adjustment instruction sent by the second target user through the first target device. The adjustment instruction includes at least one of the following: volume adjustment, preset language adjustment, and translation mode adjustment;

[0125] Adjust the second target device according to the adjustment instruction;

[0126] Wherein, the first target device is an operation device of the user side, and the second target device is an audio output device and / or a translation device.

[0127] In this implementation, the above-mentioned second target user can send the above-mentioned adjustment instruction through the above-mentioned first target device according to his own needs. The above-mentioned adjustment instruction is used to adjust the volume, preset language or translation mode, and is responded by the above-mentioned second target device, so as to complete the adjustment process. Through this method, the needs of the above-mentioned second target user can be met, thereby improving the user experience in the simultaneous interpretation process.

[0128] It should be noted that in some alternative embodiments, the above-mentioned second target user first enters a virtual meeting environment through a Virtual Reality (VR) device, which can be understood as the above-mentioned second target device. Then, the above-mentioned second target user can adjust the volume, preset language, and translation mode through the VR device.

[0129] Optionally, the target object further includes at least one of the following:

[0130] The objects in the area, and other users in the area except the first target user.

[0131] In this embodiment, there can be multiple different types of target objects in the above-mentioned target virtual scene. The target object can be a static background object or an object that can undergo overall spatial changes. The target object can also be an object that undergoes partial spatial changes. The embodiments of the present invention do not limit this. For example, when the above-mentioned first target user includes user A and user B, then user A and user B can be constructed simultaneously in the construction of the above-mentioned target virtual scene. Similarly, during sound collection and translation, user A and user B can also be processed simultaneously. Through this solution, a complete scene of the area where the above-mentioned first target user is located can be constructed, thereby improving the translation effect and the user experience effect during the overall simultaneous interpretation process.

[0132] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of a simultaneous interpretation device provided by an embodiment of the present invention. As Figure 2 shown, the simultaneous interpretation device 200 includes:

[0133] A first acquisition module 201, configured to acquire target information of the area where the first target user is located, and build a target virtual scene in the area where the second target user is located. The target information includes at least one of the following: image information of the target object in the area. The target virtual scene includes a virtual image of the target object, and the target object includes the first target user;

[0134] An acquisition module 202, configured to collect first voice information of the first target user when the target virtual scene is built;

[0135] A first processing module 203, configured to convert the first voice information into first target text information, and translate the first target text information into second target text information;

[0136] The second processing module 204 is configured to generate second voice information based on the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display actions matching the second voice information.

[0137] Optionally, the first acquisition module 201 includes:

[0138] A first acquisition unit, configured to acquire the attribute features of multiple target objects in the target virtual scene to be built, where the attribute features include at least two of the following: field of view range feature, three-dimensional feature, spatial interaction feature, movement duration feature, movement logic feature, and sound feature;

[0139] A first determination unit, configured to determine a target manner for creating the target object based on the attribute features of the target object, where the target manner includes one of a panoramic image and a panoramic video;

[0140] A building unit, configured to create the target object based on the target manner to build the target virtual scene.

[0141] Optionally, the acquisition module 202 includes:

[0142] An acquisition unit, configured to acquire the first voice information of the first target user, where the first voice information includes the speech content of the first target user;

[0143] A disassembling unit, configured to disassemble the speech content of the first target user to obtain phoneme information, where the phoneme information includes at least one word phoneme and / or at least one pronunciation feature phoneme;

[0144] A second determination unit, configured to determine the speech feature of the first target user according to the phoneme information.

[0145] Optionally, the first processing module 203 includes:

[0146] A conversion unit, configured to convert the first voice information into the first target text information and perform semantic calibration on the first target text information;

[0147] A second acquisition unit, configured to acquire the current translation method;

[0148] A translation unit, configured to translate the first target text information according to the preset language type and the translation method to obtain the second target text information.

[0149] Optionally, the second processing module 204 includes:

[0150] Generate second voice information according to the second target text and the speech feature;

[0151] Set the actions of the virtual character in the target virtual scene according to the second voice information, and output the second voice information as the output audio. The actions of the virtual character include at least one of the following: body actions and facial expressions;

[0152] Wherein, the virtual character is the virtual image of the first target user in the target virtual scene, and the language type of the first target user is different from the preset language type.

[0153] Optionally, the simultaneous interpretation device 200 further includes:

[0154] A second acquisition module, configured to acquire an adjustment instruction sent by the second target user through the first target device. The adjustment instruction includes at least one of the following: volume adjustment, preset language adjustment, and translation mode adjustment;

[0155] An adjustment module, configured to adjust the second target device according to the adjustment instruction;

[0156] Wherein, the first target device is an operation device of the user side, and the second target device is an audio output device and / or a translation device.

[0157] Optionally, the target object further includes at least one of the following:

[0158] The objects in the area, other users in the area except the first target user.

[0159] Specifically, please refer to Figure 3 , the embodiment of the present invention further provides an electronic device, including a bus 301, a transceiver 302, an antenna 303, a bus interface 304, a processor 305, and a memory 306.

[0160] The transceiver 302 is configured to acquire target information in the area where the first target user is located, and build a target virtual scene in the area where the second target user is located. The target information includes at least one of the following: image information of the target object in the area. The target virtual scene includes the virtual image of the target object, and the target object includes the first target user;

[0161] The transceiver 302 is further configured to collect the first voice information of the first target user when the target virtual scene is built;

[0162] The processor 305 is configured to convert the first voice information into first target text information, and translate the first target text information into second target text information;

[0163] The processor 305 is further configured to generate second voice information based on the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display actions matching the second voice information.

[0164] In Figure 3 it, the bus architecture (represented by bus 301), bus 301 may include any number of interconnected buses and bridges. Bus 301 links together various circuits including one or more processors represented by processor 305 and a memory represented by memory 306. Bus 301 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and thus will not be further described herein. Bus interface 304 provides an interface between bus 301 and transceiver 302. Transceiver 302 may be one element or multiple elements, such as multiple receivers and transmitters, providing units for communicating with various other devices over a transmission medium. The data processed by processor 305 is transmitted over a wireless medium via antenna 303. Further, antenna 303 also receives data and transmits the data to processor 305.

[0165] Processor 305 is responsible for managing bus 301 and general processing, and may also provide various functions including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 306 may be used to store data used by processor 305 when performing operations.

[0166] Optionally, processor 305 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or a Complex Programmable Logic Device (CPLD).

[0167] An embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a program stored on the memory and executable on the processor. When the program is executed by the processor, it implements each process of the above simultaneous interpretation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0168] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above-described embodiment of the simultaneous interpretation method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here. Among them, the computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0169] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement each process of the method embodiment shown in FIG. A or FIG. B above and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0170] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including that element.

[0171] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment method can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0172] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. A simultaneous interpretation method, characterized in that: include: Acquire target information of the area where the first target user is located, and build a target virtual scene in the area where the second target user is located, wherein the target information includes at least one of the following: image information of a target object in the area, the target virtual scene includes a virtual image of the target object, and the target object includes the first target user; When the target virtual scene is built, collecting first voice information of the first target user; Converting the first speech information into first target text information, and translating the first target text information into second target text information; Generate second voice information according to the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display actions matching the second voice information.

2. The simultaneous interpretation method according to claim 1, characterized in that: The step of building a target virtual scene in the area where the second target user is located includes: Acquire attribute characteristics of multiple target objects in the target virtual scene to be constructed, wherein the attribute characteristics include at least two of the following: field of view characteristics, three-dimensional characteristics, spatial interaction characteristics, motion duration characteristics, motion logic characteristics, and sound characteristics; Based on the attribute characteristics of the target object, determining a target mode for creating the target object, the target mode comprising one of a panoramic image and a panoramic video; The target object is created based on the target mode to build the target virtual scene.

3. The simultaneous interpretation method according to claim 2, characterized in that: The collecting the first voice information of the first target user includes: Collecting the first voice information of the first target user, wherein the first voice information includes speech content of the first target user; Decomposing the speech content of the first target user to obtain phoneme information, wherein the phoneme information includes at least one word phoneme and / or at least one pronunciation feature phoneme; The speech characteristics of the first target user are determined according to the phoneme information.

4. The simultaneous interpretation method according to claim 3, characterized in that: The converting the first voice information into first target text information, and translating the first target text information into second target text information, comprises: Converting the first speech information into the first target text information, and performing semantic calibration on the first target text information; Get the current translation mode; The first target text information is translated according to the preset language type and the translation method to obtain the second target text information.

5. The simultaneous interpretation method according to claim 3, characterized in that: The steps of generating the second voice information according to the second target text information, outputting the second voice information, and controlling the virtual image display of the first target user in the target virtual scene to match the second voice information include: generating second speech information according to the second target text and the speech feature; Setting the action of the virtual character in the target virtual scene according to the second voice information, and outputting the second voice information as output audio, wherein the action of the virtual character includes at least one of the following: body movements and facial expressions; The virtual character is a virtual image of the first target user in the target virtual scene, and the language type of the first target user is different from the preset language type.

6. The simultaneous interpretation method according to any one of claims 1 to 5, characterized in that: After the steps of generating the second voice information according to the second target text information, outputting the second voice information, and controlling the virtual image display of the first target user in the target virtual scene to match the second voice information, the method further includes: Acquire an adjustment instruction sent by the second target user through the first target device, where the adjustment instruction includes at least one of the following: volume adjustment, preset language adjustment, and translation mode adjustment; adjusting the second target device according to the adjustment instruction; The first target device is an operating device at the user end, and the second target device is an audio output device and / or a translation device.

7. The simultaneous interpretation method according to any one of claims 1 to 5, characterized in that: The target object also includes at least one of the following: objects in the area, and other users in the area except the first target user.

8. A simultaneous interpretation device, characterized in that: The device comprises: A first acquisition module is used to acquire target information of an area where a first target user is located, and to build a target virtual scene in an area where a second target user is located, wherein the target information includes at least one of the following: image information of a target object in the area, the target virtual scene includes a virtual image of the target object, and the target object includes the first target user; A collection module, used for collecting the first voice information of the first target user when the target virtual scene is built; A first processing module, configured to convert the first voice information into first target text information, and translate the first target text information into second target text information; The second processing module is used to generate second voice information according to the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display an action matching the second voice information.

9. The simultaneous interpretation method according to claim 8, characterized in that: The first acquisition module includes: A first acquisition unit is used to acquire attribute characteristics of multiple target objects in the target virtual scene to be constructed, wherein the attribute characteristics include at least two of the following: field of view characteristics, three-dimensional characteristics, spatial interaction characteristics, motion duration characteristics, motion logic characteristics, and sound characteristics; A first determining unit, configured to determine a target mode for creating the target object based on an attribute feature of the target object, wherein the target mode includes one of a panoramic image and a panoramic video; A construction unit is used to create the target object based on the target mode to build the target virtual scene.

10. The simultaneous interpretation method according to claim 9, characterized in that: The acquisition module comprises: A collecting unit, configured to collect the first voice information of the first target user, wherein the first voice information includes a speech content of the first target user; a disassembling unit, configured to disassemble the speech content of the first target user to obtain phoneme information, wherein the phoneme information includes at least one word phoneme and / or at least one pronunciation feature phoneme; The second determining unit is used to determine the speech characteristics of the first target user according to the phoneme information.

11. The simultaneous interpretation method according to claim 10, characterized in that: The first processing module comprises: A conversion unit, configured to convert the first speech information into the first target text information, and perform semantic calibration on the first target text information; A second obtaining unit, used for obtaining the current translation mode; The translation unit is used to translate the first target text information according to the preset language type and the translation method to obtain the second target text information.

12. The simultaneous interpretation method according to claim 10, characterized in that: The second processing module comprises: generating second speech information according to the second target text and the speech feature; Setting the action of the virtual character in the target virtual scene according to the second voice information, and outputting the second voice information as output audio, wherein the action of the virtual character includes at least one of the following: body movements and facial expressions; The virtual character is a virtual image of the first target user in the target virtual scene, and the language type of the first target user is different from the preset language type.

13. The simultaneous interpretation method according to any one of claims 8 to 12, characterized in that: The device also includes: A second acquisition module is used to acquire an adjustment instruction sent by the second target user through the first target device, wherein the adjustment instruction includes at least one of the following: volume adjustment, preset language adjustment, and translation mode adjustment; An adjustment module, used for adjusting the second target device according to the adjustment instruction; The first target device is an operating device at the user end, and the second target device is an audio output device and / or a translation device.

14. The simultaneous interpretation method according to any one of claims 8 to 12, characterized in that: The target object also includes at least one of the following: objects in the area, and other users in the area except the first target user.

15. An electronic device, characterized in that: Including transceiver and processor, The transceiver is used to obtain target information of the area where the first target user is located, and to build a target virtual scene in the area where the second target user is located, wherein the target information includes at least one of the following: image information of a target object in the area, the target virtual scene includes a virtual image of the target object, and the target object includes the first target user; The transceiver is further used to collect the first voice information of the first target user when the target virtual scene is built; The processor is used to convert the first voice information into first target text information, and translate the first target text information into second target text information; The processor is further configured to generate second voice information based on the second target text information, output the second voice information, and control the virtual image of the first target user in the target virtual scene to display an action matching the second voice information.

16. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the simultaneous interpretation method according to any one of claims 1 to 7 are implemented.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the simultaneous interpretation method according to any one of claims 1 to 7 are implemented.

18. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the simultaneous interpretation method according to any one of claims 1 to 7.

Images