Game-based multilingual real-time translation method and device

Abstract

The invention provides a multilingual real-time translation method and device based on a game, and the method comprises the steps: determining a first text corresponding to speech data input by a game user and each target language which the first text needs to be translated into in response to the speech data, carrying out the standardization of the first text according to a predetermined term library, and carrying out the translation of the first text; and determining a to-be-translated text, inputting the to-be-translated text into a pre-trained translation model, converting the to-be-translated text into a translated text corresponding to each target language through the translation model, outputting the translated text, and returning the translated text to each target user corresponding to each target language in the game. Through the method, multilingual real-time translation in the game can be accurately carried out.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a game-based multi-language real-time translation method and device. Background Art

[0002] At present, with the rapid development of Internet technology, online games are widely popular around the world. However, although gamers from all over the world can interact and communicate in the game, language barriers have become a major problem that restricts the deep interaction and cooperation between players. Especially in game types that require high coordination and instant communication, such as multiplayer online battle arena (MOBA) and massively multiplayer online role-playing games (MMORPG), the negative impact of language differences on the gaming experience is particularly significant.

[0003] Although there are many translation tools on the market, which have alleviated the problem of language communication to a certain extent, these translation tools are difficult to accurately capture and translate the game's unique terms, slang, and vocabulary that changes with game updates, resulting in translation results that often deviate from the actual context, affecting players' understanding and communication efficiency.

[0004] Therefore, how to accurately perform real-time translation within the game has become a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of this specification provide a game-based multi-language real-time translation method and device to partially solve the problems existing in the above-mentioned prior art.

[0006] The embodiments of this specification adopt the following technical solutions: This specification provides a game-based multi-language real-time translation method, the method comprising: In response to speech data input by a game user, determining a first text corresponding to the speech data and each target language into which the first text needs to be translated; Standardizing the first text according to a predetermined terminology database to determine a text to be translated; Inputting the text to be translated into a pre-trained translation model, so as to convert the text to be translated into translation texts corresponding to the target languages ​​through the translation model, and outputting the translation texts; The translated text is returned to each target user corresponding to each target language in the game.

[0007] Optionally, the first text is standardized according to a predetermined term library, specifically including: Determining each colloquial vocabulary in the first text according to a predetermined term library; Determining each standard vocabulary corresponding to each spoken vocabulary from the predetermined term library; Each colloquial word in the first text is replaced with the corresponding standard word to determine the text to be translated.

[0008] Optionally, the first text is standardized according to a predetermined term library to determine the text to be translated, specifically including: Inputting the first text into a pre-trained term conversion model to standardize the first text through the term conversion model and output a text to be translated; The term conversion model is pre-trained and determined according to the predetermined term library.

[0009] Optionally, pre-training the term conversion model specifically includes: determining a first sample text containing each spoken vocabulary; According to the predetermined term library, each colloquial word in the first sample text is replaced with a corresponding standard word, and a first annotated text is determined as an annotation; Input the first sample text as a sample into the term conversion model, and output a standardized text corresponding to the first sample text; The term conversion model is trained with the goal of minimizing the difference between the first annotated text and the standardized text.

[0010] Optionally, after determining the text to be translated and before inputting the text to be translated into a pre-trained translation model, the method further includes: Returning the text to be translated to the game user, so that the game user can judge whether the semantics of the text to be translated is accurate; If it is inaccurate, obtain the historical speech data of the game user in the game; The first text and the historical speech data are input into the pre-trained term conversion model, so that the term conversion model re-outputs text according to the first text under the prompt of the historical speech data as a new text to be translated.

[0011] Optionally, pre-training the translation model specifically includes: determining a second sample text containing each standard vocabulary; According to the predetermined sample languages ​​into which the second sample text needs to be translated, determining correct translation texts corresponding to the sample languages ​​and having the same semantics as the second sample text as the second annotation; The second sample text is used as a sample and input into the translation model, so as to translate the second sample text through the translation model and output training translation texts of each sample language corresponding to the second sample text; The translation model is trained with the goal of minimizing the difference between the training translation text and the second annotation.

[0012] Optionally, returning the translated text to each target user corresponding to each target language in the game specifically includes: By calling the speech generation plug-in, the translation texts corresponding to the target languages ​​are converted into corresponding translation speech; For the translation text corresponding to each target language, determine the target language corresponding to each target user; The translated text and translated speech corresponding to the target language are returned to each target user corresponding to the target language.

[0013] This specification provides a game-based multilingual real-time translation device, the device comprising: A response module, configured to respond to speech data input by a game user, determine a first text corresponding to the speech data, and each target language into which the first text needs to be translated; A determination module, used to standardize the first text according to a predetermined term library to determine a text to be translated; A translation module, used for inputting the text to be translated into a pre-trained translation model, so as to convert the text to be translated into translation texts corresponding to the target languages ​​through the translation model, and output the translation texts; The return module is used to return the translated text to each target user corresponding to each target language in the game.

[0014] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned game-based multi-language real-time translation method is implemented.

[0015] The present specification provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned game-based multi-language real-time translation method when executing the program.

[0016] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: The embodiment of this specification discloses a multi-language real-time translation method based on a game, which responds to the speech data input by the game user, determines the first text corresponding to the speech data, and the target languages ​​to which the first text needs to be translated, and standardizes the first text according to a predetermined term library, determines the text to be translated, and inputs the text to be translated into a pre-trained translation model, so that the text to be translated is converted into the translation text corresponding to the target languages ​​through the translation model, and outputs, and returns the translation text to the target users corresponding to the target languages ​​in the game. Through this method, after the speech data of the player is captured in real time in the game and the first text is determined, the first text can be translated into multiple languages ​​in real time and efficiently, and based on the term library and the pre-trained translation model, it can ensure that while translating efficiently, the unique terms, slang and other words in the game are accurately captured and translated, thereby improving the accuracy of translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A flowchart of a multi-language real-time translation method based on a game provided in an embodiment of this specification; Figure 2 A schematic diagram of the structure of a game-based multi-language real-time translation device provided in an embodiment of this specification; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0019] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0020] Figure 1 The flowchart of a multi-language real-time translation method based on a game in this specification may include the following steps: S100: In response to speech data input by a game user, determining a first text corresponding to the speech data and target languages ​​into which the first text needs to be translated.

[0021] Specifically, in the embodiments of the present specification, when a game user plays a multiplayer online game with other game users from various countries, it may be necessary to communicate with each other game user by voice or text, so that other game users can perform operations based on the speech data of the game user or provide feedback to the game user, thereby achieving interaction and communication among the game users, i.e., players in the game, and improving the gaming experience.

[0022] Furthermore, since the game users in the same online game may come from different countries and use different languages, the server in the background of the multiplayer online game can record the player information and determine the language preference set by each game player on the client when each game user logs in to the game through the client. If the server detects that all game users in the current game environment use the same language, that is, the language preferences set by all game users are consistent, the game server will allow all game users to communicate directly. Specifically, the speech data input by the game user can be directly received, and after determining that the speech data meets the pre-set risk control rules, the speech data is returned to all other game users.

[0023] Furthermore, if the server of the online game detects that the language preferences set by the game users in the current game environment are different, it will respond to the speech data input by the game user and determine the first text corresponding to the speech data, which will be used for subsequent translation based on the first text to obtain texts in the target languages ​​with the same semantics as the speech data and return them to the corresponding other game players.

[0024] The speech data mentioned here includes voice information or text information input by the game user. If the speech data is voice information, the text information corresponding to the voice information is determined as the first text through a pre-trained voice translation model or voice translation plug-in. If the speech data is text information, the text information can be directly used as the first text corresponding to the speech data. Of course, the text information input by the user can be pre-processed, including removing irrelevant symbols, converting uppercase and lowercase, and word segmentation to obtain the first text. The specific determination method of the first text is not limited in the embodiments of this specification.

[0025] Furthermore, while determining the first text corresponding to the speech data, the language recognition module in the natural language processing technology can also be used to automatically detect the language of the speech data input by the user, and based on the nationality information or set language preferences of other game users in the game, determine the target languages ​​into which the speech data needs to be translated.

[0026] For example, in a multiplayer online game, there are three game users A, B, and C online at the same time. The language preference set by game user A is Chinese, the language preference set by game user B is English, and the language preference set by game user C is Japanese. After game user A inputs speech data, the target languages ​​can be determined as English and Japanese based on the language preference settings of game users B and C. Subsequently, based on the speech data input by game user A, a translation text with the same semantics as the speech data and in English can be returned to game user B, and a translation text with the same semantics as the speech data and in Japanese can be returned to game user C.

[0027] It should be noted that, in response to the speech data input by the game user, the first text and the target languages ​​can be determined by a terminal device such as a laptop computer, a tablet computer, etc. Of course, it can also be a server connected to a multiplayer online game. For the sake of ease of description, this specification only takes the server as an example of the execution subject to illustrate the game-based multi-language real-time translation method provided in this specification.

[0028] S102: Standardizing the first text according to a predetermined term library to determine a text to be translated.

[0029] Furthermore, after determining the first text, since there may be some colloquial expressions or slang words input by the game user in the text, which will affect the subsequent translation, the first text can be standardized according to a predetermined term library. Then, the text after the standardization process, that is, the text with the term replacement and unified format, is called the text to be translated. The text to be translated is the basis for subsequent translation work. It ensures the consistency and accuracy of the terminology during the translation process and reduces the possibility of translation errors and misunderstandings.

[0030] Specifically, standardizing the first text can adjust or replace the words, phrases or sentences in the first text according to the standards in the predetermined term library, identify the colloquial expressions in the first text, find the corresponding standard vocabulary in the term library, and then replace the colloquial expressions in the first text with these standardized in-game professional names to improve the accuracy of subsequent translations. For example, the first text is "I want to play crocodile", "crocodile" is a colloquial expression vocabulary, which can be replaced with the corresponding in-game professional name "Rekton", then the standardized vocabulary to be translated is "I want to play Renekton", or "I hope to choose the hero character as Renekton". Of course, standardization can also unify the text format, punctuation, date and time format, etc., to further ensure the accuracy and consistency of the translation, and the specific standardization steps are not limited in the embodiments of this specification.

[0031] Among them, the terminology library mentioned here stores a large number of game terms, professional terms, colloquial expressions and slang databases. This database is specially designed to assist the translation system to improve the accuracy and relevance of the translation. Specifically, to create a terminology library in advance, you can first clarify the scope of the terminology and the target user group, define which types of game terms need to be included, such as character names, skill names, game mechanisms, equipment props, etc., and determine which types of players or developers the terminology library mainly serves. Then, you can manually collect terms and corresponding colloquial expressions through game forums, communities, official documents, in-game chat logs, etc., develop or use ready-made APIs to obtain data from official game websites, social media, forums, etc., and use web crawler technology to automatically capture relevant information. Of course, you can also enable players to submit new terms and corresponding colloquial vocabulary through websites or applications, and set up a review mechanism to ensure the accuracy and applicability of the terms.

[0032] Then, after collecting the term-related information, clean the collected data, including removing noise, standardizing the format, marking tags, etc., and store the relevant term information in the database, including colloquial vocabulary such as slang and corresponding terms or professional names in the game, etc. Of course, machine learning classification algorithms (such as decision trees, random forests, support vector machines, etc.) can also be used to classify terms. The basis for classification can be the semantic features, contextual features or user-defined labels of the terms. By training the classification model, new terms can be automatically assigned to the corresponding categories. For unlabeled or difficult to classify terms, clustering algorithms (such as K-means, hierarchical clustering, etc.) can be used to group them. Clustering algorithms group terms based on similarity or distance between terms, which helps to discover potential term groups or topics.

[0033] Furthermore, the term base can be automatically updated, including adding new terms, updating term definitions, adjusting term classifications, etc. The automatic update process can be set to be executed regularly to ensure that the term base is always up to date. It can also be iterated and optimized through user feedback on terminology submitted on the interface, such as pointing out errors and making suggestions for improvement, to continuously improve the algorithm and model performance.

[0034] Furthermore, predetermining the term library can also be specifically as follows: according to a preset period, each historical speech data of each historical game user is obtained, and the historical speech text corresponding to each historical speech data is determined, and the historical speech text is input into a pre-trained event prediction model to obtain the historical event corresponding to the historical speech text. If the historical event corresponds to at least one historical operation behavior of the historical game user within a preset time after inputting the historical speech data, then according to the historical event, the historical term text corresponding to the historical speech text is determined, and according to the historical speech text and the historical term text, the standard vocabulary corresponding to each colloquial vocabulary in the historical speech text is determined and added to the term library. Among them, the standard vocabulary is derived from the historical term text. It should be noted that the specific method of predetermining the term library is not limited in the embodiments of this specification.

[0035] Specifically, the historical speech data of all players can be collected from the game log system at a frequency of monthly or any preset period. These data include but are not limited to chat messages, team voice-to-text records, etc. The collected historical speech data is pre-processed, such as removing invalid characters, normalizing punctuation marks, and text segmentation, to determine the historical speech text for subsequent analysis. Then, the processed historical speech text can be analyzed using a pre-trained event prediction model to predict the historical operation behavior of the historical game user corresponding to the historical speech text after inputting the historical speech data, such as "forming a team to brush a copy", "selling equipment", etc., as a historical event.

[0036] If the historical event corresponds to at least one historical operation behavior of the historical game user within a preset time after inputting the historical speech data, the historical event predicted by the model is compared with the actual operation behavior of the player within a preset time after speaking, such as 5 minutes, 10 minutes, etc. If the predicted event is highly correlated with the actual behavior of the player, then the event prediction is considered to be valid. For example, the event prediction model predicts the historical event "Team up to fight the dragon" based on the historical speech data "Team up to play a copy", and the historical game user teams up with other game users to play a copy within the preset time, then the prediction is correct.

[0037] Furthermore, based on the predicted historical events, the historical terminology texts containing standard vocabulary corresponding to the historical speech texts are determined, so as to identify the standard vocabulary derived from the historical terminology texts corresponding to the colloquial vocabulary in the historical speech texts and add them to the terminology library. Using the above example, the historical terminology text "Team up to play a copy" can be determined based on the historical event "Team up to play a copy", and the colloquial vocabulary "Play a dragon" can be determined based on the historical speech data "Team up to play a copy", and the standard vocabulary "Play a copy" can be determined based on the historical terminology text "Team up to play a copy", and then the terminology library can be updated based on the colloquial vocabulary "Play a dragon" and the standard vocabulary "Play a copy".

[0038] Correspondingly, pre-training the event prediction model can obtain each historical sample speech data of each historical sample game user, and determine the sample speech text corresponding to each historical sample speech data. For each historical sample speech data of each historical sample game user, according to each historical operation behavior of the historical sample game user within a preset time period after inputting the historical sample speech data, determine the historical annotation event as the annotation. The sample speech text is input into the event prediction model as a sample to obtain the predicted event. The event prediction model is trained with the goal of minimizing the difference between the predicted event and the historical annotation event. In this way, the event prediction model can learn and predict what kind of historical event the historical speech data corresponds to based on the historical speech data of the game user, so as to further determine and verify the standardized expression corresponding to the historical speech data, which is used to extract and update the terminology library.

[0039] Specifically, to train the event prediction model, a large amount of speech data of historical game users can be collected from the game system. These data may include text chat, voice-to-text, etc., as historical sample speech data, and then the sample speech text corresponding to each historical sample speech data is determined. At the same time, the game operation behaviors of these historical sample game users within a period of time after speaking, such as moving, attacking, purchasing equipment, etc., are recorded. Based on these historical operation behaviors, historical annotation events are determined as model annotations during training. These historical annotation events can be any user behavior patterns with analytical value, such as "prepare to attack", "seek help", "express dissatisfaction", etc.

[0040] Furthermore, the collected historical sample speech texts are used as samples and input into the event prediction model to predict the events corresponding to the historical sample speech texts to obtain predicted events. Then, based on the predicted events and the actual historical annotated events, the model parameters are adjusted to minimize the difference between the two. This process usually uses a loss function to quantify the difference and updates the model parameters through optimization algorithms such as gradient descent. After training, the event prediction model can predict possible game events based on new game user speech data.

[0041] S104: Input the text to be translated into a pre-trained translation model, so that the text to be translated is converted into translation texts corresponding to the target languages ​​through the translation model, and then output.

[0042] Furthermore, the text to be translated will be provided as input data to a pre-trained translation model. The translation text can be in any language. When the text to be translated is input into the translation model, the model will convert the source language text, i.e. the text to be translated, into the target language text according to the mapping relationship it has learned, including word segmentation, encoding (converting the text into a numerical form understandable to the model), decoding (generating the text in the target language according to the encoding result), etc. For models that support multilingual translation, the translation can distinguish different target languages ​​and adjust the parameters or paths in the translation process accordingly to ensure that the translation text is generated in accordance with the grammar and expression habits of the target language.

[0043] Next, after converting the text to be translated into the translation text corresponding to each target language, the pre-trained translation model will output the translated text. If there are multiple target languages, a nearly natural and fluent translation text corresponding to each target language will be output separately. It should be noted that since the text to be translated contains some standard vocabulary such as professional terms in the game, the pre-trained translation model needs to learn how to accurately translate the standard vocabulary in the predetermined term library during the training phase, so that the model can accurately translate the terms in the text to be translated during the use phase.

[0044] Specifically, the translation model is pre-trained through a large amount of bilingual or multilingual comparison data, which usually contains a large number of source language sentences and their corresponding target language translations. During the training process, the translation model will learn how to map the text of the source language to the text of the target language by identifying features such as sentence structure, grammatical rules, and vocabulary selection. Among them, based on existing pre-trained models with better performance, such as RNN, LSTM, Transformer, etc., fine-tuning or prompt fine-tuning can be performed through samples containing a large number of standard vocabularies to obtain a translation model suitable for the game field. It is also possible to directly retrain a translation model through samples in the game field. The specific training process of the translation model is not limited in the embodiments of this specification.

[0045] S106: Returning the translated text to each target user corresponding to each target language in the game.

[0046] Furthermore, after the translation model outputs the translation texts corresponding to the target languages, one or more target users corresponding to the target language are determined for the translation texts corresponding to the target languages ​​according to the game preference settings of other game users in the game. Then, the translation texts corresponding to the target languages ​​are returned to these target users through various methods such as the chat system, message push, and bulletin board in the game. The user interface survey can also be used to ensure that the display of the translation results is consistent with the style of the game environment and does not interfere with the game experience. The specific method of returning the translation texts to the target users corresponding to the target languages ​​in the game is not limited in the embodiments of this specification.

[0047] For example, suppose that in an online strategy game, a player from China and a player from Spain are playing on the same server. They need to collaborate to complete a task, but due to language barriers, direct communication is difficult. In response to the instructions issued by the Chinese player in Chinese, it is determined that the text to be translated is "We need to build more arrow towers to defend against the enemy first." Based on the pre-trained translation model, the translated text in Spanish is output: "Necesitamos construir más torres de flechas para defendernos de losenemigos." Then, the translated text is returned to the player whose language preference is set to Spanish. After seeing the translation, the Spanish player can understand the need for collaboration and respond accordingly.

[0048] Of course, the translation quality and user experience can be continuously optimized based on the target users’ feedback on the translated text. Such feedback includes direct user comments, satisfaction surveys, or automatically analyzed user behavior data.

[0049] based on Figure 1 A game-based multilingual real-time translation method is provided, which can capture the speech data of the player in real time in the game and determine the first text, and then translate the first text into multiple languages ​​in real time and efficiently. In addition, based on the terminology library and the pre-trained translation model, it can ensure that while translating efficiently, the specific terms, slang and other vocabulary in the game are accurately captured and translated, thereby improving the accuracy of the translation.

[0050] Furthermore, the first text is standardized according to a predetermined terminology library, which can be specifically as follows: according to the predetermined terminology library, each colloquial word in the first text is determined, each standard word corresponding to each colloquial word is determined from the predetermined terminology library, each colloquial word in the first text is replaced with the corresponding standard word, and the text to be translated is determined.

[0051] Specifically, after obtaining the first text, colloquial words that may not meet standards or specifications but are widely used in the field can be identified in the first text to be processed. These words may vary depending on the region, player group or the characteristics of the game itself. Then, for each identified colloquial word, its corresponding standard word is searched in a predetermined term library by string matching, synonym search or context-based reasoning, and all colloquial words in the first text are replaced with their respective standard words found in the term library to obtain the text to be translated.

[0052] For example, suppose the server is processing the chat records of game users, i.e. players, in an online role-playing game (MMORPG), which contains some colloquial words. The original text is: "Newbie, please help me, how can I quickly reach level 60? Also, why can't I use the skill 'Firewall'?" In the terminology, the colloquial word "newbie" corresponds to the standard word "newbie", the colloquial word "how" corresponds to the standard word "how", and the skill name "Firewall" is already a standard word and does not need to be replaced. The standardized text to be translated is: "Newbie, please help me, how can I quickly reach level 60? Also, why can't I use the skill 'Firewall'?". By replacing the colloquial words according to the terminology library in the game field, a more standardized and easy-to-understand text to be translated is obtained, which improves the accuracy and consistency of the translation results.

[0053] Furthermore, in addition to simply replacing the colloquial vocabulary in the first text according to a predetermined term library, standardizing the first text according to the predetermined term library to determine the text to be translated, the first text can also be input into a pre-trained term conversion model to standardize the first text through the term conversion model and output the text to be translated, wherein the term conversion model is pre-trained and determined according to the predetermined term library.

[0054] Specifically, the pre-trained term conversion model is a model built based on machine learning or deep learning technology, which can learn and understand the conversion rules between words, especially the conversion between different languages ​​or different expression styles. Based on this, the first text can be used as the input of the model, so that the term conversion model determines the characteristics of the first text according to the input first text, and automatically identifies the colloquial words in the text according to the learned characteristics of the first text, and converts these words into corresponding standard words according to the learned conversion rules, so that the output text to be translated is more standardized, which is convenient for subsequent accurate translation.

[0055] Compared with simple word replacement, the term conversion model can handle more complex word conversion relationships, including polysemy, context dependency, etc. Moreover, through large-scale data training, the model can learn more accurate conversion rules and improve the accuracy of conversion. At the same time, the term conversion model is also scalable. With the update of the term library, the term conversion model can be continuously trained and optimized to adapt to the ever-changing terms and expressions, and output more accurate and standard texts to be translated.

[0056] Accordingly, in order to obtain the term conversion model, the term conversion model is trained in advance, which can be specifically as follows: a first sample text containing various colloquial words is determined, and each colloquial word in the first sample text is replaced with a corresponding standard word according to the predetermined term library, and a first annotation text is determined as an annotation. The first sample text is used as a sample and input into the term conversion model, and a standardized text corresponding to the first sample text is output. The term conversion model is trained with the goal of minimizing the difference between the first annotation text and the standardized text.

[0057] Specifically, a set of texts containing colloquial vocabulary can be collected or generated as training samples, namely, first sample texts. These texts can come from various sources, such as social media posts in the gaming field, online forum discussions, game chat records, etc. Next, using a pre-established terminology library, each colloquial vocabulary in the first sample text is replaced with a corresponding standard vocabulary. The replacement process is to replace each colloquial vocabulary in the text with its standard vocabulary defined in the terminology library, thereby generating an annotated text, namely, the first annotated text.

[0058] Furthermore, the original first sample text that has not been replaced is used as input and sent to a term conversion model, which can be a rule-based model, a statistical model (such as a hidden Markov model, a conditional random field, etc.) or a deep learning model (such as a neural network), so that the term conversion model automatically converts the colloquial vocabulary in the input text into standard vocabulary according to the characteristics of the input text, and outputs the required standardized text to be translated. In order to evaluate the performance of the model and optimize it, it is necessary to compare the standardized text output by the model with the annotated text obtained by replacing the term library, that is, the first annotated text, so as to minimize the difference between the two texts. This difference can be evaluated by a variety of metrics, such as accuracy, recall, F1 score, edit distance, etc.

[0059] Based on the above comparison results, an appropriate optimization algorithm, such as gradient descent, is used to update the parameters of the term conversion model to reduce the difference between the annotated text and the output text. This process is iterative, that is, the term conversion model processes the training samples multiple times and gradually adjusts its internal parameters according to each evaluation result until the predetermined performance standard or the upper limit of the training round is reached. During the training process, a portion of independent sample texts is usually reserved as a validation set and a test set to evaluate the performance of the model on unseen data, to prevent overfitting, and to ensure that the model has generalization ability.

[0060] Of course, you can also use pre-trained models, such as BERT, GPT, etc. These models are trained on a large amount of text data and can learn rich language knowledge and semantic information. On the basis of these pre-trained models, the pre-trained data is fine-tuned according to the first sample text and the first annotated text to obtain a term conversion model, which can fully capture and understand the contextual information of sentences, paragraphs and even the entire document, so as to better standardize the input first text during use to obtain the text to be translated. The specific training method of the term conversion model is not limited in the embodiments of this specification.

[0061] Furthermore, in addition to simply standardizing the language by replacing words in the known terminology library, the pre-trained term conversion model can also understand and analyze the semantics of the context, so as to more intelligently handle new words or colloquial expressions that are not in the terminology library. At the same time, a user feedback mechanism can be introduced to allow the term conversion model to continuously learn and improve in actual applications, and the term conversion model can be iterated and optimized through the evaluation and correction of the replacement results of the term conversion model by game users. By continuously optimizing the model parameters and structure, and updating the terminology library, it can be ensured that the term conversion model always maintains high performance when processing new words and colloquial expressions.

[0062] Furthermore, after determining the text to be translated, before inputting the text to be translated into the pre-trained translation model, the text to be translated can also be returned to the game user so that the game user can judge whether the semantics of the text to be translated is accurate. If it is inaccurate, the historical speech data of the game user in the game is obtained, and the first text and the historical speech data are input into the pre-trained terminology conversion model, so that the terminology conversion model re-outputs the text according to the first text under the prompt of the historical speech data as a new text to be translated.

[0063] Specifically, after determining the text to be translated through the term conversion model, the text to be translated can be returned to the game user who inputs the speech data. When the game user finds that the text to be translated automatically generated by the term conversion model fails to accurately convey the semantics of the original sentence, the game user can make a judgment or supplement. In response to the feedback of the game user, the server can collect the historical speech data of the game user in the game game, which includes chat text or language information with other game users.

[0064] Then, these historical speech data are input into the term conversion model together with the original first text, that is, the text that needs to be converted input by the user, so that the term conversion model can use the context information, idioms and semantic associations in these historical data to perform more refined analysis and conversion, and generate new text to be translated that is closer to the user's actual intention and expression style. In addition, the re-output text to be translated can be returned to the game user so that the game user can make another judgment. If the game user believes that the re-returned text to be translated is correct, the correct text to be translated is input into the pre-trained translation model for subsequent translation, and the term library is updated immediately.

[0065] Furthermore, the translation model is trained in advance, which may be specifically as follows: determining a second sample text containing each standard vocabulary, and determining a correct translation text corresponding to each sample language and having the same semantics as the second sample text according to each sample language that is predetermined to be translated into the second sample text, as the second annotation. The second sample text is input into the translation model as a sample, so that the second sample text is translated by the translation model, and the training translation text of each sample language corresponding to the second sample text is output, and the translation model is trained with the goal of minimizing the difference between the training translation text and the second annotation.

[0066] Specifically, a second sample text containing each standard vocabulary in a predetermined term library can be determined first. Since the second sample text contains each standard vocabulary in the game field, such as the name of the character in the game, the name of the skill, etc., the trained translation model can recognize each standard vocabulary in the text to be translated and accurately translate it. For each second sample text, it is necessary to determine the correct translation text translated into other sample languages ​​(such as English, French, German, etc.). These translation texts are generated by professional translators or verified automatic translation systems to ensure that the semantics are the same as or as close as possible to the original text. These correct translation texts are used as annotations of the translation model to evaluate the performance of the translation model during the training process.

[0067] Furthermore, a predetermined second sample text can be input into the translation model, and features of the second sample text can be extracted from the translation model, and training translation texts of each sample language corresponding to the second sample text can be generated based on the features. Then, based on the annotations and output results corresponding to the second sample text, i.e., the second sample text and the training translation text, the loss function of the translation model is obtained, and the gradient of the loss function is calculated. Finally, according to the descending gradient of the loss function, the parameters are adjusted in the opposite direction. This is an optimization iteration process of the translation model during the training process, and multiple iterations can be performed to obtain the final trained translation model.

[0068] It should be noted that the above embodiment is a supervised learning training process of the translation model. A translation model that does not require sample annotation may also be selected for training. The specific translation model training process is not limited in the embodiments of this specification.

[0069] Further, after determining the translation text corresponding to the speech data, the translation text is returned to each target user corresponding to each target language in the game, which can be specifically as follows: By calling a speech generation plug-in, the translation texts corresponding to the target languages ​​are converted into corresponding translation speech. For the translation texts corresponding to the target languages, the target users corresponding to the target languages ​​are determined, and the translation texts and translation speech corresponding to the target languages ​​are returned to the target users corresponding to the target languages.

[0070] Specifically, in order to convey the translated text to the target users in a more intuitive and easy-to-understand manner, the system can convert the translated text into corresponding voices by calling a voice generation plug-in, such as a text-to-speech plug-in (TTS). This plug-in can receive the translated text and output the corresponding voice file or real-time voice stream, that is, the translated text corresponding to each target language is converted into the corresponding translated voice.

[0071] Furthermore, it is possible to determine which game users belong to a specific language group, that is, for each target language corresponding to the translation text, determine the target language corresponding to each target user, and send the translation text corresponding to each target language and each translated voice to all target users corresponding to the language, so that other game users in the game can receive the translation results and perform game operations or communication based on the information. At the same time, the front-end display module can be optimized to seamlessly display text translations, and provide voice playback controls so that game users can choose whether to play the translated voice.

[0072] The above is a game-based multi-language real-time translation method provided in the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0073] Figure 2 A schematic diagram of a multi-language real-time translation device based on a game is provided in an embodiment of this specification, and the device includes: The response module 200 is used to respond to the speech data input by the game user, determine the first text corresponding to the speech data, and each target language into which the first text needs to be translated; A determination module 202 is used to standardize the first text according to a predetermined term library to determine a text to be translated; The translation module 204 is used to input the text to be translated into a pre-trained translation model, so as to convert the text to be translated into translation texts corresponding to the target languages ​​through the translation model, and output the translation texts; The return module 206 is used to return the translated text to each target user corresponding to each target language in the game.

[0074] Optionally, the determination module 202 is specifically used to determine each colloquial vocabulary in the first text according to a predetermined term library; determine each standard vocabulary corresponding to each colloquial vocabulary from the predetermined term library; replace each colloquial vocabulary in the first text with the corresponding standard vocabulary to determine the text to be translated.

[0075] Optionally, the determination module 202 is specifically used to input the first text into a pre-trained term conversion model to standardize the first text through the term conversion model and output a text to be translated; wherein the term conversion model is pre-trained and determined according to the predetermined term library.

[0076] Optionally, the determination module 202 is specifically used to determine a first sample text containing various colloquial words; according to the predetermined terminology library, replace each colloquial word in the first sample text with a corresponding standard word, and determine a first annotated text as an annotation; input the first sample text as a sample into the terminology conversion model, and output a standardized text corresponding to the first sample text; and train the terminology conversion model with the goal of minimizing the difference between the first annotated text and the standardized text.

[0077] Optionally, the determination module 202 is also used to return the text to be translated to the game user so that the game user can judge whether the semantics of the text to be translated are accurate; if inaccurate, obtain the historical speech data of the game user in the game; input the first text and the historical speech data into the pre-trained term conversion model, so that the term conversion model re-outputs the text according to the first text under the prompt of the historical speech data as a new text to be translated.

[0078] Optionally, the translation module 204 is specifically used to determine a second sample text containing various standard vocabulary; according to the predetermined sample languages ​​into which the second sample text needs to be translated, determine the correct translation text corresponding to each sample language and having the same semantics as the second sample text as the second annotation; input the second sample text as a sample into the translation model, so as to translate the second sample text through the translation model, and output training translation texts of each sample language corresponding to the second sample text; and train the translation model with the goal of minimizing the difference between the training translation text and the second annotation.

[0079] Optionally, the return module 206 is specifically used to convert the translation texts corresponding to the target languages ​​into corresponding translation voices by calling a voice generation plug-in; determine the target users corresponding to the target language for the translation texts corresponding to the target languages; and return the translation texts and translation voices corresponding to the target languages ​​to the target users corresponding to the target languages.

[0080] This specification also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, it can be used to perform the above Figure 1 A game-based multilingual real-time translation method is provided.

[0081] based on Figure 1 The multi-language real-time translation method based on the game is shown, and the embodiment of this specification also provides Figure 3 The structural diagram of the electronic device shown in FIG. Figure 3 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The game-based multilingual real-time translation method.

[0082] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0083] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0084] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0085] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0086] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0087] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0089] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0091] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0092] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0093] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0094] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0095] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0097] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0098] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A game-based multilingual real-time translation method, characterized in that: The method comprises: In response to speech data input by a game user, determining a first text corresponding to the speech data and each target language into which the first text needs to be translated; Standardizing the first text according to a predetermined terminology database to determine a text to be translated; Inputting the text to be translated into a pre-trained translation model, so as to convert the text to be translated into translation texts corresponding to the target languages ​​through the translation model, and outputting the translation texts; The translated text is returned to each target user corresponding to each target language in the game.

2. The method according to claim 1, characterized in that The first text is standardized according to a predetermined term base, specifically including: Determining each colloquial vocabulary in the first text according to a predetermined term library; Determining each standard vocabulary corresponding to each spoken vocabulary from the predetermined term library; Each colloquial word in the first text is replaced with the corresponding standard word to determine the text to be translated.

3. The method according to claim 1, characterized in that The first text is standardized according to a predetermined terminology database to determine the text to be translated, specifically including: Inputting the first text into a pre-trained term conversion model to standardize the first text through the term conversion model and output a text to be translated; The term conversion model is pre-trained and determined according to the predetermined term library.

4. The method according to claim 3, characterized in that Pre-training the term conversion model includes: determining a first sample text containing each spoken vocabulary; According to the predetermined term library, each colloquial word in the first sample text is replaced with a corresponding standard word, and a first annotated text is determined as an annotation; Input the first sample text as a sample into the term conversion model, and output a standardized text corresponding to the first sample text; The term conversion model is trained with the goal of minimizing the difference between the first annotated text and the standardized text.

5. The method according to claim 3, characterized in that After determining the text to be translated, before inputting the text to be translated into the pre-trained translation model, the method further includes: Returning the text to be translated to the game user, so that the game user can judge whether the semantics of the text to be translated is accurate; If it is inaccurate, obtain the historical speech data of the game user in the game; The first text and the historical speech data are input into the pre-trained term conversion model, so that the term conversion model re-outputs text according to the first text under the prompt of the historical speech data as a new text to be translated.

6. The method according to claim 1, characterized in that Pre-training the translation model specifically includes: determining a second sample text containing each standard vocabulary; According to the predetermined sample languages ​​into which the second sample text needs to be translated, determining correct translation texts corresponding to the sample languages ​​and having the same semantics as the second sample text as the second annotation; The second sample text is used as a sample and input into the translation model, so as to translate the second sample text through the translation model and output training translation texts of each sample language corresponding to the second sample text; The translation model is trained with the goal of minimizing the difference between the training translation text and the second annotation.

7. The method according to claim 1, characterized in that Returning the translated text to each target user corresponding to each target language in the game, specifically including: By calling the speech generation plug-in, the translation texts corresponding to the target languages ​​are converted into corresponding translation speech; For the translation text corresponding to each target language, determine the target language corresponding to each target user; The translated text and translated speech corresponding to the target language are returned to each target user corresponding to the target language.

8. A multi-language real-time translation device based on a game, characterized in that: The device comprises: A response module, configured to respond to speech data input by a game user, determine a first text corresponding to the speech data, and each target language into which the first text needs to be translated; A determination module, used to standardize the first text according to a predetermined term library to determine a text to be translated; A translation module, used for inputting the text to be translated into a pre-trained translation model, so as to convert the text to be translated into translation texts corresponding to the target languages ​​through the translation model, and output the translation texts; The return module is used to return the translated text to each target user corresponding to each target language in the game.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images