A game-based multilingual real-time translation method and device

Abstract

The specification provides a game-based multilingual real-time translation method and device, in response to speech data input by a game user, determining a first text corresponding to the speech data, and each target language required to be translated by the first text, standardizing the first text according to a predetermined term library, determining a text to be translated, inputting the text to be translated into a pre-trained translation model, converting the text to be translated into a translation text corresponding to each target language through the translation model, and outputting, returning the translation text to each target user corresponding to each target language in the game. Through this method, accurate multilingual real-time translation in the game can be performed.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer, and particularly relates to a multi-lingual real-time translation method and device based on games. BACKGROUND

[0002] At present, with the rapid development of Internet technology, online games are widely popular in the world. However, although game players from various countries can interact and communicate in the game, language barriers become a major problem restricting in-depth interaction and cooperation among players. Especially in the game types such as Multiplayer Online Battle Arena (MOBA), Massively Multiplayer Online Role-Playing Game (MMORPG) and the like, which require high coordination and instant communication, the negative impact of language differences on game experience is particularly significant.

[0003] Although there are many translation tools on the market at present, which to some extent alleviate the problem of language communication, these translation tools are difficult to accurately capture and translate the specific terms, slang and constantly changing vocabulary in the game, resulting in that the translation results often deviate from the actual context, affecting the understanding and communication efficiency of players.

[0004] Therefore, how to accurately perform real-time translation in the game becomes a problem to be solved. SUMMARY

[0005] Embodiments of the present specification provide a multi-lingual real-time translation method and device based on games to partially solve the problems existing in the prior art.

[0006] Embodiments of the present specification adopt the following technical solutions:

[0007] The present specification provides a multi-lingual real-time translation method based on games, the method comprising:

[0008] In response to speech data input by a game user, determining a first text corresponding to the speech data, and each target language required to be translated by the first text;

[0009] According to a predetermined term library, standardizing the first text to determine a text to be translated;

[0010] Inputting the text to be translated into a pre-trained translation model, to convert the text to be translated into a translation text corresponding to each target language through the translation model, and outputting;

[0011] Returning the translation text to each target user corresponding to each target language in the game.

[0012] Optionally, according to the predetermined term library, the standardization of the first text specifically comprises:

[0013] determine each colloquial vocabulary in the first text according to a predetermined term library;

[0014] determine each standard vocabulary corresponding to each colloquial vocabulary from the predetermined term library;

[0015] replace each colloquial vocabulary in the first text with the corresponding standard vocabulary to determine a text to be translated.

[0016] Optionally, the first text is standardized according to the predetermined term library to determine the text to be translated, specifically including:

[0017] input the first text into a pre-trained term conversion model to standardize the first text by the term conversion model and output the text to be translated;

[0018] wherein the term conversion model is pre-trained and determined according to the predetermined term library.

[0019] Optionally, the term conversion model is pre-trained, specifically including:

[0020] determine a first sample text containing each colloquial vocabulary;

[0021] replace each colloquial vocabulary in the first sample text with the corresponding standard vocabulary according to the predetermined term library to determine a first labeled text as a label;

[0022] input the first sample text as a sample into the term conversion model to output a standardized text corresponding to the first sample text;

[0023] train the term conversion model with the goal of minimizing the difference between the first labeled text and the standardized text.

[0024] Optionally, after determining the text to be translated, the method further includes:

[0025] return the text to be translated to the game user to determine whether the semantics of the text to be translated is accurate by the game user;

[0026] if not, obtain historical speech data of the game user in the game;

[0027] input the first text and the historical speech data into the pre-trained term conversion model to make the term conversion model output a new text to be translated according to the first text under the prompt of the historical speech data.

[0028] Optionally, the translation model is pre-trained, specifically including:

[0029] Identify a second sample text containing each standard vocabulary;

[0030] Based on the pre-determined sample languages ​​into which the second sample text needs to be translated, determine the correct translation text with the same semantics as the second sample text for each sample language, and use it as the second annotation;

[0031] The second sample text is used as a sample and input into the translation model to translate the second sample text and output the training translation text for each sample language corresponding to the second sample text.

[0032] The translation model is trained with the goal of minimizing the difference between the training translated text and the second annotation.

[0033] Optionally, the translated text is returned to the target users corresponding to each target language within the game, specifically including:

[0034] By calling the speech generation plugin, the translated text corresponding to each target language is converted into the corresponding translated speech;

[0035] For each target language, the corresponding target user is determined based on the translated text.

[0036] The translated text and audio corresponding to the target language are returned to the respective target users for that target language.

[0037] This specification provides a game-based multilingual real-time translation device, the device comprising:

[0038] The response module is used to respond to the speech data input by the game user, determine the first text corresponding to the speech data, and the target languages ​​into which the first text needs to be translated.

[0039] The determination module is used to standardize the first text according to a pre-determined terminology database to determine the text to be translated;

[0040] The translation module is used to input the text to be translated into a pre-trained translation model, so as to convert the text to be translated into the corresponding translation text for each target language through the translation model, and output it.

[0041] The return module is used to return the translated text to the target users corresponding to each target language in the game.

[0042] The computer readable storage medium provided by the specification stores a computer program, and the computer program is executed by a processor to implement the game-based multilingual real-time translation method.

[0043] The electronic device provided by the specification includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the game-based multilingual real-time translation method when executing the program.

[0044] The above at least one technical solution adopted by the embodiments of the specification can achieve the following beneficial effects:

[0045] The embodiments of the specification disclose a game-based multilingual real-time translation method, which determines first text corresponding to speech data input by a game user and each target language to which the first text needs to be translated, standardizes the first text according to a predetermined term library, determines text to be translated, inputs the text to be translated into a pre-trained translation model, converts the text to be translated into translation text corresponding to the each target language through the translation model, and outputs the translation text to each target user corresponding to the each target language in the game. Through the method, after capturing the speech data of the player and determining the first text in the game, the first text can be efficiently translated in multiple languages in real time, and based on the term library and the pre-trained translation model, the accuracy of the translation can be ensured while efficiently translating, and the accuracy of the translation can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings described herein are used to provide further understanding of the specification, form a part of the specification, and the illustrative embodiments of the specification and the description thereof are used to explain the specification, and do not constitute an improper limitation on the specification. In the drawings:

[0047] Figure 1 A flowchart of a game-based multilingual real-time translation method provided by the embodiments of the specification is shown in the figure;

[0048] Figure 2 A structural diagram of a game-based multilingual real-time translation device provided by the embodiments of the specification is shown in the figure;

[0049] Figure 3 A structural diagram of an electronic device provided by the embodiments of the specification is shown in the figure. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present specification clearer, the technical solutions of the present specification will be described clearly and completely in the following with specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present specification, rather than all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present specification.

[0051] The technical solutions provided by the embodiments of the present specification will be described in detail below with reference to the drawings.

[0052] Figure 1 The flowchart of the method for real-time translation of multiple languages based on games in the present specification can specifically include the following steps:

[0053] S100: in response to the speech data input by the game user, determining the first text corresponding to the speech data, and the target languages required to be translated by the first text.

[0054] Specifically, in the embodiments of the present specification, when the game user plays the multiplayer online game with other game users from different countries, the game user may need to communicate with other game users through voice or text, so that other game users can operate or give feedback to the game user according to the speech data of the game user, realize the interaction and communication between each game user in the game, that is, the player, and improve the game experience.

[0055] Further, since each game user in the same online game may come from different countries, the language used may also be different. Therefore, the server in the background of the multiplayer online game can record the player information when each game user logs in the game through the client, and determine the language preference set by each game player in the client. If the server detects that all game users in the current game environment use the same language, that is, the language preferences set by all game users are consistent, the game server will allow all game users to communicate directly. Specifically, the speech data input by the game user can be directly received, and after it is determined that the speech data meets the pre-set risk control rules, the speech data is returned to all other game users.

[0056] Further, if the server of the online game detects that the language preferences set by each game user in the current game environment are different, it will respond to the speech data input by the game user to determine the first text corresponding to the speech data, which is used to subsequently translate the text of each target language with the same semantic as the speech data and return it to the corresponding each other game player.

[0057] The speech data mentioned herein includes voice information or text information input by a game user. If the speech data is voice information, the text information corresponding to the voice information is determined as the first text through a pre-trained voice translation model or voice translation plug-in. If the speech data is text information, the text information can be directly used as the first text corresponding to the speech data. Of course, the text information input by the user can be preprocessed, including removing irrelevant symbols, converting case, word segmentation processing, etc. to obtain the first text. The specific determination manner of the first text is not limited in the embodiments of the present specification.

[0058] Further, while determining the first text corresponding to the speech data, a language recognition module in natural language processing technology can be used to automatically detect the language of the speech data input by the user, and based on the nationality information or language preference set by each other game user in the game session, determine the target languages into which the speech data needs to be translated.

[0059] For example, in a certain multi-player online game session, three game users A, B and C are online at the same time. The language preference set by game user A is Chinese, the language preference set by game user B is English, and the language preference set by game user C is Japanese. After game user A inputs speech data, the target languages can be determined as English and Japanese according to the language preference settings of game user B and game user C. Subsequently, a translation text with the same semantics as the speech data and in English can be returned to game user B, and a translation text with the same semantics as the speech data and in Japanese can be returned to game user C.

[0060] It should be noted that the terminal device, such as a notebook computer or a tablet computer, can be used to determine the first text and the target languages in response to the speech data input by the game user. Of course, the server connected to the multi-player online game can also be used. For ease of description, the server is taken as an example to describe the game-based multi-language real-time translation method provided in the present specification.

[0061] S102: standardizing the first text according to a pre-determined term library to determine a text to be translated.

[0062] Further, after determining the first text, since there can be some colloquial expressions or slang in the text input by the game user, which can affect the subsequent translation, the first text can be standardized according to a predetermined term library. Then, the text after standardization, i.e. the text after term replacement and format unification, is called the text to be translated, which is the basis for subsequent translation work. It ensures the consistency and accuracy of the terms in the translation process and reduces the possibility of translation errors and misunderstandings.

[0063] Specifically, standardizing the first text can adjust or replace the words, phrases or sentences in the first text according to the standards in the predetermined term library, identify the colloquial expressions in the first text, find the corresponding standard words in the term library, and then replace the colloquial expressions in the first text with these standardized in-game professional names to improve the accuracy of subsequent translation. For example, the first text is "I want to play crocodile", and "crocodile" is a colloquial expression word, which can be replaced by the corresponding in-game professional name "Raketon". Then, the standardized text to be translated is "I want to play Raketon" or "I hope to choose the hero character Raketon". Of course, standardization can also unify the text format, punctuation, date and time format, etc. to further ensure the accuracy and consistency of translation. The specific standardization steps are not limited by the embodiments of the present specification.

[0064] Among them, the term library mentioned here stores a large database of game terms, professional names, colloquial expressions and slang, which is specially designed to assist the translation system to improve the accuracy and relevance of translation. Specifically, the term library can be created in advance, and the scope of the terms and the target user group can be defined, and it can be defined which types of game terms need to be included, such as character names, skill names, game mechanisms, equipment props, etc., and it can be determined which types of players or developers the term library mainly serves. Then, the terms and corresponding colloquial expressions can be manually collected through game forums, communities, official documents, in-game chat logs, etc. Pre-existing APIs can be developed or used to obtain data from game official websites, social media, forums, etc. using web crawler technology to automatically crawl relevant information. Of course, players can also submit new terms and corresponding colloquial words through the website or application, and an audit mechanism can be set up to ensure the accuracy and applicability of the terms.

[0065] Then, after collecting the term-related information, the collected data is cleaned, including removing noise, standardizing format, labeling tags, etc., and the relevant term information, including slang and other colloquial vocabulary, and corresponding terms or in-game professional names, etc. is stored in the database. Of course, machine learning classification algorithms (such as decision trees, random forests, support vector machines, etc.) can also be used to classify terms, and the basis for classification can be semantic features, contextual features, or user-defined labels of the terms. By training the classification model, new terms can be automatically assigned to the corresponding categories. For unannotated or difficult-to-classify terms, clustering algorithms (such as K-means, hierarchical clustering, etc.) can be used to group them, and clustering algorithms group based on the similarity or distance between terms, which helps to discover potential term groups or topics.

[0066] Further, the term library can be automatically updated, including adding new terms, updating term definitions, adjusting term classification, etc. The automatic update process can be set to execute periodically to ensure that the term library is always up-to-date. User feedback on terms submitted through the interface, such as pointing out errors, suggesting improvements, etc., can also be used for iterative optimization to continuously improve algorithm and model performance.

[0067] Further, the term library can be pre-determined, and specifically, the historical speech data of each historical game user can be obtained at a pre-set period, and the historical speech text corresponding to the historical speech data is determined, the historical speech text is input into a pre-trained event prediction model to obtain a historical event corresponding to the historical speech text, if the historical event corresponds to at least one historical operation behavior of the historical game user within a pre-set time period after inputting the historical speech data, then according to the historical event, the historical term text corresponding to the historical speech text is determined, and each standard vocabulary corresponding to each colloquial vocabulary in the historical speech text is determined and added to the term library according to the historical speech text and the historical term text. Wherein, the standard vocabulary is derived from the historical term text. It should be noted that the specific method of pre-determining the term library is not limited by the embodiments of the present disclosure.

[0068] Specifically, historical speech data of all players can be collected from the game log system at a frequency of every month or any pre-set period. These data include but are not limited to chat messages, team voice-to-text records, etc. The collected historical speech data is pre-processed, such as removing invalid characters, standardizing punctuation, text segmentation, etc., to determine the historical speech text for subsequent analysis. Then, a pre-trained event prediction model can be used to analyze the processed historical speech text to predict the historical operation behavior of the historical game user after inputting the historical speech data, such as "team up to brush the copy", "sell equipment", etc., as a historical event.

[0069] If the historical event corresponds to at least one historical operation behavior of the historical game user within a preset time period after inputting the historical speech data, i.e., comparing the model-predicted historical event with the actual operation behavior of the player within a preset time period, such as 5 minutes, 10 minutes, etc., after the speech, if the predicted event is highly related to the actual behavior of the player, it is considered that the event prediction is effective. For example, according to the historical speech data "team up to fight the dragon", the predicted historical event is "team up to brush the copy", and the historical game user and other game users team up to brush the copy within a preset time, which indicates that the prediction is correct.

[0070] Further, according to the predicted historical event, the historical speech text corresponding to the historical term text containing the standard vocabulary is determined, so as to identify each standard vocabulary corresponding to each colloquial vocabulary in the historical speech text and added to the term library. According to the above example, the historical term text "team up to brush the copy" can be determined according to the historical event "team up to brush the copy", and the colloquial vocabulary "fight the dragon" can be determined according to the historical speech data "team up to fight the dragon", the standard vocabulary "brush the copy" can be determined according to the historical term text "team up to brush the copy", and then the term library can be updated according to the colloquial vocabulary "fight the dragon" and the standard vocabulary "brush the copy".

[0071] Correspondingly, the event prediction model is pre-trained, each historical sample speech data of each historical sample game user can be obtained, and sample speech text corresponding to the historical sample speech data can be determined. According to each historical operation behavior of the historical sample game user within a preset time period after inputting the historical sample speech data, the historical labeled event is determined as a label. The sample speech text is input into the event prediction model as a sample, and the predicted event is obtained. The event prediction model is trained to minimize the difference between the predicted event and the historical labeled event. The event prediction model is trained to learn and predict the historical event corresponding to the historical speech data of the game user according to the historical speech data of the game user, so as to further determine and verify the standardized expression corresponding to the historical speech data, which is used for extracting and updating the term library.

[0072] Specifically, the event prediction model is trained by collecting a large amount of historical game user speech data from the game system, which can include text chat, voice-to-text, and the like, as historical sample speech data, and then determining the sample speech text corresponding to each historical sample speech data. At the same time, the game operation behavior of the historical sample game user within a certain period of time after the speech is recorded, such as movement, attack, purchase of equipment, etc. According to these historical operation behaviors, historical labeled events are determined as model labels during training. These historical labeled events can be any user behavior pattern with analytical value, such as "preparing to attack", "seeking help", "expressing dissatisfaction", etc.

[0073] Further, the collected historical sample speech text is input into the event prediction model as a sample to predict the event corresponding to the historical sample speech text to obtain a predicted event, and then the model parameters are adjusted according to the predicted event and the actual historical labeled event to minimize the difference between them. This process usually uses a loss function to quantify the difference and updates the model parameters through gradient descent and other optimization algorithms. After training is completed, the event prediction model can predict possible game events according to new game user speech data.

[0074] S104: input the text to be translated into the pre-trained translation model to convert the text to be translated into the translation text corresponding to each target language through the translation model, and output.

[0075] Further, the text to be translated is provided as input data to the pre-trained translation model, and the translation text can be any language. When the text to be translated is input into the translation model, the model will convert the source language text, i.e. the text to be translated, into the target language text according to the mapping relationship it has learned, including word segmentation, encoding (converting text into numerical form understandable by the model), decoding (generating target language text according to encoding results), etc. For models supporting multi-language translation, the translation can distinguish different target languages and adjust the parameters or paths in the translation process accordingly to ensure that the translation text generated conforms to the grammar and expression habits of the target language.

[0076] Next, after the text to be translated is converted into the translation text corresponding to each target language, the pre-trained translation model outputs the translated text. If the target language is multiple, the corresponding natural and fluent translation text of each target language will be output respectively. It should be noted that since the text to be translated contains some in-game professional terms and other standard vocabularies, the pre-trained translation model needs to learn how to accurately translate the standard vocabularies in the pre-determined term library during the training phase, so that the model can accurately translate the terms in the text to be translated during the use phase.

[0077] Specifically, the translation model is obtained by training a large amount of bilingual or multilingual parallel data in advance, which usually contains a large number of source language sentences and their corresponding target language translations. During the training process, the translation model learns how to map the source language text to the target language text by identifying features such as sentence structure, grammar rules, and vocabulary selection. Based on existing pre-trained models with good performance, such as RNN, LSTM, Transformer, etc., fine-tuning or prompt fine-tuning can be performed on samples containing a large number of standard vocabularies to obtain a translation model suitable for the game field. Alternatively, a translation model can be retrained directly using game field samples. The specific training process of the translation model is not limited by the embodiments of the present specification.

[0078] S106: Return the translated text to each target user corresponding to each target language in the game.

[0079] Further, after the translation model outputs the translated text corresponding to each target language, it determines one or more target users corresponding to each target language according to the game preferences of other game users in the game, and then returns the translated text corresponding to each target language to these target users through various ways such as in-game chat systems, message push, bulletin board, etc. User interface surveys can also be used to ensure that the display of the translation results is consistent with the style of the game environment and does not interfere with the game experience. The specific way of returning the translated text to each target user corresponding to each target language in the game is not limited by the embodiments of the present specification.

[0080] For example, assume that in an online strategy game, a player from China and a player from Spain are playing on the same server. They need to work together to complete a task, but direct communication is difficult due to the language barrier. In response to the instruction issued by the Chinese player in Chinese, the text to be translated is determined to be "We need to build more arrow towers to defend against enemies." Based on the pre-trained translation model, the translated text in Spanish is output: "Necesitamos construir más torres de flechas para defendernos de los enemigos." Then, the translated text is returned to the player whose language preference is set to Spanish. After seeing the translation, the Spanish player can understand the need for collaboration and respond accordingly.

[0081] Of course, subsequent optimization of translation quality and user experience can be based on feedback from target users on translated text, including direct user comments, satisfaction surveys, or automatically analyzed user behavior data, etc.

[0082] based on Figure 1 The provided game-based multilingual real-time translation method can capture the player's speech data in real time and determine the first text in the game, and then efficiently translate the first text in real time in multiple languages. Based on the term library and the pre-trained translation model, it can ensure efficient translation while accurately capturing and translating terms, slang and other vocabulary specific to the game, improving the accuracy of translation.

[0083] Further, according to the pre-determined term library, the first text is standardized, which can be specifically determined according to the pre-determined term library. Each colloquial vocabulary in the first text is determined from the pre-determined term library, each standard vocabulary corresponding to each colloquial vocabulary in the first text is determined, each colloquial vocabulary in the first text is replaced with the corresponding standard vocabulary, and the translated text is determined.

[0084] Specifically, after obtaining the first text, colloquial vocabulary that may not conform to the standard or specification but is widely used in the field can be identified in the first text to be processed. These words may vary by region, player group or game characteristics. Then, for each identified colloquial vocabulary, the corresponding standard vocabulary is found in the pre-determined term library through string matching, synonym searching or context-based reasoning, and all colloquial vocabulary in the first text is replaced with the standard vocabulary found in the term library, and the translated text is obtained.

[0085] For example, assume that the server is processing a game user, i.e. a player, in an online role-playing game (MMORPG), the chat record contains some colloquial vocabulary. The original text is: "Newbie for teaching, how to quickly upgrade to level 60? Also, how to put out that 'firewall skill'?". In the term, the colloquial vocabulary "newbie" corresponds to the standard vocabulary "newbie", the colloquial vocabulary "how" corresponds to the standard vocabulary "how", and the skill name "firewall skill" is a standard vocabulary and does not need to be replaced. The standardized translated text is: "Newbie for teaching, how to quickly upgrade to level 60? Also, how to put out that 'firewall skill'?". By replacing the colloquial vocabulary according to the term library of the game field, a more standardized and easily understood translated text is obtained, improving the accuracy and consistency of the translation result.

[0086] Further, in addition to simply replacing colloquial vocabulary in the first text according to the predetermined terminology library, the first text can be input into a pre-trained terminology conversion model to standardize the first text according to the predetermined terminology library and output the text to be translated, wherein the terminology conversion model is determined according to the predetermined terminology library.

[0087] Specifically, the pre-trained terminology conversion model is a model constructed based on machine learning or deep learning technology, which can learn and understand the conversion rules between words, especially between different languages or different expression styles. Based on this, the first text can be input as a model to make the terminology conversion model determine the features of the first text according to the input first text, and automatically identify colloquial vocabulary in the text according to the learned features of the first text, and convert these words to corresponding standard words according to the learned conversion rules, so that the output text to be translated is more standardized, facilitating subsequent accurate translation.

[0088] Compared with simple word replacement, the terminology conversion model can handle more complex word conversion relationships, including polysemy and context dependence. And through large-scale data training, the model can learn more accurate conversion rules to improve conversion accuracy. At the same time, the terminology conversion model also has scalability, and as the terminology library is updated, the terminology conversion model can be continuously trained and optimized to adapt to changing terminology and expression methods, output more accurate and standardized text to be translated.

[0089] Correspondingly, to obtain the terminology conversion model, the terminology conversion model is pre-trained, which can specifically be determined as follows: determining a first sample text containing each colloquial vocabulary, replacing each colloquial vocabulary in the first sample text with a corresponding standard word according to the predetermined terminology library, determining a first labeled text as a label. Input the first sample text as a sample into the terminology conversion model, output the standardized text corresponding to the first sample text, and minimize the difference between the first labeled text and the standardized text as the goal of the terminology conversion model.

[0090] Specifically, a set of texts containing colloquial words can be collected or generated as training samples, i.e., first sample texts. These texts can come from various sources, such as social media posts, online forum discussions, game chat records, etc. Next, using a pre-established term library, each colloquial word in the first sample text is replaced with the corresponding standard word, and the replacement process is to replace each colloquial word in the text with its standard word defined in the term library, thereby generating a labeled text, i.e., the first labeled text.

[0091] Further, the original first sample text without replacement is input into the term conversion model, which can be a rule-based model, a statistical model (such as Hidden Markov Model, Conditional Random Field, etc.) or a deep learning model (such as neural network), so that the term conversion model automatically converts the colloquial words in the input text into standard words according to the features of the input text, and outputs the required standardized translation text. In order to evaluate the performance of the model and optimize it, the standardized text output by the model needs to be compared with the labeled text obtained by replacing the term library, i.e., the first labeled text, so as to minimize the difference between the two texts. This difference can be evaluated by various metrics, such as accuracy, recall, F1 score, edit distance, etc.

[0092] Based on the above comparison results, the parameters of the term conversion model are updated using appropriate optimization algorithms, such as gradient descent, to reduce the difference between the labeled text and the output text. This process is iterative, i.e., the term conversion model processes the training samples multiple times and gradually adjusts its internal parameters according to the evaluation results each time, until a predetermined performance standard or training round limit is reached. During the training process, a portion of independent sample texts are usually reserved as validation and test sets to evaluate the performance of the model on unseen data, prevent overfitting, and ensure the model has generalization ability.

[0093] Of course, pre-trained models such as BERT, GPT, etc. can also be used. These models have been trained on a large amount of text data and can learn rich language knowledge and semantic information. Based on these pre-trained models, the pre-trained data is fine-tuned according to the first sample text and the first labeled text to obtain the term conversion model, which can fully capture and understand the context information of sentences, paragraphs, or even entire documents, thereby better standardizing the input first text during use to obtain the translation text. The specific training method of the term conversion model is not limited by the embodiments of the present specification.

[0094] Further, the pre-trained term conversion model can understand and analyze the semantics of the context in addition to simply standardizing the language by replacing the vocabulary in the known term library, thereby more intelligently processing new words or colloquial expressions that are not in the term library. At the same time, a user feedback mechanism can be introduced to allow the term conversion model to continuously learn and improve in actual application, and through the evaluation and correction of the replacement results of the term conversion model by the game user, the term conversion model is iterated and optimized. By continuously optimizing the model parameters and structure, as well as updating the term library, it can be ensured that the term conversion model always maintains high performance when processing new words and colloquial expressions.

[0095] Further, after determining the text to be translated, the text to be translated can be returned to the game user before being input to the pre-trained translation model, so as to judge whether the semantics of the text to be translated is accurate through the game user, if not, the historical speech data of the game user in the game is obtained, the first text and the historical speech data are input into the pre-trained term conversion model, so that the term conversion model re-outputs the text according to the first text as a new text to be translated under the prompt of the historical speech data.

[0096] Specifically, after determining the text to be translated by the term conversion model, the text to be translated can be returned to the game user who inputs the speech data, and when the game user finds that the text to be translated automatically generated by the term conversion model fails to accurately convey the semantics of the original sentence, the game user can make a judgment or supplement. The server can collect the historical speech data of the game user in the game session in response to the feedback of the game user, and these data contain the chat text or language information with other game users.

[0097] Then, the historical speech data and the original first text, i.e., the text input by the user and needing to be converted, are input into the term conversion model, so that the term conversion model can utilize the context information, idioms and semantic associations in the historical data to perform more refined analysis and conversion, and generate a new text to be translated that is closer to the actual intention and expression style of the user. In addition, the re-output text to be translated can be returned to the game user again for further judgment. If the game user thinks that the returned text to be translated is correct, the correct text to be translated is input into the pre-trained translation model for subsequent translation, and the term library is updated.

[0098] Further, the translation model is pre-trained, which can specifically include determining a second sample text containing each standard vocabulary, determining, according to each sample language to which the second sample text needs to be translated, a correct translation text corresponding to each sample language and having the same semantics as the second sample text as a second label. The second sample text is input into the translation model, and the translation model is used to translate the second sample text, and a training translation text of each sample language corresponding to the second sample text is output. The training translation text and the second label are minimized to train the translation model.

[0099] Specifically, the second sample text containing each standard vocabulary in the pre-determined term library can be determined first. Since the second sample text contains each standard vocabulary in the game field, such as in-game character names and skill names, the trained translation model can recognize each standard vocabulary in the text to be translated and accurately translate it. For each second sample text, a correct translation text translated into other sample languages (such as English, French, German, etc.) needs to be determined. These translation texts are generated by professional translators or verified automatic translation systems, ensuring that the semantics are the same as or as close as possible to the original text. These correct translation texts are used as labels of the translation model to evaluate the performance of the translation model during the training process.

[0100] Further, the pre-determined second sample text can be input into the translation model, the features of the second sample text can be extracted from the translation model, and the training translation text of each sample language corresponding to the second sample text can be generated according to the features. Then, the loss function of the translation model is obtained according to the label and the output result of the second sample text, i.e., the second sample text and the training translation text, and the gradient of the loss function is calculated. Finally, the parameters are adjusted in the opposite direction according to the descending gradient of the loss function. This is one optimization iteration process of the translation model during the training process, which can be iterated multiple times to obtain the final trained translation model.

[0101] It should be noted that the above embodiment is a supervised learning training process of the translation model, and the translation model without sample labels can also be selected for training. The specific translation model training process is not limited by the embodiments of the present disclosure.

[0102] Further, after determining the translation text corresponding to the speech data, the translation text is returned to each target user corresponding to each target language in the game, which can specifically include

[0103] The translation text corresponding to each target language is converted into corresponding translation speech by calling a voice generation plug-in, for the translation text corresponding to each target language, each target user corresponding to the target language is determined, and the translation text and translation speech corresponding to the target language are returned to each target user corresponding to the target language.

[0104] Specifically, in order to convey the translation text to the target user in a more intuitive and easy-to-understand manner, the system can call a voice generation plug-in, such as a text-to-speech plug-in (Text-To-Speech, TTS), to convert the translation text into corresponding speech. Through this plug-in, the translation text can be received, and the corresponding speech file or real-time speech stream can be output, that is, the translation text corresponding to each target language is converted into corresponding translation speech.

[0105] Further, it can be determined which game users belong to a specific language group, that is, for the translation text corresponding to each target language, each target user corresponding to the target language is determined, and the translation text corresponding to each target language is sent to all target users corresponding to the language, so that other game users in the game can receive the translation result and perform game operation or communication according to the information. At the same time, the front-end display module can be optimized to seamlessly display the text translation and provide a voice playback control to enable game users to select whether to play the translation speech.

[0106] The above is a game-based multilingual real-time translation method provided by the embodiments of the present specification. Based on the same idea, the present specification also provides corresponding devices, storage media and electronic equipment.

[0107] Figure 2 A structure diagram of a game-based multilingual real-time translation device provided by the embodiments of the present specification, the device comprises:

[0108] The response module 200 is configured to determine a first text corresponding to the speech data input by the game user and each target language to which the first text needs to be translated in response to the speech data input by the game user.

[0109] The determination module 202 is configured to standardize the first text according to a predetermined term library to determine a to-be-translated text.

[0110] The translation module 204 is configured to input the to-be-translated text into a pre-trained translation model, so as to convert the to-be-translated text into translation text corresponding to each target language through the translation model and output.

[0111] The return module 206 is configured to return the translation text to each target user corresponding to each target language in the game.

[0112] Optionally, the determining module 202 is specifically configured to determine each colloquial vocabulary in the first text according to a predetermined vocabulary library; determine each standard vocabulary corresponding to the each colloquial vocabulary from the predetermined vocabulary library; replace each colloquial vocabulary in the first text with the corresponding each standard vocabulary to determine the text to be translated.

[0113] Optionally, the determining module 202 is specifically configured to input the first text into a pre-trained vocabulary conversion model to standardize the first text by the vocabulary conversion model and output the text to be translated; wherein the vocabulary conversion model is determined according to the predetermined vocabulary library.

[0114] Optionally, the determining module 202 is specifically configured to determine a first sample text containing each colloquial vocabulary; replace each colloquial vocabulary in the first sample text with a corresponding standard vocabulary according to the predetermined vocabulary library to determine a first labeled text as a label; input the first sample text as a sample into the vocabulary conversion model to output a standardized text corresponding to the first sample text; and train the vocabulary conversion model with the goal of minimizing the difference between the first labeled text and the standardized text.

[0115] Optionally, the determining module 202 is further configured to return the text to be translated to the game user to determine whether the semantics of the text to be translated is accurate by the game user; if not, obtain historical utterance data of the game user in the game; and input the first text and the historical utterance data into the pre-trained vocabulary conversion model to make the vocabulary conversion model output a new text to be translated according to the first text under the prompt of the historical utterance data.

[0116] Optionally, the translation module 204 is specifically configured to determine a second sample text containing each standard vocabulary; determine correct translation texts corresponding to each sample language of the second sample text according to a predetermined vocabulary library, as a second label; input the second sample text as a sample into the translation model to translate the second sample text by the translation model and output training translation texts of each sample language corresponding to the second sample text; and train the translation model with the goal of minimizing the difference between the training translation texts and the second label.

[0117] Optionally, the return module 206 is specifically used to: convert the translated text corresponding to each target language into corresponding translated speech by calling the speech generation plugin; determine the target user corresponding to each target language for the translated text corresponding to each target language; and return both the translated text and the translated speech corresponding to the target language to the target user corresponding to the target language.

[0118] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the above-described actions. Figure 1 Provides a game-based, real-time multilingual translation method.

[0119] based on Figure 1 The game-based multilingual real-time translation method shown in this specification also provides embodiments that... Figure 3 The diagram shows the structure of the electronic device. Figure 3 At the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The aforementioned game-based multilingual real-time translation method.

[0120] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0121] In the 1990s, it was possible to distinguish whether an improvement in a technology was a hardware improvement (e.g., an improvement in the circuit structure of a diode, transistor, switch, etc.) or a software improvement (an improvement in a method flow). However, as technology has advanced, many improvements in method flows today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into a hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it himself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, instead of manually manufacturing an integrated circuit chip, this programming is now mostly implemented using "logic compiler" software, which is similar to the software compiler used when developing a program, and the original code before compilation must also be written in a specific programming language, which is called a hardware description language (HDL), and there are many types of HDL, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that it is very easy to obtain a hardware circuit that implements a logical method flow by simply logically programming the method flow in one of the above-mentioned hardware description languages and programming it into an integrated circuit.

[0122] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by means of logic programming of the method steps. The controller can thus be considered a hardware component, and the means comprised therein for performing the various functions can be considered structures within the hardware component. Alternatively, or even additionally, the means for performing the various functions can be considered both software modules implementing the method and structures within the hardware component.

[0123] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0124] For the sake of description, the above apparatuses are described in functional division and are described respectively as various units. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware in the implementation of the specification.

[0125] Those skilled in the art will understand that the embodiments of the specification can be provided as a method, a system or a computer program product. Therefore, the specification can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the specification can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0126] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes ​ The steps of the function specified in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0130] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0131] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0132] It should also be noted that the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0133] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present specification can be described in the general context of computer-executable instructions, such as program modules, executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media, including storage devices.

[0135] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different but related aspects of the description. Each of the various embodiments can stand on its own, and each can be combined with the subject matter of other embodiments to produce further embodiments. Where the same numbers appear in different embodiments, such numbers are used for the sake of ease of understanding only and do not imply that the embodiments in which such numbers appear must be the same.

[0136] The above description is embodied in the form of only a description of embodiments of the present specification, and is not intended to limit the present specification. Various changes and modifications can be made by those skilled in the art based on the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.

Claims

1. A game-based multilingual real-time translation method, characterized in that, The method comprises: determining first text corresponding to speech data input by a game user in response to the speech data, and target languages to which the first text needs to be translated; standardizing the first text according to a predetermined term library to determine a text to be translated; wherein the term library is constructed according to a corresponding relationship between colloquial words and standard words determined by an event prediction model based on historical speech text of a user and at least one historical operation of the user within a preset time period; the event prediction model is used to output a predicted event based on the historical speech text of the user; and the event prediction model is trained to minimize the difference between the predicted event and the historical operation; inputting the text to be translated into a pre-trained translation model to convert the text to be translated into translation text corresponding to the target languages by the translation model and output the translation text; returning the translation text to each target user corresponding to the target languages in the game.

2. The method of claim 1, wherein, The standardization of the first text according to the predetermined term library specifically comprises: determining each colloquial word in the first text according to the predetermined term library; determining each standard word corresponding to each colloquial word from the predetermined term library; replacing each colloquial word in the first text with the corresponding standard word to determine the text to be translated.

3. The method of claim 1, wherein, The standardization of the first text according to the predetermined term library to determine the text to be translated specifically comprises: inputting the first text into a pre-trained term conversion model to standardize the first text by the term conversion model and output the text to be translated; wherein the term conversion model is determined according to the pre-determined term library.

4. The method of claim 3, wherein, The pre-training of the term conversion model specifically comprises: determining a first sample text containing each colloquial word; replacing each colloquial word in the first sample text with a corresponding standard word according to the predetermined term library to determine a first labeled text as a label; inputting the first sample text as a sample into the term conversion model to output a standardized text corresponding to the first sample text; training the term conversion model to minimize the difference between the first labeled text and the standardized text.

5. The method of claim 3, wherein, After determining the text to be translated, before inputting the text to be translated into a pre-trained translation model, the method further comprises: returning the text to be translated to the game user to determine whether the semantics of the text to be translated is accurate by the game user; if not, obtaining historical speech data of the game user in the game; inputting the first text and the historical speech data into the pre-trained term conversion model to make the term conversion model output a text according to the first text under the prompt of the historical speech data as a new text to be translated.

6. The method of claim 1, wherein, The pre-training of the translation model specifically comprises: determining a second sample text containing each standard word; determine, according to the second sample text, a correct translation text corresponding to each sample language, as a second annotation, the correct translation text having the same semantics as the second sample text in the each sample language; input the second sample text into the translation model as a sample, and translate the second sample text by the translation model to output a training translation text corresponding to each sample language of the second sample text; train the translation model by minimizing the difference between the training translation text and the second annotation.

7. The method of claim 1, wherein, return the translation text to each target user corresponding to each target language in the game, specifically including: convert the translation text corresponding to each target language into corresponding translation speech by calling a voice generation plug-in; determine each target user corresponding to each target language for the translation text corresponding to the target language; return the translation text and the translation speech corresponding to the target language to each target user corresponding to the target language.

8. A game-based multilingual real-time translation device, characterized by, The apparatus includes: a response module configured to determine a first text corresponding to speech data input by a game user and each target language required to be translated by the first text in response to the speech data; a determination module configured to normalize the first text according to a predetermined term library to determine a text to be translated, wherein the term library is constructed according to a corresponding relationship between colloquial vocabulary and standard vocabulary by an event prediction model based on historical speech text of a user and at least one historical operation of the user within a preset time period, the event prediction model is configured to output a predicted event based on the historical speech text of the user, and the event prediction model is trained by minimizing the difference between the predicted event and the historical operation; a translation module configured to input the text to be translated into a pre-trained translation model to convert the text to be translated into a translation text corresponding to each target language by the translation model and output the translation text; a return module configured to return the translation text to each target user corresponding to each target language in the game.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any one of claims 1-7.

Images