Continuous learning method, device, terminal and storage medium for multilingual translation system

Abstract

The present invention discloses a continuous learning method, device, terminal and storage medium for a multilingual translation system. The method includes: pre-processing a translation request according to a processing strategy; distributing a corresponding translation model according to the processed translation request, and performing translation reasoning on the processed translation request through the translation model; performing confidence assessment based on the translation result, and triggering a "Please translate" mode based on the confidence assessment result, dispatching the processed translation request to an online expert or other user side; receiving translation reference answers fed back by the online expert or other user side, and incentivizing the translation model to perform answer fusion on the fed back translation reference answers; receiving translation result scores fed back by other user sides, and continuously learning the translation model based on the translation result scores and the fed back translation reference answers to expand the translation analysis data and continuous learning data of the translation model. The present invention improves the translation performance of the system through human-machine collaboration of feedback scoring and "Please translate".

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a continuous learning method, device, terminal and storage medium for a multilingual translation system. Background Art

[0002] In the field of natural language processing, multilingual translation models trained on large-scale data, such as m2m100, already possess powerful many-to-many machine translation capabilities, meaning a single model supports translation in multiple directions simultaneously. However, real-world translation systems, such as Baidu Translate and Google Translate, remain complex systems engineering tasks, involving tedious and repetitive data collection and organization, offline training of multiple bilingual translation models, and online deployment of these models. These processes require extensive rule processing, system engineering, and manual intervention, lacking the ability for human-machine collaboration and continuous learning.

[0003] Human-computer collaboration places users and systems on equal footing, acting as both providers and receivers of information. This shifts away from the traditional paradigm of users passively receiving information, enhancing user engagement and influence on the system. Through two-way interaction and mutual reinforcement between humans and machines, the system's usability is continuously improved. However, current human-computer interaction in translation systems is typically one-way and simplistic. Translation systems only output translation results, without focusing on user feedback or responses to these results. This makes it impossible to leverage the feedback process of human-computer collaboration to improve translation system performance.

[0004] Continuous learning refers to the ability of a model to continuously learn new tasks or knowledge. In real-world scenarios, data is typically not acquired all at once but generated as it is served. This requires the system to efficiently and continuously learn from this streaming data to expand capabilities and enhance performance. However, current translation systems typically employ a process paradigm of data collection, offline model training, and online model deployment. Under this paradigm, expanding model capabilities requires re-collecting data, designing new models, and re-training them offline. This prevents automated and continuous learning to expand the capabilities and performance of translation models, resulting in low efficiency and poor real-time performance.

[0005] Therefore, the existing technology needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that, in response to the defects of the existing technology, the present invention provides a continuous learning method, device, terminal and storage medium for a multilingual translation system to solve the technical problems of low efficiency and poor real-time performance caused by the inability of existing translation systems to continuously learn.

[0007] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0008] In a first aspect, the present invention provides a continuous learning method for a multilingual translation system, comprising:

[0009] Obtaining an input translation request and preprocessing the translation request according to a preset processing strategy to obtain a processed translation request;

[0010] Distributing a corresponding translation model according to the processed translation request, performing translation reasoning on the processed translation request through the translation model, and outputting a translation result;

[0011] Performing a confidence assessment based on the translation result, triggering a request-for-translation mode based on the confidence assessment result, and dispatching the processed translation request to an online expert or other user side;

[0012] receiving translation reference answers fed back by the online expert or other user side, and motivating the translation model to perform answer fusion on the fed-back translation reference answers;

[0013] The translation result scores fed back by the other users are received, and the translation model is continuously learned based on the translation result scores and the fed-back translation reference answers, so as to expand the translation analysis data and continuous learning data of the translation model.

[0014] In one implementation, obtaining an input translation request and preprocessing the translation request according to a preset processing strategy to obtain a processed translation request includes:

[0015] Get the input translation request;

[0016] The translation request is subjected to language detection, abnormal character removal, and uppercase / lowercase conversion processing according to the preset processing strategy to obtain the processed translation request.

[0017] In one implementation, distributing a corresponding translation model according to the processed translation request, performing translation reasoning on the processed translation request using the translation model, and outputting a translation result includes:

[0018] Matching and distributing corresponding translation models according to the language detection result of the processed translation request;

[0019] Perform translation reasoning on the processed translation request according to the translation analysis data corresponding to the translation model, and output a translation result.

[0020] In one implementation, performing a confidence assessment based on the translation result, triggering a request-for-translation mode based on the confidence assessment result, and dispatching the processed translation request to an online expert or other user side includes:

[0021] inputting the translation result and the processed translation request into a self-assessment model;

[0022] Performing confidence assessment on the translation result using the self-assessment model to obtain a confidence assessment score;

[0023] The request-for-translation mode is triggered according to the confidence evaluation score, and the processed translation request is dispatched to an online expert or other user side.

[0024] In one implementation, triggering the request-for-translation mode based on the confidence evaluation score and dispatching the processed translation request to an online expert or other user side includes:

[0025] Determining whether the confidence assessment score is lower than a preset threshold;

[0026] If the confidence evaluation score is lower than the preset threshold, the "Please translate" mode is triggered, and the processed translation request is dispatched to an online expert or other user side.

[0027] In one implementation, receiving the translation reference answers fed back by the online expert or the other user, and incentivizing the translation model to perform answer fusion on the fed-back translation reference answers, includes:

[0028] receiving translation reference answers from the online expert or other user-side feedback;

[0029] The translation reference answer and the processed translation request are input into the translation model, and fusion translation reasoning is performed to obtain a fusion translation result.

[0030] In one implementation, the step of inputting the translation reference answer and the processed translation request into the translation model to perform fusion translation reasoning further includes:

[0031] Performing language detection, removing abnormal characters, and case conversion on the fused translation results, and sorting the processed translation results according to confidence assessment scores;

[0032] The sorted translation results are sent to the other user sides.

[0033] In one implementation, receiving the translation result scores fed back by the other users, and continuously learning the translation model based on the translation result scores and the fed-back translation reference answers, includes:

[0034] Receive translation result scores fed back by other users;

[0035] The translation reference answer based on the translation result score and feedback is input into a continuous learning algorithm, and the translation model is subjected to translation data enhancement learning, language expansion learning, and translation field enhancement learning operations through the continuous learning algorithm.

[0036] In one implementation, the receiving of translation result scores fed back by the other users and the continuous learning of the translation model based on the translation result scores and the fed-back translation reference answers may then include:

[0037] Obtain data sources at certain time intervals;

[0038] Performing clustering, evaluation, cleaning, and mining operations on the data source to mine corpus data that meets the conditions in the data source;

[0039] The corpus data is distributed and scheduled to the online expert or the other user side to update the translation database of the online expert or the other user side.

[0040] In one implementation, the method further includes:

[0041] Continuously training the translation model through continuous learning techniques including regularization, parameter isolation, and data replay;

[0042] Continuously evaluate the translation quality of the translation model, adjust the continuous learning strategy based on the translation quality evaluation results, and generate and deploy the adjusted translation model.

[0043] In a second aspect, the present invention provides a continuous learning device for a multilingual translation system, comprising:

[0044] A preprocessing module, configured to obtain an input translation request and preprocess the translation request according to a preset processing strategy to obtain a processed translation request;

[0045] A translation inference module, configured to distribute a corresponding translation model according to the processed translation request, perform translation inference on the processed translation request using the translation model, and output a translation result;

[0046] A confidence assessment module is configured to perform a confidence assessment based on the translation result, trigger a request-for-translation mode based on the confidence assessment result, and dispatch the processed translation request to an online expert or other user side;

[0047] A translation fusion module, configured to receive translation reference answers fed back by the online expert or other user side, and to stimulate the translation model to perform answer fusion on the fed-back translation reference answers;

[0048] The continuous learning module is used to receive the translation result scores fed back by the other users, and continuously learn the translation model based on the translation result scores and the fed-back translation reference answers to expand the translation analysis data and continuous learning data of the translation model.

[0049] In a third aspect, the present invention provides a terminal comprising: a processor and a memory, wherein the memory stores a continuous learning program for a multi-language translation system, and when the continuous learning program for the multi-language translation system is executed by the processor, it is used to implement the operation of the continuous learning method for the multi-language translation system as described in the first aspect.

[0050] In a fourth aspect, the present invention further provides a storage medium, which is a computer-readable storage medium and stores a continuous learning program for a multi-language translation system. When the continuous learning program for the multi-language translation system is executed by a processor, it is used to implement the operation of the continuous learning method for the multi-language translation system as described in the first aspect.

[0051] The present invention adopts the above technical solution to achieve the following effects:

[0052] The present invention enhances the two-way interaction and influence between users and the translation system through feedback scoring and human-computer collaboration, thereby improving the usability of the translation system. In addition, the present invention continuously enhances the capabilities and performance of the translation system based on continuous learning, intelligently drives the translation system to continuously adapt to the real world and continuously evolve, thereby linking human-computer collaboration with continuous learning, and using the data generated by human-computer collaboration as data for continuous learning of the translation model, so that the translation model is no longer a static component that is trained once and used all the time in the system, but a process of continuous learning as the translation system continuously interacts with the external world, including users. In this process, human intelligence and machine intelligence empower each other and continuously improve, dynamically improving the translation performance of the translation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0054] Figure 1 The present invention is a flowchart of a continuous learning method for a multilingual translation system in one implementation mode.

[0055] Figure 2 It is a schematic diagram of the overall framework of a multilingual translation system in one implementation of the present invention.

[0056] Figure 3 This is a block diagram of the continuous learning principle in one implementation of the present invention.

[0057] Figure 4 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0060] Exemplary Methods

[0061] Current translation systems typically adopt a process paradigm of data collection, offline model training, and online model deployment. Under this paradigm, in order to expand the capabilities of the model, it is necessary to collect data again and design a new model for re-offline training. It is impossible to automatically and continuously perform continuous learning to expand the capabilities and performance of the translation model, resulting in low efficiency and poor real-time performance.

[0062] In response to the above technical problems, this embodiment provides a continuous learning method for a multilingual translation system. Through feedback scoring and human-computer collaboration such as asking you to translate, the two-way interaction and influence between users and the translation system are enhanced, thereby improving the usability of the translation system. In addition, based on the continuous learning method, the capabilities and performance of the translation system are continuously enhanced, and the translation system is intelligently driven to continuously adapt to the real world and continue to evolve, thereby linking human-computer collaboration and continuous learning. The data generated by human-computer collaboration is used as data for continuous learning of the translation model, so that the translation model is no longer a static component that is trained once and used all the time in the system, but a process of continuous learning as the translation system continuously interacts with the external world, including users. In this process, human intelligence and machine intelligence empower each other and continuously improve, dynamically improving the translation performance of the translation system.

[0063] like Figure 1 As shown, an embodiment of the present invention provides a continuous learning method for a multilingual translation system, comprising the following steps:

[0064] Step S100: obtaining an input translation request, and preprocessing the translation request according to a preset processing strategy to obtain a processed translation request.

[0065] In this embodiment, the continuous learning method of the multilingual translation system is applied to a terminal, which includes but is not limited to a computer and other devices.

[0066] Current multilingual translation systems require complex system engineering to continuously improve the capabilities of the translation system. This is expensive and inefficient, and cannot meet users' personalized needs for human-computer collaboration. In this embodiment, based on the existing multilingual translation system, human-computer collaboration and continuous learning are adopted. On the one hand, it can enhance user participation and influence on the system to improve the usability of the system. On the other hand, the streaming data generated during the human-computer collaboration process can be used for continuous learning of the multilingual translation model to expand capabilities and enhance performance. Therefore, in this embodiment, the design of an effective human-computer collaboration and continuous learning solution for the multilingual translation system can greatly improve the usability of the system, improve the system's evolution efficiency, and reduce labor costs.

[0067] In this embodiment, the two-way interaction and influence between users and the translation system are enhanced through feedback scoring and human-computer collaboration such as asking users to translate. The streaming data generated by the human-computer collaboration process is further used for continuous learning of the multilingual translation model, thereby achieving continuous interaction and continuous learning between the translation system and users.

[0068] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0069] Step S101, obtaining an input translation request;

[0070] Step S102 : performing language detection, removing abnormal characters, and case conversion on the translation request according to the preset processing strategy to obtain the processed translation request.

[0071] like Figure 2 As shown, in this embodiment, in a multilingual translation system based on human-computer collaborative continuous learning, the overall framework consists of a translation system, a translation model, a data warehouse, and a processing strategy; wherein the translation system includes two parts: human-computer collaboration and multilingual translation; the translation model includes two parts: continuous learning and model cluster; the data warehouse includes three parts: human-computer collaboration log, data processing, and continuous evolution; the processing strategy includes four parts: distribution, preprocessing, post-processing, and fusion sorting.

[0072] In this embodiment, the translation system provides users with multilingual translation services. Based on feedback scoring and a human-computer collaboration method of "ask you to translate", the system enhances the two-way interaction and influence between users and the translation system, thereby improving the usability of the translation system. In the translation system, users can input sentences they want to translate, thereby forming a translation request in the translation system. The translation system receives the translation request input by the user and pre-processes the translation request according to a preset processing strategy, such as language detection, removal of abnormal characters, and case conversion, to obtain a translation request that conforms to the system's translation rules.

[0073] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the continuous learning method of the multilingual translation system further includes the following steps:

[0074] Step S200 : distributing a corresponding translation model according to the processed translation request, performing translation reasoning on the processed translation request through the translation model, and outputting a translation result.

[0075] In this embodiment, the translation process of the multilingual translation system involves the participation and real-time interaction of multiple parties including intelligent models, users, and experts. Both users and the system serve as providers and receivers of information, abandoning the traditional paradigm of users passively receiving information. This enhances user participation and influence on the system. Through the two-way interaction and mutual promotion between humans and machine systems, the usability of the system is continuously improved in the process of human-machine interaction.

[0076] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0077] Step S201, matching and distributing corresponding translation models according to the language detection result of the processed translation request;

[0078] Step S202 : performing translation reasoning on the processed translation request according to the translation analysis data corresponding to the translation model, and outputting a translation result.

[0079] In this embodiment, translation is performed by a translation model, which is an intelligent agent that performs translation reasoning. This model cluster, supported by continuous learning, provides translation reasoning services. The model cluster consists of a series of multilingual translation models with different characteristics. These models differ primarily in language support, translation focus, model parameter size, and reasoning efficiency. All models support continuous learning for dynamic expansion and performance enhancement. Continuous learning is a set of algorithmic and strategic components that support continuous learning for translation models, including data-enhanced learning, language expansion, and domain-enhanced learning.

[0080] During the translation process of the translation model, the translation system first matches and distributes the corresponding translation model based on the language detection results of the processed translation request. Then, it performs translation inference on the processed translation request based on the translation analysis data corresponding to the translation model and outputs the translation result. During the entire translation process of the translation model, the translation analysis data used is the data in the data warehouse.

[0081] It's understood that the data in the data warehouse can be data from the initial release or data obtained during the previous round of human-computer interaction learning. In other words, the data warehouse in this embodiment serves as the data source for the continuous learning of the translation model. Based on the data generated during the human-computer collaboration process, it continuously provides a continuous stream of data for the translation model's continuous learning. Data generated during multi-party human-computer collaboration, such as translation requests, model translation results, and user translation modifications, is used to construct the continuous data.

[0082] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the continuous learning method of the multilingual translation system further includes the following steps:

[0083] Step S300: performing a confidence evaluation based on the translation result, triggering a translation request mode based on the confidence evaluation result, and dispatching the processed translation request to an online expert or other user side.

[0084] In this embodiment, the translation model's data processing involves clustering, evaluating, cleaning, and mining operations through human-machine collaboration to uncover valuable corpus data. Continuous Evolution intelligently annotates, filters, and updates the processed data to form a continuously evolving data warehouse for the translation model's continuous learning. Intelligent annotation triggers the human-machine collaborative "Request Your Translation" process, intelligently dispatching data with model translation results and user translation modification confidence levels below a certain threshold to online experts or user-requested translation annotation.

[0085] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0086] Step S301: inputting the translation result and the processed translation request into a self-assessment model;

[0087] Step S302: performing confidence assessment on the translation result using the self-assessment model to obtain a confidence assessment score;

[0088] Step S303: triggering a translation request mode according to the confidence evaluation score, and dispatching the processed translation request to an online expert or other user side.

[0089] In this embodiment, the translation model performs translation reasoning on the translation request, outputs the translation result, and automatically evaluates the confidence of the translation result. If the confidence of the translation result is lower than a certain threshold, the translation request process is triggered, and the translation request is intelligently dispatched to an online expert or other user-side request to provide a real-time translation reference answer, which encourages the translation model to integrate the reference answer to improve the performance of translation reasoning.

[0090] Specifically, in one implementation of this embodiment, step S303 includes the following steps:

[0091] Step S303a, determining whether the confidence assessment score is lower than a preset threshold;

[0092] Step S303b: If the confidence evaluation score is lower than the preset threshold, the "Please translate" mode is triggered, and the processed translation request is dispatched to an online expert or other user side.

[0093] In this embodiment, confidence assessment is performed by a pre-set and trained automatic evaluation and scoring model (i.e., a self-evaluation model). In this automatic evaluation and scoring model, the input is a translation request text and a translation result text pair, and the output is a corresponding score (0 to 1 point). By judging whether the confidence assessment score is lower than a preset threshold (for example, the preset threshold is 0.85 points), the please translate mode can be triggered according to the judgment result; specifically, if the confidence assessment score is lower than the preset threshold, the please translate mode is triggered, and the translation request is dispatched to the online expert or other user side to request the online expert or other user side to provide real-time translation reference answers, thereby encouraging the translation model to fuse the reference answers to improve the performance of translation reasoning.

[0094] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the continuous learning method of the multilingual translation system further includes the following steps:

[0095] Step S400: receiving translation reference answers fed back by the online expert or other user side, and stimulating the translation model to perform answer fusion on the fed-back translation reference answers.

[0096] In this embodiment, based on multi-party feedback scoring and human-computer collaboration, through the intelligent interaction process of users, online experts, and translation systems, the translation process is translated, feedbacked, scored, modified, and evaluated, to build a highly available and high-performance translation system based on intelligent interactive human-computer collaboration.

[0097] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0098] Step S401: receiving translation reference answers fed back by the online expert or other user side;

[0099] Step S402: Input the translation reference answer and the processed translation request into the translation model, perform fusion translation reasoning, and obtain a fusion translation result;

[0100] Step S403: performing language detection, removing abnormal characters, and case conversion on the fused translation results, and sorting the processed translation results according to confidence evaluation scores;

[0101] Step S404: Send the sorted translation results to the other user side.

[0102] In this embodiment, after obtaining the translation reference answer fed back by the user, the translation reference answer and the processed translation request are input into the translation model for fusion translation reasoning to obtain a fused translation result. During the translation model fusion process, the translation model supports translation reasoning with reference answers to improve translation quality. During fusion, the reference answer and the translation request text are input into the translation model for translation reasoning. If multiple translation models generate output translation results, the multiple translation results are fused and sorted to output the final translation result. Then, the fused translation result is subjected to language detection, abnormal character removal, and case conversion, and the processed translation results are sorted according to the confidence assessment score. Finally, the sorted translation results are sent to other user sides for scoring or correction by other users.

[0103] In one implementation of the present embodiment, feedback is a two-way feedback interaction process between the user and the translation system. First, the translation system feeds back the translation result and the confidence level of the translation result to the user's translation request. Secondly, the user scores the translation result and feeds it back to the translation system after correction. Finally, the translation system automatically evaluates and scores the result of the user's correction and feeds it back to the user, while storing the result of the user's feedback in a log for analysis and continuous data construction. The whole process involves the real-time participation and interaction of the user and the translation system. The user and the system both act as providers and receivers of information, abandoning the traditional paradigm of users passively receiving information, enhancing the user's participation and influence on the system, and continuously improving the availability of the system in the process of human-computer interaction through the two-way interaction and mutual promotion between the human and machine systems.

[0104] The human-machine collaboration process in "Please Translate" primarily manifests itself in two aspects: first, assisting the translation model's online translation reasoning during the translation process; second, supporting the intelligent annotation process in continuous data construction. The online reasoning of the assisted translation model involves intelligently dispatching translation requests to online experts or other users to provide real-time translation reference answers when the translation model's translation result evaluation score confidence falls below a certain threshold. This encourages the translation model to integrate the reference answers to improve translation reasoning performance. The intelligent annotation process in continuous data construction involves further revising the automatic annotation results through the "Please Translate" process with online users and experts when the evaluation score confidence of the automatic annotation results based on the translation model falls below a certain threshold, thereby improving the annotation quality of the corpus data.

[0105] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the continuous learning method of the multilingual translation system further includes the following steps:

[0106] Step S500: receiving translation result scores fed back by other users, and continuously learning the translation model based on the translation result scores and the fed-back translation reference answers to expand the translation analysis data and continuous learning data of the translation model.

[0107] In this embodiment, the translation results output by the translation model are processed by the processing strategy module and then submitted to the user. The user can score the translation results and submit them for revision. The system will automatically evaluate and score the user's revised results and provide feedback to the user. The feedback results will be stored in the log for analysis and continuous data construction.

[0108] Specifically, in one implementation of this embodiment, step S500 includes the following steps:

[0109] Step S501: receiving translation result scores fed back by other users;

[0110] Step S502: input the translation result score and the feedback translation reference answer into a continuous learning algorithm, and perform translation data enhancement learning, language expansion learning, and translation domain enhancement learning on the translation model through the continuous learning algorithm.

[0111] In this embodiment, the translation model can be continuously learned based on the translation result scores and the translation reference answers fed back. The continuous learning module is a set of algorithms and strategy components that support the continuous learning of the translation model. The translation model is subjected to translation data enhancement learning, language expansion learning, and translation domain enhancement learning through the continuous learning algorithm. Data enhancement learning refers to the process of expanding translation data, data enhancement is the learning of data increments, and domain enhancement learning refers to the learning of domain increments.

[0112] Specifically, in an implementation of this embodiment, step S502 further includes the following steps:

[0113] Step S503, obtaining a data source at a certain time interval;

[0114] Step S504: performing clustering, evaluation, cleaning, and mining operations on the data source to mine corpus data that meets the conditions in the data source;

[0115] Step S505 : Distribute and dispatch the corpus data to the online expert or the other user side to update the translation database of the online expert or the other user side.

[0116] In one implementation of this embodiment, based on continuous learning of data constructed by human-machine collaboration, services are provided in a continuous learning manner. First, an initial model is trained based on existing data, and then the initial model is used to provide online translation services. During the translation process, the translation quality is improved through human-machine collaboration. At the same time, the online data is used to continuously update and enhance the model in an unsupervised / supervised manner after continuous data evolution. The principle block diagram of the overall continuous learning process is as follows: Figure 3 shown.

[0117] The initial translation model is used for the cold start of the translation system. Initial translation training data is obtained through data collection and production, and then cleansed to improve data quality. Key cleaning operations include rapid alignment, removal of sentence pairs where the source / target sentence length exceeds a specified threshold, language detection, removal of HTML tags and excess whitespace, and deduplication. The high-quality data from this cleansing process serves as the base dataset for training the initial multilingual translation model. The resulting initial translation model is used for translation reasoning in the translation system, and through continuous learning based on the continuous data construction of human-machine collaboration, its capabilities and performance are continuously improved.

[0118] Continuous data construction is based on the data logs generated during the human-machine collaboration process. Data sources for construction are extracted from the human-machine collaboration logs at regular intervals. Data cleaning operations such as clustering, evaluation, cleaning, and mining are then performed to mine valuable corpus data. The cleaned corpus data is then dispatched through intelligent data distribution to online users and experts in the translation system for intelligent annotation. This intelligent annotation process evaluates and scores the annotated corpus and performs human-machine collaborative annotation on unannotated corpus. Human-machine collaborative annotation uses the translation model to obtain an initial translation result, which is then revised through the translation process of online users and experts. Finally, the intelligently annotated corpus data is filtered according to the evaluation and scoring to form new, continuously constructed data.

[0119] In one implementation of the embodiment of the present invention, the continuous learning method of the multilingual translation system further includes the following steps:

[0120] Step S600, continuously training the translation model through continuous learning techniques such as regularization, parameter isolation, and data playback;

[0121] Step S700 : performing continuous translation quality evaluation on the translation model, adjusting the continuous learning strategy according to the translation quality evaluation result, and generating and deploying the adjusted translation model.

[0122] In this embodiment, continuous model learning is the continuous learning of the translation model or the initial translation model based on the generated continuously constructed data to continuously improve its capabilities and performance. Based on the new data that is continuously constructed periodically, continuous learning techniques such as regularization, parameter isolation, and data playback are applied to continuously train the model while maintaining its existing capabilities and performance. During and after training, the model will be continuously evaluated for translation quality. Based on the evaluation results, the continuous learning strategy is adjusted and a translation model for online deployment is generated. Because each continuous training only trains the newly constructed data and does not require the full amount of historical data, the continuous learning process is efficient and fast, without the need for a large amount of rule processing, system engineering, and manual participation, which improves the system's evolution efficiency and reduces labor costs.

[0123] This embodiment achieves the following technical effects through the above technical solution:

[0124] This embodiment supports continuous learning through continuous interaction between the translation system and the outside world. A multi-party human-machine collaborative process is constructed, and the two-way interaction and mutual promotion between humans and machines are enhanced through intelligent interactive feedback scoring to continuously improve the usability of the system. The translation performance of the online translation model and the quality of intelligently annotated data are enhanced through multi-party participation in "Please translate". A continuous learning process for the model based on continuous data construction through human-machine collaboration is constructed, and the translation model is automatically and continuously learned to continuously improve its capabilities and performance. Through the mutual empowerment of human intelligence and machine intelligence, continuous capability and performance improvement is achieved, without the need for a large amount of rule processing, system engineering and manual participation, and it is efficient and low-cost.

[0125] Exemplary devices

[0126] Based on the above embodiments, the present invention further provides a continuous learning device for a multilingual translation system, comprising:

[0127] A preprocessing module, configured to obtain an input translation request and preprocess the translation request according to a preset processing strategy to obtain a processed translation request;

[0128] A translation inference module, configured to distribute a corresponding translation model according to the processed translation request, perform translation inference on the processed translation request using the translation model, and output a translation result;

[0129] A confidence assessment module is configured to perform a confidence assessment based on the translation result, trigger a request-for-translation mode based on the confidence assessment result, and dispatch the processed translation request to an online expert or other user side;

[0130] A translation fusion module, configured to receive translation reference answers fed back by the online expert or other user side, and to stimulate the translation model to perform answer fusion on the fed-back translation reference answers;

[0131] The continuous learning module is used to receive the translation result scores fed back by the other users, and continuously learn the translation model based on the translation result scores and the fed-back translation reference answers to expand the translation analysis data and continuous learning data of the translation model.

[0132] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 4 shown.

[0133] The terminal includes: a processor, memory, interface, display screen and communication module connected via a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0134] When the computer program is executed by a processor, it is used to implement the operation of a continuous learning method for a multi-language translation system.

[0135] It will be understood by those skilled in the art that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0136] In one embodiment, a terminal is provided, comprising: a processor and a memory, wherein the memory stores a continuous learning program for a multi-language translation system, and when the continuous learning program for the multi-language translation system is executed by the processor, it is used to implement the operations of the continuous learning method for the multi-language translation system as described above.

[0137] In one embodiment, a storage medium is provided, wherein the storage medium stores a continuous learning program for a multi-language translation system. When the continuous learning program for the multi-language translation system is executed by a processor, it is used to implement the operations of the continuous learning method for the multi-language translation system as described above.

[0138] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory.

[0139] In summary, the present invention provides a continuous learning method, device, terminal and storage medium for a multilingual translation system, the method comprising: pre-processing a translation request according to a processing strategy; distributing a corresponding translation model according to the processed translation request, and performing translation reasoning on the processed translation request through the translation model; performing a confidence assessment based on the translation result, and triggering a "Please translate" mode based on the confidence assessment result, dispatching the processed translation request to an online expert or other user side; receiving translation reference answers fed back by the online expert or other user side, incentivizing the translation model to perform answer fusion on the fed back translation reference answers; receiving translation result scores fed back by other user sides, and continuously learning the translation model based on the translation result scores and the fed back translation reference answers to expand the translation analysis data and continuous learning data of the translation model. The present invention improves the translation performance of the system through human-machine collaboration of feedback scoring and "Please translate".

[0140] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A continuous learning method for a multilingual translation system, characterized in that: include: Obtaining an input translation request and preprocessing the translation request according to a preset processing strategy to obtain a processed translation request; Distributing a corresponding translation model according to the processed translation request, performing translation reasoning on the processed translation request through the translation model, and outputting a translation result; Performing a confidence assessment based on the translation result, triggering a request-for-translation mode based on the confidence assessment result, and dispatching the processed translation request to an online expert or other user side; receiving translation reference answers fed back by the online expert or other user side, and motivating the translation model to perform answer fusion on the fed-back translation reference answers; Receive translation result scores fed back by the other users, and continuously learn the translation model based on the translation result scores and the fed-back translation reference answers to expand the translation analysis data and continuous learning data of the translation model; The receiving of the translation reference answers fed back by the online expert or other user side, and motivating the translation model to perform answer fusion on the fed-back translation reference answers, includes: receiving translation reference answers from the online expert or other user-side feedback; Inputting the translation reference answer and the processed translation request into the translation model, performing fusion translation reasoning, and obtaining a fused translation result; if multiple translation models generate translation results, performing fusion ranking decision on the multiple translation results and outputting a final translation result; The step of inputting the translation reference answer and the processed translation request into the translation model to perform fusion translation reasoning further includes: Performing language detection, removing abnormal characters, and case conversion on the fused translation results, and sorting the processed translation results according to confidence assessment scores; Sending the sorted translation results to the other user side; The method further comprises: Continuously training the translation model through continuous learning techniques including regularization, parameter isolation, and data replay; Continuously evaluate the translation quality of the translation model, adjust the continuous learning strategy based on the translation quality evaluation results, and generate and deploy the adjusted translation model.

2. The continuous learning method for a multilingual translation system according to claim 1, characterized in that: The step of obtaining an input translation request and preprocessing the translation request according to a preset processing strategy to obtain a processed translation request includes: Get the input translation request; The translation request is subjected to language detection, abnormal character removal, and uppercase / lowercase conversion processing according to the preset processing strategy to obtain the processed translation request.

3. The continuous learning method for a multilingual translation system according to claim 1, wherein: The distributing a corresponding translation model according to the processed translation request, performing translation reasoning on the processed translation request through the translation model, and outputting a translation result includes: Matching and distributing corresponding translation models according to the language detection result of the processed translation request; Perform translation reasoning on the processed translation request according to the translation analysis data corresponding to the translation model, and output a translation result.

4. The continuous learning method for a multilingual translation system according to claim 1, wherein: The confidence evaluation is performed based on the translation result, and the request-for-translation mode is triggered based on the confidence evaluation result, and the processed translation request is dispatched to an online expert or other user side, including: inputting the translation result and the processed translation request into a self-assessment model; Performing confidence assessment on the translation result using the self-assessment model to obtain a confidence assessment score; The request-for-translation mode is triggered according to the confidence evaluation score, and the processed translation request is dispatched to an online expert or other user side.

5. The continuous learning method for a multilingual translation system according to claim 4, characterized in that: The triggering of the request-for-translation mode according to the confidence evaluation score and dispatching the processed translation request to an online expert or other user side includes: Determining whether the confidence assessment score is lower than a preset threshold; If the confidence evaluation score is lower than the preset threshold, the "Please translate" mode is triggered, and the processed translation request is dispatched to an online expert or other user side.

6. The continuous learning method for a multilingual translation system according to claim 1, characterized in that: The receiving of translation result scores fed back by the other users and continuously learning the translation model based on the translation result scores and the fed-back translation reference answers includes: Receive translation result scores fed back by other users; The translation reference answer based on the translation result score and feedback is input into a continuous learning algorithm, and the translation model is subjected to translation data enhancement learning, language expansion learning, and translation field enhancement learning operations through the continuous learning algorithm.

7. The continuous learning method for a multilingual translation system according to claim 1, wherein: The step of receiving the translation result scores fed back by the other users and continuously learning the translation model according to the translation result scores and the fed-back translation reference answers may further include: Obtain data sources at certain time intervals; Performing clustering, evaluation, cleaning, and mining operations on the data source to mine corpus data that meets the conditions in the data source; The corpus data is distributed and scheduled to the online expert or the other user side to update the translation database of the online expert or the other user side.

8. A continuous learning device for a multilingual translation system, used to implement the continuous learning method for a multilingual translation system according to any one of claims 1 to 7, characterized in that: include: A preprocessing module, configured to obtain an input translation request and preprocess the translation request according to a preset processing strategy to obtain a processed translation request; A translation inference module, configured to distribute a corresponding translation model according to the processed translation request, perform translation inference on the processed translation request using the translation model, and output a translation result; A confidence assessment module is configured to perform a confidence assessment based on the translation result, trigger a request-for-translation mode based on the confidence assessment result, and dispatch the processed translation request to an online expert or other user side; A translation fusion module, configured to receive translation reference answers fed back by the online expert or other user side, and to stimulate the translation model to perform answer fusion on the fed-back translation reference answers; The continuous learning module is used to receive the translation result scores fed back by the other users, and continuously learn the translation model based on the translation result scores and the fed-back translation reference answers to expand the translation analysis data and continuous learning data of the translation model.

9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a continuous learning program for a multi-language translation system, and when the continuous learning program for the multi-language translation system is executed by the processor, it is used to implement the operation of the continuous learning method for a multi-language translation system as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, which stores a continuous learning program for a multi-language translation system. When the continuous learning program for a multi-language translation system is executed by a processor, it is used to implement the operation of the continuous learning method for a multi-language translation system according to any one of claims 1 to 7.

Images