Server and method for facilitating providing translations

Abstract

Aspects concern a server comprising a processor configured to: receive new source texts in a first language; and translate the new source texts into a second language using a trained translation model, wherein the trained translation model is generated by collecting feedback data regarding published translations in which source texts are translated from the first language into the second language by a translation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

Description

SERVER AND METHOD FOR FACILITATING PROVIDING TRANSLATIONSTECHNICAL FIELD

[0001] Various embodiments relate to a server and a method for facilitating providing translations. In particular, various embodiments relate to the server and the method for facilitating providing the translations for an on-demand service.BACKGROUND

[0002] Due to development of information technology, a user (who in some contexts herein may also be referred to as a “consumer”, a “requester” or a “Pax”) may request an on-demand service by accessing an on-demand service application. The on-demand service may allow the user to fulfil the user’ s demand via an immediate access to goods and / or services. The user may request the on-demand service, for example, a delivery service or a transport sendee, via the on-demand service application. After a server for the on-demand service receives an order (also referred to as a “booking”) for the on-demand service from the user, the senrer may allocate a driver (who in some contexts herein may also be referred to as a “delivery service provider”, a “delivery partner” or a “delivery agent”) to the received order, and the driver may perform the on-demand service for the received order.

[0003] The on-demand service application may have become an integral part of daily life for millions of users worldwide, including tourists. Therefore, the on-demand service application may need to serve diverse users speaking different languages, and thus may require effective translations of a user interface, including merchant names (for example, restaurant names) anditem names (for example, menu names, food names or cuisine names) to provide a seamless and user-friendly experience.

[0004] FIG. 1 illustrates an exemplary diagram showing a user interface of a computing device associated with the user according to the conventional technology. As shown in FIG. 1, the computing device may display a user interface screen of the on-demand service application. The user may select an icon of “menu language”, to select a menu language. For example, if the menu language selected is Chinese, the menu names may be displayed in Chinese, as shown in FIG. 1. In this manner, the user, for example, the tourist may read the local language menu.

[0005] However, translating the menu names accurately into multiple languages pose challenges. For example, conventional translation functions often use general-purpose translation algorithms that may not be optimised for a specific context of the on-demand service application. This may lead to inaccurate, confusing, and / or contextually inappropriate translations, especially when dealing with culturally specific terms, idiomatic expressions, and / or unique service names.

[0006] To improve a quality of the translations, it may be required to identify inaccurate translations, and generate enough training data with a limited budget for a human translator. However, conventionally, the inaccurate translations have been identified by a human operator for a manual review to correct the inaccurate translations, which may be costly, time- consuming, and not scalable with the growing number of daily updated translations. In addition, conventional solutions have not considered feedback on the translations.

[0007] Therefore, there is a need to provide a solution for facilitating providing accurate translations for the on-demand service.SUMMARY

[0008] According to various embodiments, there is a server for facilitating providing translations, the server comprising: a memory configured to store instructions; and a processor for executing the stored instructions and configured to: receive new source texts in a first language; and translate the new source texts in the first language into a second language using a trained translation model, wherein the trained translation model is generated by collecting feedback data regarding published translations in which source texts are translated from the first language into the second language by a translation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

[0009] In some embodiments, the processor is further configured to: classify each of the published translations into either a first group of the published translations or a second group of the published translations based on the score of each of the published translations; and assign the first group of the published translations to the one or more annotators, to generate the annotated translations for the first group of the published translations.

[0010] In some embodiments, the processor is further configured to: check the score of a certain published translation of the published translations; and if the score of the certain published translation is less than the predetermined threshold, classify the certain published translation into the first group of the published translations; and if the score of the certain published translation is equal to or greater than the predetermined threshold, classify the certain published translation into the second group of the published translations.

[0011] In some embodiments, the processor is further configured to: score the collected feedback data, based on characteristics of the collected feedback data; and select the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data.

[0012] In some embodiments, the one or more annotators include a human annotator and a teacher LLM (Large Language Model).

[0013] In some embodiments, the processor is further configured to: provide the part of the published translations to the human annotator; request the human annotator to modify and / or correct the part of the published translations; and generate the annotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator.

[0014] In some embodiments, the processor is further configured to: provide the part of the published translations to the teacher LLM; request the teacher LLM to translate the part of the published translations; and generate the annotated translations for the part of the published translations based on the translation received from the teacher LLM.

[0015] In some embodiments, the processor is further configured to: tune the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations.

[0016] In some embodiments, the processor is further configured to: remove at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed are translated using the trained translation model.

[0017] In some embodiments, the processor is further configured to: determine if a new source text of the new source texts originally includes an item name in both the first language and the second language; and if the new source text originally includes the item name in both the firstlanguage and the second language, present the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

[0018] According to various embodiments, there is a method for facilitating providing translations, the method comprising: receiving new source texts in a first language; and translating the new source texts in the first language into a second language using a trained translation model, wherein the trained translation model is generated by collecting feedback data regarding published translations in which source texts are translated from the first language into the second language by a translation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

[0019] In some embodiments, the method further comprises: classifying each of the published translations into either a first group of the published translations or a second group of the published translations based on the score of each of the published translations; and assigning the first group of the published translations to the one or more annotators, to generate the annotated translations for the first group of the published translations.

[0020] In some embodiments, the method further comprises: checking the score of a certain published translation of the published translations; if the score of the certain published translation is less than the predetermined threshold, classifying the certain published translation into the first group of the published translations; and if the score of the certain publishedtranslation is equal to or greater than the predetermined threshold, classifying the certain published translation into the second group of the published translations.

[0021] In some embodiments, the method further comprises: scoring the collected feedback data, based on characteristics of the collected feedback data; and selecting the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data.

[0022] In some embodiments, the one or more annotators include a human annotator and a teacher LLM (Large Language Model).

[0023] In some embodiments, the method further comprises: providing the part of the published translations to the human annotator; requesting the human annotator to modify and / or correct the part of the published translations; and generating the annotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator.

[0024] In some embodiments, the method further comprises: providing the part of the published translations to the teacher LLM; requesting the teacher LLM to translate the part of the published translations; and generating the annotated translations for the part of the published translations based on the translation received from the teacher LLM.

[0025] In some embodiments, the method further comprises: tuning the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations.

[0026] In some embodiments, the method further comprises: removing at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed are translated using the trained translation model.

[0027] In some embodiments, the method further comprises: determining if a new source text of the new source texts originally includes an item name in both the first language and the second language; and if the new source text originally includes the item name in both the first language and the second language, presenting the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

[0028] According to various embodiments, a data processing apparatus configured to perform the method of any one of the above embodiments is provided.

[0029] According to various embodiments, a computer program element comprising program instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of any one of the above embodiments is provided.

[0030] According to various embodiments, a computer-readable medium comprising program instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of any one of the above embodiments is provided. The computer-readable medium may include a non-transitory computer-readable medium.BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The invention will be better understood with reference to the detailed description when considered in conjunction with the non-limiting examples and the accompanying drawings, in which:- FIG. 1 illustrates an exemplary' diagram showing a user interface of a computing device associated with a user according to the conventional technology.- FIG. 2 illustrates an infrastructure of a system including a server for facilitating providing translations according to various embodiments.- FIG. 3 illustrates a block diagram of a server for facilitating providing translations according to various embodiments.- FIG. 4 illustrates a flowchart for a method for facilitating providing translations according to various embodiments.- FIG. 5 illustrates a data flow diagram of a server for facilitating providing translations according to various embodiments.- FIG. 6 illustrates a data flow diagram of a translation component of FIG. 5 according to various embodiments.- FIG. 7 illustrates a data flow diagram of a bilingual treatment of FIG. 6 according to various embodiments.DETAILED DESCRIPTION

[0032] The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the ait to practice the disclosure. Other embodiments may be utilized and structural, and logical changes may be made without departing from the scope of the disclosure. The various embodiments are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

[0033] Embodiments described in the context of one of a server and a method are analogously valid for the other server and method. Similarly, embodiments described in the context of a server are analogously valid for a method, and vice-versa.

[0034] Features that are described in the context of an embodiment may correspondingly be applicable to the same or similar' features in the other embodiments. Features that are describedin the context of an embodiment may correspondingly be applicable to the other embodiments, even if not explicitly described in these other embodiments. Furthermore, additions and / or combinations and / or alternatives as described for a feature in the context of an embodiment may correspondingly be applicable to the same or similar feature in the other embodiments.

[0035] In the context of various embodiments, the articles “a”, “an” and “the” as used with regard to a feature or element include a reference to one or more of the features or elements.

[0036] As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0037] Throughout the description, the term “module” may be understood as an application specific integrated circuit (ASIC), an electronic circuit, a combinational logic circuit, a field programmable gate array (FPGA), a processor which executes code, other suitable hardware components which provide the described functionality, or any combination thereof. The term of “module” may include a memory which stores code executed by the processor.

[0038] In the following, embodiments will be described in detail.

[0039] FIG. 2 illustrates an infrastructure of a system 200 including a server 100 for facilitating providing translations according to various embodiments.

[0040] As shown in FIG. 2, the system 200 may include, but is not limited to, the server 100, a database system 140, a network 150, a first computing device 160 associated with a user 161 (who in some contexts herein may also be referred to as a “consumer”, a “requester” or a “Pax”), and a plurality of second computing devices 170 (not shown) each associated with a plurality of merchants 171 (who in some contexts herein may also be referred to as a “retailer”, a “restaurant” or a “Mcx”).

[0041] In some embodiments, an on-demand service may be a service allowing the user 161 to fulfil the user’s demand via an immediate access to items and / or services. The user 161 may request the on-demand service, such as a transport service or an item delivery service, using auser interface presented on the first computing device 160. The user 161 may make an order for the on-demand service.

[0042] In some embodiments, the user 161 may use an on-demand service application, for example, a mobile application, provided by the server 100. For example, the server 100 may be controlled and / or managed by an on-demand service platform provider. The on-demand sendee application may be installed in the first computing device 160 associated with the user 161, to interact with the server 100. In some embodiments, the user interface provided by the on-demand service application may display texts, including, but not limited to, merchant names (for example, restaurant names), item names (for example, menu names, food names or cuisine names), service options, and features, in a certain language (also referred to as a “first language” or a “source language”). For example, the first language may be a default language (also referred to as an “original language”) that the on-demand service application provides by default. In some embodiments, the on-demand service application may provide a translation function to the user 161. Upon the user’s request to change the language, the user interface provided by the on-demand service application may display translations of the texts, including, but not limited to, translations of the merchant names, the item names, the service options, and the features, in the changed language (also referred to as a “second language” or a “target language”). In this manner, the on-demand service application may provide a seamless and user-friendly experience to the user 161.

[0043] In some embodiments, the system 200 may further include a plurality of third computing devices 180 (not shown) each associated with a plurality of human annotators 181 (who in some contexts herein may also be referred to as “human translators”). In some embodiments, the plurality of human annotators 181 may receive a request to modify and / or correct published translations (for example, published translations of the merchant names and / or the item names), modify and / or correct the published translations, and provide themodified and / or corrected translations to the server 100. In some embodiments, the published translations may be generated by a translation model (as will be described with reference to FIG. 3).

[0044] In some embodiments, the network 150 may include, but is not limited to, a Local Area Network (LAN), a Wide Area Network (WAN), a Global Area Network (GAN), or any combination thereof. The network 150 may provide a wireline communication, a wireless communication, or a combination of the wireline and wireless communication between the server 100 and the first computing device 160, between the server 100 and the plurality of second computing devices 170, and between the server 100 and the plurality of third computing devices 180.

[0045] In some embodiments, the first computing device 160 may be connectable to the server 100 via the network 150. In some embodiments, the first computing device 160 may be arranged in data or signal communication with the server 100 via the network 150. In some embodiments, the first computing device 160 may include, but is not limited to, at least one of the following: a mobile phone, a tablet computer, a laptop computer, a desktop computer, a head-mounted display and a smart watch. In some embodiments, the first computing device 160 may be associated with the user 161. For example, the first computing device 160 may belong to the user 161. Although not shown, it may be appreciated that the system 200 may further include a plurality of first computing devices each associated with, for example, belonging to, a plurality of users.

[0046] In some embodiments, the first computing device 160 may include a location sensor. In some embodiments, the location sensor may communicate with at least one of a global positioning satellite (GPS) server, a network server, and a Wi-Fi server, to detect a location of the first computing device 160. In some embodiments, the first computing device 160 may generate information about the location of the first computing device 160.

[0047] In some embodiments, the server 100, for example, implemented by a server computer, may include a communication interface 110, a processor 120, and a memory 130 (as will be described with reference to FIG. 3).

[0048] In some embodiments, the server 100 may communicate with the first computing device 160 via the network 150. In some embodiments, the first computing device 160 may receive a request (hereinafter, referred to as an “order”) from the user 161 for the on-demand service. The first computing device 160 may send the order to the server 100 via the network 150. In some embodiments, the first computing device 160 may send the information about the location of the first computing device 160 to the server 100 via the network 150. The location of the first computing device 160 may be considered as a location of the user 161. In some embodiments, the on-demand service application, provided by the server 100, may show the merchant names and the item names for available merchants 171 based on the location of the first computing device 160.

[0049] In some embodiments, the system 200 may further include a database 141. In some embodiments, the database 141 may be a part of the database system 140 which may be external to the server 100. The server 100 may communicate with the database 141. In some other embodiments, although not shown, the database 141 may be implemented locally in the memory 130 of the server 100.

[0050] In some embodiments, the server 100 may communicate with the plurality of second computing devices 170 via the network 150. In some embodiments, the plurality of second computing devices 170 may be arranged in data or signal communication with the server 100 via the network 150. In some embodiments, the plurality of second computing devices 170 may include, but is not limited to, at least one of the following: a mobile phone, a tablet computer, a laptop computer, a desktop computer, a head-mounted display and a smart watch. The plurality of second computing devices 170 may be associated with the plurality of merchants171 respectively. For example, the plurality of second computing devices 170 may belong to the plurality of merchants 171 respectively.

[0051] In some embodiments, the server 100 may communicate with the plurality of third computing devices 180 via the network 150. In some embodiments, the plurality of third computing devices 180 may be arranged in data or signal communication with the server 100 via the network 150. In some embodiments, the plurality of third computing devices 180 may include, but is not limited to, at least one of the following: a mobile phone, a tablet computer, a laptop computer, a desktop computer, a head-mounted display and a smart watch. The plurality of third computing devices 180 may be associated with the plurality of human annotators 181 respectively. For example, the plurality of third computing devices 180 may belong to the plurality of human annotators 181 respectively.

[0052] FIG. 3 illustrates a block diagram of a server 100 for facilitating providing translations according to various embodiments.

[0053] As shown in FIG. 3, the server 100, for example, implemented by a server computer, may include a communication interface 110, a processor 120, and a memory 130.

[0054] In some embodiments, the memory 130 (also referred to as a “database”) may store input data and / or output data temporarily or permanently. In some embodiments, the memory 130 may be configured to store instructions. In some embodiments, the memory 130 may store program code which allows the server 100 to perform a method 300 (as will be described with reference to FIG. 4). In some embodiments, the program code may be embedded in a Software Development Kit (SDK). The memory 130 may include an internal memory of the server 100 and / or an external memory. The external memory may include, but is not limited to, an external storage medium, for example, a memory card, a flash drive, and a web storage.

[0055] In some embodiments, the communication interface 110 may allow one or more computing devices, including a first computing device 160, to communicate with the processor120 of the server 100 via a network 150, as shown in FIG. 2. In some embodiments, the first computing device 160 may belong to a user 161 who wants to make an order for an on-demand service. In some embodiments, the communication interface 110 may transmit signals to the first computing device 160, and / or receive signals from the first computing device 160 via the network 150.

[0056] In some embodiments, the communication interface 110 may allow a plurality of second computing devices 170 to communicate with the processor 120 of the server 100 via the network 150, as shown in FIG. 2. In some embodiments, each of the plurality of second computing devices 170 may belong to each of a plurality of merchants 171 who may prepare an item, for example, food, for the order. In some embodiments, the communication interface 110 may transmit signals to the plurality of second computing devices 170, and / or receive signals from the plurality of second computing devices 170, via the network 150.

[0057] In some embodiments, the communication interface 110 may allow a plurality of third computing devices 180 to communicate with the processor 120 of the server 100 via the network 150, as shown in FIG. 2. In some embodiments, each of the plurality of third computing devices 180 may belong to each of a plurality of human annotators 181 who may receive a request to modify and / or correct published translations (for example, published translations of merchant names and item names), modify and / or correct the published translations, and provide the modified and / or corrected translations to the server 100. In some embodiments, the communication interface 110 may transmit signals to the plurality of third computing devices 180, and / or receive signals from the plurality of third computing devices 180, via the network 150.

[0058] The processor 120 may include, but is not limited to, a microprocessor, an analogue circuit, a digital circuit, a mixed-signal circuit, a logic circuit, an integrated circuit, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP),a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), or any combination thereof. Any other kind of implementation of the respective functions, which will be described below in further detail, may also be understood as the processor 120.

[0059] In some embodiments, the processor 120 may be connectable to the conununication interface 110. In some embodiments, the processor 120 may be arranged in data or signal communication with the communication interface 110 to transmit / receive the signals.

[0060] In some embodiments, a user interface of an on-demand service application provided by the processor 120 may display source texts (for example, source sentences) in a certain language (also referred to as a “first language” or a “source language”). In some embodiments, the source texts may include, but are not limited to, merchant names (for example, restaurant names), item names (for example, menu names, food names or cuisine names), service options, and features. For example, the first language may be a default language (also referred to as an “original language”) that the on-demand service application provides by default. In some embodiments, the processor 120 may provide a translation function to the user 161. Upon the user’s request to change the language, the processor 120 may translate the source texts from the first language to the changed language (also referred to as a “second language” or a “target language”) using a translation model, and publish the translations of the source texts in the second language via the on-demand service application. In some embodiments, the processor 120 may include the translation model. In some other embodiments, the translation model may be external to the server 100 and may communicate with the processor 120 via the communication interface 110 over the network 150. In some embodiments, the translation model may be a trained translation model. In some embodiments, the processor 120 may display the published translations on the user interface of the on-demand service application.

[0061] In some embodiments, the processor 120 may collect feedback data regarding the published translations. In some embodiments, the processor 120 may collect the feedback dataregarding the published translations via the communication interface 110. In some embodiments, the communication interface 110 may receive the feedback data regarding the published translations from the user 161 and / or at least one of the plurality of merchants 171. For example, the processor 120 may display the published translation of an item name (for example, a menu name), and the user 161 and / or the merchant 171a who serves the menu may provide feedback data regarding the published translation of the menu name (for example, a complaint about the published translation of the menu name) to the communication interface 110.

[0062] In some embodiments, the feedback data regarding the published translations may be gathered from one or more sources. For example, the one or more sources may include a merchant portal and the on-demand service application for the merchants 171 where the merchants 171 are able to review the published translations and edit them where necessary. As another example, the one or more sources may include a backend where operations team (Ops) (for example, a human operator) of the on-demand service platform provider may manage and edit the published translations. As another example, the one or more sources may include the on-demand service application for the user 161 where the user 161 may flag out inappropriate translations. The feedback data may then be stored in the memory 130, and used as input data for a next stage.

[0063] In some embodiments, the processor 120 may score each of the published translations based on at least a part of the collected feedback data. In some embodiments, a scoring and ranking model (for example, this model may be a small online model) may score each of the published translations based on the at least the part of the collected feedback data. In some embodiments, the processor 120 may score each of the published translations based on the at least the part of the collected feedback data selected from the collected feedback data.

[0064] In some embodiments, the processor 120 may score each of the collected feedback data, based on characteristics of the collected feedback data. In some embodiments, the scoring and ranking model may score and rank each of the collected feedback data. In some embodiments, the characteristics of the collected feedback data may include at least one of a relevance and reliability of the feedback data. In some embodiments, the processor 120 may determine whether the feedback data is reliable, for example, based on historical data of a person or an entity that left the feedback data. As an example, if the user 161 who left the feedback data is recorded as a “low-trust user” or a “blacklisted user” in the memory 130, the processor 120 may determine that the feedback data is unreliable, and assign a low score to the feedback data. For example, the processor 120 may assign the score which is less than a predetermined threshold (hereinafter, referred to as a “first predetermined threshold”) to the feedback data. As another example, the processor 120 may assign a different score to the feedback data depending on a trust level of the person or the entity that left the feedback data. In some other embodiments, the processor 120 may determine whether the feedback data is relevant to a quality of the corresponding published translation. As an example, if the feedback data does not relate to the quality of the corresponding published translation (for example, “the food does not taste good”), the processor 120 may determine that the feedback data is not relevant, and assign a low score to the feedback data. For example, the processor 120 may assign the score which is less than the first predetermined threshold to the feedback data.

[0065] In some embodiments, the processor 120 may select the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data. In some embodiments, the processor 120 may select the at least the part of the collected feedback data having the score which is equal to or greater than a predetermined threshold (hereinafter, referred to as a “second predetermined threshold”). For example, the first predetermined threshold may be the same as the second predetermined threshold. Asanother example, the first predetermined threshold may be different from (for example, less than) the second predetermined threshold. In some other embodiments, the processor 120 may rank the collected feedback data based on the score of each of the collected feedback data, and select the at least the part of the collected feedback data based on the ranking of each of the collected feedback data.

[0066] In some embodiments, after selecting the at least the part of the collected feedback data, the processor 120 may score each of the published translations based on the at least the part of the collected feedback data. In some embodiments, the processor 120 may score each of the published translations based on contents of the corresponding feedback data. In some embodiments, if it is determined that a certain published translation includes an error and is an inaccurate translation based on the contents of the corresponding feedback data, the processor 120 may assign a low score to the certain published translation. For example, the processor 120 may assign the score which is less than a predetermined threshold (hereinafter, referred to as a “third predetermined threshold”) to the certain published translation. In some other embodiments, the processor 120 may assign a different score to the certain published translation, based on a level of accuracy (for example, a level of the error) of the certain published translation. In some other embodiments, if it is determined that the certain published translation does not include an error and is an accurate translation based on the contents of the corresponding feedback data, the processor 120 may assign a high score to the certain published translation. For example, the processor 120 may assign the score which is equal to or greater than the third predetermined threshold to the certain published translation.

[0067] As described above, in some embodiments, the processor 120 may filter and process the collected feedback data, based on the score of each of the collected feedback data. In this manner, the other part of the collected feedback data having the low scores may be filtered out, and thus the at least the part of the collected feedback data which may be clean feedback datamay be selected and used to score each of the published translations. Advantageously, the scoring and ranking model (for example, this model may be a small online model) may be refined and improved by using the clean feedback data.

[0068] In some embodiments, the processor 120 may assign a part of the published translations having the score less than the third predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations. In some embodiments, the one or more annotators may include, but are not limited to, a teacher LLM (Large Language Model) and at least one of a plurality of human annotators 181 (for example, a human annotator 181a) (as described with reference to FIG. 2). In some embodiments, the teacher LLM is a large language model which may perform translations and provide instructions and / or examples relating to the translations to one or more student LLMs, so that the student LLMs may learn from the teacher LLM and perform the translations accordingly. In some embodiments, the processor 120 may include the teacher LLM. In some other embodiments, the teacher LLM may be external to the server 100 and may communicate with the processor 120 via the communication interface 110 over the network 150. As an example, the certain published translation with a low quality (for example, the inaccurate translation) may be assigned to the one or more annotators.

[0069] In some embodiments, the processor 120 may provide the part of the published translations having the score less than the third predetermined threshold to the human annotator 181a. As an example, the certain published translation with the low quality (for example, the inaccurate translation) may be provided to the human annotator 181a. In some embodiments, the processor 120 may provide the part of the published translations with the score of each of the part of the published translations to the human annotator 181a. In some embodiments, the processor 120 may request the human annotator 181a to modify and / or correct the part of the published translations. In some embodiments, the human annotator 181a may review the pailof the published translations (for example, with the score of each of the part of the published translations), and modify and / or correct the part of the published translations. In some embodiments, the processor 120 may generate the annotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator 181a.

[0070] In some embodiments, the processor 120 may provide the pail of the published translations having the score less than the third predetermined threshold to the teacher LLM. As an example, the certain published translation with the low quality (for example, the inaccurate translation) may be provided to the teacher LLM. In some embodiments, the processor 120 may provide the part of the published translations with the score of each of the part of the published translations to the teacher LLM. In some embodiments, the processor 120 may request the teacher LLM to translate the part of the published translations. In some embodiments, the teacher LLM may perform the translation for the part of the published translations. In some embodiments, the processor 120 may generate the annotated translations for the pail of the published translations based on the translation received from the teacher LLM.

[0071] In some embodiments, the processor 120 may retain the other part of the published translations having the score equal to or greater than the third predetermined threshold, for example, in the memory 130.

[0072] In some embodiments, the processor 120 may classify each of the published translations into either a first group of the published translations or a second group of the published translations based on the score of each of the published translations. In some embodiments, the processor 120 may check the score of the certain published translation of the published translations. In some embodiments, if the score of the certain published translation is less than the third predetermined threshold, the processor 120 may classify the certain publishedtranslation into the first group of the published translations. As an example, the certain published translation with the low quality (for example, the inaccurate translation) may be classified into the first group of the published translations. In some embodiments, if the score of the certain published translation is equal to or greater than the third predetermined threshold, the processor 120 may classify the certain published translation into the second group of the published translations. As an example, the certain published translation with the high quality (for example, the accurate translation) may be classified into the second group of the published translations.

[0073] In some embodiments, the processor 120 may assign the first group of the published translations to the one or more annotators, to generate the annotated translations for the first group of the published translations. In some embodiments, the processor 120 may provide the first group of the published translations (for example, with the score of each of the first group of the published translations) to the human annotator 181a. In some embodiments, the processor 120 may request the human annotator 181a to modify and / or correct the first group of the published translations. In some embodiments, the human annotator 181a may review the first group of the published translations (for example, with the score of each of the first group of the published translations), and modify and / or correct the first group of the published translations. In some embodiments, the processor 120 may generate the annotated translations for the first group of the published translations based on the modification and / or the correction received from the human annotator 181a. In some embodiments, the processor 120 may provide the first group of the published translations (for example, with the score of each of the first group of the published translations) to the teacher LLM. In some embodiments, the processor 120 may request the teacher LLM to translate the first group of the published translations. In some embodiments, the teacher LLM may perform the translation for the first group of the published translations. In some embodiments, the processor 120 may generate the annotatedtranslations for the first group of the published translations based on the translation received from the teacher LLM.

[0074] In some embodiments, the processor 120 may train the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the fourth predetermined threshold (for example, high scoring data). In some embodiments, the processor 120 may aggregate the annotated translations for the part of the published translations together with the other part of the published translations, and perform a sampling to filter good quality translation data pairs that may be used to train (for example, re-train) the translation model.

[0075] In some embodiments, the processor 120 may receive new source texts in the first language. In some embodiments, the processor 120 may translate the new source texts in the first language into the second language using the trained (for example, re-trained) translation model. In some embodiments, after training the translation model, the trained translation model may then be used to translate the new source texts. In some embodiments, at the same time, the good quality translation data pairs may be stored in the memory 130.

[0076] In some embodiments, the processor 120 may tune the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations. For example, when a sufficient size of the good quality translation data pairs is reached, the good quality translation data pairs may then be used to tune the teacher LLM. Advantageously, the teacher LLM, along with the one or more student LLMs, may also be improved by using the good quality translation data pairs.

[0077] In some embodiments, the processor 120 may remove at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed are translated using the trained translation model. For example, the at least one predetermined text component may include, but is not limited to,Unicode, special (irrelevant) characters, special spaces, and unnecessary symbols that may disrupt the translation process. Advantageously, the trained translation model may perform the translation for the new source texts which are clean and standardised as possible, and this may lead to more accurate translations.

[0078] In some embodiments, the processor 120 may determine if a new source text of the new source texts originally includes an item name in both the first language and the second language. In some embodiments, if the new source text originally includes the item name in both the first language and the second language, the processor 120 may present the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

[0079] In some embodiments, the processor 120 may break down the item name originally included in the new source text into a plurality of tokens. In some embodiments, the processor 120 may identify each of the plurality of tokens, to determine if the new source text originally includes the item name in both the first language and the second language.

[0080] For example, the item name of “4. Salmon + Chicken bentomay be originally included in the new source text. The processor 120 may break down the item name into the plurality of tokens, and identify a language of each token as follows: { ‘token_langs’: [(‘4’, ‘num’), (‘.’ ‘symbol’), (‘Salmon’, en), (“+’, ‘symbol’), (‘Chicken’, en), (‘Bento’, en),

[0081] In some embodiments, the processor 120 may classify each of the plurality of tokens into one of a plurality of language segment groups (for example, a first language segment group and a second language segment group) based on a language of each of the plurality of tokens.

[0082] For example, the processor 120 may classify tokens of “Salmon + Chicken bento” into the first language segment group (for example, English segment group), and tokens of “4.” andinto the second language segment group (for example, Chinese segmentgroup), as follows: [(‘4,’, zh), (Salmon + ‘Chicken bento’, en),

[0083] In some embodiments, the processor 120 may translate the first language segment group into the second language. Tn some embodiments, the processor 120 may determine if the first language segment group translated into the second language is similar to the second language segment group. In some embodiments, if the first language segment group translated into the second language is similar to the second language segment group, the processor 120 may retain the second language segment group. In some embodiments, the processor 120 may present the second language segment group, without presenting the first language segment group translated into the second language.

[0084] For example, the processor 120 may determine that the Chinese translation of the English segment group of “Salmon + Chicken bento” is the same as or similar to the Chinese segment group ofIn this case, the processor 120 may present theChinese segment group ofwithout presenting the Chinese translation of the English segment group of “Salmon + Chicken bento”, on the user interface in which a menu language is changed to Chinese.

[0085] FIG. 4 illustrates a flowchart for a method 300 for facilitating providing translations according to various embodiments. According to various embodiments, the method 300 for facilitating providing the translations may be provided.

[0086] In some embodiments, the method 300 may include a step 301 of collecting feedback data regarding published translations in which source texts are translated from a first language into a second language by a translation model.

[0087] In some embodiments, the method 300 may include a step 302 of scoring each of the published translations based on at least a part of the collected feedback data.

[0088] In some embodiments, the method 300 may include a step 303 of assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations.

[0089] In some embodiments, the method 300 may include a step 304 of training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

[0090] In some embodiments, the method 300 may include a step 305 of translating new source texts in the first language into the second language using the trained translation model.

[0091] FIG. 5 illustrates a data flow diagram of a server 100 for facilitating providing translations according to various embodiments.

[0092] As shown in FIG. 5, the server 100 may include three components including a feedback component, a QC (quality check) component, and a translation component.

[0093] In some embodiments, in the feedback component, feedback data regarding published translations may be gathered from one or more sources. For example, the one or more sources may include a merchant portal 201 and an on-demand service application for merchants 171 (also referred to as a “merchant app 202”) where the merchants 171 are able to review the published translations and edit them where necessary. As another example, the one or more sources may include a backend 203 where operations team (Ops) of an on-demand service platform provider may manage and edit the published translations. As another example, the one or more sources may include the on-demand service application for a user 161 (also referred to as a “consumer app 204”) where the user 161 may flag out inappropriate translations. These feedback data may then be stored in a database 205, and used as input data for a next stage. For example, the database 205 may be included in the memory 130 (as described with reference to FIG. 3).

[0094] In some embodiments, in the QC component, the collected feedback data may go through a filter (also referred to as a “sieve”) and get processed. Advantageously, this stage may allow to refine and improve a scoring and ranking model (for example, this may be a small online model) 206 with the filtered (also referred to as “selected”) feedback data. In a first stage, the collected feedback data may be scored and ranked, so that clean feedback data may be selected subsequently. In some embodiments, scoring and ranking the collected feedback data may be performed by the scoring and ranking model 206 that may assign scores to the published translations. With the scores assigned to the collected feedback data, the published translations may be scored. The published translations having low scores (also referred to as “at least a part of the published translations” or “the published translations having low confidence”) may go through a data sampling process 207 and then be assigned to a human annotator 208 and / or a teacher LLM 209. In some embodiments, in the data sampling process 207, the server 100 may filter out non-ideal published translations from the published translations having the low scores, based on one or more predetermined rules as follows. For example, the server 100 may filter out those with unmatched numbers translation between a first language (i.e. a source language) and a second language (i.e. a target language). As another example, the server 100 may use predetermined token ratio rules to filter out a translation sentence which is much shorter or much longer than a name in the source language (for example, based on whether the difference between the number of tokens in the source language and the number of tokens in the target language is higher than a predetermined number). As another example, the server 100 may filter out translations which are repetitive. Thereafter, the server 100 may assign the published translations having the low scores, excluding the nonideal published translations, to the human annotator 208 and / or the teacher LLM 209. The published translations having high scores (also referred to as “the other part of the published translations” or “the published translations having high confidence”) may be retained fortraining a translation model later on. In a next step, the human annotator 208 may review the published translations having the low scores, and modify and / or correct the published translations having the low scores, so that they are accurate. Similarly, the published translations having the low scores may be translated by the teacher LLM 209. The published translations having the high scores may be retained. In this manner, annotated translations may be generated. For example, the annotated translations may include the the modification and / or the correction received from the human annotator 208 and / or the translation received from the teacher LLM 209. In a next stage, the annotated translations together with the published translations having high scores may be aggregated, and further go through a sampling process to filter good quality translation data pairs that may be used to train (for example, re -train) the translation model 210. In some embodiments, in the sampling process, the server 100 may filter out non-ideal translation data pairs from the aggregated annotated translations and the published translations having high scores, based on one or more predetermined rules as follows. For example, the server 100 may filter out those with unmatched numbers translation between the source language and the target language. As another example, the server 100 may use predetermined token ratio rules to filter out a translation sentence which is much shorter or much longer than a name in the source language (for example, based on whether the difference between the number of tokens in the source language and the number of tokens in the target language is higher than a predetermined number). As another example, the server 100 may filter out translations which are repetitive. In this manner, the server 100 may filter out the nonideal translation data pairs, and obtain the good quality translation data pairs that may be used to train the translation model. Finally, after the training, the trained (for example, re-trained) translation model may then be used in a next component. At the same time, all the good quality translation data pairs may be stored in the memory 130, for example, the database 205. Whena sufficient size of the good quality translation data pairs is reached, the good quality translation data pairs may then be used to tunc the teacher LLM 209.

[0095] In some embodiments, the translation component may be a component responsible for translating new source texts, for example, food menu names, into multiple languages. The trained translation model may translate the menu names 211, and publish the translations of the menu names 212. The trained translation model may utilise a transformer model architecture which may include multi-headed encoders and decoders, and translate from a first language (also referred to as a “source language”) into a second language (also referred to as a “target language”). However, there may be a problem where a new source text, for example, a new source sentence, may include characters which are not from the source language (for example, including words from other languages, other symbols and / or cmojis). To solve the problem, the server 100 may provide an augmented pipeline for a translation process including several stages (as will be described with reference to FIG. 6).

[0096] FIG. 6 illustrates a data flow diagram of a translation component of FIG. 5 according to various embodiments.

[0097] As shown in FIG. 6, the translation component may perform pre-processing, model inference, and post-processing.

[0098] In some embodiments, the pre-processing stage may involve cleaning new source texts, for example, an item name 401 , for efficient processing. At least one predetermined text component, including, but not limited to, Unicode 402, special characters 403 and special spaces 404, that may disrupt a translation process, may be removed from the item name, so that the item name with the at least one predetermined text component removed is translated using a trained translation model. Advantageously, the trained translation model may perform the translation for the new source texts which are clean and standardised as possible, and this may lead to more accurate translations.

[0099] In some embodiments, during the model inference stage, batch inference 405 using the trained translation model may be implemented. Multiple menu names may be translated simultaneously, instead of individually, to make the translation process more computationally efficient. Machine learning models trained on one or more extensive bilingual item (for example, menu) databases as well as QC data may be utilised, to provide accurate translations.

[0100] In some embodiments, the post-processing stage may include at least one of following steps 406, 407, 408, 409, to provide the translation of the item name 410:• Bilingual Treatment 406: If the item name is bilingual (for example, “Boiled Drunken Soup Prawnsa processor 120 may acknowledge and retain the item’s original name and combine the original name with its translated name to provide a user 161 with comprehensive information. More details about the bilingual treatment will be described with reference to FIG. 7.• Ad-hoc Map Inference 407 (also referred to as “Ad-hoc Map Replacement”): The technique may involve creating a dictionary-like mapping of “item names” to “correct translations”. When the item name matches a key in the map, the processor 120 may readily output the stored “correct translation”. This technique may serve as an effective solution for quickly addressing faulty translations without having to re-train the entire translation model.• Special Words Replacement 408: As different cultural contexts may associate different meanings with different words, the processor 120 may handle potentially sensitive and / or controversial translations. A rule -based process may be used to handle such potentially sensitive and / or controversial translations. For example, in a certain country, “pork” may be a sensitive word due to religious considerations. The process may involve a specific rule of “if “babi” is not present in the item name but “pork” appearsin the translation, the word “pork” in the translation is deleted”. This process may achieve a respect for regional preferences and sentiments.• Profanity Filtering 409: The profanity filtering model may incorporate mechanisms to filter out any profanity and / or inappropriate language in the translated texts. This step may improve appropriateness and user-friendliness of the translated output.

[0101] FIG. 7 illustrates a data flow diagram of a bilingual treatment of FIG. 6 according to various embodiments. FIG . 7 illustrates an exemplary data flow diagram, where an item name 501 includes both English and Chinese characters.

[0102] In some embodiments, the bilingual treatment may be a crucial process in a translation pipeline provided by a server 100, by addressing a common occurrence of item names appearing in mixed languages, for example the item name including both English and Chinese words. This process may allow that an original source language segment is retained in the translations and redundant translations are reduced. The procedure is as follows:• Tokenisation: The item name may first be broken down into smaller tokens or units. This step may allow each word or term in the item name to be analysed and processed individually.• Language Detection 502: The language of each token may be identified. This recognition step may allow that every language within a mixed-language item name is properly noted. For example, the item name of “4. Salmon + Chicken bentomay be originally included in a new source text. The item name may be broken down into a plurality of tokens, and language of each token may be identified as follows: { ‘token_langs’: [(‘4’, ‘num’), (‘.’ ‘symbol’), (‘Salmon’, en), (‘+’, ‘symbol’), (‘Chicken’, en), (‘Bento’, en),• Language Segment 503 (also referred to as “Segmentation”): Tokens of the same language may be grouped together to form a single segment. This step may prepare segments for the following translation process. For example, tokens of “Salmon + Chicken bento” may be classified into a first language segment group (for example, English segment group), and tokens of “4.” and may be classifiedinto a second language segment group (for example, Chinese segment group), as follows: [(‘4,’, zh), (Salmon + ‘Chicken bento’, en),• Translation of Source Language Segment: Only the segment identified as being in the source language (also referred to as a “source language segment”) may be translated.• Segment Checking 504 (also referred to as “Retention of Source Language Segment”):If the translation of the source language segment is noticeably similar to the original source language segment, the original source language segment may be retained. This step may reduce redundancy in the translated item name. For example, it is determined that the Chinese translation of the English segment group of “Salmon + Chicken bento” is the same as or similar to the Chinese segment group of In thiscase, the Chinese segment group ofis presented, without presenting the Chinese translation of the English segment group of “Salmon + Chicken bento”, on a user interface in which a menu language is changed to Chinese.

[0103] In some embodiments, by following the above mentioned procedure, the bilingual treatment may effectively handle mixed-language item names, by preserving an original source language segment and improving accuracy and readability of translated item names. In some embodiments, using the above mentioned pipeline, all the item names may be translated and then published online automatically through an offline task. Finally, to complete a feedbackQC loop, the published translations may be notified to merchants 171 and displayed to a user161 to gather a next round of feedback data.

[0104] As described above, the server 100 according to various embodiments may utilise several feedback channels (from merchants 171, Ops, the user 161, etc.) on the published translations. The server 100 may incorporate the feedback data, and include a QC component into the hanslation model refinement process, in order to learn from edge cases and / or adapt to changes in the item names, for example, menu names. Further, the server 100 may make use of an LLM as a teacher model, to guide and train one or more student models for an online use. In addition, the server 100 may propose an augmented translation pipeline as a flexible way, to cater for source texts, for example, source sentences, including symbols, emojis and words not from the source language.

[0105] Advantageously, the server 100 may adapt and improve translation quality over time, and may also be effective in optimising resources. By focusing on translation data that a conventional algorithm is unsure about, the server 100 may allow to prevent wastage of resources on high-confidence translations.

[0106] In the following, various examples of this disclosure are illustrated:

[0107] Example 1 is a server for facilitating providing translations, the server comprising: a memory configured to store instructions; and a processor for executing the stored instructions and configured to: receive new source texts in a first language; and translate the new source texts in the first language into a second language using a trained translation model, wherein the trained translation model is generated by collecting feedback data regarding published translations in which source texts arc translated from the first language into the second language by a translation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotatedtranslations for the part of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

[0108] Example 2 is the server of Example 1, wherein the processor is further configured to: classify each of the published translations into either a first group of the published translations or a second group of the published translations based on the score of each of the published translations; and assign the first group of the published translations to the one or more annotators, to generate the annotated translations for the first group of the published translations.

[0109] Example 3 is the server of Example 2, wherein the processor is further configured to: check the score of a certain published translation of the published translations; and if the score of the certain published translation is less than the predetermined threshold, classify the certain published translation into the first group of the published translations; and if the score of the certain published translation is equal to or greater than the predetermined threshold, classify the certain published translation into the second group of the published translations.

[0110] Example 4 is the server of any one of Examples 1 to 3, wherein the processor is further configured to: score the collected feedback data, based on characteristics of the collected feedback data; and select the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data.

[0111] Example 5 is the server of any one of Examples 1 to 4, wherein the one or more annotators include a human annotator and a teacher LLM (Large Language Model).

[0112] Example 6 is the server of Example 5, wherein the processor is further configured to: provide the part of the published translations to the human annotator; request the human annotator to modify and / or correct the part of the published translations; and generate theannotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator.

[0113] Example 7 is the server of Example 5 or Example 6, wherein the processor is further configured to: provide the part of the published translations to the teacher LLM; request the teacher LLM to translate the part of the published translations; and generate the annotated translations for the part of the published translations based on the translation received from the teacher LLM.

[0114] Example 8 is the server of any one of Examples 5 to 7, wherein the processor is further configured to: tune the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations.

[0115] Example 9 is the server of any one of Examples 1 to 8, wherein the processor is further configured to: remove at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed are translated using the trained translation model.

[0116] Example 10 is the server of any one of Examples 1 to 9, wherein the processor is further configured to: determine if a new source text of the new source texts originally includes an item name in both the first language and the second language; and if the new source text originally includes the item name in both the first language and the second language, present the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

[0117] Example 11 is a method for facilitating providing translations, the method comprising: receiving new source texts in a first language; and translating the new source texts in the first language into a second language using a trained translation model, wherein the trained translation model is generated by collecting feedback data regarding published translations in which source texts are translated from the first language into the second language by atranslation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

[0118] Example 12 is the method of Example 11, further comprising: classifying each of the published translations into either a first group of the published translations or a second group of the published translations based on the score of each of the published translations; and assigning the first group of the published translations to the one or more annotators, to generate the annotated translations for the first group of the published translations.

[0119] Example 13 is the method of Example 12, further comprising: checking the score of a certain published translation of the published translations; if the score of the certain published translation is less than the predetermined threshold, classifying the certain published translation into the first group of the published translations; and if the score of the certain published translation is equal to or greater than the predetermined threshold, classifying the certain published translation into the second group of the published translations.

[0120] Example 14 is the method of any one of Examples 11 to 13, further comprising: scoring the collected feedback data, based on characteristics of the collected feedback data; and selecting the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data.

[0121] Example 15 is the method of any one of Examples 11 to 14, wherein the one or more annotators include a human annotator and a teacher LLM (Large Language Model).

[0122] Example 16 is the method of Example 15, further comprising: providing the part of the published translations to the human annotator; requesting the human annotator to modifyand / or correct the part of the published translations; and generating the annotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator.

[0123] Example 17 is the method of Example 15 or Example 16, further comprising: providing the part of the published translations to the teacher LLM; requesting the teacher LLM to translate the part of the published translations; and generating the annotated translations for the part of the published translations based on the translation received from the teacher LLM.

[0124] Example 18 is the method of any one of Examples 15 to 17, further comprising: tuning the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations.

[0125] Example 19 is the method of any one of Examples 11 to 18, further comprising: removing at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed are translated using the trained translation model.

[0126] Example 20 is the method of any one of Examples 11 to 19, further comprising: determining if a new source text of the new source texts originally includes an item name in both the first language and the second language; and if the new source text originally includes the item name in both the first language and the second language, presenting the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

[0127] While the disclosure has been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is thus indicated bythe appended claims and all changes which come within the meaning and range of equivalency of the claims arc therefore intended to be embraced.

Claims

CLAIMS1. A server for facilitating providing translations, the server comprising: a memory configured to store instructions; and a processor for executing the stored instructions and configured to: receive new source texts in a first language; and translate the new source texts in the first language into a second language using a trained translation model, wherein the trained translation model is generated by: collecting feedback data regarding published translations in which source texts are translated from the first language into the second language by a translation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the pail of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

2. The server according to claim 1, wherein the processor is further configured to: classify each of the published translations into cither a first group of the published translations or a second group of the published translations based on the score of each of the published translations; andassign the first group of the pubiished translations to the one or more annotators, to generate the annotated translations for the first group of the published translations.

3. The server according to claim 2, wherein the processor is further configured to: check the score of a certain published translation of the published translations; and if the score of the certain published translation is less than the predetermined threshold, classify the certain published translation into the first group of the published translations; and if the score of the certain published translation is equal to or greater than the predetermined threshold, classify the certain published translation into the second group of the published translations.

4. The server according to claim 1, wherein the processor is further configured to: score the collected feedback data, based on characteristics of the collected feedback data; and select the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data.

5. The server according to claim 1, wherein the one or more annotators include a human annotator and a teacher LLM (Large Language Model).

6. The server according to claim 5, wherein the processor is further configured to: provide the part of the published translations to the human annotator; request the human annotator to modify and / or correct the part of the published translations; andgenerate the annotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator.

7. The server according to claim 5, wherein the processor is further configured to: provide the part of the published translations to the teacher LLM; request the teacher LLM to translate the part of the published translations; and generate the annotated translations for the part of the published translations based on the translation received from the teacher LLM.

8. The server according to claim 5, wherein the processor is further configured to: tunc the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations.

9. The server according to claim 1, wherein the processor is further configured to: remove at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed are translated using the trained translation model.

10. The server according to claim 1 , wherein the processor i further configured to: determine if a new source text of the new source texts originally includes an item name in both the first language and the second language; and if the new source text originally includes the item name in both the first language and the second language, present the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

11. A method for facilitating providing translations, the method comprising: receiving new source texts in a first language; and translating the new source texts in the first language into a second language using a trained translation model, wherein the trained translation model is generated by: collecting feedback data regarding published translations in which source texts are translated from the first language into the second language by a translation model; scoring each of the published translations based on at least a part of the collected feedback data; assigning a part of the published translations having the score less than a predetermined threshold to one or more annotators, to generate annotated translations for the part of the published translations; and training the translation model using the annotated translations for the part of the published translations and the other part of the published translations having the score equal to or greater than the predetermined threshold.

12. The method according to claim 11, further comprising: classifying each of the published translations into either a first group of the published translations or a second group of the published translations based on the score of each of the published translations; and assigning the first group of the published translations to the one or more annotators, to generate the annotated translations for the first group of the published translations.

13. The method according to claim 12, further comprising: checking the score of a certain published translation of the published translations;if the score of the certain published translation is less than the predetermined threshold, classifying the certain published translation into the first group of the published translations; and if the score of the certain published translation is equal to or greater than the predetermined threshold, classifying the certain published translation into the second group of the published translations.

14. The method according to claim 11, further comprising: scoring the collected feedback data, based on characteristics of the collected feedback data; and selecting the at least the part of the collected feedback data from the collected feedback data, based on the score of each of the collected feedback data.

15. The method according to claim 11, wherein the one or more annotators include a human annotator and a teacher LLM (Large Language Model).

16. The method according to claim 15, further comprising: providing the part of the published translations to the human annotator; requesting the human annotator to modify and / or correct the part of the published translations; and generating the annotated translations for the part of the published translations based on the modification and / or the correction received from the human annotator.

17. The method according to claim 15, further comprising: providing the part of the published translations to the teacher LLM;requesting the teacher LLM to translate the part of the published translations; and generating the annotated translations for the part of the published translations based on the translation received from the teacher LLM.

18. The method according to claim 15, further comprising: tuning the teacher LLM using the annotated translations for the part of the published translations and the other part of the published translations.

19. The method according to claim 11, further comprising: removing at least one predetermined text component from the new source texts, so that the new source texts with the at least one predetermined text component removed arc translated using the trained translation model.

20. The method according to claim 11, further comprising: determining if a new source text of the new source texts originally includes an item name in both the first language and the second language; and if the new source text originally includes the item name in both the first language and the second language, presenting the item name in the second language originally included in the new source text, without presenting the item name translated into the second language.

Images