TRANSFERRING TEXT FORMATTING TO TRANSLATED CONTENT USING NEURAL NETWORKS

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for transferring style formatting to translated texts using neural networks. In particular, in some embodiments, the disclosed systems receive an input text string, wherein the input text string includes a style formatting element. Furthermore, in some embodiments, the disclosed systems generate a modified input text string from the input text string, wherein the modified input text string includes an encoded tag that identifies the style formatting element. Additionally, in some embodiments, the disclosed systems generate a translated text string from the modified input text string using a neural machine translation model.Furthermore, in some embodiments, the disclosed systems apply the style formatting element to a word of the translated text string based on the encoded tag of the modified input text string.

Description

BACKGROUND

[0001] In recent years, there have been developments in hardware and software platforms that implement translation models for translating texts from one language to another. For example, state-of-the-art translation systems such as neural machine translation and large language models are capable of generating translated texts and, in some cases, even applying stylistic devices (such as emphasis, italics, underlining, etc.) to convey the stylistic features originally used in the source text. Despite these developments, state-of-the-art systems exhibit a number of technical shortcomings, including inaccuracies in generating translated texts with stylistic devices. BRIEF SUMMARY OF THE INVENTION

[0002] Embodiments of the present disclosure offer advantages and / or solve one or more prior art problems by means of systems, non-transitory computer-readable media, and methods for generating formatted translated text content using machine learning models. In some embodiments, for example, the disclosed systems determine a stylization that is applied to a text string in a particular language. From the formatted text string, the disclosed systems, in some implementations, generate a translated text string in another language using a neural transformer network. Furthermore, in some embodiments, the disclosed systems determine attention head values ​​generated by the neural transformer network that indicate relationships between words in the source text and words in the translated text.Thus, in some embodiments, the disclosed systems use the attention heads to determine which translated words of the translated text string are to be formatted.

[0003] Furthermore, in some implementations, the disclosed systems employ an alternative technique for style transmission to translated text without determining attention scores. For example, in some embodiments, the disclosed systems use a neural machine translation model and / or a large language model to translate text and apply style changes from input text to the translated text. For example, in some implementations, the disclosed systems modify input text with coded tags or delimiters to mark the beginning and end of style formatting in the input text.Furthermore, in some embodiments, the disclosed systems process the modified text (for example, the text including the tags / separators) using the neural machine translation model or the large language model to generate a translated text that preserves the encoded tags or separators. In addition, in some embodiments, the disclosed systems apply style formatting elements to the translated text based on (for example, between) the encoded tags or separators.

[0004] In some implementations, the disclosed systems use a hybrid model that employs the neural machine translation model to translate input text and the large language model to determine which translated words in the translated text should be formatted. For example, in some embodiments, the disclosed systems use unigram mappings of formatted words in the input text to identify which translated words in the translated text should be formatted. Furthermore, in some implementations, the disclosed systems apply a style of the formatted input words to the translated words to produce a formatted translated text.

[0005] The following description sets out additional features and advantages of one or more embodiments of the disclosed methods, non-transitory computer-readable media, and systems. In some cases, such features and advantages are obvious to a person skilled in the art or can be learned by carrying out the disclosed embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The detailed description includes one or more embodiments with additional specifications and details using the accompanying drawings, as briefly described below. Fig. Figure 1 shows a diagram of an environment in which a system for preserving formatting operates according to one or more embodiments. Fig. Figure 2 shows an overview of the system for preserving formatting, which, according to one or more embodiments, generates a formatted translated text string from an input text string. Fig. Figure 3 shows the system for preserving formatting, which, according to one or more embodiments, generates an attention head matrix. Fig. Figures 4A-4B show the formatting preservation system that extracts a style from an input text string and uses attention heads to apply the style to a translated text string according to one or more embodiments. Fig. Figures 5A-5B illustrate the formatting preservation system that provides a formatted input text string and a formatted translated text string for display via a graphical user interface according to one or more embodiments. Fig. Figure 6 shows an overview of alternative techniques for preserving the text style according to one or more embodiments. Fig. Figure 7 illustrates the formatting preservation system, which uses a neural machine translation model to generate translated text with coded tags in order to format the translated text according to one or more embodiments. Fig. Figure 8 illustrates the formatting preservation system, which uses a large language model to generate translated text with delimiters in order to format the translated text according to one or more embodiments. Fig. Figure 9 shows the system for preserving formatting, which uses a neural machine translation model to generate a translated text from an input text and a large language model to determine translated words for formatting according to one or more embodiments. Fig. Figures 10A-10B illustrate the formatting preservation system, which provides an input graphic with formatted text and a translated graphic with formatted translated text for display via a graphical user interface according to one or more embodiments. Fig. Figure 11 shows a diagram of an exemplary architecture of the system for preserving formatting according to one or more embodiments. Fig. Figure 12 shows a flowchart of a series of operations for preserving styles of translated texts using attention head values ​​according to one or more embodiments. Fig. Figure 13 shows a flowchart of a series of operations for preserving styles of translated texts using neural networks according to one or more embodiments. Fig. Figure 14 shows a block diagram of an exemplary computer system for implementing one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0007] The present disclosure describes one or more embodiments of a formatting preservation system that generates formatted translated text content using machine learning models. For example, in some embodiments, the formatting preservation system receives an input text string in a first language and extracts a style formatting element that specifies or defines a style applied to the input text string. Furthermore, in some implementations, the formatting preservation system generates a translated text string in a second language, different from the first, using a neural transformer network. In addition, in some embodiments, the formatting preservation system determines attention head values ​​generated by the neural transformer network during the translation process.In some embodiments, the formatting preservation system further generates an attention head matrix from the attention head values, where the matrix defines relationships between words in the source text and words in the translated text. Accordingly, in some embodiments, the formatting preservation system uses the attention head matrix to determine which translated words in the translated text string should be formatted. In some implementations, the formatting preservation system generates a formatted translated text string by applying the style formatting element to the translated words.

[0008] Furthermore, in some implementations, the formatting preservation system uses an alternative technique for transferring style to translated text without generating an attentional head matrix. For example, in some embodiments, the formatting preservation system uses a neural machine translation model and / or a large language model to translate text and apply formatting from input text to the translated text. In some implementations, the formatting preservation system modifies input text with coded tags or delimiters to identify style formatting elements in the input text.Furthermore, in some embodiments, the formatting preservation system processes the modified text using the neural machine translation model or the large language model to produce a translated text that preserves the coded tags or delimiters. Additionally, in some embodiments, the formatting preservation system applies style formatting elements to the translated text based on the coded tags or delimiters.

[0009] In some implementations, the formatting preservation system uses a hybrid model that employs the neural machine translation model to translate input text and the large language model to determine which translated words in the translated text need formatting. For example, in some implementations, the formatting preservation system uses unigram mappings of formatted words in the input text to identify the translated words in the translated text that need formatting. Furthermore, in some implementations, the formatting preservation system applies a style of the formatted input words to the translated words to produce a formatted translated text.

[0010] In some implementations, the formatting preservation system facilitates the creation of designs in multiple languages ​​(for example, one version in English, one in Spanish, etc.). For instance, design content creators often choose to expand a graphic design from its original language to one or more other languages. This allows the target audience of a design to encompass linguistic variations within a country, a field, or even globally. The globalization of graphic designs (for example, for marketing, magazines, etc.) is becoming increasingly important for communicating with a broad audience. To achieve this, text content in graphic designs must be accurately translated, and the text formatting must be maintained so that it blends seamlessly into the overall design.Maintaining text formatting often requires a high degree of accuracy in word alignment between the original text and the translated text.

[0011] The formatting preservation system offers multilingual capabilities for converting a design from one language to another. Furthermore, in some implementations, the formatting preservation system generates translated content with preserved formatting in multiple languages ​​beyond the source language. For example, the formatting preservation system receives a graphic containing English text. It then generates translated graphics (for example, in German, French, Italian, etc.) that retain the graphic and stylistic elements of the original graphic. For instance, the corresponding German, French, and Italian texts in the translated graphics have styles that match those of the original English text.

[0012] Although prior art systems generate translated text from an input text string, such systems exhibit several problems regarding the accuracy of stylistic formatting for the translated text. For example, some prior art systems apply text styles to translated text content that do not reflect the formatting of the input text. In some cases, prior art systems apply stylizations to incorrect sections of a translated text. In some cases, prior art systems apply stylizations to incorrect words, particularly when different sections of the words to be formatted should be separated or subdivided by unformatted words in the translated text.

[0013] Beyond inaccurate formatting, certain state-of-the-art systems generate machine translations into plain text without any stylistic formatting. In these cases, the user must manually format the text after the machine translation. Due to the inaccuracy of such systems, they are also inefficient and often require too many steps (e.g., input, selections, clicks, etc.) to format the translated text.

[0014] At least in part due to their inaccuracies and inefficient formatting, prior art systems often waste computing resources. For example, while some prior art neural transformer networks generate attention head values ​​during machine translation, these attention head values ​​are not used beyond the internal processes of the machine translation. Thus, some prior art systems use extensive computations to generate the attention head values ​​but ignore them for other computational tasks, such as determining formatted words within an input text string.

[0015] The formatting preservation system offers several technical advantages over prior art systems. For example, it improves accuracy by using attention heads of a neural transformer network to determine which translated words in a translated text string correlate with formatted input words in the input text string. By using an attention head matrix, the formatting preservation system generates, for example, a formatted translated text that accurately reflects the formatting of the input text string.Furthermore, the formatting preservation system accurately translates texts and determines precise formatting for the translated text that matches the formatting of the input text by using a neural machine translation model to generate the translated text and a large language model to determine which translated words to format.

[0016] In particular, using an attention-head matrix, a neural machine translation model, and / or a large language model, the formatting preservation system formats the translated text string to match the formatting of the input text string, even if the translated text string has a different number of words and / or a different word order than the input text string. In some implementations, the formatting preservation system formats the translated text string exactly to match the formatting of the input text string, including text strings with multiple types of formatting in multiple places (for example, multiple formatted words with different styles or formatting in different parts of a sentence).Furthermore, in some implementations, the formatting preservation system accurately formats translated text strings that have a different number of words than the corresponding input text strings (for example, a translated German sentence with a compound word that corresponds to several input words in an English input sentence). Additionally, in some implementations, the formatting preservation system accurately formats translated text strings even when the translated text uses a different tokenization strategy than the input text (for example, when translating from a language with word spaces, such as English, to a language without word spaces, such as Chinese).

[0017] Furthermore, the formatting preservation system provides a user interface that requires less user input compared to prior art systems, for example, less user interaction with multiple different subsystems. In some embodiments, the formatting preservation system performs both machine translation and formatting based on a single input of a formatted input text string. As another example, in some embodiments, the formatting preservation system integrates a neural machine translation model and a large language model to produce a formatted translated text based on a formatted input text without requiring the user to interact with multiple computer applications.In comparison to previous systems that only produce unformatted translations, the formatting preservation system thus reduces the interactions required to format the translated text (for example, to zero interactions with the translated text).

[0018] Furthermore, in some implementations, the formatting preservation system increases computational efficiency by utilizing attention-head values ​​from a neural transformer network for additional functions beyond machine translation. For example, the formatting preservation system uses attention-head values ​​to identify translated words in a translated text string that correlate with formatted words in an input text string, thus making better use of the computational resources required to generate the attention-head values. Accordingly, compared to previous systems that ignored attention-head values ​​for computational tasks outside of machine translation, the formatting preservation system uses computational resources efficiently by extracting relational information for the input text string and the translated text string from the attention-head matrix.More precisely, the system for preserving formatting uses the attention head values ​​generated by the neural transformer network to determine this relational information, thus avoiding the need to redetermine relational information in other ways, which would otherwise require further calculations.

[0019] Further details are provided below with illustrative figures that depict exemplary embodiments and implementations of a formatting preservation system. For example, [Figure 1] shows Fig. 1. A system 100 (or an environment) in which a formatting preservation system 102 operates according to one or more embodiments. As shown, the system 100 comprises one or more servers 106, a network 112, and a client device 108. As also shown, the servers 106 and the client device 108 communicate with each other via the network 112.

[0020] As in Fig. As shown in Figure 1, the server 106 comprises a digital media management system 104, which further comprises the formatting preservation system 102. In some embodiments, the formatting preservation system 102 uses one or more machine learning models (for example, a translation model 114 and / or a formatting preservation model 116) to translate text and / or preserve the style formatting for the text. For example, in some implementations, the formatting preservation system 102 uses the machine learning models to generate a translated text string from an input text string and to apply a style formatting element to one or more translated words of the translated text string. In some embodiments, the server(s) 106 include, among other things, a computer (as shown below with reference to Figure 1). Fig. 14 explained).

[0021] A machine learning model comprises a computer representation that can be tuned (for example, trained) based on inputs to approximate unknown functions used to generate corresponding outputs. Specifically, a machine learning model, in one or more embodiments, is a computer-implemented model that uses algorithms to learn from known data and make predictions by analyzing the known data to learn to generate outputs that reflect patterns and attributes of the known data.In some cases, a machine learning model may include, but is not limited to, a neural network (for example, a convolutional network, a recurrent neural network, or another deep learning network), a decision tree (for example, a gradient-enhanced decision tree), support vector learning, Bayesian networks, a transformer-based model, a diffusion model, or a combination thereof.

[0022] Similarly, a neural network comprises a machine learning model that can be trained and / or tuned based on inputs to determine classifications and / or evaluations or to approximate unknown functions. In some cases, a neural network includes, for example, a model of interconnected artificial neurons (for example, organized in layers) that communicate and learn to approximate complex functions and generate outputs based on inputs provided to the neural network. In other cases, a neural network refers to an algorithm (or set of algorithms) that implements deep learning techniques to model high-level abstractions in data.A neural network comprises various layers, such as an input layer, one or more hidden layers, and an output layer, each performing tasks for data processing by the neural network. Examples of neural networks include deep neural networks, convolutional neural networks, diffusion neural networks, recurrent neural networks (such as LSTMs), graph neural networks, transformer neural networks, and generative adversarial neural networks.

[0023] A neural transformer network comprises a neural network that utilizes attentional mechanisms to generate embeddings for sequential data. Specifically, a neural transformer network includes a self-attentional mechanism (for example, attentional heads) to generate representations (or embeddings) that account for extensive dependencies and contextual information between different sections of data within sequential data (for example, via tokens).

[0024] A neural machine translation model comprises one or more neural networks, each consisting of multiple layers of interconnected nodes, to process input text and generate translations of that input text. In some embodiments, a neural machine translation model includes an encoder-decoder architecture that transforms input text into a dense, high-dimensional context vector and then transforms that context vector into a translated text in a target language. In some implementations, a neural machine translation model includes a neural transformer network that uses an attention mechanism to focus on multiple parts of an input text when generating each word of the translated text. Furthermore, in some embodiments, a neural machine translation model includes a general-purpose neural network for machine translation.In some embodiments, a neural machine translation model includes a special neural network that is tailored to a specific machine translation application (for example, for a specific language, a specific format of the source text, etc.).

[0025] In some cases, the formatting preservation system 102 receives a request (for example, from the client device 108) to translate input text and transfer the style from the input text to the translated text. For example, the formatting preservation system 102 receives the input text and a request to translate the input text and retain the formatting of the input text. Some embodiments of servers 106 perform a variety of functions via the digital media management system 104 on the servers 106. For illustration, the server or...The servers 106 (via the formatting preservation system 102 on the digital media management system 104) perform functions such as, but not limited to, extracting a style formatting element from an input text string, generating a translated text string from the input text string, generating an attention head matrix for words in the input text string, and generating a formatted translated text string. In some embodiments, the servers 106 use the translation model 114 and / or the formatting preservation model 116 to generate formatted translated text strings. In some embodiments, the servers 106 train the translation model 114 and / or the formatting preservation model 116.

[0026] Furthermore, the system includes 100, as in Fig. Figure 1 shows the client device 108. In some embodiments, the client device 108 includes, among other things, a mobile device (for example, a smartphone, a tablet), a laptop computer, a desktop computer, or any other type of computer, including those described below with reference to Fig. 14 will be explained. Some embodiments of the client device 108 perform a variety of functions on the client device 108 via a client application 110. For example, the client device 108 (via the client application 110) performs functions such as, but not limited to, extracting a style formatting element from an input text string, generating a translated text string from the input text string, generating an attention head matrix for words in the input text string, and generating a formatted translated text string. In some embodiments, the client device 108 uses the translation model 114 and / or the formatting preservation model 116 to generate formatted translated text strings. In some embodiments, the client device 108 trains the translation model 114 and / or the formatting preservation model 116.

[0027] To access the functions of the formatting preservation system 102 (as described above and in more detail below), a user interacts with the client application 110 on the client device 108 in one or more embodiments. For example, the client application 110 comprises one or more software applications (for example, for transferring formatting or styles from the input text to the translated text according to one or more embodiments described herein) installed on the client device 108, such as a digital media management application, a word processing application, and / or a graphic design application. In certain cases, the client application 110 is hosted on the server(s) 106.If the client application 110 is hosted on the server(s) 106, the client device 108 accesses it via a web browser and / or another online interface platform and / or online interface tool. Furthermore, in some embodiments, the client device 108, the server(s) 106, or another system hosts one or more databases containing digital data.

[0028] As in Fig. As shown in Figure 1, in some embodiments the formatting preservation system 102 is hosted by the client application 110 on the client device 108 (for example, in addition to or as an alternative to hosting by the digital media management system 104 on the server(s) 106). For example, the formatting preservation system 102 performs the text style and formatting preservation techniques described herein on the client device 108. In some implementations, the formatting preservation system 102 uses the server(s) 106 to train and implement machine learning models (such as the translation model 114 and / or the formatting preservation model 116).In one or more embodiments, the formatting preservation system 102 uses the server 106 to train machine learning models (such as the translation model 114 and / or the formatting preservation model 116) and uses the client device 108 to implement or apply the machine learning models.

[0029] Although in Fig. 1. Where the formatting preservation system 102 is depicted as being implemented by a specific component and / or device within the system 100 (for example, the server 106 and / or the client device 108), in some embodiments the formatting preservation system 102 is implemented wholly or partly by other computing devices and / or components within the system 100. For example, in some embodiments the formatting preservation system 102 is implemented on a different client device. More precisely, in one or more embodiments, the description of the formatting preservation system 102 (and the operations it performs) is implemented (or executed) by the client application 110 on a different client device.

[0030] In some embodiments, the client application 110 includes a web hosting application that allows the client device 108 to interact with content and services hosted on the server(s) 106. For illustration, in one or more implementations, the client device 108 accesses a web page or computer application supported by the server(s) 106. The client device 108 provides input to the server(s) 106 (for example, a request to translate text while preserving formatting). In response, the formatting preservation system 102 on the server(s) 106 performs the operations described here to produce formatted translated text. The server(s) 106 deliver the output or results of the operations (for example, formatted translated text strings, graphic designs with formatted translated text, etc.) to the client device 108.As another example, in some implementations, the formatting preservation system 102 on client device 108 performs the operations described here to generate formatted translated text. Client device 108 provides the output or results of the operations (for example, formatted translated text strings, graphic designs with formatted translated text, etc.) via a display on client device 108 and / or transmits the output or results of the operations to another device (for example, server 106 and / or another client device).

[0031] Furthermore, the system comprises 100, as in Fig. Figure 1 shows the network 112. As mentioned above, the network 112 enables communication between components of the system 100 in some cases. In certain embodiments, the network 112 comprises a suitable network and communicates using any communication platforms and technologies suitable for transporting data and / or communication signals, examples of which are shown with reference to Fig. 14 are described. Although in Fig. 1 Server 106 and client device 108 communicate via the network 112; in certain embodiments, the various components of the system 100 communicate and / or interact via other methods (for example, server 106 and client device 108 communicate directly with each other).

[0032] As explained above, in some implementations, the formatting preservation system 102 preserves the style formatting for translated text. For example, it shows Fig. 2 An exemplary overview of the system for preserving formatting 102, which, according to one or more embodiments, generates a formatted translated text string from an input string. Further details on the various aspects relating to Fig. The processes introduced in 2 are provided below with reference to the following figures.

[0033] In particular, it shows Fig. 2, how the formatting preservation system 102 receives an input text string 202 in a first language. The input text string 202 contains a style formatting element. A style formatting element comprises a data construct or a computer code segment that defines formatting for one or more characters (for example, one or more words) of a text string, such as font, font size, color, underline, italics, bold, emphasis, hyperlinks, outline, capitalization, strikethrough, alignment, shape, subscript, and superscript. In addition, in some embodiments, a style formatting element includes formatting for a paragraph, such as indentation, spacing, tabs, bullets and numbering, justification, and text alignment.Furthermore, in some implementations, a style formatting element includes formatting for a section or page, such as page margins, page alignment, page breaks, columns, headers, and footers. In the example below... Fig. In example 2, the input text string 202 is “The coffee was cold as ice” with a style formatting element in the form of an underline of the word “cold”.

[0034] Furthermore, it shows Fig. 2 The formatting preservation system 102 processes the input text string 202 through a translation and formatting preservation model 204 to produce a formatted translated text 206. For example, the formatting preservation system 102 uses the translation and formatting preservation model 204 to produce a translated text string in a second language that is different from the first language and to apply a translated style formatting element to the translated text string. A translated style formatting element includes a style formatting element for a translated text string, for example, to preserve formatting of the input text for the translated text. In the Fig. In the example shown, the formatted translated text 206 (in German) is “Der Kaffee war kalt wie Eis” (The coffee was cold as ice) with a translated style formatting element in the form of an underline of the word “kalt” (cold). Depending on the implementation, the translation and formatting preservation model 204 can take the form of various types of machine learning models, such as a neural transformer network, a neural machine translation model, a large language model, and / or a combination of one or more of the above-mentioned models.

[0035] As previously mentioned, in some embodiments, the Formatting Preservation System 102 uses the Translation and Formatting Preservation Model 204 to translate and / or format the input text string 202. For illustration, in some implementations, the Formatting Preservation System 102 uses a machine learning model, such as a neural transformer network, to perform machine translation and generate translated text strings in one or more languages ​​different from the first language. Furthermore, in some embodiments, the Formatting Preservation System 102 uses a neural machine translation model to generate translated text strings in one or more languages ​​different from the first language.Furthermore, in some embodiments, the formatting preservation system 102 uses a large language model to generate translated text strings in one or more languages ​​that differ from the first language.

[0036] In this context, the formatting preservation system 102 uses a machine learning model in some implementations to generate formatting (for example, a translated style formatting element) and apply the formatting to a translated text string (for example, to match the style formatting element applied to the input text string 202). Furthermore, in some implementations, the formatting preservation system 102 uses a hybrid approach, employing a neural machine translation model to generate a translated text string from an input text string and a large language model to determine and apply formatting to the translated text string.

[0037] As mentioned, in some implementations, the formatting preservation system 102 determines attention head values ​​for words in a text string. For example, it shows Fig. 3 the system for preserving formatting 102, which generates an attention head matrix according to one or more embodiments.

[0038] In particular, it shows Fig. 3, how the formatting preservation system 102 generates an attention head matrix 300 from attention head values ​​of a neural transformer network. Specifically, the attention head matrix 300 comprises rows 301–306, which are associated with words of an input text string (for example, "The coffee was as cold as ice"). Furthermore, the attention head matrix 300 comprises columns 311–316, which are associated with words of a translated text string (for example, "The coffee was as cold as ice"). For example, each word of the input text string is assigned a row of the attention head matrix 300, with the row containing cells that hold attention head values ​​for the word of the input text string with respect to each word of the translated text string.

[0039] In some embodiments, the Formatting Preservation System 102 determines the attention head values ​​from a neural transformer network. For example, the Formatting Preservation System 102 uses the neural transformer network to generate the translated text string from the input text string. During the generation of the translated text string, the neural transformer network generates attention head values. An attention head value comprises a metric or measure of the focus that the neural transformer network places on a portion of the input text string.

[0040] As mentioned, in some implementations an attention value indicates a relationship between a word in the input text string and a translated word in the translated text string. For example, a relatively large attention value indicates a close relationship (e.g., a high correlation) between the input word and the translated word. Conversely, a relatively small attention value (e.g., close to zero or zero) indicates a distant relationship (e.g., a low correlation) between the input word and the translated word. In the Fig. In the example shown, the Attention-Head matrix uses 300 light colors, including white, to indicate larger Attention-Head values, representing higher correlations between words. Similarly, the Attention-Head matrix uses 300 dark colors, including black, to indicate smaller Attention-Head values, representing lower correlations between words.

[0041] As in Fig. Figure 3 shows the correlation between the example input string and the translated string. The input word "coffee" has a high correlation with the translated word "Kaffee" (shown by the cell in row 302 and column 312) and a low correlation with the translated word "Eis" (shown by the cell in row 302 and column 316). As another example, the input word "ice" has a high correlation with the translated word "Eis" (shown by the cell in row 306 and column 316), a medium correlation with the translated word "kalt" (shown by the cell in row 306 and column 314), and a low correlation with the word "Kaffee" (shown by the cell in row 306 and column 312).As described in more detail below, in some embodiments, the Formatting Preservation System 102 uses the attention head values ​​to determine formatting for translated text strings by determining correlations between formatted words of input text strings and translated words of the translated text strings.

[0042] While the exemplary attention-head matrix in Fig. While the input text string is typically a square matrix, in some embodiments the Formatting Preservation System 102 generates a non-square attention head matrix. For example, in some cases the input text string contains more words than the translated text string. In this case, the Formatting Preservation System 102 generates an attention head matrix with more rows than columns. Conversely, in some cases the input text string has fewer words than the translated text string. In this case, the Formatting Preservation System 102 generates an attention head matrix with more columns than rows. Accordingly, the Formatting Preservation System 102 determines correlations between input words and translated words, where one input word is associated with many translated words and / or many input words are associated with one translated word.In some cases, the Formatting Preservation System 102 determines an attention head threshold to assign a correlation (for example, a threshold relationship strength) between words. The Formatting Preservation System 102 thus determines a correlation between words (for example, one-to-one, one-to-many, or many-to-one) for attention head values ​​that meet the threshold.

[0043] As explained above, in some embodiments, the formatting preservation system 102 uses attention heads to preserve the formatting of an input text string for a translated text string. For example, the Fig. 4A - 4B the formatting preservation system 102, which extracts formatting from an input text string and uses attention heads to apply the formatting according to one or more embodiments to a translated text string.

[0044] In particular, it shows Fig. 4A, how the formatting preservation system 102 receives an input text string 402 and processes the input text string 402 using a formatting extractor 404. For example, the formatting preservation system 102 determines a style formatting element in the input text string 402. Furthermore, in some implementations, the formatting preservation system 102 extracts the style formatting element from the input text string 402 using the formatting extractor 404. In the implementation described in Fig. In the example shown in Figure 4A, the formatting preservation system 102 uses the formatting extractor to extract an underline style formatting element from the word "cold" in the input text string 402. In some embodiments, a formatting extractor includes a neural network that identifies the style formatting of text and interprets the style formatting as a numerical representation. For example, a formatting extractor includes an encoder that converts a style formatting element into a vector representation of the style formatting element's formatting. In some embodiments, a formatting extractor includes computer code, such as a routine or function, that reads style information directly from the text formatting (for example, HTML formatting, font encoding, etc.) to determine a style formatting element.

[0045] In some embodiments, the formatting preservation system 102 processes the input text string and the style formatting element through a formatting preservation model 406. In some implementations, the formatting preservation model 406 is a neural transformer network that uses attention heads. As in Fig. As shown in Figure 4A, the formatting preservation system 102 processes the input text string and formatted words (for example, typed words containing the style formatting element) through the formatting preservation model 406. As shown below in conjunction with Fig. As described in more detail in Section 4B, the Formatting Preservation System 102 uses the Formatting Preservation Model 406 in some implementations to produce translated text and translated formatted words. For example, the Formatting Preservation System 102 uses the Neural Transformer network to generate the translated text string from the input text string 402.

[0046] Furthermore, in some implementations, the Formatting Preservation System 102 uses the neural transformer network to determine attention head values ​​for the words of the input text string 402 relative to the words of the translated text string. As explained in more detail below, the Formatting Preservation System 102 uses these attention head values ​​to map the formatted words of the input text string 402 to the translated words of the translated text string. From these translated words, the Formatting Preservation System 102 generates formatted translated words using the style formatting element(s) that correspond to the formatted words of the input text string 402.

[0047] Furthermore, in some embodiments, the formatting preservation system 102 uses a formatting applicator 408 to generate a formatted translated text string 410. For example, the formatting preservation system 102 generates the formatted translated text string 410 by applying the style formatting element to the translated text string using the formatting applicator 408. In the Fig. In the example shown in Figure 4A, the formatting preservation system 102 uses the formatting applicator 408 to apply the "underline" formatting element to the word "cold" in the translated text string, thus producing the formatted translated text string 410. In some embodiments, a formatting applicator includes a neural network that interprets numerical representations of the formatting to add the formatting to the text. For example, a formatting applicator includes a decoder that converts a vector representation of the formatting into a style formatting element and adds the style formatting element to a translated text. In some embodiments, a formatting applicator includes computer code, such as a routine or function, that copies style information (for example, HTML code, rule-based formatting, etc.) in a style formatting element into a text string.Furthermore, in some embodiments, a formatting applicator is a separate model from a machine translation model. For example, in some embodiments, the Formatting Preservation System 102 uses a machine translation model (such as a neural machine translation model or a large language model) to translate text and uses a formatting extractor and a formatting applicator to format the text.

[0048] As already mentioned, shows Fig. 4B Additional details of the formatting preservation model 406. In particular, in some implementations, the formatting preservation system 102 uses the formatting preservation model 406 to translate the input text string and produce the translated text string. Furthermore, in some implementations, the formatting preservation system 102 uses the formatting preservation model 406 to generate the formatted translated words for the formatted translated text string from the formatted words of the input text string.

[0049] For illustration, in some embodiments, the formatting preservation system 102 performs an operation 420 to translate the input text string 402 and generate an attention head matrix (for example, attention head matrix 300). For example, the formatting preservation system 102 uses a neural transformer network to generate the translated text string. For example, the formatting preservation system 102 uses an encoder to determine intermediate representations for words in the input text string 402 and compares the intermediate representations to determine correlations between words.

[0050] In some implementations, the Formatting Preservation System 102 uses a neural transformer network, which has layers of encoders and decoders. For example, the encoders take each word of the input text string, process the word into an intermediate representation, and compare this intermediate representation to the other intermediate representations of the other words in the input text string. The results of these comparisons are attention scores that indicate each word's contribution to a keyword in the input text string. The Formatting Preservation System 102 uses these attention scores as weights for word representations, which are passed to a fully connected network that generates a new representation for the keyword.The Formatting Preservation System 102 performs this process for each word in the input text string and transmits the new representation, along with the attention scores, to the decoders. The decoders use these new representations and attention scores to generate predictions. Additionally, each encoder generates a weighted sum of its previous encoder states. The Formatting Preservation System 102 processes this weighted sum across the decoders to produce a final machine translation, along with the attention head matrix.

[0051] Furthermore, the decoders have access to hidden states of the encoders, which are used to predict the translated words. The Formatting Preservation System 102 weights different hidden states differently, as not all hidden states are relevant in every step. The Formatting Preservation System 102 uses the neural transformer network to focus on the relevant parts of the input text string. In each iteration, the Formatting Preservation System 102 uses the decoder to receive input from the encoder and the previous decoder output for use in the next step.

[0052] Furthermore, in some embodiments, the formatting preservation system 102 retains positioning information for each word of the input text string by generating a word position index based on sine and cosine functions. The formatting preservation system 102 adds this information as position encoding to the embedding vector and processes the position encoding through the encoders.

[0053] As mentioned, in some embodiments, the Formatting Preservation System 102 generates the attention head matrix with attention head values ​​for the words of the input text string 402. For example, the Formatting Preservation System 102 determines the attention head values ​​generated by the neural transformer network for the words of the input text string 402 as part of generating the translated text string in the second language. For example, the Formatting Preservation System 102 maps relationships between the words of the input text string 402 and translated words of the translated text string. More precisely, the Formatting Preservation System 102 compares encoder states for the input text string 402 to predict the relationships between the words of the input text string 402 and the translated words of the translated text string.

[0054] As also in Fig. As shown in Figure 4B, in some implementations, the Format Preservation System 102 uses byte-pair encoding to generate the formatted translated words from the formatted words of the input text string 402. For example, the Format Preservation System 102 performs operation 422, which applies byte-pair encoding to the formatted words. To illustrate, the Format Preservation System 102 generates a byte-pair encoding for a formatted word (for example, "cold") in the input text string 402. Furthermore, the Format Preservation System 102 performs operation 424, which determines the n (for example, 3, 5, or another number) highest attention head values ​​for the byte-pair encoding for the formatted word from the attention head matrix.Furthermore, in some implementations, the Formatting Preservation System 102 performs a string search in each row of the attention head matrix to find the most strongly correlated translated words (from the columns of the attention head matrix) for each element of the byte pair encoding for each formatted word. For example, the Formatting Preservation System 102 generates a list of translated formatted words as byte pair encodings.

[0055] To further illustrate, in some embodiments, the formatting preservation system 102 performs an operation 426 in which, for each byte pair encoding of the formatted words, a target byte pair encoding for the translated words of the translated text string is found. For example, the formatting preservation system 102 determines a distance between the byte pair encoding for the formatted word and a translated byte pair encoding of a translated word of the translated text string. Furthermore, in some implementations, the formatting preservation system 102 performs an operation 428 to remove the byte pair encoding from the translated byte pair encoding to generate the translated word for formatting.

[0056] Furthermore, in some implementations, the Formatting Preservation System 102 applies formatting of the formatted word to the translated word of the translated text string based on the embedding distance. For example, the Formatting Preservation System 102 uses the Formatting Applicator 408 to add formatting of the formatted word based on the embedding distance to the translated word. For example, the Formatting Preservation System 102 compares a first attention head value for a formatted word of the input text string relative to a first translated word of the translated text string, and a second attention head value for the formatted word of the input text string relative to a second translated word of the translated text string.In addition, the formatting preservation system 102 uses the formatting applicator 408 to add the formatting of the formatted word to the first word of the translated text string, based on a comparison of the first attention head value and the second attention head value (for example, based on the fact that the first translated word has a shorter embedding distance to the formatted input word than the second translated word).

[0057] As mentioned, in some embodiments, the Formatting Preservation System 102 generates a translated style formatting element for a translated text string. For example, the Formatting Preservation System 102 uses the attention head values ​​to map a translated word in the translated text string to a formatted word in the input text string using the style formatting element. Furthermore, in some embodiments, the Formatting Preservation System 102 applies the style formatting element to the translated word in the translated text string. Thus, in some embodiments, the Formatting Preservation System 102 generates the formatted translated text string by applying the style formatting element to the translated text string according to the attention head matrix.In addition, in some embodiments, the formatting preservation system 102 reworks the formatted translated text by removing the formatting of some of the formatted translated words in order to approximate the style structure of the translated text to that of the input text.

[0058] Furthermore, in some implementations, the formatting preservation system 102 maps an input word to more than one translated word, or vice versa. For example, the formatting preservation system 102 maps a word in the input text string to a multitude of translated words in the translated text string based on corresponding attention head values ​​that exceed an attention head threshold. As described above in conjunction with Fig. As described in section 3, the attention head matrix is ​​not square in some cases because the relationships between input words and translated words are not one-to-one. For example, if the attention head matrix has more columns than rows, the translated text string contains more words than the input text string. In these cases, for example, one word in the input text string correlates with several words in the translated text string (or some words in the input text string correlate with more than some words in the translated text string).

[0059] Similarly, in some examples, the formatting preservation system 102 maps a multitude of words in the input text string to a translated word in the translated text string based on appropriate attention head values ​​that exceed an attention head threshold. For example, the formatting preservation system 102 determines that several words in the input text string have a short embedding distance to the translated word, and thus all of these multiple input words correspond to the translated word. Therefore, in some embodiments, the formatting preservation system 102 matches the formatting of this translated word with the formatting of these multiple input words.

[0060] As mentioned, in some implementations, the formatting preservation system 102 provides a formatted translated text string for display. For example, the Fig. 5A - 5B the formatting preservation system 102, which provides a formatted input text string and a formatted translated text string for display via a graphical user interface according to one or more embodiments.

[0061] In particular, it shows Fig. 5A a computer (for example, a client device 108) with a graphical user interface. In some implementations, the formatting preservation system 102 provides an input text string 502 for display via the graphical user interface, which includes a style formatting element. In the example shown, the input text string is "a report from the Department of Work also showed this week that the number of available jobs fell below 10 million in February for the first time in almost two years." Additionally, the input text string 502 has a style formatting element that includes bold and italic text for the words "fell below 10 million in February," while the rest of the input text string 502 is plain English text.

[0062] As described above, in some embodiments, the formatting preservation system 102 processes the input text string 502 through a translation and formatting preservation model to produce a formatted translated text string 504. Furthermore, it shows Fig. 5B, the formatting preservation system 102, provides the formatted translated text string 504 for display via the graphical user interface. In the example shown, the formatted translated text string 504 reads, "A report from the Department of Labor this week also showed that the number of available jobs fell below 10 million in February for the first time in nearly two years." Additionally, the formatted translated text string 504 has a formatting element for the translation style that includes bold and italic text for the words "in February" and "fell below 10 million," while the rest of the formatted translated text string 504 contains plain German text.

[0063] As mentioned above, in some implementations, the formatting preservation system 102 uses a variety of approaches for formatting preservation for machine translation. For example, it shows Fig. Six alternative formatting preservation techniques according to one or more embodiments. Furthermore, in some implementations, the Formatting Preservation System 102 uses these formatting preservation techniques for translating graphic designs. For example, the Formatting Preservation System 102 translates text within a graphic design into another language and applies stylistic elements of the original text to the translated text to maintain the appearance of the original graphic design in the translated graphic designs.

[0064] In particular, it shows Fig. 6. A first formatting preservation technique using attention heads (for example, as above in conjunction with Fig. 3 - 5B described in detail), a second formatting preservation technique using a neural machine translation model, a third formatting preservation technique using a large language model, and a fourth formatting preservation technique using a hybrid approach with a neural machine translation model and a large language model.

[0065] As described in detail above, the Formatting Preservation System 102 employs the first formatting preservation technique using attention heads. It processes source text through a neural machine translation model (for example, a Neural Transformer network), generates candidates for attention heads, evaluates these candidates, and determines which sections of the translated text should be formatted based on the evaluations of the attention head values. As mentioned, this technique uses attention head values ​​from the Neural Transformer network. However, in some cases, the attention head values ​​may not be readily available for evaluation.Therefore, in some embodiments, the formatting preservation system 102 uses an alternative formatting preservation technique, for example, using the neural machine translation model (NMT), the large language model (LLM), or the hybrid approach.

[0066] In some implementations, the formatting preservation system 102 uses the NMT approach to formatting preservation. For example, the formatting preservation system 102 receives the source text (for example, the input text string 202), inserts markers into the source text, and processes the marked text through a neural machine translation model to generate the text for formatting. Further details of this approach are given below in conjunction with Fig. 7 explained.

[0067] Furthermore, in some implementations, the Formatting Preservation System 102 uses the LLM approach to formatting preservation. For example, the Formatting Preservation System 102 retrieves the source code (for example, the input text string 202), inserts markup into the source code, generates a prompt wrapper for a large language model, and processes the marked text and the prompt wrapper through the large language model to generate the text for formatting. Further details of this approach are given below in conjunction with Fig. 8 explained.

[0068] Furthermore, in some embodiments, the Formatting Preservation System 102 employs a hybrid NMT and LLM approach to formatting preservation. For example, the Formatting Preservation System 102 receives the source text (e.g., input text string 202), processes the source text through a neural machine translation model to generate a translated text, inserts markups into the translated text, generates a prompt wrapper for a large language model, and processes the marked-up translated text and the prompt wrapper through the large language model to generate the text for formatting. Further details of this approach are provided below in conjunction with Fig. 9 is indicated.

[0069] As previously mentioned, the Formatting Preservation System 102, in some cases, preserves text formatting in graphic designs with high accuracy in word alignment between the original and translated text. Furthermore, the Formatting Preservation System 102 uses one of these four techniques (i.e., attention heads, NMT, LLM, and / or hybrid NMT+LLM) to preserve text formatting within graphic designs.

[0070] As mentioned, in some implementations, the formatting preservation system 102 uses the NMT approach to formatting preservation. For example, it illustrates Fig. 7 the formatting preservation system 102, which uses a neural machine translation model to generate translated text with coded tags in order to format the translated text according to one or more embodiments.

[0071] In particular, it shows Fig. 7. The system for preserving formatting 102 receives an input text string 702. The input text string 702 contains a style formatting element. In the Fig. In example 7, the input text string 702 reads “Job cuts have also soared nearly fivefold so far this year from a year ago”, where the words “nearly fivefold” are formatted with an italic style element.

[0072] Furthermore, it shows Fig. 7, how the formatting preservation system 102 generates a modified input text string 704 from the input text string 702. In particular, the formatting preservation system 102 generates the modified input text string 704 with a coded tag that identifies the style formatting element. For example, the formatting preservation system 102 generates a first coded tag at the beginning of a text segment containing the style formatting element (for example, before the word "almost") and a second coded tag at the end of the text segment containing the style formatting element (for example, after the word "fivefold"). Furthermore, in some embodiments, the formatting preservation system 102 generates multiple pairs of coded tags to mark multiple style formatting elements.

[0073] Furthermore, it shows Fig. 7 The formatting preservation system 102 processes the modified input text string 704 using a neural machine translation model 706. For example, the formatting preservation system 102 uses the neural machine translation model 706 to generate a translated text string 708 from the modified input text string 704. The formatting preservation system 102, for example, generates the translated text string 708 and retains the first encoded tag and the second encoded tag in the translated text string 708. In the Fig. In example 7, the translated text string 708 (in German) reads: “Even job cuts have decreased this year compared to last year.” <s 1>"almost fivefold." As shown, the first and second coded tags enclose the translated words "almost fivefold", which correspond to the formatted words "almost fivefold" in the input text string 702.

[0074] Furthermore, it shows Fig. 7, how the formatting preservation system 102 produces a formatted translated text string 710 by applying the style formatting element to a word in the translated text string 708 based on the coded tag of the modified input text string 704 (for example, via the formatting applicator 408 described above). For example, the formatting preservation system 102 removes the first and second coded tags from the translated text string 708 and formats the translated text string by applying the style formatting element to a translated text segment that corresponds to the text segment. In the example shown, the translated words “almost five times” in the formatted translated text string 710 are italicized according to the italic style formatting element from the input text string 702.

[0075] Furthermore, as mentioned, the formatting preservation system 102 uses the LLM approach to formatting preservation in some implementations. For example, it illustrates Fig. 8 the formatting preservation system 102, which uses a large language model to generate translated text with delimiters in order to format the translated text according to one or more embodiments.

[0076] In particular, it shows Fig. 8, how the formatting preservation system 102 receives an input text string 802. The input text string 802 contains a style formatting element. In the Fig. In example 8, the input text string 802 reads “Job cuts have also soared nearly fivefold so far this year from a year ago”, with the words “nearly fivefold” formatted with an italic style element.

[0077] Furthermore, it shows Fig. 8, how the formatting preservation system 102 generates a modified input text string 804 from the input text string 802. In particular, the formatting preservation system 102 generates the modified input text string 804 with a first delimiter that marks the beginning of the style formatting element and a second delimiter that marks the end of the style formatting element. For example, the formatting preservation system 102 generates the first delimiter at the beginning of a text passage that includes the style formatting element (for example, before the word "almost"), and the second delimiter at the end of the text passage that includes the style formatting element (for example, after the word "fivefold").

[0078] Furthermore, it shows Fig. 8 The formatting preservation system 102 processes the modified input text string 804 through a large language model 806. For example, the formatting preservation system 102 uses the large language model 806 to generate a translated text string 808 from the modified input text string 804. The formatting preservation system 102, for example, generates the translated text string 808 and retains the first and second delimiters in the translated text string 808. In the Fig. In the example shown, the translated text string 808 (in German) reads: “Even job cuts have increased this year compared to last year ##start##fast fivefold##end##.” As shown, the first and second separators enclose the translated words “fast fivefold”, which correspond to the formatted words “fast fivefold” in the input text string 802. In some implementations, the large language model 806 identifies the correct words in the translated text string 808 for stylization but does not identify a formatting type to apply (for example, italics). As explained in more detail above and below, in some implementations, the formatting preservation system 102 determines the formatting type (for example, using a formatting extractor) and applies the formatting type (for example, using a formatting applicator) to the translated words for formatting.

[0079] Furthermore, it shows Fig. 8, how the formatting preservation system 102 produces a formatted translated text string 810 by applying the style formatting element to a word in the translated text string 808 based on the first and second delimiters of the modified input text string 804. For example, the formatting preservation system 102 removes the first and second delimiters from the translated text string 808 and formats the translated text string by applying the style formatting element to a translated text segment that corresponds to the text segment. In the example shown, the translated words “almost five times” in the formatted translated text string 810 are italicized according to the italic formatting element from the input text string 802.

[0080] As part of the instruction for the large language model 806, the formatting preservation system 102, in some embodiments, generates a prompt for the large language model 806. For example, the formatting preservation system 102 generates a prompt that defines the first and second delimiters. To illustrate, the prompt explains that the first delimiter marks the beginning of a special style around the formatted text passage, and that the second delimiter marks the end of the special style around the formatted text passage. Furthermore, in some embodiments, the formatting preservation system 102 generates multiple pairs of delimiters to mark multiple style formatting elements.In addition, in some embodiments, the prompt contains instructions for the large language model 806 to translate the input text string 802, retaining the first separator and the second separator in the translated text string 808.

[0081] Furthermore, in some embodiments, the formatting preservation system 102 generates the translated text string 808 by processing the prompt through the large language model 806 with the modified input text string 804. For example, the formatting preservation system 102 provides the modified input text string 804 to the large language model 806 and instructs the large language model (via the prompt) to translate the modified input text string 804 while preserving the delimiters.

[0082] As mentioned, in some implementations, the formatting preservation system 102 uses the hybrid NMT+LLM approach. For example, it illustrates Fig. 9 the formatting preservation system 102, which uses a neural machine translation model to generate a translated text from an input text, and a large language model to determine translated words for formatting according to one or more embodiments.

[0083] In particular, it shows Fig. 9, how the formatting preservation system 102 receives an input text string 902. The input text string 902 contains a style formatting element. In the Fig. In example 9, the input text string 902 reads “Job cuts have also soared nearly fivefold so far this year from a year ago”, with the words “nearly fivefold” formatted with an italic style element.

[0084] Furthermore, it shows Fig. 9 the formatting preservation system 102, which processes the input text string 902 through a neural machine translation model 904 to generate a translated text string 906. In the Fig. In example 9, the translated text string 906 (in German) reads: "Even job cuts have increased this year compared to last year, almost fivefold." However, at this stage, the Formatting Preservation System 102 does not retain the formatting in the translated text string 906. Instead, the Formatting Preservation System 102 uses a large language model 910 to determine the formatting for the translated text string.

[0085] As just mentioned, in some embodiments, the Formatting Preservation System 102 uses a large language model to determine the formatting for a translated text string, while in other embodiments, it uses a neural machine translation model to generate the translated text string. For example, the Formatting Preservation System 102 processes the input text string 902 and the translated text string 906 through a large language model 910 to determine which words should be translated and formatted. For example, the Formatting Preservation System 102 uses the large language model 910 to process the translated text string 906 and determine a translated word within the translated text string 906 that corresponds to a formatted word in the input text string 902.

[0086] Furthermore, in some implementations, the Formatting Preservation System 102 uses unigram mappings to determine which translated words should be formatted. A unigram mapping comprises a single, self-contained unit of a text string. For example, a unigram mapping might comprise a word of a sentence or a token representing a single element of the sentence. In some embodiments, the Formatting Preservation System 102 generates a unigram mapping for each formatted word of the input text string 902. In the implementation described in Fig. In the example shown, the formatting preservation system 102 generates unigram mappings 908 for the words "fast" and "fivefold", which are formatted (italic) in the input text string 902. The formatting preservation system 102 processes the unigram mappings 908 through the large language model 910 with the input text string 902 and the translated text string 906 to generate translated unigram mappings 912 of the translated words to be formatted.

[0087] In the Fig. In the example shown, the formatting preservation system 102 uses the large language model 910 to generate unigram mappings for the translated words "fast" and "fivefold" that correspond to the formatted input words "fast" and "fivefold". Thus, in some implementations, the formatting preservation system 102 determines the translated words of the translated text string 906 for formatting based on the unigram mappings of the formatted words of the input text string 902. For illustration, the formatting preservation system 102 generates the unigram mappings 908 by identifying the formatted words of the input text string 902 and processes the unigram mappings 908 through the large language model 910 to determine the translated unigram mappings 912 of the translated words.

[0088] Additionally, in some embodiments, the formatting preservation system 102 applies the style formatting element to the translated words of the translated text string. For example, the formatting preservation system 102 produces a formatted translated text string 914 by applying the style formatting element to the translated words of the translated text string 906 that correspond to the translated unigram mappings 912. In the Fig. In example 9, the formatted translated text string 914 reads “Even job cuts have almost quintupled this year compared to the previous year”, where the translated words “almost” and “quintupled” are formatted by italics.

[0089] In particular, in some embodiments, the formatting preservation system 102 generates a prompt for the large language model 910. For example, the formatting preservation system 102 generates a prompt that defines the unigram mappings. Furthermore, the prompt instructs the large language model to provide translated unigram mappings of the relevant translated words (i.e., the translated words that correspond to the formatted input words). To illustrate, the prompt explains that the unigram mappings correspond to the input words of the input text string 902 and that the large language model should provide translated unigram mappings of translated words that correspond to the input words with unigram mappings.

[0090] Furthermore, in some implementations, the formatting preservation system 102 processes the prompt, the input text string 902, and the translated text string 906 via the large language model 910 to generate the translated unigram mappings 912 of the translated words. Using the large language model 910, the formatting preservation system 102 thus identifies the corresponding translated words for stylization.

[0091] As mentioned previously, in some embodiments, the formatting preservation system 102 provides a formatted translated text string for display. Furthermore, in some embodiments, the formatting preservation system 102 provides a graphic with formatted translated text for display. For example, the Fig. 10A-10B the formatting preservation system 102, which provides an input graphic with formatted text and a translated graphic with formatted translated text for display via a graphical user interface according to one or more embodiments.

[0092] In particular, it shows Fig. 10A a computer (for example, a client device 108) with a graphical user interface. In some implementations, the formatting preservation system 102 provides an input graphic 1002 for display via the graphical user interface, which includes a style formatting element for text. In the example shown, the input graphic 1002 contains, among other things, the text "YOUNG STAR". In addition, the text has a style formatting element with a specific font, shape, color, and capitalization, as well as other style formats.

[0093] In some embodiments, the formatting preservation system 102 processes the input graphic 1002 through a translation and formatting preservation model to produce a translated graphic 1004 with formatted translated text. For example, shows Fig. 10B, the formatting preservation system 102, provides the translated graphic 1004 for display via the graphical user interface. In the example shown, the translated graphic 1004 contains, among other things, the text "YOUNG STAR". As shown, the translated text has a style formatting element that matches the source code of the input graphic 1002 ("YOUNG STAR"). For example, the font, shape, color, and capitalization of the translated text in the translated graphic 1004 correspond to the style formats of the source code of the input graphic 1002.

[0094] In Fig. Section 11 now provides further details on the components and capabilities of one or more embodiments of the system for preserving formatting. In particular, it shows Fig. 11 An example of a formatting preservation system 102, which is executed by one or more computing units 1100 (for example, the server units 106 or the client unit 108). As in the embodiment of Fig. As shown in Figure 11, the computing facility(ies) 1100 includes or hosts the digital media management system 104 and / or the formatting preservation system 102. As shown in Figure 11, the computing facility(ies) 1100 includes or hosts the digital media management system 104 and / or the formatting preservation system 102. Fig. As shown in Figure 11, the formatting preservation system 102 further includes a text string manager 1102, a translation manager 1104, an attention head manager 1106, a style formatting manager 1108 and a memory manager 1110.

[0095] As in Fig. As shown in Figure 11, the formatting preservation system 102 includes a text string manager 1102. In some embodiments, the text string manager 1102 receives an input text string that includes a style formatting element. Furthermore, in some implementations, the text string manager 1102 extracts the style formatting element from the input text string. In some implementations, the text string manager 1102 produces a modified input text string that includes coded tags or delimiters for use with a neural network.

[0096] Furthermore, the system for preserving formatting includes 102, as described in Fig. Figure 11 shows a translation manager 1104. In some embodiments, the translation manager 1104 generates a translated text string from the input text string in a language different from the language of the input text string. Furthermore, in some embodiments, the translation manager 1104 uses a neural network, such as a neural transformer network, a neural machine translation model, or a large language model, to generate the translated text string.

[0097] Furthermore, the system for preserving formatting includes 102, as described in Fig. Figure 11 shows an attention head manager 1106. In some implementations, the attention head manager 1106 determines attention head values ​​generated by a neural transformer network for words in the input text string. Furthermore, in some implementations, the attention head manager 1106 generates an attention head matrix from these attention head values. For example, in some embodiments, the attention head manager 1106 maps relationships between words in the input text string and translated words in the translated text string.

[0098] Furthermore, the system for preserving formatting includes 102, as described in Fig. Figure 11 shows a style formatting manager 1108. In some embodiments, the style formatting manager 1108 extracts a style formatting element from an input text string. In some embodiments, the style formatting manager 1108 generates a translated style formatting element for a translated text string. Furthermore, in some implementations, the style formatting manager 1108 creates a formatted translated text string by applying a style of a formatted input word to a translated word that corresponds to the formatted input word. In some embodiments, the style formatting manager 1108 applies the style formatting element to a translated word of the translated text string based on an encoded tag, a separator, or a unigram mapping that identifies formatted words of the input text string.

[0099] Additionally, the system for preserving formatting includes 102, as described in Fig. Figure 11 shows a memory manager 1110. In some implementations, the memory manager 1110 stores information (for example, about one or more storage devices) on behalf of the system for preserving formatting 102. For example, the memory manager 1110 includes an input text string, a style formatting element, a modified input text string, a translated text string, a formatted translated text string, an encoded tag, a separator, and / or a unigram mapping. In some implementations, the memory manager 1110 includes parameters of one or more neural networks, such as a neural transformer network, a neural machine translation model, and / or a large language model.

[0100] Each of the components 1102-1110 of the Format Preservation System 102 comprises software, hardware, or both. For example, components 1102-1110 comprise one or more instructions stored on a computer-readable storage medium that can be executed by processors of one or more computers, such as a client device or a server. When executed by the one or more processors, the computer-executable instructions of the Format Preservation System 102, in some implementations, cause the computer(s) to perform the procedures described herein. Alternatively, components 1102-1110, in one or more implementations, comprise hardware, such as a special processing device for performing a particular function or group of functions.Alternatively, in some implementations, components 1102 - 1110 of the System for Preserving Formatting 102 comprise a combination of computer-executable instructions and hardware.

[0101] Furthermore, components 1102–1110 of the System for Preserving Formatting 102 are implemented, for example, as one or more operating systems, as one or more standalone applications, as one or more modules of an application, as one or more plug-ins, as one or more library functions, as one or more functions callable by other applications, and / or as a cloud computing model. Therefore, in some implementations, components 1102–1110 are implemented as a standalone application, such as a desktop or mobile application. Additionally, in various implementations, components 1102–1110 are implemented as one or more web-based applications hosted on a remote server. In some implementations, components 1102–1110 are implemented in a suite of applications for mobile devices, or "apps."In some implementations, components 1102–1110 are implemented in an application, including, but not limited to, Adobe Acrobat, Adobe Creative Cloud, Adobe Express, Adobe Fresco, Adobe Illustrator, Adobe InCopy, Adobe InDesign, and Adobe Photoshop. The foregoing designations are either registered trademarks or trademarks of Adobe in the United States and / or other countries.

[0102] The Fig. 1-11, the corresponding text and examples show a series of different methods, systems, devices and non-transitory computer-readable media of the system for preserving formatting 102. In addition to the foregoing, one or more embodiments are described by means of flowcharts that include steps for achieving a particular result, as in the Fig. 12 and Fig. Figure 13 illustrates this. In some implementations, the system's formatting preservation processes are performed with more or fewer steps. Furthermore, the steps are performed in different sequences across different implementations. Additionally, in some implementations, the steps described here are repeated, performed concurrently, or performed in parallel with different instances of the same or similar steps.

[0103] As already mentioned, it shows Fig. 12 a flowchart of a series of operations 1200 for preserving formatting of translated texts using attention head values ​​according to one or more implementations. While Fig. 12 processes according to one implementation show that alternative implementations allow any of the in Fig. The 12 operations shown are removed, added, rearranged, and / or modified. In one or more implementations, the steps of Fig. 12 as part of a procedure (for example, a computer-implemented procedure). Alternatively, in one or more implementations, a non-transitory, computer-readable storage medium comprises instructions which, when executed by one or more processors, cause a computer to perform the steps from Fig. 12. In some implementations, a system executes the steps. Fig. 12 out.

[0104] As in Fig. As shown in Figure 12, the series of operations 1200 includes an operation 1202 to preserve an input text string containing a style formatting element, an operation 1204 to generate a translated text string from the input text string, an operation 1206 to determine attention head values ​​for words in the input text string, and an operation 1208 to generate a translated style formatting element for the translated text string. Furthermore, as shown in Fig. Figure 12 shows a process 1202a for extracting the style formatting element from the input text string, a process 1204a for using a neural transformer network to process the input text string, a process 1206a for generating an attention head matrix from the attention head values, and a process 1208a for generating a formatted translated text string by applying the style formatting element to the translated text string.

[0105] In some implementations, step 1202 specifically involves obtaining an input text string in a first language, where the input text string includes a style formatting element; step 1204 involves generating a translated text string in a second language, different from the first language, using a neural transformer network to process the input text string; step 1206 involves determining attention head values ​​generated by the neural transformer network for words of the input text string as part of generating the translated text string in the second language; and step 1208 involves generating a translated style formatting element for the translated text string based on the attention head values ​​for the words of the input text string.

[0106] For example, in some implementations, the sequence of operations 1200 includes determining the attention head values ​​for the words of the input text string by generating an attention head matrix that maps relationships between the words of the input text string and translated words in the translated text string. Furthermore, in some implementations, the sequence of operations 1200 includes generating the attention head matrix by comparing encoder states for the input text string to predict the relationships between the words of the input text string and the translated words in the translated text string.

[0107] Furthermore, in some implementations, the series of operations 1200 includes generating the translated style formatting element for the translated text string by: using the attention head values ​​to map a word of the translated text string to a formatted word of the input text string that has been formatted by the style formatting element; and applying the style formatting element to the word of the translated text string.Furthermore, in some implementations, the series of operations 1200 includes generating the translated style formatting element for the translated text string by: comparing a first attention head value for a word of the input text string relative to a first word of the translated text string and a second attention head value for the word of the input text string relative to a second word of the translated text string; and applying the style formatting element to the first word of the translated text string based on the comparison of the first attention head value and the second attention head value.

[0108] Furthermore, in some implementations, the 1200 operation set includes generating the translated style formatting element for the translated text string by: generating a byte pair encoding for a formatted word in the input text string; determining an embedding distance between the byte pair encoding for the formatted word and a translated byte pair encoding of a translated word in the translated text string; and applying a formatting of the formatted word to the translated word in the translated text string based on the embedding distance. Additionally, in some implementations, the 1200 operation set includes providing the translated text string in the second language, with the translated style formatting element applied to a word in the translated text string according to the attention head values, for display via a client device user interface.Furthermore, in some implementations, the series of operations 1200 includes generating the translated text string in the second language by: using an encoder to determine an intermediate representation for a word in the input text string; and comparing the intermediate representation for that word with another intermediate representation for a different word in the input text string.

[0109] Furthermore, in some implementations, the series of operations 1200 includes extracting a style formatting element from an input text string in a first language using a formatting extractor; generating a translated text string in a second language, different from the first language, using a neural transformer network to process the input text string; generating an attention head matrix for words of the input text string from attention head values ​​of the neural transformer network by mapping relationships between the words of the input text string and translated words of the translated text string; and generating a formatted translated text string by applying the style formatting element to the translated text string according to the attention head matrix.

[0110] In some implementations, the series of operations 1200 includes, for example, extracting the style formatting element from the input text string using the formatting extractor to determine a formatted word in the input text string; and generating the formatted translated text string using a formatting applicator to add a style of the formatted word to a translated word in the translated text string.Furthermore, in some implementations, the series of operations 1200 includes generating the formatted translated text string by: using the attention head values ​​to map a word of the translated text string to a formatted word of the input text string that has been formatted by the style formatting element; and using a formatting applicator to add a style of the formatted word to a translated word of the translated text string.

[0111] Furthermore, in some implementations, the series of operations 1200 includes generating the formatted translated text string by: comparing a first attention head value for a formatted word of the input text string relative to a first word of the translated text string and a second attention head value for the formatted word of the input text string relative to a second word of the translated text string; and using a formatting applicator to add a style of the formatted word to the first word of the translated text string, based on the comparison of the first attention head value and the second attention head value.Furthermore, in some implementations, the sequence of operations 1200 includes generating the formatted translated text string by: generating a byte-pair encoding for a formatted word in the input text string; determining an embedding distance between the byte-pair encoding for the formatted word and a translated byte-pair encoding of a translated word in the translated text string; and using a formatting applicator to add formatting of the formatted word to the translated word in the translated text string, based on the embedding distance. Additionally, in some implementations, the sequence of operations 1200 includes generating the translated word by removing the byte-pair encoding from the translated byte-pair encoding.

[0112] Furthermore, in some implementations, the series of operations 1200 includes extracting a style formatting element from an input text string in a first language using a formatting extractor; generating a translated text string in a second language, different from the first, using a neural transformer network to process the input text string; determining attention head values ​​generated by the neural transformer network for words in the input text string as part of generating the translated text string in the second language; and generating a formatted translated text string using a formatting applicator on the translated text string to apply the style formatting element to translated words specified by the attention head values ​​of the neural transformer network.

[0113] In some implementations, for example, the sequence of operations 1200 includes generating the formatted translated text string by mapping a word from the input text string to a multitude of translated words in the translated text string based on corresponding attention head values ​​that exceed a threshold for the attention head value. Furthermore, in some implementations, the sequence of operations 1200 includes generating the formatted translated text string by mapping a multitude of words from the input text string to a translated word in the translated text string based on corresponding attention head values ​​that exceed a threshold for the attention head value.

[0114] Furthermore, in some implementations, the Operation 1200 set includes extracting the style formatting element from the input text string using the formatting extractor to determine a formatted word in the input text string. Additionally, in some implementations, the Operation 1200 set includes generating the formatted translated text string using the formatting applicator to add formatting to a translated word in the translated text string. Finally, in some implementations, the Operation 1200 set includes determining the attention head values ​​for the words in the input text string by generating an attention head matrix through comparison of encoder states for the input text string to predict relationships between the words in the input text string and translated words in the translated text string.

[0115] As mentioned, shows Fig. 13 A flowchart of a series of steps 1300 for preserving formatting of translated text using neural networks according to one or more implementations. While Fig. 13 steps according to one implementation show that alternative implementations omit steps, add steps, rearrange steps, and / or modify steps that are in Fig. Figure 13 shows the steps of... In one or more implementations, the steps of... Fig. 13 as part of a procedure (for example, a computer-implemented procedure). Alternatively, in one or more implementations, a non-transitory, computer-readable storage medium comprises instructions which, when executed by one or more processors, cause a computer to perform the steps of Fig. 13. In some implementations, a system performs the steps of Fig. 13 out.

[0116] As in Fig. As shown in Figure 13, the sequence of steps 1300 comprises a step 1302 to preserve an input text string containing a style formatting element, a step 1304 to generate a modified input text string from the input text string, a step 1306 to generate a translated text string from the modified input text string, and a step 1308 to apply the style formatting element to a word of the translated text string. Furthermore, the sequence of operations 1300, as shown in Fig. Figure 13 shows an operation 1304a to generate an encoded tag, separator or unigram mapping, an operation 1306a to use a neural machine translation model or a large language model to generate the translated text string, and an operation 1308a to apply the style formatting element based on the encoded tag, separator or unigram mapping.

[0117] In particular, in some implementations, process 1302 includes obtaining an input text string, wherein the input text string includes a style formatting element; step 1304 includes generating a modified input text string from the input text string, wherein the modified input text string includes an encoded tag that identifies the style formatting element; step 1306 includes generating a translated text string from the modified input text string using a neural machine translation model; and step 1308 includes applying the style formatting element to a word of the translated text string based on the encoded tag of the modified input text string.

[0118] For example, in some implementations, the sequence of operations 1300 includes generating the modified input text string by: generating the coded tag at the beginning of a text segment containing the style formatting element; and generating an additional coded tag at the end of the text segment containing the style formatting element. Furthermore, in some implementations, the sequence of operations 1300 includes generating the translated text string by preserving the coded tag and the additional coded tag in the translated text string.Furthermore, in some implementations, the series of operations 1300 includes applying the style formatting element to the word of the translated text string by: removing the coded tag and the additional coded tag from the translated text string; and formatting the translated text string by applying the style formatting element to a translated text section that corresponds to the text section.

[0119] Furthermore, in some implementations, the 1300 operation sequence includes applying the style formatting element to the word in the translated text string by creating a graphical design element for the translated text string. Additionally, in some implementations, the 1300 operation sequence includes making the translated text string, with the style formatting element applied to the word in the translated text string as a graphical design element, available for display via a client device user interface.

[0120] Furthermore, in some implementations, the series of operations 1300 includes obtaining an input text string, wherein the input text string contains a style formatting element; generating a modified input text string from the input text string, wherein the modified input text string contains a first delimiter that identifies the beginning of the style formatting element and a second delimiter that identifies the end of the style formatting element; generating a translated text string from the modified input text string using a large language model, wherein the translated text string contains the first delimiter and the second delimiter; and applying the style formatting element to a word of the translated text string based on the first delimiter and the second delimiter.

[0121] In some implementations, the sequence of operations 1300 includes, for example, generating a prompt that defines the first and second delimiters and instructs the large language model to translate the input text string, preserving the first and second delimiters in the translated text string. Furthermore, in some implementations, the sequence of operations 1300 includes generating the translated text string by having the large language model process the prompt and the modified input text string to produce the translated text string.

[0122] Furthermore, in some implementations, the sequence of operations 1300 includes generating the modified input text string by: generating the first delimiter at the beginning of a text segment containing the style formatting element; and generating the second delimiter at the end of the text segment containing the style formatting element. Additionally, in some implementations, the sequence of operations 1300 includes generating the translated text string by preserving the first and second delimiters in the translated text string.Furthermore, in some implementations, the series of operations 1300 includes applying the style formatting element to the word of the translated text string by: removing the first and second separators from the translated text string; and formatting the translated text string by applying the style formatting element to a translated text segment that corresponds to a text segment of the input text string that includes the style formatting element.

[0123] Furthermore, in some implementations, the 1300 operation sequence includes applying the style formatting element to the word in the translated text string by creating a graphical design element for the translated text string. Additionally, in some implementations, the 1300 operation sequence includes making the translated text string, with the style formatting element applied to the word in the translated text string as a graphical design element, available for display via a client device user interface.

[0124] Furthermore, in some implementations, the series of operations 1300 includes obtaining an input text string, where the input text string includes a style formatting element; generating a translated text string from the input text string using a neural machine translation model; determining a translated word of the translated text string that corresponds to a formatted word of the input text string using a large language model to process the translated text string based on a unigram mapping of the formatted word of the input text string; and applying the style formatting element to the translated word of the translated text string.

[0125] For example, in some implementations, the 1300 operation sequence includes generating the unigram mapping by identifying the formatted word of the input text string. Furthermore, in some implementations, the 1300 operation sequence includes processing the unigram mapping through the large language model to determine a translated unigram mapping of the translated word.

[0126] Furthermore, in some implementations, the 1300 operation sequence includes generating a prompt that defines the unigram mapping and instructs the large language model to provide a translated unigram mapping of the translated word. Additionally, in some implementations, the 1300 operation sequence includes determining the translated word of the translated text string by having the large language model process the prompt, the input text string, and the translated text string to generate the translated unigram mapping of the translated word.

[0127] Furthermore, in some implementations, the 1300 operation sequence includes applying the style formatting element to the translated word of the translated text string by generating a graphical design for the translated text string. Additionally, in some implementations, the 1300 operation sequence includes providing the translated text string, with the style formatting element applied to the word of the translated text string as a graphical design element, for display via a client device user interface.

[0128] Embodiments of the present disclosure may include or utilize a specialized or general-purpose computer comprising computer hardware, such as one or more processors and system memory, as explained in more detail below. Embodiments within the scope of the present disclosure also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. In particular, at least some of the processes described herein may be implemented as instructions embodied in a non-transitory, computer-readable medium that can be executed by one or more computers (for example, one of the media access devices described herein).In general, a processor (for example, a microprocessor) receives instructions from a non-volatile, computer-readable medium (for example, memory) and executes these instructions, thereby running one or more processes, including one or more of the processes described here.

[0129] Computer-readable media can be any available media accessible to a general-purpose or specialized computer system. Computer-readable media that store computer-executable instructions are non-transient computer-readable storage media (devices). Computer-readable media that contain computer-executable instructions are transmission media. Thus, embodiments of the disclosure may, for example, but are not limited to, include at least two distinctly different types of computer-readable media: non-volatile computer-readable storage media (devices) and transmission media.

[0130] Non-transitory computer-readable storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid-state drives (“SSDs”) (for example, RAM-based), flash memory, phase-change memory (“PCM”), other types of storage, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.

[0131] A "network" is defined as one or more data connections that enable the transport of electronic data between computer systems and / or generators and / or other electronic devices. When information is transmitted or provided to a computer via a network or other communication link (either wired, wireless, or a combination of both), the computer properly considers the link to be a transmission medium. Transmission media may include a network and / or data links that can be used to transmit desired program code resources in the form of computer-executable instructions or data structures, and which a general-purpose or specialized computer can access. Combinations of the above elements should also fall within the scope of computer-readable media.

[0132] Furthermore, program code resources in the form of computer-executable instructions or data structures can be automatically transferred from transmission media to non-transient, computer-readable storage media (devices) upon reaching various computer system components (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in the RAM of a network interface generator (for example, a "NIC") and then eventually transferred to the RAM of a computer system and / or to less volatile computer storage media (devices) within that computer system. Therefore, it should be clear that non-volatile, computer-readable storage media (devices) can be included in computer system components that also (or even primarily) use transmission media.

[0133] Computer-executable instructions include, for example, instructions and data which, when executed by a processor, cause a general-purpose computer, a specialized computer, or a specialized processing device to perform a specific function or group of functions. In some embodiments, computer-executable instructions are executed by a general-purpose computer to transform the general-purpose computer into a specialized computer that implements elements of the disclosure. The computer-executable instructions may be, for example, binary files, instructions in intermediate formats such as assembly language, or even source code. Although the subject matter has been described in language relating to structural features and / or methodological actions, it should be noted that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above.Rather, the described characteristics and actions are revealed as examples of how the claims can be implemented.

[0134] A person skilled in the art will recognize that the disclosure can be performed in network computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframes, mobile phones, PDAs, tablets, pagers, routers, switches, and the like. The disclosure can also be practiced in distributed system environments where local and remote computer systems, connected via a network (either by wired data links, wireless data links, or a combination of both), perform both tasks. In a distributed system environment, program generators can reside in both local and remote storage devices.

[0135] Embodiments of the present disclosure can also be implemented in cloud computing environments. As used here, the term “cloud computing” refers to a model that enables on-demand network access to a shared pool of configurable computing resources. For example, cloud computing can be used in the market to offer ubiquitous and convenient on-demand access to the shared pool of configurable computing resources. The shared pool of configurable computing resources can be rapidly provisioned via virtualization and released with minimal management effort or interaction with the service provider, and then scaled accordingly.

[0136] A cloud computing model can have various characteristics, such as on-demand self-service, broad network access, resource pooling, high elasticity, metered services, and so on. A cloud computing model can also provide different service models, such as Software as a Service (SaaS), a web service, Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). Furthermore, a cloud computing model can be deployed using various deployment models, such as private cloud, community cloud, public cloud, hybrid cloud, etc. In this document, the term "cloud computing environment" refers to an environment in which cloud computing is used.

[0137] Fig. Figure 14 shows a block diagram of an example computer system 1400, which can be configured to perform one or more of the processes described above. It is understood that one or more computers, such as Computer 1400, can represent the computers described above (for example, Computer 1100, Server 106, or Client Device 108). In one or more embodiments, Computer 1400 can be a mobile device (for example, a mobile phone, smartphone, PDA, tablet, laptop, camera, tracker, watch, wearable device, etc.). In some embodiments, Computer 1400 can be a non-mobile device (for example, a desktop computer or another client). Furthermore, Computer 1400 can be a server that includes cloud-based processing and storage capabilities.

[0138] As in Fig. As shown in Figure 14, the computing unit 1400 can comprise one or more processors 1402, a memory 1404, a storage device 1406, input / output interfaces 1408 (or "I / O interfaces 1408"), and a communication interface 1410, which can be communicatively coupled via a communication infrastructure (for example, a bus 1412). The Fig. 14 shown computer device 1400 and the one in Fig. The 14 components shown are not to be understood as limiting. Additional or alternative components may be used in other embodiments. Furthermore, in certain embodiments, the Computer 1400 comprises fewer components than shown. Fig. 14 shown. The in Fig. The 14 components of the Computer 1400 shown here will now be described in more detail.

[0139] In certain embodiments, the processor(s) 1402 includes hardware for executing instructions, for example, those comprising a computer program. By way of example, and without limitation, the processor(s) 1402 can retrieve (or fetch) instructions from an internal register, an internal cache, a memory 1404, or a storage device 1406, and decode and execute them.

[0140] The computing unit 1400 includes the memory 1404, which is coupled to the processor(s) 1402. The memory 1404 can be used to store data, metadata, and programs for execution by the processor(s). The memory 1404 can include one or more volatile and non-volatile memory types, such as random-access memory (RAM), read-only memory (ROM), a solid-state drive (SSD), flash memory, phase-change memory (PCM), or other types of data storage. The memory 1404 can be internal or distributed.

[0141] The computing unit 1400 includes the storage unit 1406 for storing data or instructions. As an example, and without limitation, the storage unit 1406 can include a non-volatile storage medium as described above. The storage unit 1406 can include a hard disk drive (“HDD”), flash memory, a universal serial bus (“USB”) drive, or a combination of these or other storage devices.

[0142] As shown, the Computer 1400 includes one or more I / O interfaces 1408, which are provided to enable a user to make inputs (for example, user inputs), receive outputs, and otherwise transfer data to and from the Computer 1400. These I / O interfaces 1408 can include a mouse, a keyboard or keypad, a touchscreen, a camera, an optical scanner, a network interface, a modem, other known I / O devices, or a combination of such I / O interfaces 1408. The touchscreen can be activated with a stylus or a finger.

[0143] The 1408 I / O interfaces can include one or more devices for displaying output to a user, including, but not limited to, a graphics engine, a display (for example, a screen), one or more output drivers (for example, display drivers), one or more loudspeakers, and one or more audio drivers. In certain embodiments, the 1408 I / O interfaces are configured to supply graphical data to a display for presentation to a user. The graphical data can represent one or more graphical user interfaces and / or other graphical content that may serve a particular implementation.

[0144] The Computer 1400 may also include a Communication Interface 1410. The Communication Interface 1410 may include hardware, software, or both. The Communication Interface 1410 provides one or more interfaces for communication (such as packet-based communication) between the computer and one or more other computers or one or more networks. For example, and without limitation, the Communication Interface 1410 may include a Network Interface Controller (NIC) or network adapter for communication with an Ethernet or other wired network, or a Wireless NIC (WNIC) or wireless adapter for communication with a wireless network, such as Wi-Fi. The Computer 1400 may also include the Bus 1412. The Bus 1412 may include hardware, software, or both that connect components of the Computer 1400.

[0145] The use of the terms "first," "second," "third," etc., in the preceding description and in the accompanying claims does not necessarily imply a specific order or number of elements. Generally, the terms "first," "second," "third," etc., are used to distinguish different elements as generic identifiers. Unless it is demonstrated that the terms "first," "second," "third," etc., denote a specific order, they should not be understood as denoting a specific order. Similarly, unless it is demonstrated that the terms "first," "second," "third," etc., denote a specific number of elements, they should not be understood as denoting a specific number of elements. For example, a first widget may be described as a widget with a first page, and a second widget as a widget with a second page.The use of the term "second page" in relation to the second widget can serve to distinguish this page of the second widget from the "first page" of the first widget, and does not necessarily mean that the second widget has two pages.

[0146] In the foregoing description, the invention has been described with reference to specific exemplary embodiments thereof. Various embodiments and aspects of the invention(s) are described with reference to the details discussed herein, and the accompanying drawings illustrate the different embodiments. The above description and the drawings serve to illustrate the invention and are not to be understood as limiting the invention. Numerous specific details are described to provide a comprehensive understanding of the various embodiments of the present invention.

[0147] The present invention can be embodied in other specific forms without departing from its spirit or essential features. The described embodiments are to be considered in every respect merely illustrative and not limiting. For example, the methods described herein can be carried out with fewer or more steps / actions, or the steps / actions can be carried out in different sequences. Furthermore, the steps / actions described herein can be repeated, carried out in parallel, or carried out in parallel with different instances of the same or similar steps / actions. The scope of the invention is therefore defined by the appended claims and not by the foregoing description. All modifications that fall within the meaning and scope of the claims are included therein.< / s>

Claims

[1] Computer-implemented method which includes: Receiving an input text string, where the input text string includes a style formatting element; Generating a modified input text string from the input text string, wherein the modified input text string includes an encoded tag that identifies the style formatting element; Generating a translated text string from the modified input text string using a neural machine translation model; and Applying the style formatting element to a word of the translated text string based on the encoded tag of the modified input text string. [2] Computer-implemented method according to claim 1, wherein generating the modified input text string comprises: Generating the coded tag at the beginning of a text passage that includes the style formatting element; and Generating an additional coded tag at the end of the text section that includes the style formatting element. [3] Computer-implemented method according to claim 2, wherein generating the translated text string comprises preserving the encoded tag and the additional encoded tag in the translated text string. [4] Computer-implemented method according to claim 2 or 3, wherein applying the style formatting element to the word of the translated text string comprises: Removing the coded tag and the additional coded tag from the translated text string; and Formatting the translated text string by applying the style formatting element to a translated text section that corresponds to the text section. [5] Computer-implemented method according to one of the preceding claims, wherein applying the style formatting element to the word of the translated text string comprises generating a graphic design element for the translated text string. [6] Computer-implemented method according to claim 5, further comprising providing the translated text string with the style formatting element applied to the word of the translated text string as a graphic design element for display via a user interface of a client device. [7] System which includes: a storage component; and one or more processing devices coupled to the storage component, wherein the one or more processing devices perform operations which include: Receiving an input text string, where the input text string includes a style formatting element; Generating a modified input text string from the input text string, wherein the modified input text string includes a first delimiter that identifies a beginning of the style formatting element and a second delimiter that identifies an end of the style formatting element; Generating a translated text string from the modified input text string using a large language model, wherein the translated text string includes the first delimiter and the second delimiter; and Applying the style formatting element to a word in the translated text string based on the first separator and the second separator. [8] System according to claim 7, wherein the operations further comprise generating a prompt that defines the first separator and the second separator and instructs the large language model to translate the input text string while preserving the first separator and the second separator in the translated text string. [9] System according to claim 8, wherein generating the translated text string comprises processing the prompt and the modified input text string by the large language model to generate the translated text string. [10] System according to any one of claims 7 to 9, comprising generating the modified input text string: Generating the first separator at the beginning of a text passage that includes the style formatting element; and Generating the second separator at the end of the section that includes the style formatting element. [11] System according to any one of claims 7 to 10, wherein generating the translated text string comprises preserving the first separator and the second separator in the translated text string. [12] System according to any one of claims 7 to 11, wherein applying the style formatting element to the word of the translated text string comprises: Removing the first and second separators from the translated text string; and Formatting the translated text string by applying the style formatting element to a translated text segment that corresponds to a text segment of the input text string that includes the style formatting element. [13] System according to any one of claims 7 to 12, wherein applying the style formatting element to the word of the translated text string comprises generating a graphic design element for the translated text string. [14] System according to claim 13, wherein the operations further comprise providing the translated text string with the style formatting element applied to the word of the translated text string as a graphic design element for display via a user interface of a client device. [15] Non-transitory computer-readable medium that stores executable instructions which, when executed by a processing device, cause the processing device to perform operations which include: Receiving an input text string, where the input text string includes a style formatting element; Generating a translated text string from the input text string using a neural machine translation model; Determining a translated word from the translated text string that corresponds to a formatted word from the input text string, using a large language model to process the translated text string based on a unigram mapping of the formatted word from the input text string; and Applying the style formatting element to the translated word of the translated text string. [16] Non-transitory computer-readable medium according to claim 15, wherein the processes further comprise: Generating the unigram mapping by identifying the formatted word of the input text string; and Processing the unigram mapping using the large language model to determine a translated unigram mapping of the translated word. [17] Non-transitory computer-readable medium according to claim 15 or 16, wherein the processes further comprise generating a prompt that defines the unigram mapping and instructs the large language model to provide a translated unigram mapping of the translated word. [18] Non-transitory computer-readable medium according to claim 17, wherein determining the translated word of the translated text string comprises processing the prompt, the input text string and the translated text string by the large language model to generate the translated unigram mapping of the translated word. [19] Non-transitory computer-readable medium according to any one of claims 15 to 18, wherein applying the style formatting element to the translated word of the translated text string comprises generating a graphic design element for the translated text string. [20] Non-transitory computer-readable medium according to claim 19, wherein the operations further comprise providing the translated text string with the style formatting element applied to the translated word of the translated text string as a graphic design element for display via a user interface of a client device.

Images