Document translation method and device, equipment, storage medium and program product

Abstract

The invention provides a document translation method and device, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a multi-modal document and a translation demand uploaded by a user; based on a document analysis mode corresponding to the file type of the multi-modal document, obtaining text content and format metadata of the multi-modal document; based on the format metadata, the text content is disassembled, and at least one piece of sub-text content is obtained; based on the translation demand, translating the sub-text content by using a large model to obtain at least one translated text; based on the format metadata, blank documents are filled with the translated texts, translated documents of the multi-modal document are obtained, and the format of the translated documents is consistent with that of the multi-modal document. According to the method, the translation efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a document translation method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] A multi-modal document refers to a document that integrates multiple information modalities, such as a document containing text, images, tables, charts, and the like. Traditional multi-modal document translation relies on a translator to manually translate the text in the multi-modal document into a translated document. Due to the low accuracy of manual translation, the related art uses a human-machine collaborative translation method to translate documents.

[0003] The human-machine collaborative translation method extracts text from a multi-modal document manually, translates the extracted text into a translated document using a translation machine, and finally manually arranges the translated document into a translated document. The document translation method in the related art is limited by the low efficiency of manual text extraction and document arrangement, resulting in low document translation efficiency in the related art. SUMMARY

[0004] The present application provides a document translation method, device, equipment, storage medium and program product to solve the technical problem of low translation efficiency in the related art.

[0005] In a first aspect, the present application provides a document translation method, comprising:

[0006] obtaining a multi-modal document and a translation requirement uploaded by a user;

[0007] obtaining text content and format metadata of the multi-modal document based on a document parsing method corresponding to a file type of the multi-modal document;

[0008] based on the format metadata, the text content is disassembled to obtain at least one sub-text content;

[0009] based on the translation requirement, the sub-text content is translated using a large model to obtain at least one translated text;

[0010] based on the format metadata, each translated text is filled into a blank document to obtain a translated document of the multi-modal document, and the format of the translated document is consistent with the format of the multi-modal document.

[0011] Optionally, the method further comprises:

[0012] According to the format metadata, the text content of the same modality is split according to a split structure to obtain at least one sub-text content; the split structure comprises a paragraph structure and / or a sentence structure.

[0013] Optionally, the method further comprises:

[0014] If the split structure is a paragraph structure and the length of the split sub-text content is greater than a preset length, the split sub-text content is split again based on a sentence structure until there is no sub-text content with a length greater than the preset length.

[0015] Optionally, the filling of the translated text into the blank document based on the format metadata to obtain the translation document of the multi-modal document comprises:

[0016] According to the format metadata, a format constraint corresponding to each modality data in the multi-modal document is generated; the format constraint at least comprises a style parameter of the multi-modal document.

[0017] According to the format constraint corresponding to each modality data and the corresponding filling mode, the translated text corresponding to each modality is filled into the blank document to obtain the translation document of the multi-modal document.

[0018] Optionally, the obtaining of the text content and the format metadata of the multi-modal document based on the document parsing mode corresponding to the file type of the multi-modal document comprises:

[0019] For the editable structured multi-modal document, the multi-modal document is decompressed to obtain a text content file and a style file;

[0020] The text content is extracted from the text content file, and the format metadata is extracted from the style file; the format metadata comprises at least one of the following: a paragraph style, a table structure, and an embedded object.

[0021] Optionally, the obtaining of the text content and the format metadata of the multi-modal document based on the document parsing mode corresponding to the file type of the multi-modal document comprises:

[0022] For the multi-modal document in a portable file format, a text layer, a graphics layer, and structure metadata are separated;

[0023] The text content is extracted from the text layer and the graphics layer;

[0024] The format metadata is extracted from the structure metadata.

[0025] Optionally, the obtaining the text content and the format metadata of the multi-modal document based on the document parsing manner corresponding to the file type of the multi-modal document comprises:

[0026] For an image type document, a text block in the multi-modal document is obtained.

[0027] Text in the text block is extracted based on optical character recognition to obtain the text content.

[0028] Text in the text content is matched with a plurality of text formats, and a matched text format is determined as the format metadata.

[0029] Optionally, the obtaining the multi-modal document uploaded by the user comprises:

[0030] The multi-modal document uploaded by the user through a client interface is received.

[0031] Optionally, the method further comprises:

[0032] Translation information is displayed on the client interface, the translation information comprising at least one of:

[0033] translation progress, preview information of the translated document, display of the translated document and the multi-modal document, and a download channel of the translated document.

[0034] In a second aspect, the application provides a document translation device, comprising:

[0035] An obtaining module is configured to obtain a multi-modal document uploaded by a user and a translation requirement.

[0036] An analyzing module is configured to obtain text content and format metadata of the multi-modal document based on a document parsing manner corresponding to a file type of the multi-modal document.

[0037] A disassembling module is configured to disassemble the text content based on the format metadata to obtain at least one sub-text content.

[0038] A translation module is configured to translate the sub-text content based on the translation requirement by using a large model to obtain at least one translated text.

[0039] A filling module is configured to fill each of the translated texts into a blank document based on the format metadata to obtain a translated document of the multi-modal document, the format of the translated document being consistent with the format of the multi-modal document.

[0040] In a third aspect, the application provides an electronic device, comprising a processor and a memory connected in communication with the processor.

[0041] The memory stores computer-executable instructions;

[0042] The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of the first aspect.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method according to any one of the first aspect.

[0044] In a fifth aspect, the present application provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the method according to any one of the first aspect.

[0045] The document translation method, device, equipment, storage medium and program product provided by the present application obtain a multi-modal document uploaded by a user and a translation requirement, obtain text content and format metadata of the multi-modal document based on a document analysis manner corresponding to a file type of the multi-modal document, disassemble the text content based on the format metadata to obtain at least one sub-text content, translate the sub-text content based on the translation requirement by using a large model to obtain at least one translated text, and fill each translated text into a blank document based on the format metadata to obtain a translated document of the multi-modal document, wherein the format of the translated document is consistent with the format of the multi-modal document. The method provided by the present application realizes automatic acquisition of text content and format metadata in a multi-modal document, and generates a translated document of the multi-modal document with a format consistent with the format of the multi-modal document based on the text content and the format metadata, thereby improving translation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0047] Figure 1 A flowchart of a document translation process in the related art;

[0048] Figure 2 A flowchart of a document translation method provided by an embodiment of the present application;

[0049] Figure 3 A flowchart of a document analysis process provided by an embodiment of the present application;

[0050] Figure 4 A flowchart of a specific embodiment of a document translation process provided by an embodiment of the present application;

[0051] Figure 5A structural schematic diagram of a document translation device provided by an embodiment of the present application is shown in the figure.

[0052] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in the figure.

[0053] The specific embodiments of the present application have been shown in the above figures, and will be described in more detail hereinafter. These figures and the written description are not intended to limit the scope of the present application in any way, but to illustrate the present application by reference to specific embodiments. DETAILED DESCRIPTION

[0054] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0055] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for users to choose authorization or refusal.

[0056] And the present application involves big data analysis of user information (including but not limited to personal biological characteristics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and uses artificial intelligence technology for automatic decision-making, and makes technical solutions based on automatic decision-making results that have a significant impact on personal rights and interests, provides appropriate operation portals for users to choose to agree or refuse automatic decision-making results; if the user chooses to refuse, the expert decision-making process is entered.

[0057] It should be noted that the document translation method, device, equipment, storage medium and program product provided by the present application can be used in the field of artificial intelligence technology, and can also be used in any field other than the field of artificial intelligence technology. The application field of the document translation method, device, equipment, storage medium and program product provided by the present application is not limited.

[0058] Figure 1 A flowchart of document translation in the related art is shown in the figure. Figure 1As shown, in the related art, the text in the document to be translated is manually extracted, and the extracted text is input into a translation machine, and the extracted text is translated into a translation by the translation machine, and finally the translation is manually arranged into a translation document. The document translation method in the related art is limited by the low efficiency of manual extraction of text and arrangement of documents, resulting in low document translation efficiency in the related art.

[0059] Therefore, an embodiment of the present application proposes a document translation method. The text content and format metadata of a multi-modal document are obtained by a document parsing method corresponding to the file type of the multi-modal document. The text content is translated using a large model, and each translated text is filled into a blank document according to the format metadata, to obtain a translation document with a format consistent with that of the multi-modal document. The method of the present application automatically obtains the text content and format metadata in the multi-modal document, and generates a translation document of the multi-modal document with a format consistent with that of the multi-modal document based on the text content and format metadata, thereby improving the translation efficiency.

[0060] An embodiment of the present application is applied to an application program, a website or an applet with a document translation function. On the application program, website or applet, the document translation function is implemented, for example, a website for document translation implements the function of translating a user-uploaded original document into a translation document. For another example, a third-party application calls a document translation method through an application programming interface (API) to implement the document translation function. The following takes a translation system as an example to illustrate the technical solution of the present application.

[0061] The following describes the technical solution of the present application and how the technical solution of the present application solves the above technical problems in detail with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in detail in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0062] Figure 2 The flowchart of the document translation method provided by an embodiment of the present application is shown in FIG. 1. Figure 2 As shown, the document translation method can include:

[0063] S201, obtaining a multi-modal document uploaded by a user and a translation requirement.

[0064] The multi-modal document refers to a document to be translated that integrates multiple information modalities. For example, the multi-modal document can include text, images, videos, tables, charts, attachments, etc.

[0065] The translation requirement is a language used by the translated text as instructed by the user, i.e., a target language of translation. Optionally, the user can indicate a language used by the multi-modal document and a target language of translation. For example, the translation requirement is to translate Chinese into English.

[0066] Optionally, the multi-modal document uploaded by the user can be acquired by receiving the multi-modal document uploaded by the user through the client interface.

[0067] Illustratively, the client interface of the translation system can display an upload button for uploading a file and options for selecting a language of the multi-modal document and a target language of translation. The user can click the upload button, select a storage path of the multi-modal document to be translated, and select the language of the multi-modal document and the target language of translation. The translation system acquires the multi-modal document uploaded by the user according to the storage path and acquires the language of the multi-modal document and the target language of translation selected by the user, and determines the language of the multi-modal document and the target language of translation as the translation requirement.

[0068] Illustratively, the client interface of the translation system can display a drag area for uploading a file and options for selecting a language of the multi-modal document and a target language of translation. The user drags the file to the drag area, and the translation system acquires the multi-modal document based on listening to the drag event and determines the language of the acquired multi-modal document and the target language of translation as the translation requirement.

[0069] S202, based on a document parsing manner corresponding to a file type of the multi-modal document, acquiring text content and format metadata of the multi-modal document.

[0070] The file type refers to a format specification for storing and representing data of the multi-modal document in a computer system. For example, the file type can be an editable structured, a Portable Document Format (PDF), an image type document, etc. The editable structured multi-modal document can be a Word type multi-modal document.

[0071] The text content is the text in the multi-modal document. The format metadata represents the format of the content in the multi-modal document. Illustratively, the font, size, paragraph structure, and table format of the text, etc.

[0072] Illustratively, the translation system can determine the file type according to the extension of the multi-modal document. For example, for the multi-modal document with the extension of “docx” and “xlsx”, the file type of the multi-modal document is editable structured, and for the multi-modal document with the extension of “pdf”, the file type of the multi-modal document is a portable document format.

[0073] Different file types correspond to different document parsing manners. The translation system can obtain the text content and format metadata of the multi-modal document based on the document parsing manner corresponding to the file type of the multi-modal document. For example, editable structured documents are a type of document organized by standardized formats. The document parsing manner corresponding to editable structured documents can be decompression of the document, which can obtain text content files and style files, etc. For example, if the file type of the multi-modal document is editable structured, the translation system can adopt the document parsing manner corresponding to editable structured, decompress the multi-modal document, obtain document.xml and styles.xml files from the decompressed files, and obtain the text content from the document.xml file and the format metadata from the styles.xml file.

[0074] S203. Based on the format metadata, the text content is disassembled to obtain at least one sub-text content.

[0075] The translation system can disassemble the text content of the multi-modal document according to the format metadata to obtain at least one sub-text content. For example, the translation system can split the text content by paragraphs according to the paragraph structure to obtain multiple paragraph texts, and each paragraph text is a sub-text content.

[0076] For example, the translation system can split the text content by sentences according to the sentence structure to obtain multiple sentence texts, and each sentence text is a sub-text content. Alternatively, the translation system can split the text content by text length based on the splitting of the text content by the sentence structure, and in the case that the text length of multiple consecutive sentences is less than or equal to a preset text length, the multiple sentences are determined as a sub-text content.

[0077] S204. Based on the translation requirement, the sub-text content is translated using a large model to obtain at least one translated text.

[0078] For example, the translation system can translate each sub-text content one by one. Taking a large model based on a Transformer as an example, the large model learns the mapping relationship from the source language to the target language through training. In the translation process, each sub-text content is input into the large model to obtain the translated text output by the large model.

[0079] For example, the large model can be translated based on a statistical method, in which the large model uses a bilingual corpus to statistically analyze the corresponding relationship between words, phrases, and sentences in the source language and the target language. The translation system can input each sub-text content into the large model, and the large model outputs the corresponding translated text based on the words, phrases, and sentences in the input sub-text content.

[0080] S205, fill the translated texts into the blank document based on the format metadata to obtain a translation document of the multi-modal document, and the format of the translation document is consistent with the format of the multi-modal document.

[0081] For example, the translation system can fill the translated texts of each section into the blank document according to the format metadata of each section. For example, for a table, the translation system can fill the translated texts into the blank document according to the table structure.

[0082] For example, the translation system can define the font format of the content of each section according to the format metadata of each section, fill the translated texts into the generated document according to the specified font format based on the text generation method. Alternatively, the translation system can set the font style for the content of each section, and automatically fill each section into the document according to the set font style through the template engine.

[0083] The document translation method of the embodiment of the present application obtains the text content and format metadata of the multi-modal document based on the document parsing method corresponding to the file type of the multi-modal document, and fills the translated texts into the blank document based on the format metadata to obtain a translation document of the multi-modal document, and the format of the translation document is consistent with the format of the multi-modal document. The method of the present application realizes automatic acquisition of text content and format metadata in the multi-modal document, and generates a translation document of the multi-modal document with a format consistent with the format of the multi-modal document based on the text content and format metadata, thereby improving the translation efficiency.

[0084] Next, how to obtain the text content and format metadata of the multi-modal document based on the document parsing method corresponding to the file type of the multi-modal document in the embodiment of the present application is described.

[0085] Next, taking the file types including image type documents, portable document format, and editable structured as examples, the method of parsing the multi-modal document is described.

[0086] Figure 3 The flowchart of parsing the document provided by the embodiment of the present application is shown. As shown in Figure 3 The translation system can obtain the text content and format metadata of the multi-modal document according to the document parsing method corresponding to the file type.

[0087] Alternatively, the translation system can decompress the multi-modal document to obtain a text content file and a style file, extract the text content from the text content file, and extract the format metadata from the style file for the multi-modal document of editable structured.

[0088] The editable structured multi-modal document can be a Word or Excel type document. The format metadata includes at least one of the following: paragraph style, table structure, and embedded object. The paragraph style includes, for example, content of each paragraph, table style, text style, alignment, layout position, etc. The table structure can be the number of rows and columns, and the embedded object is data that is not translated, such as attachments, links, illustrations, etc.

[0089] For example, if the file type of the multi-modal document is editable structured, the translation system can decompress the multi-modal document to obtain a plurality of files, obtain a document.xml file and a styles.xml file from the plurality of decompressed files, the document.xml file being document text, and the styles.xml file being document format. The translation system can obtain text content from the document.xml file and format metadata from the styles.xml file.

[0090] Alternatively, the translation system can separate the text layer, the graphics layer, and the structure metadata for the multi-modal document in portable document format, extract text content from the text layer and the graphics layer, and extract format metadata from the structure metadata.

[0091] The text layer is the part of the PDF that stores text, usually containing text information such as characters, words, etc. in the document. The graphics layer is the part of the PDF that stores images and vector graphics, usually containing non-text content such as pictures, charts, lines, etc. in the document. The structure metadata is the part of the PDF that stores document structure, attributes, and meta information.

[0092] The translation system can separate the text layer, the graphics layer, and the structure metadata of the multi-modal document, and obtain text content from the obtained text layer and graphics layer, and extract format metadata from the structure metadata. For example, the translation system can separate the multi-modal document based on related separation technologies in the programming language tool library, and obtain the text layer, the graphics layer, and the structure metadata, obtain text content from the obtained text layer and graphics layer, and extract format metadata from the structure metadata.

[0093] Alternatively, the translation system can obtain a text block in the multi-modal document for an image type document, extract text in the text block based on optical character recognition (OCR), obtain text content, match the text in the text content with a plurality of text formats, and determine the matched text format as format metadata.

[0094] Taking a multi-modal document including text and illustrations as an example, the translation system can identify a text region and an illustration region in the document based on image processing, take the text in each paragraph in the text region as a text block, and identify the text in the image based on OCR for each text block to obtain text content.

[0095] The translation system can match the text in the text content with a plurality of text formats based on the identified text, and determine the text format with the highest matching degree as format metadata. For example, the identified "FAN" in the text block can be matched based on different fonts of "FAN", and the font with the highest matching degree can be determined as the format metadata of the text block. Alternatively, the translation system can also match the identified "FAN" in the text block with a plurality of font sizes and fonts of "FAN", and determine the font size and font with the highest matching degree as the format metadata of the text block.

[0096] The above embodiments illustrate how to obtain text content and format metadata of a multi-modal document based on a document parsing method corresponding to the file type of the multi-modal document when the file type of the multi-modal document is an image type document, a portable file format, or an editable structured document. The following describes how to split the text content based on the format metadata to obtain at least one sub-text content after obtaining the text content and format metadata.

[0097] Alternatively, the translation system can split the text content based on the format metadata according to a split structure to obtain at least one sub-text content.

[0098] The split structure includes a paragraph structure and / or a sentence structure.

[0099] For example, taking the sentence structure as an example, the translation system can split the text content according to the format metadata based on the sentence structure. For example, the translation system sets a character threshold of 800, the number of characters in sentence 1 is 400, and the number of characters in sentence 2 is 500. If the number of characters in sentence 2 is 400, the translation system splits sentence 1 and sentence 2 into two sub-text contents, and if the number of characters in sentence 2 is 400, the translation system determines sentence 1 and sentence 2 as the same sub-text content.

[0100] For example, taking the paragraph structure as an example, the translation system can split the text content according to the format metadata based on the paragraph structure, and take the text in each paragraph as a sub-text content.

[0101] Alternatively, if the split structure is a paragraph structure and the length of the split sub-text content is greater than a preset length, the translation system can split the split sub-text content again based on the sentence structure until there is no sub-text content with a text length greater than the preset length.

[0102] Exemplarily, with a preset length of 800 characters, the length of paragraph 1 is 400 characters, and the length of paragraph 2 is 2000 characters, the translation system determines the text of paragraph 1 as one subtext content, and splits paragraph 2 again. For example, the translation system can divide the text in paragraph 2 by sentence, divide the first 10 sentences in paragraph 2 as the first subtext content, and the text length is 780 characters, then continue to split the remaining 1220 characters. For example, the translation system splits the remaining 1220 characters by sentence into a subtext content with a text length of 800 characters, and a subtext content with a text length of 420 characters. In the current mode, the translation system splits paragraph 3 into 3 subtext contents.

[0103] Based on the above embodiments, the translation system can obtain at least one subtext content, and further, the translation system translates the subtext content based on the translation requirement by using a large model to obtain at least one translated text.

[0104] Exemplarily, the translation system can translate by a statistical method based on a large model, in which the large model counts the corresponding relationship between the source language and the target language of the vocabulary, phrases and sentences through bilingual corpus. In translation, the large model selects the most possible translation result according to these statistical rules. For example, taking the translation requirement of Chinese to English as an example, “she” often corresponds to “she”. Through these statistical information, the large model can generate the corresponding translation text under the condition of inputting the subtext content. In this way, the large model can also decompose the sentence into phrases and translate according to the statistical rules of the phrases in the corpus. Alternatively, the large model can also combine rule translation, decompose the subtext content according to the language rules such as grammar, vocabulary and the like of the source language and the target language, and realize translation combined with statistical corresponding relationship.

[0105] Exemplarily, the translation system can perform text translation based on a large model constructed by a neural network. The model learns the mapping relationship from the source language to the target language through training. Taking a Transformer-based model as an example, in the translation process, after inputting the sentence in the subtext content into the large model, the large model encodes the sentence into a vector representation. The encoding process combines the relationship between words, and through the self-attention mechanism, each word can pay attention to other words in the sentence. For example, in the translation of the sentence “Translating documents is the technique of translating the text……”, the model can consider the information associated with “documents” such as “Translating” “is the technique” and “the text” when encoding “documents”. The decoder of the large model generates translated text step by step according to the encoded vector. This architecture can better capture long-distance dependencies, such as in long sentences, the information at the beginning of the sentence is related to the translation at the end of the sentence.

[0106] The above embodiments illustrate the method of obtaining translated text by the translation system based on the large model, and then the translation system can fill each translated text into the blank document based on the format metadata to obtain the translated document of the multi-modal document.

[0107] Optionally, the translation system can generate format constraints corresponding to each modality data in the multi-modal document according to the format metadata, fill the translated text corresponding to each modality into the blank document according to the format constraints corresponding to each modality data and the corresponding filling mode, and obtain the translated document of the multi-modal document.

[0108] The format constraints at least include style parameters of the multi-modal document.

[0109] The translation system can generate format constraints corresponding to each modality data in the multi-modal document according to the format metadata.

[0110] Exemplarily, the translation system generates format constraints corresponding to each modality data in the multi-modal document according to the format metadata, using a markup language to mark the font format of the text. For example, by <h1>The label represents the title, The label represents a paragraph.

[0111] Illustratively, the translation system can utilize the programming interface of the document processing library to dynamically generate the format constraints corresponding to each modality data in the multi-modal document. For example, a paragraph is added through the add_paragraph method, and the font style is set through the style attribute.

[0112] Illustratively, the translation system can use the document generation tools in the related art to define the structure and format of the document, and generate the format constraints corresponding to each modality data in the multi-modal document.

[0113] Figure 4 The flowchart of the specific embodiment of the document translation provided by the embodiments of the present application is shown in the figure. Figure 4 As shown, taking a multi-modal document of the Word type as an example, the multi-modal document includes text, illustrations, tables, and attachments, which can include the following steps:

[0114] S401, obtaining a multi-modal document uploaded by a user and a translation requirement.

[0115] S402, for an editable structured multi-modal document, decompressing the multi-modal document to obtain a text content file and a style file.

[0116] S403, extracting text content from the text content file, and extracting format metadata from the style file.

[0117] The text content includes text and table text, and the format metadata includes paragraph structure, text font, text size, table row and column number, and embedded objects, i.e., illustrations and attachments.

[0118] S404, based on the format metadata, disassembling the text content to obtain at least one sub-text content.

[0119] S405, based on the translation requirement, using a large model to translate the sub-text content to obtain at least one translated text.

[0120] S406, according to the format metadata, generating format constraints corresponding to text, format constraints corresponding to tables, and format constraints of embedded objects in the multi-modal document.

[0121] Illustratively, the format constraints of the embedded objects include illustration size, display position, and attachment display position.

[0122] S407, according to the format constraints corresponding to each modality data and the corresponding filling method, filling the translated text corresponding to each modality into a blank document to obtain a translated document of the multi-modal document.

[0123] According to the format constraint corresponding to the text, the translation text corresponding to the text is filled into the blank document, a table is generated in the document according to the format constraint corresponding to the table, the translation text corresponding to the table text is filled into the document in the format of the table, according to the format constraint of the embedded object, the illustrations and attachments are inserted into the document, and the translation document of the multi-modal document is obtained.

[0124] It should be noted that, in Figure 4 The various processing steps (S401-S407) shown in the embodiments can refer to the specific implementation of the same or similar steps in the above embodiments, and Figure 4 The various processing steps shown in the embodiments do not constitute a specific limitation on the document translation process. In other embodiments of the present application, the document translation process can include more or fewer steps than Figure 4 the embodiments. For example, the document translation process can include Figure 4 part of the steps in the embodiments, or Figure 4 some steps in the embodiments can be replaced by steps with the same function, or some steps in the embodiments can be split into multiple steps, etc.

[0125] Optionally, the translation system can also display translation information on the client interface, the translation information including at least one of the following: translation progress, preview information of the translated document, display of the translated document and the multi-modal document, and download channel of the translated document.

[0126] Illustratively, the translation system can also display the download channel of the translated document on the client interface. For example, the translation website can transmit the translated document to the storage location specified by the user in response to the user's operation of the download button. In this way, the translation system displays a button representing download on the client interface, and when the user clicks the button, the translation system captures the operation and transmits the translated document to the storage location specified by the user.

[0127] Optionally, the translation system can provide the user with multiple file formats for downloading the translated document, and convert the file format of the translated document to the file format specified by the user, and then store it in the storage location specified by the user.

[0128] Exemplarily, the translation system can also display preview information of the translated document and a comparison display of the translated document and the multimodal document on the client interface. For example, in response to a user clicking a preview button, the translation system can display the translated document in a text box on the client interface. For example, the translation system can display a comparison display button on the client interface. When the user clicks the button, the translation system captures the operation and creates two side-by-side text boxes, which are displayed on the client interface. One text box is used to display the multimodal document, and the other text box is used to display the translated document. Optionally, when the user clicks the next page of any text box, the content displayed in the other text box will turn pages synchronously.

[0129] Optionally, the translation system can also establish an index association between each original sentence in the multimodal document and the corresponding translation. In this way, when the user selects an original sentence in the multimodal document in a text box, the index of the original sentence is obtained, and the corresponding translation sentence is determined based on the index, and the translation sentence is highlighted in another text box.

[0130] For example, the translation system can also display the translation progress of the translated document on the client interface. For example, during the translation process, the translation system can calculate the ratio of the number of currently translated paragraphs to the total number of paragraphs in the original document and display the ratio on the user interface.

[0131] In this way, the translated information is displayed on the client interface, which can improve user friendliness.

[0132] The document translation method of the embodiment of the present application obtains a multimodal document uploaded by a user and a translation requirement, obtains the text content and format metadata of the multimodal document based on the document parsing method corresponding to the file type of the multimodal document, decomposes the text content based on the format metadata to obtain at least one sub-text content, translates the sub-text content using a large model based on the translation requirement, obtains at least one translated text, and fills each translated text into a blank document based on the format metadata to obtain a translation document of the multimodal document, the format of the translation document being consistent with the format of the multimodal document. The method of the present application realizes the automatic acquisition of the text content and format metadata in the multimodal document, and generates a translation document of the multimodal document with a format consistent with the format of the multimodal document based on the text content and format metadata, thereby improving translation efficiency.

[0133] Figure 5 This is a schematic diagram of the structure of a document translation device provided in an embodiment of the present application. Figure 5 As shown, the document translation apparatus 500 may include, for example: an acquisition module 501, a parsing module 502, a decomposition module 503, a translation module 504, and a filling module 505. Optionally, a display module may also be included.

[0134] The acquisition module 501 is configured to acquire a multi-modal document uploaded by a user and a translation requirement;

[0135] The analysis module 502 is configured to acquire text content and format metadata of the multi-modal document based on a document analysis manner corresponding to a file type of the multi-modal document.

[0136] The disassembly module 503 is configured to disassemble the text content based on the format metadata to obtain at least one sub-text content.

[0137] The translation module 504 is configured to translate the sub-text content based on the translation requirement by using a large model to obtain at least one translated text.

[0138] The filling module 505 is configured to fill each of the translated texts into a blank document based on the format metadata to obtain a translation document of the multi-modal document, and a format of the translation document is consistent with a format of the multi-modal document.

[0139] In a possible implementation, the disassembly module 503 is specifically configured to:

[0140] The text content of the same modality is split according to a split structure based on the format metadata to obtain at least one sub-text content, and the split structure includes a paragraph structure and / or a sentence structure.

[0141] In a possible implementation, the disassembly module 503 is further configured to:

[0142] If the split structure is the paragraph structure and the sub-text content after splitting includes text with a length greater than a preset length, the sub-text content after splitting is split again based on the sentence structure until there is no sub-text content with a text length greater than the preset length.

[0143] In a possible implementation, the filling module 505 is specifically configured to:

[0144] The format constraints corresponding to each modality data in the multi-modal document are generated according to the format metadata, and the format constraints at least include style parameters of the multi-modal document.

[0145] The translated texts corresponding to each modality are filled into the blank document according to the format constraints corresponding to each modality data and the corresponding filling manner to obtain a translation document of the multi-modal document.

[0146] In a possible implementation, the analysis module 502 is specifically configured to:

[0147] For the editable structured multi-modal document, the multi-modal document is decompressed to obtain a text content file and a style file.

[0148] extracting the text content from the text content file, and extracting the format metadata from the style file; the format metadata comprises at least one of the following: paragraph style, table structure, and embedded object.

[0149] In a possible implementation, the parsing module 502 is specifically configured to:

[0150] separating a text layer, a graphics layer, and structure metadata for the multi-modal document in the portable file format;

[0151] extracting the text content from the text layer and the graphics layer;

[0152] extracting the format metadata from the structure metadata.

[0153] In a possible implementation, the parsing module 502 is specifically configured to: for an image type document, obtaining a text block in the multi-modal document;

[0154] extracting text in the text block based on optical character recognition to obtain the text content;

[0155] matching characters in the text content with a plurality of character formats, and determining a matched character format as the format metadata.

[0156] In a possible implementation, the obtaining module 501 is specifically configured to: receiving the multi-modal document uploaded by the user through a client interface.

[0157] In a possible implementation, the display module is specifically configured to:

[0158] displaying translation information on the client interface, the translation information comprising at least one of the following:

[0159] translation progress, preview information of the translated document, display of the translated document and the multi-modal document, and a download channel of the translated document.

[0160] Figure 6 A structural schematic diagram of an electronic device is provided in the embodiments of the present application. As shown in the figure, the electronic device can include at least one processor 601 and a memory 602. Figure 6

[0161] The memory 602 is configured to store a program. Specifically, the program can include program code, and the program code includes computer operation instructions.

[0162] The memory 602 can include a high-speed RAM memory, and can also include a non-volatile memory.​

[0163] The processor 601 is configured to execute the computer-executable instructions stored in the memory 602, so as to implement the method of the foregoing method embodiments. The processor 601 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0164] Optionally, the electronic device can further include a communication interface 603. In a specific implementation, if the communication interface 603, the memory 602, and the processor 601 are independently implemented, the communication interface 603, the memory 602, and the processor 601 can be connected to each other through a bus and complete communication therebetween.

[0165] Optionally, in a specific implementation, if the communication interface 603, the memory 602, and the processor 601 are integrated on a chip, the communication interface 603, the memory 602, and the processor 601 can complete communication through an internal interface.

[0166] The present application further provides a computer-readable storage medium, which can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), and various storage program codes. Specifically, the computer-readable storage medium stores program instructions, which are used to implement the actions of the method embodiments.

[0167] The present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the actions of the method embodiments are implemented.

[0168] In addition, each functional unit / module in each embodiment of the present application can be integrated in one unit / module, or each unit / module can exist physically, or two or more units / modules can be integrated together. The integrated unit / module can be implemented in the form of hardware or in the form of a software program module.

[0169] If the integrated units / modules are implemented in the form of hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors. Unless otherwise specified, the processor can be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit can be any appropriate storage medium, such as a variable resistance memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0170] If the integrated units / modules are implemented in the form of software program modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the essential part of the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk and various program code storage media.

[0171] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments. The technical features of the above embodiments can be combined arbitrarily, and in order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0172] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes ROM, RAM and various storage media that can store program codes. < / h1>

Claims

1. A document translation method, characterized in that: include: Obtain multimodal documents and translation requirements uploaded by users; Acquiring text content and format metadata of the multimodal document based on a document parsing method corresponding to the file type of the multimodal document; Based on the format metadata, the text content is disassembled to obtain at least one sub-text content; Based on the translation requirement, the sub-text content is translated using the large model to obtain at least one translated text; Based on the format metadata, each of the translated texts is filled into a blank document to obtain a translation document of the multimodal document, wherein the format of the translation document is consistent with the format of the multimodal document.

2. The method according to claim 1, characterized in that The step of decomposing the text content based on the format metadata to obtain at least one sub-text content includes: Based on the format metadata, the text content of the same modality is split according to a split structure to obtain at least one sub-text content; the split structure includes: a paragraph structure and / or a sentence structure.

3. The method according to claim 2, characterized in that The method further comprises: If the split structure is a paragraph structure, and the text length of the split sub-text content is greater than the preset length, the split sub-text content is split again based on the sentence structure until there is no sub-text content with a text length greater than the preset length.

4. The method according to claim 1, wherein Filling each of the translated texts into a blank document based on the format metadata to obtain a translated document of the multimodal document includes: Generating format constraints corresponding to each modal data in the multimodal document according to the format metadata; the format constraints at least include: style parameters of the multimodal document; According to the format constraints corresponding to each modality data and the corresponding filling method, the translation text corresponding to each modality is filled into the blank document to obtain the translation document of the multimodal document.

5. The method according to any one of claims 1 to 4, characterized in that The acquiring of the text content and format metadata of the multimodal document based on the document parsing method corresponding to the file type of the multimodal document includes: For the editable structured multimodal document, decompress the multimodal document to obtain a text content file and a style file; The text content is extracted from the text content file, and the format metadata is extracted from the style file; the format metadata includes at least one of the following: paragraph style, table structure and embedded object.

6. The method according to any one of claims 1 to 4, characterized in that The acquiring of the text content and format metadata of the multimodal document based on the document parsing method corresponding to the file type of the multimodal document includes: For the multimodal document in a portable document format, separating a text layer, a graphics layer, and structural metadata; Extracting the text content from the text layer and the graphics layer; The format metadata is extracted from the structure metadata.

7. The method according to any one of claims 1 to 4, characterized in that The acquiring of the text content and format metadata of the multimodal document based on the document parsing method corresponding to the file type of the multimodal document includes: For an image-type document, obtaining a text block in the multimodal document; extracting text from the text block based on optical character recognition to obtain the text content; The characters in the text content are matched with a plurality of character formats, and the matched character formats are determined as the format metadata.

8. The method according to any one of claims 1 to 4, characterized in that The step of obtaining the multimodal document uploaded by the user includes: Receive the multimodal document uploaded by the user through the client interface.

9. The method according to any one of claim 8, characterized in that: The method further comprises: Displaying translation information on the client interface, the translation information including at least one of the following: Translation progress, preview information of the translated document, comparison display of the translated document and the multimodal document, and download channel of the translated document.

10. A document translation device, characterized in that: include: The acquisition module is used to obtain multimodal documents and translation requirements uploaded by users; a parsing module, configured to obtain text content and format metadata of the multimodal document based on a document parsing method corresponding to the file type of the multimodal document; a disassembly module, configured to disassemble the text content based on the format metadata to obtain at least one sub-text content; A translation module, configured to translate the sub-text content using a large model based on the translation requirement to obtain at least one translated text; A filling module is used to fill each of the translated texts into a blank document based on the format metadata to obtain a translation document of the multimodal document, wherein the format of the translation document is consistent with the format of the multimodal document.

11. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.

Images