Document translation method and device and related product

Abstract

The invention discloses a document translation method, a document translation device and related products, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. Translating the document text to obtain a text translation result; extracting a first picture from the first document, and identifying a picture text from a plurality of text regions in the first picture; translating the picture text to obtain a picture translation result; if the plurality of text areas comprise a first text area and a second text area which are intersected in the longitudinal coordinate interval, and the text length of a first translation result filled in the first text area in the picture translation result is greater than the target text length, reducing the word size of the first translation result; filling a plurality of text regions in the first picture with the adjusted picture translation result to obtain a second picture; and replacing the document text with the text translation result, and replacing the first picture with the second picture to obtain a second document. In this way, the original format of the document is maintained, and the document translation effect is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a document method, device and related products. Background Art

[0002] Document translation is the process of converting a document written in one natural language (one language) into another natural language (another language). With the advancement of globalization, more and more people need to communicate and collaborate across language barriers. For example, to adapt to the reading habits of employees who speak different languages, document translation is necessary during transmission.

[0003] Documents such as Word, Excel, and PPT may contain both text and images, and images may also contain text that needs to be translated. Related document translation methods can translate both the text in the document and the text in the image, but require the user to manually specify the image to be translated. The image containing the translation results is often displayed in a separate interface, requiring the user to compare and view them, or manually insert the image containing the translation results into the document. Furthermore, the translation effect of text in images is poor, which can easily affect the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a document translation method, device and related products, the purpose of which is to translate both the document text and image text in a document while maintaining the original format of the document, improving the document translation effect, and thereby enhancing the user experience.

[0005] A first aspect of the present application provides a document translation method, wherein a first document includes document text and image text, and the image text is the text of a first image in the first document. The method includes:

[0006] Translating the document text to obtain a text translation result;

[0007] Extracting the first picture from the first document, and identifying the picture text from multiple text areas in the first picture;

[0008] Translating the image text to obtain an image translation result;

[0009] If the plurality of text areas include a first text area and a second text area whose longitudinal coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result; the target text length is the distance between the longitudinal boundaries of the first text area and the second text area on the same side;

[0010] Filling the adjusted image translation result into multiple text areas in the first image to obtain a second image;

[0011] The document text is replaced with the text translation result, and the first image is replaced with the second image to obtain a second document.

[0012] A second aspect of the present application provides a document translation device, wherein a first document includes document text and image text, and the image text is the text of a first image in the first document, including:

[0013] A document text translation module, configured to translate the document text to obtain a text translation result;

[0014] An image text recognition module is configured to extract the first image from the first document and recognize the image text from multiple text areas in the first image;

[0015] A picture text translation module, used to translate the picture text to obtain a picture translation result;

[0016] a font size adjustment module configured to reduce the font size of the first translation result if the plurality of text areas include a first text area and a second text area whose longitudinal coordinate intervals intersect, and if the length of the text of the first translation result filled in the first text area in the image translation result is greater than a target text length; the target text length being the distance between longitudinal boundaries of the first text area and the second text area on the same side;

[0017] A result filling module, configured to fill the adjusted image translation result into multiple text areas in the first image to obtain a second image;

[0018] The document acquisition module is used to replace the document text with the text translation result and replace the first picture with the second picture to obtain a second document.

[0019] A third aspect of the present application provides a document translation device, the device comprising a processor and a memory:

[0020] The memory is used to store a computer program and transmit the computer program to the processor;

[0021] The processor is configured to execute the steps of the document translation method provided in the first aspect according to the instructions in the computer program.

[0022] A fourth aspect of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a document translation device, the steps of the document translation method provided in the first aspect are implemented.

[0023] A fifth aspect of the present application provides a computer program product, comprising a computer program, which, when executed by a document translation device, implements the steps of the document translation method provided in the first aspect.

[0024] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0025] In the technical solution of the present application, a first document includes document text and a first image, and the first image includes image text. When translating the first document, the document text can be first translated to obtain a text translation result; the first image is extracted from the first document, and the image text is identified from multiple text areas in the first image; the image text in the first image is then translated to obtain an image translation result; when the multiple text areas in the first image include a first text area and a second text area, the longitudinal coordinate area of ​​the first text area and the longitudinal coordinate area of ​​the second text area intersect, indicating that if the image translation result is filled into the two text areas, text overlap may occur, then the text length of the first translation result filled into the first text area and the target text length can be determined. The target text length refers to the distance between the longitudinal boundaries of the first text area and the second text area on the same side. If the target text length is greater than the target text length, indicating that text overlap may occur, the font size of the first translation result can be reduced; then, the adjusted image translation result is filled into the multiple text areas in the first image to obtain a second image; then, the document text is replaced with the text translation result, and the first image is replaced with the second image to obtain a second document. Since this application can not only generate a second document including text translation results and a second image with image translation results, but also avoid the situation where text overlaps in the second image of the image translation results corresponding to the image text, it can maintain the original format of the document text and image text in the first document, improve the document translation effect, and do not require users to compare and view or manually operate, which greatly improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A scenario architecture diagram of a document translation method provided in an embodiment of the present application;

[0027] Figure 2 A flowchart of a document translation method provided in an embodiment of the present application;

[0028] Figure 3 A flowchart of a document text translation process provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a text area provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of a data structure of a data unit provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of another text area provided in an embodiment of the present application;

[0032] Figure 7 A flowchart of a picture-to-text translation process provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram comparing a first image and a second image provided in an embodiment of the present application;

[0034] Figure 9 A schematic diagram of a comparison between a first document and a second document provided in an embodiment of the present application;

[0035] Figure 10 A schematic diagram of a page of a picture folder provided in an embodiment of the present application;

[0036] Figure 11 A schematic diagram of a translation platform page provided in an embodiment of the present application;

[0037] Figure 12 A schematic diagram of the structure of a document translation device provided in an embodiment of the present application;

[0038] Figure 13 A schematic diagram of the structure of the server in the embodiment of the present application;

[0039] Figure 14 This is a structural diagram of a terminal device in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The document translation solution in the related art can translate the text in the document. If the text in the pictures included in the document needs to be translated, the user is required to manually specify the pictures to be translated, and the pictures including the translation results are usually displayed in other interfaces. The user needs to compare and view the original document and the pictures in other interfaces, or manually insert the pictures including the translation results into the document, and the translation effect of the text in the pictures is also poor. These all affect the document translation effect and are likely to affect the user experience.

[0041] In view of the above problems, the present application provides a document translation method, device and related products, the purpose of which is to translate both the document text and the image text in the document and maintain the original format of the document, improve the document translation effect, and thus enhance the user experience. In the technical solution provided in the present application, the first document includes document text and image text, and the image text is the text of the first image in the first document. The document translation method includes: translating the document text to obtain a text translation result; extracting the first image from the first document, and identifying the image text from multiple text areas in the first image; translating the image text to obtain an image translation result; if the multiple text areas include a first text area and a second text area with an intersection in the vertical coordinate interval, and the text length of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result; the target text length is the distance between the vertical boundaries of the first text area and the second text area on the same side; filling the adjusted image translation result into multiple text areas in the first image to obtain a second image; replacing the document text with the text translation result, and replacing the first image with the second image to obtain a second document.

[0042] In this way, both the document text and the image text in the first document can be translated, and the text translation results and the image translation results are both in the second document. The user does not need to compare or manually operate. At the same time, it can avoid the situation where the generated image translation results have text overlap. Therefore, this application can maintain the original format of the document text and image text in the first document, greatly improve the document translation effect, and thus improve the user experience.

[0043] Next, the execution subject of the document translation method provided in the embodiment of the present application is introduced.

[0044] The execution subject of the document translation method provided in the embodiment of the present application can be a terminal device. The terminal device can execute the document translation method to translate a first document including document text and image text. For example, the terminal device translates the document text to obtain a text translation result. As an example, the terminal device can specifically include but is not limited to a mobile phone, a desktop computer, a tablet computer, a laptop computer, a PDA, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc. The execution subject of the document translation method provided in the embodiment of the present application can also be a server. For example, the server translates the document text to obtain a text translation result.

[0045] Furthermore, the document translation method provided in the embodiments of the present application can also be collaboratively executed by a terminal device and a server. The terminal and the server can be connected directly or indirectly via wired or wireless communication, which is not a limitation of the present application. Therefore, the embodiments of the present application do not limit the implementation of the technical solution of the present application.

[0046] Figure 1 A scenario architecture diagram of a document translation method provided in an embodiment of the present application, Figure 1 It includes servers and various forms of terminal devices. Figure 1 The server shown can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system. Furthermore, the server can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0047] The document translation method provided in this application relates to the field of artificial intelligence. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI is the study of the design principles and implementation methods of various intelligent machines, giving them the capabilities of perception, reasoning, and decision-making.

[0048] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0049] The document translation method provided in this application primarily involves computer vision technology and natural language processing. Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects, and further image processing to transform computer-generated images into images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Large model technology has brought significant changes to the development of computer vision technology. Pre-trained models in the field of vision, such as the Swin Transformer, ViT, V-MOE, and MAE, can be fine-tuned to quickly and widely apply to specific downstream tasks. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0050] Natural language processing (NLP) is an important field in the fields of computer science and artificial intelligence. It studies various theories and methods that enable effective communication between humans and computers using natural language. Natural language processing involves natural language, the language people use in daily life, and is closely related to linguistics research; it also involves computer science and mathematics. Pre-training models, an important technology for model training in the field of artificial intelligence, are developed from large language models (LLMs) in the field of NLP. After fine-tuning, large language models can be widely used in downstream tasks. Natural language processing technologies generally include text processing, semantic understanding, machine translation, robot question answering, knowledge graphs, and other technologies.

[0051] Next, the document translation method provided in the embodiment of the present application is specifically introduced from the perspective of a terminal device.

[0052] See also Figure 2 , which is a flow chart of a document translation method provided by an embodiment of the present application. The first document includes document text and image text, the image text is the text of the first image in the first document, combined with Figure 2 As shown, the document translation method may include the following steps:

[0053] S201: Translate the document text to obtain a text translation result.

[0054] In the embodiments of the present application, a document refers to a collection of information including text, pictures, sounds, data, etc., which may appear in the form of written text or dynamic web pages. For example, files generated by editing software such as Word, Excel, or PPT can be called documents, such as Word documents, Excel documents, or PPT documents. Document text refers to the actual text content contained in a document. A text translation result refers to a text with the same meaning as the document text but in a different language. Picture text refers to the text in a picture. For details, please refer to the following embodiments. Figure 9 As shown, the first document includes document text and picture text.

[0055] A first document refers to any document that includes document text and image text. A first image refers to any image that includes image text. It should be noted that this application does not limit the number of first images in a first document. For example, a first document may include one first image or multiple first images.

[0056] As an example, see Figure 3 , which is a flowchart of a document text translation process provided by an embodiment of the present application. Figure 3 As shown, S201 may specifically include the following steps:

[0057] S2011: Traverse the first document to obtain text objects.

[0058] In the embodiment of the present application, the first document may include a text object, a picture object, etc. The text object includes the text itself and the text format and position of the text in the document.

[0059] In some embodiments, the first document may be traversed through an application programming interface (API) to traverse the text object.

[0060] S2012: Extract document text from text objects.

[0061] Extract document text from text objects for subsequent translation.

[0062] S2013: Translate the document text to obtain a text translation result.

[0063] In some embodiments, the language of the document text is Chinese. If the user needs to view the text in English, the document text can be translated into English as a text translation result.

[0064] S202: Extract the first picture from the first document, and identify the picture text from multiple text areas in the first picture.

[0065] In the embodiment of the present application, the text area refers to the area in the first image that includes the text of the image. For example, the text area can be a rectangular area, see Figure 4 , which is a schematic diagram of a text area provided by an embodiment of the present application. Figure 4 As shown, the multiple rectangular areas are multiple text areas in the first picture.

[0066] In some embodiments, the text in the first image can be scanned by optical character recognition (OCR) technology to recognize the text in each text area, such as Figure 4 As shown, text 1 "Secret Treasure House" can be identified from text area 1, and text 2 "Activated 3 / 10" can be identified from text area 2. And so on, all texts (ie, picture texts) such as text 1 and text 2 can be identified from multiple text areas.

[0067] In some embodiments, the relative coordinate positions of each text area in the first image can also be obtained by OCR technology. Figure 4 As shown, taking the upper left corner of the first picture as the origin of the coordinate system, the coordinate point (x1, y1) of the upper left corner, the coordinate point (x1, y2) of the lower left corner, the coordinate point (x2, y1) of the upper right corner, and the coordinate point (x2, y2) of the lower right corner of the text area 1 can be obtained. The coordinate points related to the text area 1 can be called the coordinate information of the text area 1. Similarly, the coordinate points of each vertex of other text areas such as the text area 2 can also be obtained, which will not be repeated here.

[0068] S203: Translate the image text to obtain an image translation result.

[0069] In the embodiment of the present application, the image translation result refers to text with the same meaning as the image text but in a different language. Based on the above example, if the language of the image text is also Chinese, if the user needs to view the text in English, the image text can be translated into English as the image translation result.

[0070] In some embodiments, the terminal device can call the machine translation interface API interface to translate the image text, which is not limited in this application.

[0071] In some embodiments, the terminal device can use the image text, image translation results, and coordinate information of the text area as data units. Based on each text area extracted by OCR technology, a corresponding data unit can be generated. The terminal device can then save the data units corresponding to multiple text areas in a list format.

[0072] In this way, in subsequent steps, the image translation results can be easily filled into the corresponding text area based on each data unit, thereby improving the accuracy of filling the image text into the corresponding text area, thereby improving the image translation effect.

[0073] As an example, Python language can be used to save each data unit in the form of a dictionary in Python language. A data unit may include a picture text in a text area, the picture translation result corresponding to the picture text in the text area, and the coordinates of the upper left corner, lower left corner, upper right corner and lower right corner of the text area in the coordinate system corresponding to the first picture. For details, please refer to Figure 5 , this figure is a schematic diagram of the data structure of a data unit provided in an embodiment of the present application, wherein data identifies a data unit of a text area, source_text represents the image text of the text area (also referred to as the source text), target_text represents the image translation result corresponding to the image text of the text area (also referred to as the target text), and location represents the coordinate information of the text area in the first image.

[0074] S204: If the plurality of text areas include a first text area and a second text area whose vertical coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, reducing the font size of the first translation result.

[0075] It is understandable that the image text and the image translation result are in different languages, and the text lengths of two texts with the same meaning but in different languages ​​may differ significantly. For example, text area 1 includes text 1, which corresponds to translation result 1. The length of text area 1 is determined based on the text length of text 1. If the text length of translation result 1 exceeds the text length of text 1, then after filling translation result 1 into text area 1, it may cause translation result 1 to occupy other text areas, causing text overlap and affecting the document translation effect. In order to solve this problem, it is necessary to determine the text area that will cause text overlap from multiple text areas, and then adjust the font size of the translation result that will be filled into this text area.

[0076] In this embodiment of the present application, the first text area and the second text area refer to two text areas whose vertical coordinate intervals intersect. The target text length is the distance between the vertical boundaries of the first and second text areas on the same side. The vertical boundary refers to the left or right boundary of the text area.

[0077] For example, if the first text area is on the left side of the second text area, and when the image translation result is filled in, the first character is filled in from the left side of the first text area, then the target text length is the distance between the left longitudinal boundary of the first text area and the left longitudinal boundary of the second text area. Figure 6 shown.

[0078] As an embodiment, the vertical coordinate interval of the first text area is [1, 3], the vertical coordinate interval of the second text area is [2, 4], and the vertical coordinate areas of the two have an intersection [2, 3]. If the first text area is on the left side of the second text area, and when filling in the image translation result, the first character is filled in from the left side of the first text area, then the first translation result filled in the first text area may invade the second text area. Please refer to the following embodiment. Figure 6 shown.

[0079] For example, if the first text area is to the right of the second text area, and when the image translation result is entered, the last character is entered starting from the right side of the first text area, then the target text length is the distance between the right vertical boundary of the first text area and the right vertical boundary of the second text area. This application does not limit this.

[0080] In some embodiments, the first translation result may be drawn on a blank canvas, and an OCR technology may be used to identify a text area of ​​the blank canvas to obtain the area length of the text area as the text length of the first translation result.

[0081] In other embodiments, the first translation result may be drawn on a blank canvas, and the horizontal distance between the leftmost side of the starting character and the rightmost side of the last character may be measured as the text length of the first translation result. This application does not limit this.

[0082] In an embodiment of the present application, the text length of the first translation result is compared with the target text length. If the text length of the first translation result is greater than the target text length, it indicates that after the first translation result is filled into the first text area, the first translation result will invade the second text area. In this case, the font size of the first translation result needs to be reduced to avoid this situation as much as possible. If the text length of the first translation result is less than or equal to the target text length, it indicates that after the first translation result is filled into the first text area, it may not fill the first text area completely, or it may fill out the first text area but will not invade the second text area.

[0083] In a possible implementation of the present application, the vertex of the first image can be the origin of the coordinate system. For example, the upper left corner vertex or the lower left corner vertex of the first image can be used as the origin of the coordinate system, and the present application does not limit this.

[0084] Accordingly, S204 “if the multiple text areas include a first text area and a second text area whose vertical coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result” may include the following steps:

[0085] Step 1: If the vertical coordinate of the upper boundary of the first text region is less than or equal to the vertical coordinate of the lower boundary of the second text region, and the vertical coordinate of the lower boundary of the first text region is greater than or equal to the vertical coordinate of the upper boundary of the second text region, then determine that the vertical coordinate interval of the first text region and the vertical coordinate interval of the second text region have an intersection.

[0086] In some embodiments, for one of the multiple text regions in the first image, a text region that may cause text overlap is identified from the remaining text regions, that is, a text region whose vertical coordinate intervals intersect. Based on the vertical coordinate of the upper boundary and the vertical coordinate of the lower boundary of the text region, the remaining text regions are traversed to identify another text region whose lower boundary has a vertical coordinate greater than the vertical coordinate of the upper boundary of the text region and whose upper boundary has a vertical coordinate less than the vertical coordinate of the upper boundary of the text region. The text region can then be determined as the first text region, and the other text region as the second text region.

[0087] Based on the above example, the coordinates of each vertex in the text area can be obtained through OCR technology. The vertical coordinate of the upper boundary of the text area is the vertical coordinate of the upper left vertex or the upper right vertex, and the vertical coordinate of the lower boundary of the text area is the vertical coordinate of the lower left vertex or the lower right vertex.

[0088] As an example, see Figure 6 , which is a schematic diagram of another text area provided by an embodiment of the present application, combined with Figure 6 As shown, when text area 1 is the first text area, text area 2, text area 3 and text area 4 can all be second text areas. Based on the above example, the target text length can be the distance between the left border of the first text area and the left border of the second text area.

[0089] Step 2: If the text length of the first translation result is greater than the target text length, the font size of the first translation result is reduced.

[0090] In some embodiments, the target text length can be represented by min_distance, and the text length of the first translation result to be inserted into the first text region can be represented by init_width. If init_width is greater than min_distance, it indicates that the current first translation result will intrude into the second text region, and its font size needs to be reduced. If init_width is less than or equal to min_distance, it indicates that the current first translation result will not intrude into the second text region, and there is no need to adjust the font size.

[0091] In a possible implementation of the present application, the step of "reducing the font size of the first translation result" in S204 may include the following steps: first obtaining a length ratio of the target text length to the text length of the first translation result; then multiplying the initial font size of the first translation result by the length ratio to obtain a target font size; and then reducing the initial font size of the first translation result to the target font size.

[0092] As an implementation method, the target font size can be calculated using the following code:

[0093] {

[0094] font_size=min_distance / / init_width*font_size

[0095] }

[0096] Based on the above description, min_distance represents the target text length, init_width represents the length of the first translation result to be used in the first text area, font_size on the left represents the target font size, and font_size on the right represents the initial font size. " / / " indicates rounding down. For example, if the calculated value is 3.2, 3 is used as the result.

[0097] In this way, the font size is reduced based on the ratio of the target text length to the text length of the first translation result, ensuring that the adjusted first translation result does not invade other text areas, and the font size is rounded down to an integer, further avoiding text overlap and improving the translation effect.

[0098] It should be understood that in order to avoid the situation where the font height of the image translation result exceeds the area height of the text area after the image translation result is filled in multiple text areas, thereby causing text overlap and other situations that affect readability, the initial font size of the image translation result can be determined based on the area height of the text area.

[0099] In a possible implementation of the present application, the initial font size of the first translation result can be obtained by the following steps:

[0100] Step 3: Get the area height of the first text area.

[0101] In the embodiments of the present application, the region height refers to the vertical distance between the upper and lower boundaries of the first text region. In some embodiments, the region height of the first text region can be measured using a measuring tool, for example, by measuring the distance between the upper left corner and the lower left corner of the first text region.

[0102] In some embodiments, the coordinate information of the first text area obtained by the above OCR technology may also be calculated, for example, the vertical coordinate of the lower left vertex minus the vertical coordinate of the upper left vertex is used to obtain the area height of the first text area.

[0103] Step 4: Obtain a height ratio between the region height and the font height of the first translation result.

[0104] It should be understood that in order to avoid incomplete text recognition, the area height of the recognized text area usually needs to be higher than the font height of the text therein.

[0105] In some embodiments, if the font of the first translation result is Microsoft YaHei font, the obtained height ratio of the region height to the font height of the first translation result is 4 / 3.

[0106] Step 5: Determine the ratio of the region height to the height ratio as the initial font size of the first translation result.

[0107] As an implementation method, taking the height ratio of 4 / 3 as an example, the initial font size can be calculated using the following code:

[0108] {

[0109] font_size=height / / 4*3

[0110] }

[0111] Where font_size represents the initial font size, and height represents the height of the text area. " / / " indicates downward calculation. For example, if the calculated value is 4.2, 4 is used as the target font size.

[0112] Furthermore, in some embodiments, the above steps may be performed for the region height of each text region and the font height of the translation result filled in each text region, so as to determine an appropriate initial font size for each text region.

[0113] In this way, the calculated initial font size can ensure that the font height of the first translation result fills the text area without exceeding the area height of the first text area, thereby improving the translation effect.

[0114] S205: Filling the adjusted image translation result into multiple text areas in the first image to obtain a second image.

[0115] In the embodiment of the present application, the adjusted image translation result includes the first translation result with the font size adjusted to a smaller size, and the remaining translation results with the font size not adjusted.

[0116] In some embodiments, the adjusted image translation results can be filled into the corresponding text areas. Exemplarily, the first image includes text area 1, text area 2, and text area 3, which respectively include image text 1, image text 2, and image text 3, that is, the image text includes image text 1, image text 2, and image text 3; the translation results of image text 1, image text 2, and image text 3 are translation result 1, translation result 2, and translation result 3, respectively, that is, the image translation results include translation result 1, translation result 2, and translation result 3. If translation result 1 is the first translation result after the font size is reduced, the adjusted image translation result includes translation result 1, translation result 2, and translation result 3 after the font size is reduced. It is necessary to fill the translation result 1 after the font size is reduced into the corresponding text area 1, the translation result 2 into the corresponding text area 2, and the translation result 3 into the corresponding text area 3. The image including translation result 1, translation result 2, and translation result 3 after the font size is reduced is the second image.

[0117] It should be understood that if the adjusted image translation result is directly filled into multiple text areas, it will cause text overlap with the image text existing in the first image itself, so the image text included in the multiple text areas needs to be covered.

[0118] In a possible implementation of the present application, S205 "filling the adjusted image translation results into multiple text areas in the first image to obtain a second image" may include: first filling the multiple text areas with a preset color to cover the image text in the multiple text areas; and then filling the adjusted image translation results into the filled multiple text areas.

[0119] As an implementation manner, the preset color may be white, and multiple text areas may be filled with white. This application does not limit the preset color.

[0120] In some embodiments, the present application may not limit the number of preset colors. For example, the preset colors may include preset color 1 and preset color 2. Preset color 1 may be used to fill a portion of the text area, and preset color 2 may be used to fill another portion of the text area.

[0121] As an embodiment, the preset color can also be the background color of the text area itself. For example, the background colors corresponding to multiple text areas in the first image can be identified and used as the preset color. When filling in the image, the background colors corresponding to the multiple text areas can be used to fill in the image to cover the text.

[0122] In this way, the original image text is covered by the preset color to ensure the clarity and readability of the image translation results and improve the translation effect.

[0123] It should be understood, based on the above introduction, that the text length of the image translation result may exceed the text length of the image text, causing the image translation result to exceed the text area. If the color of the non-text area in the first image is too similar to the color of the image translation result, the user will not be able to see the image translation result clearly, affecting readability.

[0124] Therefore, in a possible implementation of the present application, “filling the multiple text areas with preset colors” may include the following steps:

[0125] Step 6: Based on the multiple text lengths filled into the multiple text regions respectively according to the adjusted image translation results, determine the filling lengths corresponding to the multiple text regions respectively.

[0126] In an embodiment of the present application, the length of the text occupied by the adjusted image translation results after filling in the corresponding text areas is first determined, and then the filling length required to fill the text areas is determined based on this.

[0127] As an example, based on the above example, the first image includes text area 1, text area 2, and text area 3, and the translation results that need to be filled in are translation result 1, translation result 2, and translation result 3, respectively. Among them, translation result 1 is the first translation result after the font size is reduced, and translation result 2 and translation result 3 are other translation results without adjusting the font size. The text length of each translation result written into the text area is: the minimum value of the corresponding target text length min_distance (that is, the text length corresponding to the translation result after the font size is reduced) and the text length init_width of the initial font size, that is, the text length of translation result 1 is min_distance, and the text length of translation result 2 and translation result 3 is init_width. The text length of each translation result written into the text area is represented by translate_width.

[0128] For example, the length of the text written into the text area for each translation result can be calculated using the following code:

[0129] {

[0130] translate_width=min(init_width, min_distance)

[0131] }

[0132] Among them, min means taking the minimum value.

[0133] It should be understood that if the text length of the translation result written into the text area is greater than the area length of the text area itself (also known as the area width), the filling length needs to be the text length of the translation result written into the text area; if the text length of the translation result written into the text area is less than or equal to the area length of the text area itself, the text area can be directly filled and covered.

[0134] If source_width is used to represent the area length of the text area itself, the maximum value of translate_width and source_width needs to be used as the padding length. For example, assuming that the translate_width corresponding to translation result 1 is smaller than the source_width of text area 1, the translate_width corresponding to translation result 2 is smaller than the source_width of text area 2, and the translate_width corresponding to translation result 3 is larger than the source_width of text area 1. Then the padding length of text area 1 and text area 2 is the area length of the text area itself, that is, it is sufficient to fill it up. The padding length of text area 3 is the translate_width corresponding to translation result 3, and the preset color needs to be filled outside text area 3.

[0135] For example, based on the above description, using OCR technology to obtain the coordinate values ​​of each vertex of a text region, the region length can be obtained by subtracting the horizontal coordinate of the upper left corner from the horizontal coordinate of the upper right corner. Alternatively, the region length can be obtained by directly measuring the distance between the left and right boundaries of the text region. This application does not limit this.

[0136] For example, the padding lengths corresponding to multiple text areas can be calculated using the following code:

[0137] {

[0138] width=max(source_width, translate_width)

[0139] }

[0140] Among them, width indicates the padding length, and max indicates the minimum value.

[0141] Step 7: Fill the multiple text areas with the preset color based on the filling lengths corresponding to the multiple text areas.

[0142] In some embodiments, based on the above example, text area 1 and text area 2 can be filled, and the preset color can be filled outside text area 3, for example, starting from the left border of the text area and filling outside the right border of the text area.

[0143] In this way, the text area is filled by fully considering the text length of the image translation result, avoiding unclear image translation results and further improving readability.

[0144] It should be understood that after the text area is filled with the preset color, the greater the difference between the preset color and the font color of the image translation result, the clearer the displayed image translation result.

[0145] Therefore, in a possible implementation of the present application, the above-mentioned "filling the adjusted image translation result into the multiple filled text areas" may include the following steps: first calculating the color difference between multiple preset font colors and the preset color, and obtaining the color difference values ​​corresponding to the multiple preset font colors; then determining the maximum value from the multiple color difference values, and determining the preset font color corresponding to the maximum value as the target font color; then, adjusting the font color of the adjusted image translation result to the target font color; and then filling the color-adjusted image translation result into the multiple filled text areas.

[0146] In some embodiments, the plurality of preset font colors may include black, gray, blue, red, etc., as long as they are different from the preset colors.

[0147] As an implementation manner, the color difference between a preset font color and a preset color may be determined by calculating the Euclidean distance between the two colors in the RGB color space. That is, the Euclidean distance represents the color difference value.

[0148] For example, the plurality of preset font colors may include preset font color 1, preset font color 2, and preset font color 3. Assuming that the RGB values ​​of preset font color 1 are (r1, g1, b1), and the RGB values ​​of the preset colors are (r2, g2, b2), the Euclidean distance between the preset font color 1 and the preset colors in the RGB color space can be calculated using the following code:

[0149] {

[0150] distance=sqrt((r1-r2)^2+(g1-g2)^2+(b1-b2)^2)

[0151] }

[0152] Where distance represents Euclidean distance and sqrt represents square root.

[0153] A larger Euclidean distance indicates a greater color difference. Assume that the Euclidean distance between the preset font color 1 and the preset color is larger than the Euclidean distance between the preset font color 2 and the preset color, and the Euclidean distance between the preset font color 3 and the preset color. That is, the color difference value corresponding to the preset font color 1 is the largest. Then, the preset font color 1 can be used as the target font color, and the font color of the adjusted image translation result can be adjusted to the target font color.

[0154] In this way, the color difference between the font color of the image translation result and its background color in the second image (that is, the fill color of the text area) is fully taken into consideration, and a clearer image translation result is presented to the user, further improving readability and enhancing user experience.

[0155] As an example, based on the introduction of the above embodiment, combined with Figure 7 As shown in the figure, this figure is a flow chart of a picture text translation process provided by an embodiment of the present application. The process of translating the picture text may include the following A1-A6:

[0156] A1: Extract a first image from a first document.

[0157] A2: Scan the first image using OCR technology to identify multiple text areas and the image texts included in the multiple text areas.

[0158] A3: Translate the image text to obtain the image translation result.

[0159] A4: Calculate the optimal font size and color for filling the image translation results into multiple text areas.

[0160] In some embodiments, the optimal font size includes the initial font size and / or target font size mentioned above, and the optimal font color is the target font color mentioned above.

[0161] A5: Fill multiple text areas with preset colors.

[0162] A6: Fill the translated image with the adjusted font size and color into the filled text areas to obtain a second image.

[0163] As an example, see Figure 8 , which is a schematic diagram comparing a first picture and a second picture provided in an embodiment of the present application. Figure 8 As shown, the first picture is a picture including picture text, and the second picture is a picture including a picture translation result corresponding to the picture text. The text format in the second picture is the same as that in the first picture.

[0164] S206: Replace the document text with the text translation result, and replace the first image with the second image to obtain a second document.

[0165] In the embodiment of the present application, the document format is maintained by replacement, so that the document format of the second document is identical to the document format of the first document, thereby improving the translation effect.

[0166] As an example, see Figure 9 , which is a schematic diagram of a comparison between a first document and a second document provided in an embodiment of the present application. Figure 9 As shown, the first document includes document text and picture text, the second picture includes the text translation result corresponding to the document text and the picture translation result corresponding to the picture text (that is, the second picture), the document format of the second document is the same as the document format of the first document, and the user can directly view the second document to meet the needs.

[0167] In a possible implementation of the present application, the step of “extracting the first image from the first document” in S202 may include the following steps:

[0168] B1: Convert the document format of the first document into a compressed package format.

[0169] For example, assuming that the first document is a Word document, the suffix "doc" or "docx" of the Word document can be changed to "zip" to obtain the first document in a compressed package format.

[0170] B2: Decompress the first document in the compressed package format to obtain a picture folder.

[0171] In an embodiment of the present application, the first document in the compressed package format contains some XML format files and picture files, etc., and the picture files exist in the picture folder.

[0172] As an example, see Figure 10 , which is a page diagram of a picture folder provided by an embodiment of the present application. Figure 10 As shown, after decompressing the first document in the compressed package format, an mdia folder (ie, a picture folder) can be obtained, which includes multiple first pictures in the first document.

[0173] B3: Extract the first picture from the picture folder.

[0174] When the image text in the first document needs to be translated, the first image can be extracted from the image folder.

[0175] Accordingly, S206 “replacing the document text with the text translation result, and replacing the first image with the second image to obtain a second document” may include the following steps:

[0176] B4: Replace the first picture in the picture folder with the second picture.

[0177] A first picture including the picture text in the picture folder is replaced by a second picture including the picture translation result.

[0178] It should be noted that if the picture folder includes multiple first pictures, it is necessary to use the second pictures corresponding to the multiple first pictures to perform corresponding replacements.

[0179] B5: Restore the document format of the first document in the compressed package format.

[0180] The document format of the first document is restored from the compressed package to the original document format of the first document. For example, assuming that the original document format of the first document is Word, the document format of the first document in the compressed package format is restored to Word.

[0181] For example, based on the above example, the suffix "zip" of the first document in the compressed package format may be changed to "doc" or "docx" to obtain the first document in the restored format.

[0182] B6: Based on the restored format of the first document, replace the document text with the text translation result to obtain the second document.

[0183] If the first document in the restored format contains the second image, the text translation result can be used to replace the document text to obtain a second document.

[0184] In this way, for the first document that can be converted into a compressed package format, only the image text and document text can be replaced in the corresponding positions without modifying other related content. The document format of the second document can be maintained to be the same as the document format of the first document, thereby improving the translation effect and thus enhancing the user experience.

[0185] In a possible implementation of the present application, the step of “extracting the first image from the first document” in S202 may include the following steps:

[0186] C1: Traverse the first document through the application programming interface to obtain the picture object.

[0187] Based on the above description, the first document includes an image object and a text object, and the image object can be obtained by traversing through the application programming interface. The image object consists of the image itself and some metadata, which may include the image location, image link, etc.

[0188] C2: Extract the first picture from the picture object.

[0189] When the image text in the first document needs to be translated, the first image can be extracted from the image object.

[0190] Correspondingly, S206 "Replace the document text with the text translation result, and replace the first picture with the second picture to obtain a second document" can be C3: For the first document, replace the document text with the text translation result, and replace the first picture in the picture object with the second picture to obtain the second document.

[0191] In this way, for the first document that cannot be converted into a compressed package format, the first image can be identified, and the image text and document text in the corresponding positions can be directly replaced in the first document. The document format of the second document can be maintained to be the same as that of the first document, thereby improving the translation effect and thus enhancing the user experience.

[0192] In some embodiments, the document translation method provided by the embodiments of the present application can be implemented through a web product or other forms of client provided to the user. Figure 11 , which is a schematic diagram of a translation platform page provided by an embodiment of the present application. Figure 11 As shown, the page includes an original language control and a translated language control, wherein the original language control is used for the user to select the language of the document text and image text that need to be translated in the first document, and the translated language control is used for the user to select the language of the text translation result included in the required second document and the image translation result in the second image. The page also includes a translation image control, and the user can select this control to enable the terminal device to translate the image text in the first document. If the user does not check it, the terminal device can only translate the document text in the first document. The page also includes a document upload control, and the user can drag the first document here, so that the terminal device determines the first document that needs to be translated, and then executes the document translation method provided in the embodiment of the present application to obtain the second document, and then returns the second document to the user so that the user can download and view it.

[0193] It should be noted that in the above examples, the triggering methods and names of the various controls are only examples, and the page may include more or fewer functions than those described above, and this application does not limit this.

[0194] Based on the document translation method provided in the above embodiment, this application also provides a document translation device. The document translation device provided in the embodiment of this application is described in detail below.

[0195] See also Figure 12 , which is a structural diagram of a document translation device provided by an embodiment of the present application. Figure 12 As shown, in the document translation device 1200, the first document includes document text and image text. The image text is the text of the first image in the first document, and specifically may include:

[0196] The document text translation module 1210 is configured to translate the document text to obtain a text translation result;

[0197] The image text recognition module 1220 is configured to extract the first image from the first document and recognize the image text from multiple text regions in the first image;

[0198] The image text translation module 1230 is used to translate the image text to obtain an image translation result;

[0199] The font size adjustment module 1240 is configured to reduce the font size of the first translation result if the plurality of text areas include a first text area and a second text area whose vertical coordinate intervals intersect, and if the length of the text of the first translation result filled in the first text area in the image translation result is greater than a target text length; the target text length is the distance between the longitudinal boundaries of the first text area and the second text area on the same side;

[0200] A result filling module 1250 is used to fill the adjusted image translation result into multiple text areas in the first image to obtain a second image;

[0201] The document acquisition module 1260 is configured to replace the document text with the text translation result, and replace the first image with the second image, to obtain a second document.

[0202] As an implementation manner, the vertex of the first image is the origin of the coordinate system, and the font size adjustment module 1240 may specifically include:

[0203] an interval intersection determining unit, configured to determine that a longitudinal coordinate interval of the first text area and a longitudinal coordinate interval of the second text area intersect if the longitudinal coordinate of the upper boundary of the first text area is less than or equal to the longitudinal coordinate of the lower boundary of the second text area, and the longitudinal coordinate of the lower boundary of the first text area is greater than or equal to the longitudinal coordinate of the upper boundary of the second text area;

[0204] The first font size reduction unit is configured to reduce the font size of the first translation result if the text length of the first translation result is greater than the target text length.

[0205] As an implementation, the font size adjustment module 1240 may specifically include:

[0206] a length ratio obtaining unit, configured to obtain a length ratio between the target text length and the text length of the first translation result;

[0207] a target font size obtaining unit, configured to obtain a target font size by multiplying the initial font size of the first translation result by the length ratio;

[0208] The second font size reduction unit is configured to reduce the initial font size of the first translation result to the target font size.

[0209] As an implementation manner, the initial font size of the first translation result is obtained by the following units:

[0210] an area height obtaining unit, configured to obtain the area height of the first text area;

[0211] a height ratio obtaining unit, configured to obtain a height ratio of a font height of the first translation result to a height of the region;

[0212] An initial font size determining unit is configured to determine a ratio of the region height to the height ratio as an initial font size of the first translation result.

[0213] As an implementation manner, the image text recognition module 1220 may specifically include:

[0214] a format conversion unit, configured to convert the document format of the first document into a compressed package format;

[0215] A document decompression unit, configured to decompress the first document in a compressed package format to obtain a picture folder;

[0216] A first picture extraction unit, configured to extract the first picture from the picture folder;

[0217] Accordingly, the document acquisition module 1260 may specifically include:

[0218] A picture replacing unit, configured to replace the first picture in the picture folder with the second picture;

[0219] a format restoration unit, configured to restore the document format of the first document in the compressed package format;

[0220] The text replacement unit is configured to replace the document text with the text translation result based on the first document in the restored format to obtain the second document.

[0221] As an implementation manner, the image text recognition module 1220 may specifically include:

[0222] A picture object traversal unit, configured to traverse the first document through an application programming interface to obtain a picture object;

[0223] a second picture extraction unit, configured to extract the first picture from the picture object;

[0224] Accordingly, the document acquisition module 1260 may be specifically configured to:

[0225] For the first document, the document text is replaced by the text translation result, and the first picture in the picture object is replaced by the second picture to obtain the second document.

[0226] As an embodiment, the result filling module 1250 may specifically include:

[0227] an area filling unit, configured to fill the plurality of text areas with a preset color;

[0228] The result filling unit is used to fill the adjusted image translation results into the filled multiple text areas.

[0229] As an implementation manner, the region filling unit may specifically include:

[0230] a filling length determining subunit, configured to determine filling lengths corresponding to the plurality of text regions respectively based on a plurality of text lengths filled into the plurality of text regions respectively according to the adjusted image translation results;

[0231] The area filling subunit is configured to fill the multiple text areas with the preset color based on the filling lengths respectively corresponding to the multiple text areas.

[0232] As an implementation manner, the result filling unit may specifically include:

[0233] a color difference value calculation subunit, configured to calculate the color differences between a plurality of preset font colors and the preset color, and obtain the color difference values ​​corresponding to the plurality of preset font colors;

[0234] a font color determination subunit, configured to determine a target font color having the largest color difference value from the plurality of preset font colors;

[0235] a font color adjustment subunit, configured to adjust the font color of the adjusted image translation result to the target font color;

[0236] The result filling subunit is used to fill the color-adjusted picture translation result into the filled multiple text areas.

[0237] Next, the structures of the document translation equipment will be introduced in terms of server form and terminal device form respectively.

[0238] An embodiment of the present application provides a document translation device, which may be a server. Figure 13 This is a schematic diagram of a server structure provided in an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 922 (for example, one or more processors) and memory 932, and one or more storage media 930 (for example, one or more mass storage devices) for storing application programs 942 or data 944. Among them, the memory 932 and the storage medium 930 can be short-term storage or persistent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 922 can be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the server 900.

[0239] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows Server 2003 or Windows Server 2003R. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0240] The CPU 922 is configured to execute the following steps:

[0241] Translating the document text to obtain a text translation result;

[0242] Extracting the first picture from the first document, and identifying the picture text from multiple text areas in the first picture;

[0243] Translating the image text to obtain an image translation result;

[0244] If the plurality of text areas include a first text area and a second text area whose longitudinal coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result; the target text length is the distance between the longitudinal boundaries of the first text area and the second text area on the same side;

[0245] Filling the adjusted image translation result into multiple text areas in the first image to obtain a second image;

[0246] The document text is replaced with the text translation result, and the first image is replaced with the second image to obtain a second document.

[0247] The embodiment of the present application also provides another document translation device, which can be a terminal device. Figure 14 For the sake of convenience, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. Take the terminal device as a mobile phone as an example:

[0248] Figure 14 The block diagram shows a partial structure of the mobile phone provided by the embodiment of the present application. Figure 14 The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will appreciate that Figure 14 The mobile phone structure shown in the figure does not constitute a limitation to the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0249] The following combination Figure 14 A detailed introduction to the various components of a mobile phone:

[0250] RF circuitry 1010 can be used to receive and transmit signals during information transmission or calls. Specifically, it receives downlink information from the base station and transmits it to processor 1080 for processing. It also transmits uplink data to the base station. Typically, RF circuitry 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, and more. RF circuitry 1010 can also communicate with the network and other devices via wireless communications. The above-mentioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (Long Term Evolution, English abbreviation: LTE), email, Short Messaging Service (SMS), etc.

[0251] The memory 1020 can be used to store software programs and modules. The processor 1080 executes the various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 1020 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0252] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch panel 1031) and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction and detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 1080. It can also receive commands sent by the processor 1080 and execute them. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may further include other input devices 1032. Specifically, the other input devices 1032 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick.

[0253] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (English full name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (English full name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 14 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0254] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the mobile phone can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0255] Audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the phone. Audio circuit 1060 converts received audio data into electrical signals and transmits them to speaker 1061, which then converts them into sound signals for output. Microphone 1062, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 1060 and converted into audio data. The audio data is then processed by processor 1080 and transmitted to, for example, another phone via RF circuit 1010, or stored in memory 1020 for further processing.

[0256] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse the web and access streaming media through the WiFi module 1070. It provides users with wireless broadband Internet access. Figure 14 A WiFi module 1070 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0257] Processor 1080 is the control center of the phone, connecting all parts of the phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 1020 and accessing data stored in memory 1020, it executes various phone functions and processes data, thereby collecting data and information about the phone as a whole. Optionally, processor 1080 may include one or more processing units; preferably, processor 1080 may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 1080.

[0258] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, thereby managing charging, discharging, and power consumption through the power management system.

[0259] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0260] In the embodiment of the present application, the processor 1080 included in the mobile phone also has the following functions:

[0261] Translating the document text to obtain a text translation result;

[0262] Extracting the first picture from the first document, and identifying the picture text from multiple text areas in the first picture;

[0263] Translating the image text to obtain an image translation result;

[0264] If the plurality of text areas include a first text area and a second text area whose longitudinal coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result; the target text length is the distance between the longitudinal boundaries of the first text area and the second text area on the same side;

[0265] Filling the adjusted image translation result into multiple text areas in the first image to obtain a second image;

[0266] The document text is replaced with the text translation result, and the first image is replaced with the second image to obtain a second document.

[0267] An embodiment of the present application further provides a computer-readable storage medium for storing a computer program. When the computer program is run on a document translation device, the document translation device executes any one of the implementations of the document translation method described in the aforementioned embodiments.

[0268] An embodiment of the present application further provides a computer program product including a computer program, which, when executed on a document translation device, enables the document translation device to execute any one of the implementations of a document translation method described in the aforementioned embodiments.

[0269] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0270] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the system is merely a logical function division. In actual implementation, there may be other division methods, such as multiple systems can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0271] The systems described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the objectives of this embodiment as needed.

[0272] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0273] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store computer programs.

[0274] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0275] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A document translation method, characterized in that: The first document includes document text and picture text, where the picture text is the text of the first picture in the first document. The method includes: Translating the document text to obtain a text translation result; Extracting the first picture from the first document, and identifying the picture text from multiple text areas in the first picture; Translating the image text to obtain an image translation result; If the plurality of text areas include a first text area and a second text area whose longitudinal coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result; the target text length is the distance between the longitudinal boundaries of the first text area and the second text area on the same side; Filling the adjusted image translation result into multiple text areas in the first image to obtain a second image; The document text is replaced with the text translation result, and the first image is replaced with the second image to obtain a second document.

2. The method according to claim 1, characterized in that The vertex of the first image is the origin of the coordinate system; if the multiple text areas include a first text area and a second text area whose vertical coordinate intervals intersect, and the length of the text of the first translation result filled in the first text area in the image translation result is greater than the target text length, then reducing the font size of the first translation result, including: If the vertical coordinate of the upper boundary of the first text region is less than or equal to the vertical coordinate of the lower boundary of the second text region, and the vertical coordinate of the lower boundary of the first text region is greater than or equal to the vertical coordinate of the upper boundary of the second text region, then determining that the vertical coordinate interval of the first text region and the vertical coordinate interval of the second text region have an intersection; If the text length of the first translation result is greater than the target text length, the font size of the first translation result is reduced.

3. The method according to claim 1, characterized in that The reducing the font size of the first translation result includes: Obtaining a length ratio of the target text length to the text length of the first translation result; obtaining a target font size by multiplying the initial font size of the first translation result by the length ratio; The initial font size of the first translation result is reduced to the target font size.

4. The method according to claim 3, characterized in that The initial font size of the first translation result is obtained by the following steps: Get the area height of the first text area; Obtaining a ratio between the height of the region and the height of the font of the first translation result; A ratio of the region height to the height ratio is determined as an initial font size of the first translation result.

5. The method according to claim 1, wherein The extracting the first picture from the first document includes: Converting the first document into a compressed file format. Decompressing the first document in a compressed format to obtain a picture folder; Extracting the first picture from the picture folder; The step of replacing the document text with the text translation result and replacing the first image with the second image to obtain a second document includes: Replacing the first picture in the picture folder with the second picture; Restoring the document format of the first document in the compressed package format; Based on the first document in the restored format, the document text is replaced with the text translation result to obtain the second document.

6. The method according to claim 1, wherein The extracting the first picture from the first document includes: Traversing the first document through an application programming interface to obtain a picture object; Extracting the first picture from the picture object; The step of replacing the document text with the text translation result and replacing the first image with the second image to obtain a second document includes: For the first document, the document text is replaced by the text translation result, and the first picture in the picture object is replaced by the second picture to obtain the second document.

7. The method according to claim 1, characterized in that Filling the adjusted image translation result into multiple text areas in the first image to obtain a second image includes: Filling the plurality of text areas with a preset color; Fill the adjusted image translation results into the filled multiple text areas.

8. The method according to claim 7, characterized in that Filling the plurality of text areas with preset colors includes: Determining the filling lengths corresponding to the multiple text areas respectively based on the multiple text lengths filled into the multiple text areas by the adjusted image translation results; The plurality of text areas are filled with the preset color based on the filling lengths respectively corresponding to the plurality of text areas.

9. The method according to claim 7, wherein Filling the adjusted image translation results into the filled multiple text areas includes: Calculating the color differences between a plurality of preset font colors and the preset color, and obtaining the color difference values ​​corresponding to the plurality of preset font colors; Determining a target font color with the largest corresponding color difference value from the plurality of preset font colors; Adjusting the font color of the adjusted image translation result to the target font color; Fill the color-adjusted image translation results into the filled multiple text areas.

10. A document translation device, characterized in that: The first document includes document text and image text, where the image text is the text of the first image in the first document, including: A document text translation module, configured to translate the document text to obtain a text translation result; An image text recognition module is configured to extract the first image from the first document and recognize the image text from multiple text areas in the first image; A picture text translation module, used to translate the picture text to obtain a picture translation result; a font size adjustment module configured to reduce the font size of the first translation result if the plurality of text areas include a first text area and a second text area whose longitudinal coordinate intervals intersect, and if the length of the text of the first translation result filled in the first text area in the image translation result is greater than a target text length; the target text length being the distance between longitudinal boundaries of the first text area and the second text area on the same side; A result filling module, configured to fill the adjusted image translation result into multiple text areas in the first image to obtain a second image; The document acquisition module is used to replace the document text with the text translation result and replace the first picture with the second picture to obtain a second document.

11. A document translation device, characterized in that: The device includes a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the steps of the document translation method according to any one of claims 1 to 9 according to instructions in the computer program.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a document translation device, the steps of the document translation method according to any one of claims 1 to 9 are implemented.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the document translation method according to any one of claims 1 to 9 when executed by a document translation device.

Images