Industrial document translation method and system based on visual positioning

Abstract

The invention discloses an industrial document translation method and system based on visual positioning, and belongs to the field of document translation. The method comprises the following steps: S1, carrying out geometric correction on an input industrial document image, and identifying a document type of the corrected industrial document image; s2, extracting an original text region from the identified document type, translating the extracted original text region through a hybrid retrieval engine, generating a corresponding translation, and performing structural consistency verification; s3, based on the page geometric information of the industrial document and the translated text, selecting placement strategies of the text box of the translated text, scoring the placement strategies of the text box of the translated text, and selecting an optimal placement strategy according to scores; and S4, according to the selected optimal placement strategy, rendering a translated text box on the industrial document, and establishing visual association between the translated text and the original text, so that the spatial layout adaptability of the industrial document translation is improved, and the problem of failure of the spatial layout adaptability of the industrial document translation in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of document translation technology, and in particular to a method and system for industrial document translation based on visual positioning. Background Technology

[0002] In the globalized division of labor in modern manufacturing industries such as apparel, the technology pack serves as a core communication link between brand designers and overseas production factories. A technology pack is typically a highly complex industrial document combining text and graphics. Its content includes not only textual descriptions of garment styles and manufacturing requirements but also a large number of visual elements such as design sketches, size charts, grading diagrams, seam pattern diagrams, and specific pattern markings. The receiving factory needs to understand the manufacturing details and produce samples based on the textual descriptions in the document, along with the corresponding drawings and dimensional data. Therefore, the visual layout and structure of the document are of equal importance to the textual content; any disruption to the layout can lead to a break in the information chain.

[0003] For example, the document translation method, apparatus, and electronic device disclosed in Chinese invention patent CN113297856B include: determining a source document, which includes text content and tags, the tags representing attribute information of associated text content; determining external tags from the tags based on the relative position information of the tags and the text content; dividing the text content into multiple text strings based on the external tags and extracting the external tags into a template file; obtaining the corresponding translation for each text string; and filling the template file with the translation corresponding to the text string based on the positional relationship between the external tags and the text string to generate the translated document.

[0004] For example, Chinese invention patent CN120579559A discloses a document translation method, apparatus, and related products, including: translating document text to obtain a text translation result; extracting a first image from a first document, identifying image text from multiple text regions in the first image; translating the image text to obtain an image translation result; if the multiple text regions include a first text region and a second text region with intersecting vertical coordinate intervals, and the text length of the first translation result filled into the first text region in the image translation result is greater than the target text length, then the font size of the first translation result is reduced; the adjusted image translation result is filled into multiple text regions in the first image to obtain a second image; the document text is replaced with the text translation result, and the first image is replaced with the second image to obtain a second document.

[0005] Currently, the processing of such industrial documents mainly relies on general machine translation, traditional computer-aided translation (CAT) tools, and simple optical character recognition (OCR) translation solutions. General machine translation tools (such as Google Translate) focus on the semantic conversion of plain text and are primarily trained on large-scale general corpora. Traditional CAT tools (such as Trados) focus on providing professional translators with terminology management and translation memory support to improve translation efficiency and ensure terminology consistency. In addition, there are simple translation solutions that combine OCR technology, aiming to extract textual information from carrier images through image recognition technology, convert it using a translation model, and finally output the result.

[0006] In text translation generation, to address the limitations of single retrieval methods, existing technologies have proposed the concept of hybrid retrieval engines. These engines typically employ multi-level retrieval strategies to balance translation accuracy and efficiency. The first layer utilizes efficient string matching algorithms such as the Aho-Corasick Automaton (AC Automaton) to perform precise matching within a pre-defined brand terminology database or general terminology database, aiming to ensure consistency in the translation of specific terms. When precise matching fails, the system enters the second layer, employing Retrieval-Augmented Generation (RAG) technology. This involves vectorizing the text using models such as Sentence-BERT (Sentence-Bidirectional Encoder Representations from Transformers) and performing an approximate nearest neighbor search in the translation memory to recall semantically similar historical translation examples for reference. Finally, in the third layer, the original text, precisely matched terms, and recalled examples are integrated to construct an enhanced prompt, which is then input into a large language model (such as GPT-4o) to generate a final translation that balances terminology accuracy and contextual fluency.

[0007] The above-mentioned technology has at least the following technical problems: In existing technologies, general models lack industry knowledge and use non-standard terms when translating specific brand terms, proprietary vocabulary, or industry technical terms, resulting in translation results that fail to meet brand specifications or industry technical standards.

[0008] Due to the text length differences (expansion or contraction) between the source and target languages ​​(such as Chinese and English), direct translation and replacement can easily cause the translated text to overflow the predefined area, resulting in text overlap, image obscuring, or exceeding page margins, severely damaging the layout and readability of the original document. When the text to be translated in the original document is tightly surrounded by non-text elements such as table lines and graphics, leaving no available blank space, conventional translation layout strategies often fail because they cannot find a legal starting point, leading to translation task interruption or loss of results, resulting in the failure of spatial layout adaptability in industrial document translation. Summary of the Invention

[0009] This invention provides a visual positioning-based industrial document translation method and system, which can improve the accuracy of industrial document translation and thus enhance the spatial layout adaptability of industrial document translation. The technical solution provided by this application is as follows: Firstly, a visual positioning-based industrial document translation method is provided. The specific implementation of this method is as follows: S1, geometric correction is performed on the input industrial document image to identify the document type of the corrected industrial document image; S2, the original text region is extracted from the identified document type, and the extracted original text region is translated through a hybrid retrieval engine to generate the corresponding translation, and the structural consistency of the translation is checked; S3, based on the page geometry information of the industrial document and the translation, a placement strategy for the translation text box is selected, and each placement strategy for the translation text box is scored based on the layout parameters of the translation text box, and the best placement strategy is selected based on the score; S4, according to the selected best placement strategy, the translation text box is rendered on the industrial document, and a visual association between the translation and the original text is established.

[0010] Secondly, a vision-based industrial document translation system is provided. This system employs a vision-based industrial document translation method and includes: an industrial document image processing module for geometric correction of the input industrial document image and identification of the document type of the corrected image; an industrial document translation module for extracting the original text region from the identified document type, translating the extracted original text region through a hybrid retrieval engine to generate the corresponding translation, and performing structural consistency verification on the translation; a vision-based positioning evaluation module for selecting a placement strategy for the translation text box based on the page geometry information and translation of the industrial document, scoring each placement strategy based on the layout parameters of the translation text box, and selecting the optimal placement strategy based on the score; and an association rendering module for rendering the translation text box on the industrial document according to the selected optimal placement strategy and establishing a visual association between the translation and the original text.

[0011] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1. By performing geometric correction on the input industrial document images, geometric distortion is eliminated, and image quality is improved based on the actual image conditions, thereby increasing the accuracy of document type recognition. After determining the document type, the region containing the original text is accurately extracted from the document, avoiding interference from irrelevant information and improving the targeting and accuracy of the translation. A hybrid retrieval engine is used to translate the original text, and the translated text undergoes structural consistency verification to ensure structural consistency with the original text, enhancing the usability of the translation. Then, based on the page geometry information of the industrial document, the content and length of the translated text, multiple possible translation text box placement strategies are considered. Each placement strategy is scored based on the layout parameters of the translation text boxes. Existing technologies rarely integrate... By comprehensively considering the geometric information of the document page and the layout parameters of the translation, a placement strategy is selected and scored. This comprehensive approach ensures a more reasonable and aesthetically pleasing layout of the translation text box on the industrial document page, avoiding issues such as overlapping text boxes and improper placement. This improves the overall visual effect and readability of the industrial document. Finally, the placement strategy with the highest score is selected as the optimal strategy. Following this optimal strategy, the translation text box is displayed in its corresponding position on the industrial document. Some existing technologies only focus on the presentation of the translation while neglecting to establish a visual connection between the translation and the original text. This invention establishes a correspondence between the translation and the original text through visual means such as line connections, rendering the translation text box to ensure the translation is clearly presented on the document.

[0012] 2. By obtaining the average logarithmic probability of each output character, the overall confidence level of the large language model in the generated characters can be comprehensively measured, providing a basis for subsequent judgment of translation quality. Using two different large language models for translation can increase the diversity of translations. At the same time, by calculating the similarity of translations, it can be determined whether the translation results of the two models are consistent. If the similarity is high, it means that the two models understand and translate the document in a consistent manner, and the translation quality is high. Then, pattern matching is performed on the original text region in the grayscale image of the industrial document. According to the pattern matching results, the corresponding visual structure attributes are marked to maintain the consistency of the translation with the original text in visual structure. Furthermore, visual constraint instructions based on visual structure attributes are added to the prompt words to be sent to the large language model, which can guide the large language model to follow specific visual structure requirements when generating translations, making the translations more in line with actual needs and improving the quality of translations. Finally, during the decoding process of the large language model, if a term is detected that is forcibly locked, the probability of the standard translation corresponding to the term is set to 1, which can ensure that the term appears accurately in the translation. Forced verification is performed on the newly generated translation to further ensure the quality of the translation.

[0013] 3. By calculating the height of the preset translation text box, an accurate dimensional basis is provided for determining the position of the translation text box, giving the layout of the translation text box on the page a basic size plan and avoiding confusion in subsequent position calculations due to unclear dimensions. Then, candidate positions of the translation text box are calculated according to the preset placement strategy set, increasing the flexibility of the translation text box placement and verifying its effectiveness. It can flexibly select the most suitable placement method according to different situations, improving the rationality and success rate of translation text box placement, and avoiding layout problems caused by a single placement method. Existing technologies, due to fixed placement methods, may not be able to adapt to various complex page and translation situations, and are prone to unreasonable layout problems. Finally, the dynamic layout adjustment mechanism is triggered based on the pixel dilation rate. Different processing methods are adopted according to different pixel dilation rate ranges. For cases with excessive size changes, a more suitable placement strategy is directly selected, improving layout efficiency. By reducing the font size, compressing the character spacing, and automatic line wrapping, the size of the translation text box can be effectively reduced, which can solve the size over-limit problem to a certain extent and further adapt to the page space. Existing technologies lack a dynamic adjustment mechanism and may not be able to effectively handle changes in translation size, resulting in poor layout effects. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the industrial document translation method and system based on visual positioning provided in the embodiments of the present invention; Figure 2 This is a flowchart of geometric correction provided in an embodiment of the present invention; Figure 3 This is an architecture diagram of the hybrid search engine provided in an embodiment of the present invention; Figure 4 This is a performance chart of the hybrid search engine provided in the embodiments of the present invention; Figure 5 This is a flowchart of the placement strategy for the translation text box provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0017] Before providing a detailed explanation of the embodiments of this application, the application scenarios of these embodiments will be described first.

[0018] This invention provides a visual positioning-based method for industrial document translation. For example... Figure 1 The flowchart shown is for a vision-based industrial document translation method. The processing flow of this method may include the following steps: S1 performs geometric correction on the input industrial document image to eliminate geometric distortion, providing a foundation for accurate text extraction and processing. The corrected image is analyzed to identify the document type, enabling the adoption of more appropriate processing strategies for different document types, thereby improving the accuracy and efficiency of industrial document translation and layout.

[0019] like Figure 2 The flowchart shown illustrates the geometric correction process. Specifically, geometric correction is performed on the input industrial document image, and the specific process is as follows: Converting industrial document images into grayscale images: A grayscale image is formed by converting the color value of each pixel in a color image into a single grayscale value (0-255, where 0 represents black and 255 represents white). It contains only brightness information and no color information. Converting color industrial document images into grayscale images that only contain brightness information removes color information, simplifies subsequent processing steps, because many image processing algorithms are easier to implement and have less computational cost on grayscale images. At the same time, it preserves the geometric structure and texture information of the image, providing a foundation for subsequent edge detection and other operations.

[0020] Edge detection algorithms (such as the Canny operator) are applied to grayscale images of industrial documents to extract document edge information. The average tilt angle of the document is calculated based on Hough transform or line fitting techniques to initially determine the document's orientation. Subsequently, using the average tilt angle as a priori condition, a contour detection algorithm is used to accurately locate the document region in the image, retaining only contours consistent with or perpendicular to the average tilt angle as candidate document edges, eliminating interference from non-document edges. The coordinates of the four corner points of the document image in the current pixel coordinate system are located and denoted as the source point set. The minimum bounding rectangle of the source point set is calculated, and its width and aspect ratio determine the corrected standard image size. The coordinates of the four vertices of this standard rectangle are defined as the target point set. Next, the rotation angle of the minimum bounding rectangle is checked for consistency with the previously calculated average tilt angle to eliminate ambiguity in the angle calculation. If the angle difference is close to 90 degrees, for example, between 75 and 105 degrees, the rotation angle of the minimum bounding rectangle is corrected by 90 degrees. The small bounding rectangle is rotated by 90 degrees to minimize its deviation from the average tilt angle, thereby determining the accurate corner arrangement order of the target mapping coordinate system (i.e., the coordinates of the industrial document page). Based on the verified corner order, a one-to-one mapping relationship is established between the source point set and the target point set. A perspective transformation matrix is ​​calculated using this mapping relationship. Finally, the perspective transformation matrix is ​​applied to the grayscale image of the industrial document. Through image interpolation and resampling technology, the original image is mapped to a standard rectangular coordinate system, thereby eliminating geometric distortion caused by the shooting angle or scanning, resulting in a front-view grayscale image of the industrial document. The perspective transformation matrix describes the perspective transformation relationship of the image. By constructing the perspective transformation matrix, the image can be transformed from one viewpoint to another, eliminating perspective distortion caused by the shooting angle. This effectively eliminates the tilt and perspective distortion of industrial document images caused by the shooting angle, restoring the text and graphics in the document to their normal geometric shape and position, improving image readability and the accuracy of subsequent processing.

[0021] To obtain the image resolution corresponding to the grayscale image of the industrial document, such as the number of pixels in width and height, if the image resolution is less than the image resolution limit, it indicates that the image size is too small. To ensure the accuracy of subsequent processing and recognition, a cubic interpolation algorithm is used to enlarge the grayscale image of the industrial document by a preset factor. The image resolution limit represents the minimum allowable image resolution set by preset personnel. The cubic interpolation algorithm works by considering the information of the surrounding pixels and using a cubic polynomial for interpolation calculation when enlarging the image. This results in a smoother and clearer enlarged image effect than algorithms such as linear interpolation, thereby improving the image resolution and making the details in the image clearer. This is beneficial for subsequent operations such as document type recognition. At the same time, the cubic interpolation algorithm can improve the quality of the enlarged image.

[0022] The Laplacian operator is used to calculate the image sharpness score corresponding to the grayscale image of the industrial document. If the image sharpness score is less than the image sharpness score limit, it indicates that the image is too blurry and cannot accurately identify the document type, triggering a quality warning and prompting the preset personnel to re-upload the industrial document image. Otherwise, the document type of the corrected industrial document image is identified. The image sharpness score limit represents the minimum allowable image sharpness score set by the preset personnel. The Laplacian operator is used for image sharpening and edge detection. By calculating the Laplacian value of the image, the image sharpness information can be obtained. The larger the Laplacian value, the sharper the image. This ensures that the image used for recognition has sufficient sharpness, improves the accuracy and reliability of document type recognition, and avoids recognition errors caused by image blurring.

[0023] Specifically, the process for identifying the document type of the corrected industrial document image is as follows: This system determines whether the input grayscale image of an industrial document is a Computer-Aided Design (CAD) drawing. CAD drawings contain geometric shapes, dimensions, and text annotations. The determination method is based on image feature extraction and pattern recognition, such as detecting the presence of typical CAD graphic elements (e.g., lines, arcs, dimensions). Accurately distinguishing different types of industrial documents provides a basis for selecting appropriate models and algorithms in subsequent processing. Incorrect determination may lead to incorrect subsequent processing.

[0024] If the image is a CAD drawing, a pre-trained image segmentation model, namely the U-Net model, is invoked. This model decomposes the image into a dimension annotation layer, a text annotation layer, and a geometry layer. It also translates the text annotation layer, converting text from different languages ​​into a unified language for easier subsequent processing and understanding. The U-Net model employs an encoder-decoder structure, fusing the encoder's feature map with the decoder's upsampled feature map through skip connections. This effectively and accurately segments different regions in the image. The U-Net model can precisely decompose CAD drawings into different levels, allowing information such as dimensions, text annotations, and geometry to be processed and analyzed separately.

[0025] If the document is not a CAD drawing, the layout analysis model for industrial documents, namely the PP-StructureV2 model, is used to identify table areas. Tables are marked as hard-constrained areas to ensure they are not obscured or interfered with in subsequent processing. Text features are extracted from non-table areas, including text position, font, size, and other information, providing a basis for subsequent text formatting judgment. The text features are input into a gradient boosting decision tree classifier to determine the text format, including titles, body text, and headers. Hard-constrained areas represent areas in industrial documents that are prohibited from being obscured. The PP-StructureV2 model can identify different areas in industrial documents, such as tables and text, and analyze and understand the layout structure of industrial documents, providing a foundation for subsequent text extraction and formatting judgment. The PP-StructureV2 model can accurately identify table regions in a document and mark them as hard-constrained regions, ensuring that table information is not destroyed in subsequent processing. Extracting text features from non-table regions can provide rich information for text format judgment and improve classification accuracy. The gradient boosting decision tree classifier can accurately classify text format based on text features, which helps to understand the structure and content organization of documents and provides a foundation for subsequent document processing and analysis.

[0026] S2, based on the identified document type, extracts the original text using OCR. Utilizing computer vision technology, it accurately records each text block—the coordinates and size of the subsequent original text region on the page—as well as the blank areas on the page. A hybrid search engine then translates the extracted original text, generating a corresponding translation, and performs structural consistency checks on the translation. The original text region represents the area containing the original text. Accurate extraction of the original text region avoids interference from irrelevant information, improving the translation's relevance. The use of a hybrid search engine enhances the accuracy and diversity of the translation, finding translations that better fit the context. Structural consistency checks ensure the quality of the translation, making it more consistent with the target language's expression style.

[0027] like Figure 3The architecture diagram of the hybrid retrieval engine shown includes three main stages: The first layer is exact matching. In this layer, the received text block to be translated is precisely matched in the brand-specific terminology database and the Leverage general terminology database using the AC automaton (Aho-Corasick algorithm) to obtain the standard translation, ensuring the forced substitution of terms; The second layer is semantic retrieval - RAG. If the first layer fails to match, the text is vectorized using models such as Sentence-BERT, and an Approximate Nearest Neighbor (ANN) search is performed in the translation memory to recall the Top-K semantically similar translation examples; The third layer is LLM generation. The text block to be translated, the standard translation matched in the first layer, and the translation examples recalled in the second layer are integrated into an Enhanced Prompt, which is submitted to a large language model, such as GPT-4o, to generate the final translation, ensuring the best balance between accuracy, fluency, and stylistic consistency in the translation.

[0028] like Figure 4 The performance graph of the hybrid search engine shown compares it with traditional single LLM search and traditional exact match search across multiple performance metrics. These metrics include translation accuracy, structural consistency, contextual adaptability, terminology accuracy, cost efficiency, and generalization ability. The hybrid search engine demonstrates superior performance across all evaluation metrics, particularly excelling in translation accuracy, contextual adaptability, and structural consistency. Compared to traditional single LLM search and traditional exact match search, the hybrid search engine, by integrating multiple search methods, better balances translation quality and computational efficiency, improving the overall performance and scalability of the translation system. The graph clearly illustrates the robust performance of the hybrid search engine across different dimensions, especially its significant advantages when handling complex and multimodal data. This result validates the potential of this approach in practical applications, particularly its ability to improve translation quality and adapt to diverse languages ​​and structural differences.

[0029] Specifically, the translation undergoes structural consistency verification, and the specific process is as follows: After translating the original text region in the grayscale image of an industrial document using a hybrid retrieval engine, the average logarithmic probability of each output character is obtained. If the average logarithmic probability of each character is not less than the logarithmic probability limit, subsequent processing continues. The logarithmic probability of each character is used to measure the degree of confidence of the large language model in each generated character. In the large language model, for each generated character, the large language model outputs a probability value, indicating the likelihood that the character is correctly generated. The logarithmic probability is the value obtained by taking the logarithm of this probability. Taking the logarithm transforms the probability multiplication operation into an addition operation, which is convenient for calculation and processing. The logarithmic probability limit represents a pre-set threshold value used to judge whether the large language model's confidence in the generated character is high enough. The hybrid retrieval engine can fully utilize the advantages of multiple retrieval technologies to accurately identify and locate text regions in the grayscale image of the industrial document, avoiding translation errors caused by inaccurate text positioning. If the average logarithmic probability of each character is high, it indicates that the translation result is relatively reliable, and no additional verification steps are needed. Continuing with subsequent processing directly can save time and computing resources, improving the processing efficiency of industrial documents.

[0030] If the average logarithmic probability of each character is less than the logarithmic probability limit, a translation consistency determination is performed. Specifically, the original text region to be translated in the grayscale image of the industrial document is input into the first translation model and the second translation model respectively to generate the first translation text and the second translation text. The Jaccard similarity is used to calculate the similarity between the generated first translation text and the second translation text. If the calculated translation similarity is greater than the translation similarity limit, subsequent processing continues; otherwise, the terminology consistency constraint process is triggered. The translation similarity limit represents a threshold set by the preset personnel to measure the similarity between two translation texts. The Jaccard similarity is used to compare the similarity between two sets. For translation text, it can be regarded as a set of characters. By introducing a translation consistency criterion and cross-validating translations generated by two different models, the accuracy of the translation results can be guaranteed, thereby improving translation quality. When the similarity between the two translated texts is high, it indicates that the translation results are close in semantics and expression, and the translation results are considered relatively reliable, allowing the subsequent processing to continue. When the similarity between the translated texts is low, there may be problems such as inaccurate translation or inconsistent terminology usage. Triggering the terminology consistency constraint process can address these issues, improving the accuracy and consistency of the translation results at the terminology level by constraining the use of terminology, thus ensuring translation quality.

[0031] The first translation model is the GPT-4o model, and the second translation model is the Claude 3.5 Sonnet model. The first and second translation models are heterogeneous models, configured to process the original text regions to be translated in parallel from the grayscale images of the input industrial documents. The first and second translated texts are used as benchmark data for cross-validation to evaluate the accuracy and robustness of the translation results.

[0032] It should be added that the terminology consistency constraint process specifically includes: By acquiring text sequences from the original text region through OCR, and using the AC automaton in the hybrid retrieval engine to match the text sequences against brand terminology databases and general terminology databases, it is possible to quickly and accurately locate professional terms in the original text within a large number of terminology databases, providing a foundation for subsequent accurate translation and improving the efficiency and accuracy of terminology recognition. The AC automaton is a multi-pattern string matching algorithm used to simultaneously find the occurrence positions of multiple pattern strings in a text. By constructing a finite-state automaton, it can efficiently perform multi-pattern matching in text with near-linear time complexity, making it suitable for quickly locating multiple specific words or phrases in a large amount of text. The brand terminology database represents a database that specifically stores professional terms, product names, brand-specific expressions, etc., related to a specific brand. These terms have brand uniqueness and need to follow specific standards during translation to ensure accurate and consistent brand information. The general terminology database represents a database containing widely used, general professional terms, industry vocabulary, etc. These terms may have relatively unified translation standards across different brands or fields and are used to handle the translation of non-brand-specific professional content.

[0033] If a match is found in the brand terminology database, the standard translation is retrieved and forcibly locked; otherwise, the general terminology database is queried. Forcible locking means that during the translation process, the large language model is not allowed to modify or generate other translations for the established standard translation, ensuring the consistency and accuracy of terminology translation. Priority is given to ensuring the accuracy and consistency of brand terminology translation because brand terminology is usually unique and important, and needs to strictly follow the translation standards stipulated by the brand. For non-brand terminology, the general terminology database is queried to expand the terminology coverage.

[0034] If a match is found in the general terminology database, the standard translation is retrieved and locked; otherwise, the translated terminology mappings in this session are queried and locked. This ensures consistency in terminology translation even when a standard translation cannot be retrieved from the terminology database, preventing different translations of the same term from appearing in the same document.

[0035] Pattern matching is performed on the original text regions in the grayscale images of industrial documents to detect whether the original text regions contain key-value pair structures, list structures, or table cell structures. Based on the pattern matching results, the corresponding visual structure attributes are labeled. Pattern matching includes key-value pair structure detection, list structure detection, and table cell detection. Key-value pair structures consist of a key and its corresponding value. List structures represent a series of items arranged in an ordered or unordered manner, such as bulleted lists or numbered lists. Tables consist of rows and columns, with each intersection called a cell, which contains specific data or text. Accurately identifying the visual structure of industrial documents provides a basis for subsequently generating translations that conform to the visual structure, making the translation closer to the original text in terms of layout and structure.

[0036] The key-value pair structure detection specifically involves defining rules that explicitly specify common separators between keys and values, such as colons, equal signs, and spaces. For a format like "Name:Product A", the rule could be defined as "[key text][separator:|=][value text]". Regular expressions are then written based on these rules. For example, for simple key-value pairs separated by colons, the regular expression could be written as (\w+)\s*:\s*([^\n]+), where (\w+) matches the key, \s* matches any spaces, : is the separator, and ([^\n]+) matches the value (up to the newline character). If the text content matches the regular expression, it is determined to be a key-value pair format. (List structure) The detection process is as follows: If a text block meets either of the following conditions: Condition 1: It begins with a bullet point or number format. For example, regular expressions can be used to match numbered lists, such as (\d+\.)\s+ which matches list items that begin with a number followed by a period. In this case, the text belongs to a list structure. Condition 2: Calculate the horizontal difference between the coordinates of the first line and the coordinates of subsequent lines in the text block. If the difference exceeds a preset pixel threshold (e.g., 15 pixels), it is determined that there is first-line indentation, and the text belongs to a list structure. Table cell detection is performed based on the table areas identified above. If the context attributes of the text block contain a table cell identifier, or if the coordinates of the text block are within the detected table rows and columns, it is marked as a table cell structure.

[0037] Visual constraint instructions based on visual structure attributes are added to the prompts to be sent to the large language model, allowing the model to consider this visual structure information when processing text. These visual constraint instructions are generated based on the document's visual structure attributes and are used to guide the large language model to consider visual structure information when processing text, making the generated translation more consistent with the visual presentation of the original text in terms of structure and semantics. This guides the large language model to generate translations that better match the visual structure of the original text, improving the quality and readability of the translations and making them more structurally and semantically compatible with the original text.

[0038] If a forcibly locked term is detected during the decoding process of the large language model, the probability of the standard translation corresponding to that term is set to 1. This ensures that the large language model prioritizes the standard translation and performs forced verification on the newly generated translation. Forced verification means strictly checking the new translation generated by the large language model to verify whether it conforms to preset rules or standards. If the verification passes, the new translation is accepted and subsequent processing continues; otherwise, the preset personnel are notified. This ensures that forcibly locked terms are accurately presented in the translation, while also improving the overall quality of the translation and reducing errors and non-compliant content through forced verification.

[0039] It should be noted that mandatory validation is performed on the newly generated translation, and the specific process is as follows: If key-value pair structures exist in the original text area of ​​the grayscale image of the industrial document, the key-value pairs in the translation are located, and the difference between the coordinates of the key-value pairs in the translation and the coordinates of the key-value pairs in the original text is recorded as the key-value pair coordinate deviation. The key-value pair coordinate deviation represents the positional offset distance between the coordinates of the key-value pair in the translation and the coordinates of the key-value pair in the original text, calculated using the Euclidean distance formula. It is used to measure the degree of deviation between the translation and the original text in terms of key-value pair positions. By locating key-value pairs, key-value pairs can be accurately identified from the translation, providing a basis for subsequent positional comparison and ensuring that the comparison objects are the correct corresponding key-value pairs, avoiding inaccurate verification results due to identification errors. By calculating the key-value pair coordinate deviation, the positional difference between the key-value pairs in the translation and the original text is quantified, thereby objectively judging whether the position of the key-value pairs in the translation is reasonable.

[0040] Specifically, locating key-value pairs involves using an OCR tool to perform text recognition on the grayscale image of an industrial document, acquiring all text information in the image and the position coordinates of each text block. Based on the characteristics of the key-value pairs, the identified text blocks are divided into regions. For example, keys and values ​​are usually arranged adjacently in the horizontal or vertical direction. By analyzing the relative positional relationship between text blocks, text blocks that may constitute key-value pairs can be divided into the same region. Within the divided regions, the text content is further analyzed to determine the specific positions of the keys and values. For each key-value pair, its key position coordinates are determined. If the key-value pair is arranged horizontally, the center point coordinates of the key text block can be selected as the representative coordinates of the key, and the center point coordinates of the value text block can be selected as the representative coordinates of the value. If the key-value pair is arranged vertically, the midpoint coordinates of the corresponding upper and lower edges are selected as the representative coordinates. Alternatively, depending on the actual needs, the coordinates of the upper left corner of the overall rectangle of the key-value pair can be selected as the coordinates of the key-value pair.

[0041] If the key-value pair coordinate deviation exceeds the key-value pair coordinate deviation limit, the verification fails; if the key-value pair coordinate deviation is not greater than the key-value pair coordinate deviation limit, the verification passes. The key-value pair coordinate deviation limit represents a pre-set threshold used to determine whether the deviation between the translated key-value pair coordinates and the original key-value pair coordinates is within an acceptable range. Based on the comparison between the calculated deviation and the limit, the system quickly and accurately determines whether the translated text meets the requirements in terms of key-value pair positions, improving the efficiency and accuracy of mandatory verification. This ensures that the key-value pair structure in the translated text is highly matched with the original text in terms of position, ensuring accurate correspondence and presentation of the translated text and the original text in industrial documents.

[0042] If line breaks exist in the original text area of ​​the grayscale image of the industrial document, a text parsing algorithm, such as using index traversal of text characters, is used to count the number of line breaks in the translation. If the deviation in the number of line breaks between the translation and the original text exceeds a line break deviation limit, the verification fails. If the deviation is not greater than the limit, the verification passes. The line break deviation represents the absolute value of the difference between the number of line breaks in the translation and the number of line breaks in the original text. The line break deviation limit is set by pre-set personnel based on experience, for example, it can be set to 2 to allow for reasonable variations in line breaks due to differences in language habits. By accurately counting the number of line breaks, the text break handling of the translation and the original text can be intuitively understood, providing data support for determining whether they are consistent. Using the equality of the number of line breaks as a criterion, the translation can be simply and effectively ensured to maintain consistency with the original text in terms of layout, avoiding text information confusion due to different line breaks, and improving the quality and usability of the translation.

[0043] S3, based on the page geometry information and translation of the industrial document, selects a placement strategy for the translation text box. Based on the layout parameters of the translation text box, it scores each placement strategy and selects the optimal strategy. By comprehensively considering page geometry information and translation characteristics when selecting a placement strategy, the layout of the translation on the document page becomes more reasonable. By scoring each placement strategy and selecting the optimal strategy, the rationality of the translation text box layout is ensured, thereby achieving automatic translation and reasonable layout of industrial document images, improving the automation level and accuracy of industrial document processing.

[0044] like Figure 5 The flowchart shown illustrates the placement strategy for selecting the translation text box. Specifically, based on the page geometry information and translation of the industrial document, the placement strategy for selecting the translation text box is as follows: The height of the preset translated text box is calculated by multiplying the height of the original text area corresponding to the original text box and the average expansion rate of the translated text relative to the original text area, and then summing the result with an error factor set by the personnel based on experience. The original text box represents the geometric boundary of the original text area, and the width of the preset translated text box is equal to the width of the original text box. Since different languages ​​require different numbers of characters and layout space to express the same meaning, the average expansion rate of the translated text relative to the original text reflects the average increase in space occupied by the translated text relative to the original text. By considering the size of the original text and the language expansion rate, the size of the translated text box can be estimated more accurately, avoiding unreasonable size of the translated text box due to language differences. This provides a basis for the reasonable placement of the translated text box, making the layout of the translated text on the page more scientific and beautiful, and preventing layout chaos or overlap with the original text due to size issues.

[0045] The system calculates candidate positions for the translation text box based on a pre-defined set of placement strategies and verifies the effectiveness of each strategy based on page geometry, eliminating invalid strategies that exceed page margins. Page margins represent the blank space between the page edge and the content area, including top, bottom, left, and right margins, used to regulate the layout of content on the page. By comparing with page margins, invalid placement strategies that exceed page boundaries are promptly eliminated, ensuring that the translation text box is always placed within the displayable area of ​​the page. This avoids issues such as incomplete translation or exceeding the page limits, improving the readability of the translation and the overall layout quality.

[0046] The placement strategy set includes a first placement strategy that is embedded above the original text area, a second placement strategy that is embedded below the original text area, a third placement strategy that is embedded to the right of the original text area, a fourth placement strategy that floats above the original text area, and a fifth placement strategy that surrounds the original text area (i.e., the sidebar annotation placement strategy). It provides a variety of possible translation text box placement schemes, enriching the layout options of the translation on the page. It can flexibly adjust the position of the translation according to different original text content and page conditions to adapt to various complex industrial document typesetting needs.

[0047] It should be noted that the effectiveness of each placement strategy is verified based on the page geometry information, specifically as follows: In the image coordinate system, for the first placement strategy, the difference between the top ordinate of the original text box and the height of the preset translation text box is recorded as the top ordinate of the translation text box. If the top ordinate of the translation text box is less than the ordinate of the top margin of the page, it is deemed invalid; otherwise, the coordinate of the corresponding translation text box is used as the candidate coordinate. The top ordinate of the original text box represents the vertical position value of the upper edge of the original text box's rectangular bounding box in the image coordinate system. The origin of the image coordinate system is located at the upper left corner of the page, with the vertical axis pointing downwards and the horizontal axis pointing to the right. By accurately calculating the top ordinate of the translation text box and comparing it with the top margin of the page, it can be ensured that the translation text box will not exceed the top margin of the page when embedded above the original text area, thus ensuring the rationality of the translation layout in the vertical direction of the page and avoiding problems such as incomplete translation display or messy layout.

[0048] For the second placement strategy, the sum of the bottom ordinate of the original text box and the height of the preset translation text box is recorded as the bottom ordinate of the translation text box. If the bottom ordinate of the translation text box is greater than the ordinate of the bottom margin of the page, it is considered invalid; otherwise, the coordinates of the corresponding translation text box are used as candidate coordinates. The bottom ordinate of the original text box represents the vertical position value of the bottom edge of the original text box's rectangular bounding box in the image coordinate system. By accurately calculating the bottom ordinate of the translation text box and comparing it with the bottom margin of the page, it can be ensured that the translation text box will not exceed the bottom margin of the page when it is embedded below the original text area, so that the layout of the translation in the bottom area of ​​the page meets the requirements and enhances the standardization and readability of the page layout.

[0049] For the third placement strategy, the sum of the right horizontal coordinate of the original text box and the width of the preset translation text box is recorded as the right horizontal coordinate of the translation text box. If the right horizontal coordinate of the translation text box is greater than the horizontal coordinate of the right margin of the page, it is considered invalid; otherwise, the coordinate of the corresponding translation text box is used as the candidate coordinate. The right horizontal coordinate of the original text box represents the horizontal position value of the right edge of the original text box's rectangular bounding box in the image coordinate system. By calculating the right horizontal coordinate of the translation text box and comparing it with the right margin of the page, it can be ensured that the translation text box will not exceed the right boundary of the page when it is embedded on the right side of the original text area. This ensures that the layout of the translation on the right side of the page in the horizontal direction is reasonable and avoids affecting the overall aesthetics and information integrity of the page due to exceeding the boundary.

[0050] For the fourth and fifth placement strategies, the situation of not exceeding the page boundary is usually taken into consideration when setting them, or their placement methods are relatively flexible. Unlike the first three strategies, they do not require strict boundary judgment. The coordinates of the corresponding translation text box are directly marked as candidate coordinates, which simplifies the verification process and improves processing efficiency. At the same time, the fourth and fifth placement strategies provide more flexible choices for the placement of translations while ensuring the rationality of the translation layout, and can adapt to different typesetting needs.

[0051] Based on the page geometry information and translation of the industrial document, a placement strategy for the translation text box is selected, which then includes: The dynamic layout adjustment mechanism is as follows: The rendering engine is used to calculate the pixel inflation rate of the translated text. The pixel inflation rate represents the percentage increase in pixel size of the translated text relative to the original text size after rendering. In text typesetting scenarios, the rendering engine is responsible for rendering the text content according to the set styles (such as font, font size, character spacing, etc.) and calculating the final pixel size and other information displayed on the screen. The pixel inflation rate represents the percentage increase in pixel size of the translated text relative to the original text size after rendering. For example, if the original text occupies 100 pixels after rendering and the translated text occupies 150 pixels after rendering, then the pixel inflation rate is (150-100) / 100=50%. This accurately quantifies the change in space occupation of the translated text relative to the original text after typesetting, providing a key basis for subsequent judgment on whether typesetting adjustments are needed.

[0052] If the pixel inflation rate is not greater than the first pixel inflation limit, the space occupied by the translated text after typesetting is considered to be within an acceptable range, and the dynamic typesetting adjustment mechanism is not triggered, thus improving typesetting efficiency.

[0053] If the pixel inflation rate exceeds the second pixel inflation limit, the first, second, or third placement strategy is abandoned, and the fourth or fifth placement strategy is selected. This quickly addresses the issue of excessive translation space usage, effectively reduces the space occupied by the translation, and avoids display abnormalities or information loss due to layout issues.

[0054] If the pixel expansion rate is greater than the first pixel expansion limit but not greater than the second pixel expansion limit, the font size is reduced by a preset ratio until the font size reaches the minimum font size set by the preset personnel based on experience or the pixel expansion rate is not greater than the first pixel expansion limit. Under the premise of ensuring the readability of the translation, the space occupation of the translation is effectively controlled by adjusting the font size, so that it is more in line with the typesetting requirements.

[0055] If the pixel expansion rate after reaching the minimum font size is greater than the first pixel expansion limit but not greater than the second pixel expansion limit, then the character spacing is compressed until the character spacing reaches the minimum character spacing set by the preset personnel based on experience or the pixel expansion rate is not greater than the first pixel expansion limit. Based on the font size adjustment, the space occupation of the translation is further optimized by adjusting the character spacing, so as to make the translation layout more compact and reasonable without excessively affecting the reading experience.

[0056] If the pixel inflation rate after reaching the minimum character spacing is greater than the first pixel inflation limit but not greater than the second pixel inflation limit, then automatic line wrapping is executed. For example, the layout is dynamically adjusted by pixel-level calculation (such as the OpenCV library) or CSS ch units (based on character width), allowing the text to automatically wrap at the text box boundaries. Otherwise, the dynamic layout adjustment mechanism is stopped. Automatic line wrapping further optimizes the text layout, making full use of vertical space and reducing horizontal space occupation, so that the translation can be better displayed in a limited area, while avoiding excessive adjustment that leads to layout chaos.

[0057] If the height expansion rate of the translated text after line breaks exceeds the limit, the first, second, or third placement strategy is abandoned, and the fourth or fifth placement strategy is selected; otherwise, subsequent processing continues. The height expansion rate is the value obtained by calculating the ratio of the difference between the height of the translated text box and the preset height of the translated text box. After automatic line breaks are executed, if the height expansion rate of the translated text exceeds the preset limit, it means that the first, second, or third placement strategy cannot meet the space occupancy requirements. Selecting the fourth or fifth placement strategy can ensure that the translated text layout meets the space occupancy requirements while making full use of the available space, thus improving the spatial layout adaptability of industrial document translation.

[0058] The first pixel expansion limit is less than the second pixel expansion limit. The first pixel expansion limit and the second pixel expansion limit are two preset thresholds used to determine the degree of pixel expansion after the translation is typed, thereby determining whether and how to trigger the dynamic typesetting adjustment mechanism, dividing the pixel expansion rate into different intervals, and adopting different typesetting adjustment strategies for different intervals.

[0059] Specifically, the placement strategies for the translated text boxes are scored, and the specific process is as follows: The Euclidean distance formula is used to calculate the distance between the center of the translated text box and the center of the original text box, as well as the minimum distance between the translated text box and each component of the industrial document. This quantifies the relative position of the translated text box with respect to the original text and other components of the industrial document. Industrial document components represent various elements contained in the industrial document, such as charts, titles, paragraphs, and footnotes. Calculating the distance between the center of the translated text box and the center of the original text box ensures that the translated text has a certain correlation with the original text, avoiding excessive dispersion of their positions within the document. Calculating the minimum distance between the translated text box and each component of the industrial document ensures that the translated text box does not get too close to or cover industrial document components, such as charts and titles, maintaining the overall layout and readability of the industrial document.

[0060] The overlap area between the translated text box and each industrial document component is calculated. The overlap areas of the translated text box and all industrial document components are summed to obtain the total overlap area, which is used to measure the degree of spatial overlap between the translated text box and the industrial document components, and to avoid excessive coverage of the translated text box or unreasonable overlap with important components. The overlap area represents the area formed by the intersection of the translated text box and the industrial document component on the two-dimensional plane, and is used to measure the degree of spatial overlap between the translated text box and the industrial document component.

[0061] The specific method for obtaining the overlapping area is as follows: assuming that both the text box and the component are approximately rectangular, the coordinates of the translated text box and each industrial document component are used to determine whether there is an overlap. If there is no overlap, the overlapping area is 0. If there is an overlap, the overlapping area is obtained by multiplying the length and width of the overlapping part.

[0062] The reciprocal of the distance between the center of the translated text box and the center of the original text box (specifically, the reciprocal of the sum of the distance between the center of the translated text box and the center of the original text box plus a constant 1 to prevent division by zero error), the minimum distance between the translated text box and each industrial document component, and the reciprocal of the total overlapping area (specifically, the reciprocal of the sum of the total overlapping area and a constant 1 to prevent division by zero error) are weighted and coupled to obtain the placement score corresponding to each placement strategy of the translated text box. The closer the translated text box is to the original text box, the farther the translated text box is from the industrial document component, and the smaller the total overlapping area, the higher the score. The weighted coupling process means that different factors (i.e., the distance between the center of the translated text box and the center of the original text box, the minimum distance between the translated text box and each industrial document component, and the total overlapping area) affect the placement score, and different weights are assigned to each factor. Then, these factors are combined and calculated according to their respective weights to obtain the comprehensive placement score. By allocating weights appropriately, the importance of certain factors can be highlighted according to actual needs. For example, if more emphasis is placed on the positional correspondence between the translation and the original text, a greater weight can be assigned to the distance between the center of the translation text box and the center of the original text box; if more emphasis is placed on avoiding overlap with industrial document components, a greater weight can be assigned to the total overlap area, so that the placement score can more comprehensively and accurately reflect the advantages and disadvantages of each placement strategy.

[0063] For the fourth and fifth placement strategies, penalty points are added to the placement score to obtain the final placement score. The penalty points are fixed negative values ​​set by the preset personnel based on experience. The additional penalty points are added to reduce the scores obtained due to certain non-compliance situations (such as excessive overlap), with the aim of putting the fourth and fifth placement strategies at a relative disadvantage in the final comparison, thereby improving the quality of the entire translation text box placement scheme.

[0064] Setting the placement strategy with the highest final placement score as the optimal placement strategy ensures that the selected solution is the best after comprehensively considering various factors.

[0065] S4. Based on the selected optimal placement strategy, use graphics rendering technology to draw the translated text box onto the industrial document image. Simultaneously, employ visual association methods, such as connecting the translated and original text with lines of the same color and adding identical numbering next to both, to establish a visual association between the two. Rendering the translated text box on the industrial document allows the translation to be presented intuitively within the document, facilitating user reading; establishing a visual association between the translated and original text helps users quickly locate the corresponding original content, improving the efficiency of comparative reading.

[0066] This invention provides a vision-based industrial document translation system, comprising: The industrial document image processing module is used to perform geometric correction on the input industrial document images and identify the document type of the corrected industrial document images. Geometric correction makes the document images regular, laying the foundation for subsequent accurate identification of document types and extraction of text regions. Accurate identification of document types helps subsequent modules to adopt more appropriate processing methods for different types of documents, improving overall processing efficiency and accuracy.

[0067] The industrial document translation module extracts the original text region from the identified document type, translates the extracted original text region through a hybrid retrieval engine, generates the corresponding translation, and performs structural consistency verification on the translation. Extracting the original text region can accurately locate the content to be translated, avoiding interference from irrelevant information; the hybrid retrieval engine utilizes the advantages of multiple retrieval methods to improve the accuracy and efficiency of translation; structural consistency verification ensures that the translation is consistent with the original text in terms of format and logic, thus improving the quality of the translation.

[0068] The visual positioning evaluation module is used to select a placement strategy for the translation text box based on the page geometry information and translation of the industrial document. Based on the layout parameters of the translation text box, it scores each placement strategy and selects the best placement strategy according to the score. It comprehensively considers the page geometry information and translation content to select a placement strategy to ensure that the translation text box is placed reasonably. The scoring mechanism quantifies the advantages and disadvantages of different strategies and scientifically selects the best strategy to make the translation more beautiful and readable on the page.

[0069] The associated rendering module is used to render the translated text box on the industrial document according to the selected optimal placement strategy and establish a visual association between the translated text and the original text; accurately render the translated text box on the industrial document so that the translated text is presented in an appropriate way; establish a visual association to facilitate users to quickly compare the original text and the translated text, and improve the efficiency of reading and understanding industrial documents.

[0070] The above-disclosed embodiments are merely some examples of the present invention and should not be construed as limiting the scope of the present invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A visual positioning-based industrial document translation method, characterized in that, The method includes: S1, Perform geometric correction on the input industrial document image and identify the document type of the corrected industrial document image; S2, extract the original text region from the identified document type, translate the extracted original text region through a hybrid retrieval engine to generate the corresponding translation, and perform structural consistency verification on the translation; S3, based on the page geometry information and translation of the industrial document, selects the placement strategy of the translation text box, scores each placement strategy of the translation text box based on the layout parameters of the translation text box, and selects the best placement strategy based on the score. S4. Based on the selected optimal placement strategy, render the translated text box on the industrial document and establish a visual association between the translated text and the original text.

2. The industrial document translation method based on visual positioning as described in claim 1, characterized in that, The specific process for performing geometric correction on the input industrial document image is as follows: Convert industrial document images to industrial document grayscale images; The average tilt angle is calculated by identifying straight line elements in the grayscale image of the industrial document using an edge detection algorithm. A perspective transformation matrix is ​​then constructed based on the average tilt angle and the coordinates of the industrial document page, and the perspective transformation matrix is ​​applied to the grayscale image of the industrial document. Obtain the image resolution corresponding to the grayscale image of the industrial document. If the image resolution is less than the image resolution limit, use the cubic interpolation algorithm to enlarge the grayscale image of the industrial document by a preset factor. The Laplacian operator is used to calculate the image sharpness score corresponding to the grayscale image of the industrial document. If the image sharpness score is less than the image sharpness score limit, a quality warning is triggered, prompting the preset personnel to re-upload the industrial document image; otherwise, the document type of the corrected industrial document image is identified.

3. The industrial document translation method based on visual positioning as described in claim 2, characterized in that, The specific process for identifying the document type of the corrected industrial document image is as follows: Determine whether the input industrial document grayscale image is a CAD drawing; If it is a CAD drawing, the pre-trained image segmentation model is called to decompose the image into a dimension annotation layer, a text annotation layer, and a geometric shape layer, and the text annotation layer is translated. If it is not a CAD drawing, the layout analysis model of the industrial document is called to identify the table area, the table is marked as a hard constraint area, and text features are extracted from the non-table area. The text features are input into the gradient boosting decision tree classifier to determine the text format including the title, body text and header.

4. The industrial document translation method based on visual positioning as described in claim 2, characterized in that, The specific process for performing structural consistency verification on the translation is as follows: After translating the original text region in the grayscale image of the industrial document using a hybrid retrieval engine, the average log probability of each output character is obtained. If the average log probability of each character is not less than the log probability limit, subsequent processing is continued. If the average log probability of each character is less than the log probability limit, then translation consistency determination is performed. Specifically, the original text region to be translated in the grayscale image of the industrial document is input into the first translation model and the second translation model respectively to generate the first translation text and the second translation text respectively. The similarity between the generated first translation text and the second translation text is calculated. If the calculated translation similarity is greater than the translation similarity limit, then the subsequent processing continues; otherwise, the terminology consistency constraint process is triggered. The first translation model and the second translation model are heterogeneous models, configured to process the original text regions to be translated in parallel in the grayscale image of the input industrial document. The first and second translated texts are used as benchmark data for cross-validation to assess the accuracy and robustness of the translation results.

5. The industrial document translation method based on visual positioning as described in claim 4, characterized in that, The terminology consistency constraint process specifically includes: The text sequence in the original text area is obtained, and the AC automaton in the hybrid retrieval engine is used to match the text sequence in the brand terminology database and the general terminology database; If a match is found in the brand terminology database, the standard translation is retrieved and locked; otherwise, the query is performed in the general terminology database. If a match is found in the general terminology database, the standard translation is retrieved and locked; otherwise, the translated terminology mappings in this session are queried and locked. Pattern matching is performed on the original text region in the grayscale image of industrial documents to detect whether the original text region contains key-value pair structure, list structure or table cell structure, and the corresponding visual structure attributes are marked according to the pattern matching results. Add visual constraint instructions based on visual structure attributes to the prompt words to be sent to the large language model; If a term that is forcibly locked is detected during the decoding process of the large language model, the probability of the standard translation corresponding to the term is set to 1, and a forced verification is performed on the newly generated translation. If the verification passes, the new translation is accepted and subsequent processing continues; otherwise, the preset personnel are notified.

6. The industrial document translation method based on visual positioning as described in claim 5, characterized in that, The specific process for performing mandatory verification on the newly generated translation is as follows: If a key-value pair structure exists in the original text area of ​​the grayscale image of the industrial document, then locate the key-value pair in the translation and record the difference between the coordinates of the key-value pair in the translation and the coordinates of the key-value pair in the original text as the key-value pair coordinate deviation. If the key-value pair coordinate deviation is greater than the key-value pair coordinate deviation limit, the verification is deemed to have failed; if the key-value pair coordinate deviation is not greater than the key-value pair coordinate deviation limit, the verification is deemed to have passed. If line breaks exist in the original text area of ​​the grayscale image of the industrial document, the number of line breaks in the translation is counted. If the deviation of the number of line breaks between the translation and the original text is greater than the line break number deviation limit, the verification is deemed to have failed. If the deviation of the number of line breaks between the translation and the original text is not greater than the line break number deviation limit, the verification is deemed to have passed.

7. The industrial document translation method based on visual positioning as described in claim 1, characterized in that, The process of selecting a placement strategy for the translation text box based on the page geometry information and translation of the industrial document is as follows: The height of the preset translation text box is calculated based on the height of the original text area corresponding to the original text box, and the average expansion rate of the translation language relative to the original language. The candidate positions of the translation text box are calculated based on the preset placement strategy set, and the effectiveness of each placement strategy is verified based on the page geometry information, eliminating invalid strategies that exceed the page margins. The placement strategy set includes a first placement strategy embedded above the original text area, a second placement strategy embedded below the original text area, a third placement strategy embedded to the right of the original text area, a fourth placement strategy floating above the original text area, and a fifth placement strategy surrounding the original text area. The validity verification of each placement strategy based on page geometry information is specifically as follows: In the image coordinate system, for the first placement strategy, the difference between the top ordinate of the original text box and the height of the preset translation text box is recorded as the top ordinate of the translation text box. If the top ordinate of the translation text box is less than the ordinate of the top margin of the page, it is determined to be invalid; otherwise, the corresponding translation text box coordinate is used as the candidate coordinate. For the second placement strategy, the sum of the bottom vertical coordinate of the original text box and the height of the preset translation text box is recorded as the bottom vertical coordinate of the translation text box. If the bottom vertical coordinate of the translation text box is greater than the vertical coordinate of the bottom margin of the page, it is determined to be invalid; otherwise, the corresponding translation text box coordinate is used as the candidate coordinate. For the third placement strategy, the sum of the right horizontal coordinate of the original text box and the width of the preset translation text box is recorded as the right horizontal coordinate of the translation text box. If the right horizontal coordinate of the translation text box is greater than the horizontal coordinate of the right margin of the page, it is determined to be invalid; otherwise, the coordinate of the corresponding translation text box is set as the candidate coordinate. For the fourth and fifth placement strategies, the corresponding translated text frame coordinates are directly marked as candidate coordinates.

8. The industrial document translation method based on visual positioning as described in claim 7, characterized in that, The process, which involves selecting a placement strategy for the translation text box based on the page geometry information and translation of the industrial document, further includes: The pixel dilation rate of the translation is calculated using the rendering engine; If the pixel inflation rate is not greater than the first pixel inflation limit, the dynamic layout adjustment mechanism will not be triggered. If the pixel dilation rate is greater than the second pixel dilation limit, then the fourth and fifth placement strategies are selected. If the pixel expansion rate is greater than the first pixel expansion limit but not greater than the second pixel expansion limit, the font size will be reduced by a preset ratio until the font size reaches the minimum font size or the pixel expansion rate is not greater than the first pixel expansion limit. If the pixel inflation rate after reaching the minimum font size is greater than the first pixel inflation limit but not greater than the second pixel inflation limit, then compress the character spacing until the character spacing reaches the minimum character spacing or the pixel inflation rate is not greater than the first pixel inflation limit. If the pixel inflation rate after reaching the minimum character spacing is greater than the first pixel inflation limit but not greater than the second pixel inflation limit, then automatic line wrapping is executed; otherwise, the dynamic layout adjustment mechanism is stopped. If the height expansion rate of the translated text after line breaks exceeds the limit of the height expansion rate of the translated text, then abandon the first, second, or third placement strategy and select the fourth or fifth placement strategy; otherwise, continue with subsequent processing. The first pixel dilation limit is less than the second pixel dilation limit.

9. The industrial document translation method based on visual positioning as described in claim 7, characterized in that, The specific process for scoring the placement strategies of the translated text boxes is as follows: The Euclidean distance formula is used to calculate the distance between the center of the translated text box and the center of the original text box, as well as the minimum distance between the translated text box and each industrial document component. Calculate the overlap area between the translation text box and each industrial document component, and sum up the overlap areas of the translation text box and all industrial document components to obtain the total overlap area; We perform weighted coupling processing on the reciprocal of the distance between the center of the translated text box and the center of the original text box, the minimum distance between the translated text box and each industrial document component, and the reciprocal of the total overlapping area to obtain the placement score corresponding to each placement strategy of the translated text box. For the fourth and fifth placement strategies, penalty points are added to the placement score to obtain the final placement score; The placement strategy that ultimately scores the highest is set as the optimal placement strategy.

10. A vision-based industrial document translation system, used to implement the vision-based industrial document translation method as described in any one of claims 1-9, characterized in that, The system includes: The industrial document image processing module is used to perform geometric correction on the input industrial document images and identify the document type of the corrected industrial document images; The industrial document translation module is used to extract the original text region from the identified document type, translate the extracted original text region through a hybrid retrieval engine, generate the corresponding translation, and perform structural consistency verification on the translation. The visual positioning evaluation module is used to select the placement strategy of the translation text box based on the page geometry information and translation of the industrial document. Based on the layout parameters of the translation text box, it scores each placement strategy of the translation text box and selects the best placement strategy based on the score. The associated rendering module is used to render translated text boxes on industrial documents based on the selected optimal placement strategy and establish a visual association between the translated text and the original text.

Images