File processing method, device and storage medium

Abstract

The application is suitable for the technical field of digital processing, and provides a file processing method, device and storage medium. In the file processing method, first, the to-be-processed file is subjected to OCR to obtain a text item set, and table information of a table in the to-be-processed file is recognized based on a visual language model; then, based on the text item set and the table information, a text block is mapped to a row-column structure defined by the table information according to coordinate information of the text block, and structured data corresponding to the to-be-processed file is constructed. In the foregoing manner, end-to-end conversion from table image input to structured data output can be completed in a single processing flow, and the problem that the prior art is difficult to process complex table structures and changes in layout, and the output result is poor in computability and reliability is solved, and the accuracy of recognition of the to-be-processed file and the usability of the structured data are significantly improved.

Description

Technical Field

[0001] This application belongs to the field of digital processing technology, and in particular relates to document processing methods, devices and storage media. Background Technology

[0002] Unprocessed documents are the core carriers for the collection, transmission, and utilization of statistical data, playing a fundamental supporting role in macroeconomic analysis, policy formulation, industry research, and academic evaluation.

[0003] Currently, when performing structured recognition on tables in documents, existing solutions struggle to handle complex structures and layout variations, resulting in poor computability and reliability of the output, which fails to meet the requirements of statistical data governance and batch digitization.

[0004] Therefore, there is an urgent need for a document processing method to address the recognition challenges posed by complex table structures and varied layouts, and to improve the reliability and computability of the output results. Summary of the Invention

[0005] This application provides a file processing method, device, and storage medium, which can solve the technical problems in existing solutions that make it difficult to handle complex structures and layout changes in tables, and result in poor computability and reliability of the output results.

[0006] In a first aspect, embodiments of this application provide a document processing method, including: Obtain the file to be processed, which contains at least one table; Optical Character Recognition (OCR) is performed on the file to be processed to obtain a set of text items in the file. The set of text items includes at least one of the following: at least one text block, coordinate information of the text block, and OCR confidence score. The document to be processed is identified based on a pre-defined visual language model to obtain table information corresponding to at least one table. The table information includes at least one of the following: the number of tables, the table title, the unit of measurement in the table, the table header level, and the table grid structure. Based on the text item set and table information, the text blocks in the text item set are mapped to the row and column structure defined by the table information according to their coordinate information, thus constructing the structured data corresponding to the file to be processed.

[0007] In one possible implementation of the first aspect, obtaining the file to be processed includes: Get the initial file; The initial file is preprocessed to obtain the file to be processed. The preprocessing includes at least one of the following: image format unification, resolution adjustment, grayscale conversion, noise reduction, tilt correction, orientation detection, edge enhancement, and image cropping.

[0008] In one possible implementation of the first aspect, after performing OCR on the file to be processed and obtaining the set of text items in the file, it further includes: Get the height of the text box of any text block in at least one text block; Text blocks whose height is greater than or equal to the height threshold are identified as abnormal text blocks; Divide the text box corresponding to the abnormal text block into at least two regions; Perform OCR on at least two regions to obtain the target text blocks of the abnormal text blocks in at least two regions; Replace the abnormal text block in the text item set with the target text block corresponding to the abnormal text block.

[0009] In one possible implementation of the first aspect, a visual language model is used to perform first-level and second-level recognition of the file to be processed. The first layer of identification is used to identify at least one of the following: table title, unit, number of tables, table header list, multi-level header relationship, and whether there is a continuation table; The second layer of recognition is used to identify the grid structure of the table, which includes at least one of the following: number of grid rows, number of columns, merged cells, header area, data area, parent-child index relationship, column examples, and blank cell positions.

[0010] In one possible implementation of the first aspect, based on the text item set and table information, text blocks in the text item set are mapped to the row and column structure defined by the table information according to their coordinate information, thereby constructing the structured data corresponding to the file to be processed, including: For any text block in at least one text block, the table header identified by the visual language model is matched with the text block based on the coordinate information of the text block; If the match is successful, the coordinate information of the text block will be used as the header anchor of the text block; Column boundaries are determined based on at least one of the following: the header anchor point corresponding to at least one text block, the spacing between adjacent header anchor points, the number of columns, and the distribution of at least one text block; and row boundaries are generated based on the column boundaries and the vertical distribution of at least one text block. Construct a candidate cell matrix corresponding to the file to be processed based on column and row boundaries; Generate structured data based on the candidate cell matrix.

[0011] In one possible implementation of the first aspect, structured data is generated based on the candidate cell matrix, including at least one of the following: Based on the coordinate information of at least one text block, fill at least one text block into the candidate cell matrix; Identify the unit of measurement in at least one text block and append the text corresponding to the unit of measurement to the header or indicator field of the candidate cell matrix; Combine multi-level headers into complete fields and populate the candidate cell matrix; Based on the positional relationship between different languages ​​in the candidate cell matrix, fill the candidate cell matrix with different languages; According to the preset configuration, multiple languages ​​are merged into the same field and filled into the candidate cell matrix, or the target language from multiple languages ​​is filled into the candidate cell matrix.

[0012] In one possible implementation of the first aspect, different languages ​​are filled into the candidate cell matrix according to the positional relationship between different languages ​​in the candidate cell matrix, including at least one of the following: When the first language text is below the second language text and does not contain an independent value, the first language text is used as a translation description of the second language text and filled into the candidate cell matrix; When the first language text and the second language text are in adjacent columns, they are merged into a bilingual comparison field and filled into the candidate cell matrix.

[0013] In one possible implementation of the first aspect, the method further includes: Based on the table information, at least one candidate table is determined from at least one table, wherein the table information of the candidate table contains a preset identifier for indicating that the candidate table is a continuation table; Based on at least one of the following criteria for candidate tables: title similarity, header similarity, unit information, number of rows and columns, indicator name, and page order relationship, candidate tables belonging to the same table are identified from at least one candidate table; at least one candidate table includes at least one of the following criteria: vertical continuation table and horizontal continuation table. For vertical continuation tables, append the data rows of the vertical continuation table to the data area of ​​the first table and complete the parent path. The first table is the previous table that belongs to the same table as the vertical continuation table and is logically or page-sequentially adjacent to it. For horizontal continuation tables, the newly added columns of the horizontal continuation table are merged into the second table, and the multi-level table headers are unified. The second table is the preceding table that belongs to the same table as the horizontal continuation table and is logically or page-sequentially adjacent to it.

[0014] Secondly, embodiments of this application provide a document processing apparatus, including: The acquisition module is used to acquire the file to be processed, which includes at least one table. The OCR recognition module is used to perform optical character recognition (OCR) on the file to be processed, and to obtain a set of text items in the file to be processed. The set of text items includes at least one of the following: at least one text block, coordinate information of the text block, and OCR confidence level. The visual language model processing module is used to recognize the file to be processed based on a preset visual language model and obtain table information corresponding to at least one table. The table information includes at least one of the following: the number of tables, the table title, the unit of measurement in the table, the table header level, and the table grid structure. The construction module is used to map text blocks in the text item set to the row and column structure defined by the table information according to their coordinate information, based on the text item set and table information, to construct the structured data corresponding to the file to be processed.

[0015] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a file processing method as described in any of the first aspects.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements a file processing method as described in any of the first aspects.

[0017] Fifthly, embodiments of this application provide a computer program product that, when run on a client, causes the client to execute any of the file processing methods described in the first aspect above.

[0018] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0019] In this embodiment, a statistical file to be processed is acquired, an OCR is performed on the file to obtain a set of text items, and table information in the file is identified based on a visual language model. Then, based on the set of text items and the table information, text blocks are mapped to the row and column structure defined by the table information according to their coordinate information, thus constructing the structured data corresponding to the file to be processed. Through this method, the system can complete the end-to-end conversion from table image input to structured data output in a single processing flow, solving the problems of existing solutions' difficulty in handling complex table structures and layout changes, and poor computability and reliability of output results. This significantly improves the accuracy of table recognition in the file to be processed and the usability of the structured data. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application; Figure 2 This is a schematic flowchart of a document processing method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a document processing method provided in another embodiment of this application; Figure 4 This is a schematic diagram of a document processing system provided in an embodiment of this application; Figure 5 This is an exemplary embodiment of the document processing method provided in one embodiment of this application; Figure 6 This is a structural block diagram of a document processing apparatus provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0023] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0026] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0027] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0028] Documents awaiting processing are the core carriers of statistical data collection, transmission, and utilization, playing a fundamental supporting role in macroeconomic analysis, policy formulation, industry research, and academic evaluation. Common forms include statistical yearbooks, statistical bulletins, compilations of statistical data, survey reports, and industry statistical handbooks.

[0029] These types of files are typically saved in Portable Document Format (PDF), as scanned images, or in a layout format, with a large amount of core data presented in tabular form. Compared to plain text, the tables in the files to be processed not only contain text and numbers, but also structured semantic information such as indicator hierarchy, time dimension, regional dimension, unit information, table continuation relationships, annotations, and bilingual comparisons. How to accurately transform the unstructured tabular content in the files to be processed into searchable, calculable, and editable structured data is a key issue in statistical data governance and digital construction.

[0030] Specifically, existing publication document processing solutions have the following shortcomings: 1. The existing solution mainly determines the row and column structure based on the position of table lines or text boxes. When the table has no line, scan skew, merged cells, or blank cells, it is easy to cause text to be filled into the wrong cell, resulting in misalignment of the table header and data.

[0031] 2. Existing solutions typically only restore the physical row and column structure and cannot identify the hierarchical relationships and combination paths between multi-level headers, resulting in output results that cannot be directly used for statistical calculations.

[0032] 3. Unit information often appears independently below the title or in the table header row. The existing solution treats the unit as ordinary text, making it difficult to accurately determine its scope of application, resulting in the loss or incorrect attribution of units.

[0033] 4. The existing solution identifies tables independently on a single page, lacking a mechanism to identify "continued table" relationships, which can easily split the same logical table into multiple unrelated tables.

[0034] 5. Lacks the ability to distinguish the semantics of Chinese and English indicators, and is prone to misidentifying English translations as independent data columns or accidentally deleting valid English descriptions.

[0035] 6. OCR may incorrectly merge text from different lines. Existing solutions do not detect or correct this, resulting in the content of multiple cells being merged into the same cell.

[0036] 7. Some solutions directly use visual language models to generate table results, which have problems such as unstable coordinates, missing null values, and misaligned numbers, making it difficult to meet high reliability requirements.

[0037] 8. Rule-based or template-based solutions rely on manual pre-configuration. When the source, year, or layout of the publication file changes, reconfiguration is required, making it difficult to process different types of files in batches.

[0038] In summary, existing technologies cannot reliably handle the unique complex structures and layout changes in the files being processed, and the output results are difficult to directly form high-quality structured statistical data.

[0039] In view of this, embodiments of this application provide a file processing method, device, and storage medium. The method involves performing OCR on the file to be processed to obtain a set of text items, recognizing table information in the file based on a visual language model, and then mapping text blocks to the row and column structure defined by the table information according to their coordinate information, thereby constructing structured data corresponding to the file to be processed. Through this method, OCR is used to obtain stable text content, character-level coordinates, and confidence levels, while the visual language model is used to understand table titles, header levels, grid structures, merged cells, and continuation table relationships. This enables the system to complete the end-to-end conversion from input file to output structured data in a single processing flow, solving the problems of existing solutions' difficulty in handling complex table structures and layout changes, and poor computability and reliability of output results. This significantly improves the accuracy of file recognition and the usability of structured data.

[0040] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. (Reference) Figure 1 The scenario includes an electronic device and at least one file to be processed.

[0041] In one possible implementation, the files to be processed include, but are not limited to, publications such as statistical yearbooks, statistical bulletins, statistical data compilations, survey reports, or industry statistical handbooks. These files may be in the format of PDF, scanned images, photographs, or formatted documents.

[0042] In one possible implementation, the electronic device can be a desktop computer, a laptop, a handheld computer, a cloud server, or a dedicated document processing device.

[0043] In this embodiment of the application, the file to be processed can be input into an electronic device. After the electronic device obtains the file to be processed, it can execute the file processing method provided in this application based on a specific file processing system (or application) and obtain the processing result (i.e., the structured data corresponding to the file to be processed).

[0044] In the above scenario, the electronic device can complete the end-to-end conversion from input of the file to be processed to output of structured data in a single processing flow, which solves the problems of existing solutions that are difficult to handle complex table structures and layout changes, and have poor computability and reliability of output results, and significantly improves the accuracy of file recognition and the usability of structured data.

[0045] Figure 2 This is a schematic flowchart illustrating a file processing method provided in an embodiment of this application. It should be understood that the file processing method in this embodiment is executed by an electronic device. Figure 2 As shown, the file processing method includes the following steps: S201, Obtain the file to be processed.

[0046] In some embodiments, "files to be processed" refers to files that require identification and structuring. These files can be PDF files, scanned images, photographs, or table images extracted from layout documents. The sources of these files include, but are not limited to, statistical yearbooks, statistical bulletins, statistical data compilations, survey reports, or industry statistical manuals. Optionally, the system obtains files to be processed in the following ways: receiving files uploaded by users, reading files from a specified storage path, retrieving files from a database, or receiving file streams via an API interface.

[0047] In some embodiments, after obtaining the file to be processed, the system first determines the file type of the input file. If the file to be processed is a PDF file, it is converted into an image. If the file to be processed is an image, the subsequent step S202 is executed.

[0048] In some embodiments, the file to be processed includes at least one table. That is, the content of the file to be processed contains at least one table area, which can be a wired table (with full border lines), a wireless table (forming a table structure only through text alignment), or a hybrid table (partially bordered and partially borderless). A file to be processed may contain multiple tables, for example, two independent statistical tables on the same page of a PDF.

[0049] In some embodiments, the file to be processed includes at least one table, including the following cases: The file to be processed is a multi-page file. Specifically, the system receives a multi-page file to be processed (e.g., a complete PDF of a statistical yearbook), in which different pages may contain the same logical table split into multiple physical pages as a continuation table, or may contain multiple independent statistical tables.

[0050] The file to be processed is a single-page file containing multiple tables. Specifically, the system receives images containing multiple table areas on the same page (e.g., two independent statistical tables arranged vertically or horizontally within the same scanned image).

[0051] S202, Perform OCR on the file to be processed to obtain the set of text items in the file to be processed.

[0052] In some embodiments, the text item set includes at least one OCR text item, each OCR text item corresponding to at least one of the following: a text block, coordinate information of the text block, and OCR confidence level. Optionally, each OCR text item may be represented as O i = {text i bboxi chars i confidence i}, where text i For the recognized text content, bbox i The coordinates of the bounding rectangle of the text block, chars i A set of character-level coordinates, confidence i OCR confidence level.

[0053] In some embodiments, when the file to be processed is a single-page file, the table in the file includes a title, unit rows, multi-level headers, merged cells, and multiple rows of data. The system can perform OCR recognition on the single-page file to obtain a set of OCR text items. For example, the recognition result includes text blocks such as "Region," "Gross Domestic Product," "Primary Industry," "Secondary Industry," "Tertiary Industry," and "Unit: Ten Thousand Yuan," along with their corresponding coordinate information. Specifically, each recognized text block includes its text content, its precise location in the image (e.g., the coordinates of its circumscribed rectangle), character-level coordinates, and recognition confidence.

[0054] In some embodiments, when the file to be processed is a multi-page file, it may include continuation tables that span multiple pages or regions, as well as complex statistical tables containing multilingual comparative content. The system performs image preprocessing and OCR recognition on each page or each table region separately, resulting in their own independent sets of OCR text items.

[0055] S203, based on a preset visual language model, identify the file to be processed and obtain table information corresponding to at least one table.

[0056] The table information includes at least one of the following: the number of tables, the table titles, the units of measurement in the tables, the table header level, and the table grid structure.

[0057] In some embodiments, when the file to be processed is a single-page file, the system invokes a visual language model to recognize table titles, units of measurement, multi-level header structures, and grid structures. For example, the visual language model recognizes header text such as "Region," "Gross Product," "Primary Industry," and "Secondary Industry," and identifies that "Gross Product" contains several sub-indicators, forming a parent-child hierarchical relationship. The system records this hierarchical relationship as: {First-level header: "Region," Second-level header: "Gross Product," Third-level header: "Primary Industry"}, for subsequent structured output.

[0058] In some embodiments, when the file to be processed is a multi-page file, the system calls a visual language model to detect continuation table identifiers. If preset identifiers such as "continued", "continued from previous table", or "continued" are identified, the system marks the corresponding table as a candidate continuation table for subsequent continuation table merging processing.

[0059] S204: Based on the text item set and table information, the text blocks in the text item set are mapped to the row and column structure defined by the table information according to their coordinate information, thus constructing the structured data corresponding to the file to be processed.

[0060] Specifically, the system determines column and row boundaries based on text block coordinates and table information, constructing a candidate cell matrix. It then calculates the overlap between the OCR text box and the candidate cells, assigning the text blocks to the corresponding cells. Optionally, when multiple text blocks exist in the same cell, they are sorted by position and merged; when a cell has no corresponding text block, empty values ​​are retained. Further, the system identifies the unit of measurement and appends it to the corresponding header or indicator field, combining multi-level headers into complete fields. Based on bilingual positional relationships and preset configurations, it processes the bilingual content, ultimately outputting structured data.

[0061] In some embodiments, the system ultimately outputs structured data, including at least one of the following: table title, table header, unit of measurement, data matrix, cell coordinates, confidence level, processing steps, anomalies, and editable results. The output format can be JavaScript Object Notation (JSON), Microsoft Excel, Comma-Separated Values ​​(CSV), or database fields, etc.

[0062] In some embodiments, when the file to be processed is a single-page file, the system outputs a structured result, which includes table title, unit of measurement, complete table header, data matrix, cell coordinates and identification confidence level.

[0063] In some embodiments, when the file to be processed is a multi-page file, the system outputs a complete logical table, which includes the source page number, original coordinates, continuation table merging relationship, bilingual processing results, and final structured data.

[0064] In this embodiment, by integrating OCR recognition and visual language model recognition into the same processing flow, the system can complete the end-to-end conversion from table images to structured data without relying on a single OCR result or a fixed template, significantly improving the accuracy and automation of table processing. Furthermore, based on the set of text items obtained by OCR (including text blocks and their coordinate information) and the table information recognized by the visual language model (including table quantity, title, unit of measurement, header level, and grid structure), the system can map text blocks to the row and column structure defined by the table information according to their coordinate information. This enables precise parsing and targeted processing of complex table structures, improving the accuracy of header-data matching and the reliability of structured output.

[0065] Meanwhile, when constructing structured data, empty values ​​are retained for cells without corresponding text blocks, avoiding data distortion caused by arbitrary completion by the visual language model; by identifying units of measurement and attaching them to corresponding fields, combining multi-level headers into complete fields, and processing bilingual content according to preset configurations, the problems of existing solutions in handling complex scenarios such as multi-level headers, misaligned units, bilingual mixed text, and cross-page continuation are solved, significantly improving the accuracy of table recognition in the documents to be processed and the usability of structured data.

[0066] Figure 3 This is a schematic flowchart illustrating a file processing method according to another embodiment of this application. It should be understood that the file processing method in this embodiment is executed by an electronic device. Figure 3 As shown, the file processing method includes the following steps: S301, Get the initial file.

[0067] In some embodiments, the initial file is the raw data source for table recognition and structuring processing. The initial file may be derived from the file to be processed, including but not limited to statistical yearbooks, statistical bulletins, statistical data compilations, survey reports, or industry statistical manuals.

[0068] In some embodiments, the format of the initial file includes, but is not limited to, PDF files, scanned images, photographs, or table images extracted from layout documents. The system can obtain the initial file by receiving user uploads, reading from a specified storage path, retrieving from a database, or receiving it through an API interface.

[0069] In some embodiments, the initial file obtained by the system can be a single file (such as a statistical yearbook PDF) or multiple files imported in batches (such as a collection of scanned images of multiple statistical bulletins).

[0070] S302, preprocess the initial file to obtain the file to be processed.

[0071] In some embodiments, the system first determines the type of the initial file. If the initial file is a PDF file, the target pages of the PDF are first converted into images, and then preprocessing is performed; if the initial file is an image, preprocessing is performed directly. The preprocessing includes at least one of the following: image format unification, resolution adjustment, grayscale conversion, noise reduction, tilt correction, orientation detection, edge enhancement, and image cropping.

[0072] In this embodiment of the application, the above preprocessing steps can improve the stability and accuracy of subsequent OCR recognition and visual language model structure recognition.

[0073] S303, Perform OCR on the file to be processed to obtain the set of text items in the file to be processed.

[0074] It should be noted that the implementation method of step S303 can be referred to Figure 2 Step S202 in the illustrated embodiment will not be described again here.

[0075] In some embodiments, the file processing method of this application may further include the following steps S3031~S3035: S3031, obtain the height of the text box of any text block in at least one text block.

[0076] S3032, Text blocks whose height is greater than or equal to the height threshold are identified as abnormal text blocks.

[0077] In some embodiments, the height threshold can be a fixed value set by the system. Alternatively, the height threshold can be obtained based on the median height of all text boxes; for example, the system can use the median height H of all text boxes as the threshold. m Determine the height threshold T h For example, the height threshold can be set as follows: T h = max(1.8 × H m , 25) In this step, when the height H of a certain text block is... i Greater than or equal to T h When the system determines that the text box may contain text that has been incorrectly merged by OCR, it identifies the text block as an abnormal text block.

[0078] S3033, divide the text box corresponding to the abnormal text block into at least two regions.

[0079] In some embodiments, the height of the text box of the abnormal text block can be compared with the height of the normal text box (e.g., the median H). mThe ratio of the text box height to the normal text box height is used to dynamically determine the number of segments. For example, when the text box height is approximately twice the normal text box height, it is divided into 2 regions; when the text box height is approximately three times the normal text box height, it is divided into 3 regions, and so on.

[0080] In other embodiments, a fixed number of segments (such as uniformly dividing into 2 regions) can be used, or the segmentation position and number of segments can be automatically determined by analyzing the vertical projection distribution of the text within the text box.

[0081] In another embodiment, the segmentation position and segmentation area can be automatically determined based on the vertical projection distribution of the text within the text box.

[0082] S3034, Perform OCR on at least two regions to obtain the target text block of the abnormal text block in at least two regions.

[0083] Specifically, each segmented region is magnified and then subjected to OCR recognition. The coordinates of the recognition results are then mapped back to the original image coordinate system to obtain the target text block corresponding to each region.

[0084] S3035, replace the abnormal text block in the text item set with the target text block corresponding to the abnormal text block.

[0085] For example, when processing a statistical yearbook document, the OCR system identifies the following text block:

[0086] Referring to the table above, the median height H of all text boxes m Set the abnormal height threshold T to 27. h = max(1.8 × 27, 25) = max(48.6, 25) = 48.6. Therefore, the height of the text box T004, 72 ≥ 48.6, is determined to be an abnormal text block.

[0087] Analysis revealed that the abnormal text box actually contained two lines of text: "Beijing" and "1200" should have been on the first line, and "Shanghai" and "1500" should have been on the second line, but they were incorrectly merged by the OCR. The text box was divided into two regions (e.g., an upper region and a lower region) using its horizontal center line as the dividing line.

[0088] Furthermore, the segmented upper and lower regions are magnified by 200% respectively, and then OCR recognition is performed separately to obtain the following two target text blocks:

[0089] Furthermore, replace the abnormal text block T004 (“Beijing 1200 Shanghai 1500”) in the original text item set with the two new target text blocks “Beijing 1200” and “Shanghai 1500” mentioned above.

[0090] By following the steps above, detecting text boxes with abnormal heights, segmenting them, and then re-recognizing them using OCR, the problem of OCR mistakenly merging text from different lines into the same text box can be effectively corrected. This avoids merging the contents of multiple cells into the same cell, thereby improving the accuracy of subsequent table structure recognition.

[0091] S304, based on a preset visual language model, recognize the file to be processed and obtain table information corresponding to at least one table.

[0092] The table information includes at least one of the following: the number of tables, the table titles, the units of measurement in the tables, the table header level, and the table grid structure. In some embodiments, the file to be processed can be input into a visual language model, which uses a hierarchical recognition method for recognition. The first layer of recognition identifies at least one of the following: table title, unit, number of tables, header list, multi-level header relationships, and whether a continuation table exists. The second layer of recognition identifies the table's grid structure, which includes at least one of the following: number of grid rows, number of columns, merged cells, header area, data area, parent-child index relationships, column examples, and blank cell positions.

[0093] In this embodiment of the application, layered recognition reduces the risk of truncation, row and column misalignment, and unstable formatting in the visual language model output in large tables.

[0094] S305, for any text block in at least one text block, match the header identified by the visual language model with the text block based on the coordinate information of the text block.

[0095] Specifically, the header text identified by the visual language model is fuzzily matched with the OCR text items. The matching process comprehensively considers text similarity, keyword overlap, character-level coordinates, candidate text positions, and the range of the header area.

[0096] S306, if the match is successful, use the coordinate information of the text block as the header anchor of the text block.

[0097] For example, when a certain header H j With OCR text item O i When a match is successful, the coordinates of the OCR text item are used as the anchor point for the table header, which can be represented as: A j = {header j ,colj x left x right y top y bottom} Among them, header j For table header text, col j x is the column index corresponding to the table header. left x right y top , y bottom This represents the coordinate boundaries of the header text in the original image.

[0098] S307, determine column boundaries based on at least one of the following: the header anchor point corresponding to at least one text block, the spacing between adjacent header anchor points, the number of columns, and the distribution of at least one text block; and generate row boundaries based on column boundaries and the vertical distribution of at least one text block.

[0099] For columns lacking anchor points, the system reconstructs complete column boundaries by interpolating or extrapolating based on the spacing between adjacent anchor points, OCR text density, and the number of columns provided by the visual language model. Subsequently, row boundaries are generated through clustering based on column boundaries, the vertical distribution of OCR text, and the number of rows provided by the visual language model.

[0100] S308: Construct a candidate cell matrix corresponding to the file to be processed based on column and row boundaries.

[0101] Specifically, the system forms a set of candidate cells based on row and column boundaries, and each candidate cell can be represented as: C rc = {row r ,col c bbox rc} Where r is the row index, c is the column index, and bbox rc This represents the coordinate range of the cell in the original graph.

[0102] In this embodiment, by matching the table header recognized by the visual language model with the OCR text block, establishing table header anchor points, and calculating column and row boundaries based on the anchor point coordinates, a candidate cell matrix is ​​constructed. This effectively solves the problem of table header and data misalignment caused by blurred table lines, non-table lines, and merged cells, and improves the matching accuracy between cell text and table header.

[0103] S309, Generate structured data based on the candidate cell matrix.

[0104] In some embodiments, structured data is generated based on the candidate cell matrix, including at least one of the following: 1. Fill at least one text block into the candidate cell matrix based on the coordinate information of at least one text block.

[0105] Specifically, the overlap between the OCR text box and the candidate cell is calculated based on the following formula: overlap = intersection area(Oi，Crc) / min(area(O i ), area(C rc The OCR text is assigned to the cells with the highest overlap and that meet the threshold condition. When multiple OCR text items exist in the same cell, they are merged after being sorted by position; when a cell has no OCR text items, the blank value is retained, and it is not arbitrarily filled in by the visual language model, thus avoiding the problems of missing blank values ​​and misaligned numbers, and ensuring the authenticity and verifiability of the output data.

[0106] 2. Identify the unit of measurement in at least one text block and append the text corresponding to the unit of measurement to the header or indicator field of the candidate cell matrix.

[0107] Specifically, identify the unit of measurement text below the title or in the table header, determine the scope of the unit, and merge the unit into the corresponding indicator or table header.

[0108] 3. Combine the multi-level headers into complete fields and populate them into the candidate cell matrix.

[0109] Specifically, multi-level headers can be broken down into several levels of items, and a correspondence between header text and column indexes can be established, combining multi-level headers into complete field names. For example, "Region-Gross Product-Primary Industry" can be parsed as: {First-level header: "Region", Second-level header: "Gross Product", Third-level header: "Primary Industry"}.

[0110] In this embodiment of the application, by identifying the unit of measurement and attaching it to the corresponding header or indicator field, and by combining multi-level headers into complete fields (such as "region-gross production value-primary industry"), the output structured data can retain the complete semantic meaning of the statistical table, which is convenient for subsequent data entry, statistical calculation and indicator analysis.

[0111] 4. Based on the positional relationships between different languages ​​in the candidate cell matrix, fill the candidate cell matrix with different languages. Specifically, this includes at least one of the following: 4.1 When the first language text is below the second language text and does not contain an independent value, the first language text is used as a translation description of the second language text and filled into the candidate cell matrix.

[0112] For example, in a bilingual statistical table, if there is the English text "Gross Domestic Product" at the corresponding position below the Chinese indicator "生产总值" and the English text does not contain an independent numerical value, the system determines that the English text is the translation description of the Chinese indicator "生产总值". During filling, "Gross Domestic Product" may be stored in association as the translation description of "生产总值", or it may be decided whether to perform combined output according to a preset configuration. For another example, if there is the English text "Primary Industry" below the Chinese indicator "第一产业" and the English text does not contain an independent numerical value, the system identifies it as a translation description.

[0113] Optionally, if the English text contains an independent numerical value, the system will not determine it as a translation description of the Chinese indicator, but process it as independent data content, and fill it into the corresponding cell normally according to the coordinate position. For example, if the content of a certain cell in the table is "GDP 2000 亿元" and contains a numerical value, the system will not identify it as a translation description of any Chinese indicator, but process it as independent English data text.

[0114] 4.2 When the first language text and the second language text are located in adjacent columns, the two are combined into a bilingual comparison field and filled into the candidate cell matrix.

[0115] For example, in a regional bilingual comparison table, if the Chinese regional name "北京" is located in a certain column and the English text "Beijing" exists in the adjacent column (right or left), it is determined that the two constitute a bilingual comparison field. During filling, "北京" and "Beijing" are combined into a bilingual comparison field, such as "北京 / Beijing" or "北京 (Beijing)", which is filled into the candidate cell matrix.

[0116] Optionally, if the English text is neither located below the Chinese indicator nor in an adjacent column with the Chinese regional name (for example, the English text is located in the comment line, footer or an independent line of the table), it will not be determined as a translation description or a bilingual comparison field, but will be filled into the corresponding candidate cell according to its actual coordinate position.

[0117] Alternatively, if the positional relationship between the English text and the Chinese text does not meet the "adjacent column" condition (for example, other columns are spaced therebetween), it is determined that the two will not be combined into a bilingual comparison field, but will be independently filled into the corresponding cells.

[0118] Alternatively, if the language type of the text cannot be determined (for example, the text is a mix of Chinese and English without a clear boundary), the text will be processed according to the original OCR recognition result, without forced splitting or combining.

[0119] 5. Based on the preset configuration, merge multiple languages ​​into the same field and fill it into the candidate cell matrix, or fill the target language from multiple languages ​​into the candidate cell matrix.

[0120] For example, the system determines how to process bilingual content based on user configuration or business needs. When bilingual information needs to be retained, the system merges Chinese and English into the same field; when only Chinese structured data is needed, the Chinese metrics are retained and the English translation is saved as supplementary explanation or metadata.

[0121] In this embodiment, the translation description or bilingual comparison field is identified based on the positional relationship between different languages, and bilingual merging or specific language priority output is supported according to the preset configuration, which can avoid data row and column disorder or accidental deletion of effective information caused by bilingual content.

[0122] In some embodiments, the file processing method further includes the following steps S1-S4: S1, Based on the table information, determine at least one candidate table from at least one table, wherein the table information of the candidate table contains a preset identifier for indicating that the candidate table is a continuation table.

[0123] For example, if the visual language model recognizes preset labels such as "continued", "following the previous table", or "continued" in the first layer of recognition, it can mark the corresponding table as a candidate continuation table.

[0124] S2, based on at least one of the following: title similarity, header similarity, unit information, number of rows and columns, indicator name, and page order relationship of at least one candidate table, determine the candidate tables that belong to the same table among at least one candidate tables.

[0125] Among them, at least one of the candidate tables includes a vertical continuation table and a horizontal continuation table.

[0126] For example, the system compares the similarity of titles, table header structure, unit information, number of rows and columns, indicator names, and page order among candidate tables to determine whether multiple tables belong to the same logical statistical table.

[0127] S3, for vertical continuation tables, append the data rows of the vertical continuation table to the data area of ​​the first table and complete the parent path. The first table is the previous table that belongs to the same table as the vertical continuation table and is logically or page-orderly adjacent to it.

[0128] Specifically, for vertical continuation tables, the header structure of the previous table can be retained, the data rows of the next table can be appended to the data area of ​​the previous table, and the parent path in the continuation table can be completed according to the parent-child indicator relationship.

[0129] S4. For horizontal continuation tables, merge the newly added columns of the horizontal continuation table into the second table and unify the multi-level table headers. The second table is the previous table that belongs to the same table as the horizontal continuation table and is adjacent to it in logical or page order.

[0130] Specifically, for horizontal continuation tables, the newly added columns of the later table can be merged into the earlier table based on the matching relationship of the indicator row, year column, or region column, and the multi-level table headers can be unified.

[0131] In this embodiment of the application, by identifying the continuation table identifier, it is determined whether the candidate tables belong to the same logical statistical table, and the vertical continuation table and the horizontal continuation table are merged respectively, which can automatically restore the complete data chain of the cross-page table without manual intervention.

[0132] In summary, in step S309, at least one of the following items is specifically executed: Identify the unit text below the title or in the table header, determine the scope of the unit, and merge the unit into the corresponding indicator or table header; Identify multi-level headers and combine them into complete field names; Identify the parent-child indicator relationship and complete the child indicator into a full indicator with a parent path; Identify line break indicators and re-merge indicator names that have been split into adjacent lines by OCR. Identify the relationships between Chinese, English, unit, and region names, and merge or clean bilingual tables; Identify continuation table identifiers and merge horizontal, vertical, or multi-page continuations; Filter page numbers, footnotes, explanatory text, and other content that is not part of the table body.

[0133] In some embodiments, when a user modifies the recognition result, the differences before and after the user's correction can be recorded to generate reusable rules. These reusable rules include at least one of the following: template rules, field mapping rules, unit processing rules, or header enhancement rules. Accordingly, when encountering similar files to be processed subsequently, the system will prioritize using the learned rules to improve processing accuracy and efficiency.

[0134] In summary, the document processing method provided in this application does not rely on fixed table templates or manually pre-configured rules, and can adapt to documents of different years, sources, formats and scanning qualities. This reduces the cost of manual modeling, data entry and proofreading, and facilitates large-scale batch processing of historical statistical data.

[0135] Figure 4 This is a schematic diagram of a document processing system provided in one embodiment of this application. Figure 4 As shown, the system includes at least one of the following core modules: The input module is used to obtain the input file to be processed; The preprocessing module is used for operations such as image conversion, correction, enhancement, and noise reduction.

[0136] The OCR recognition module is used to extract text, character-level coordinates, and confidence levels.

[0137] The OCR anomaly repair module is used to detect, crop, and perform secondary recognition of abnormally high bounding boxes, and to fix the problem of incorrect merging of text between lines.

[0138] The visual language model recognition module is used to identify table titles, headers, grid structures, hierarchical relationships, and table continuation detection using a structural hierarchical parsing method.

[0139] The header anchor point coordinate calibration module is used to fuse OCR coordinates with the semantic structure of the visual language model to achieve header-OCR matching and column boundary reconstruction.

[0140] The cell reconstruction module is used to determine row and column boundaries, retain blank values, and complete text placement.

[0141] The statistical semantic post-processing module is used to process semantic information such as unit attribution, bilingual comparison, table continuation merging, and parent-child hierarchy.

[0142] The structured output module is used to generate structured data such as JSON, Excel, and database fields.

[0143] The manual correction and template learning module is used to generate template rules after the user modifies the results, so as to realize rule accumulation and template reuse; and to prioritize the use of learned rules in subsequent similar files to be processed, forming a feedback loop.

[0144] Figure 5 This is an exemplary embodiment of a document processing method provided in this application. It should be noted that this embodiment is used to process complex single-page tables in statistical yearbooks or compilations of statistical data. The table includes a title, unit rows, multi-level headers, merged cells, and multiple rows of data.

[0145] The execution subject of this application embodiment is an electronic device (or system), such as Figure 5 As shown, the file processing method in this application embodiment includes the following steps: S501 receives the file to be processed, which includes table titles, unit descriptions, multi-level headers, and data ranges.

[0146] S502 performs image preprocessing on the file to be processed, including grayscale conversion, noise reduction, orientation correction, and resolution adjustment.

[0147] S503, perform OCR recognition to obtain a set of OCR text items. For example, the recognition result includes text such as "region", "gross production value", "primary industry", "secondary industry", "unit: 10,000 yuan" and their coordinates.

[0148] S504, Detect OCR text boxes and determine if there are any text boxes with abnormal heights. If the height of a certain text box is significantly greater than the height of most other text boxes, it is determined that it may be formed by the erroneous merging of two lines of text. The system then crops the area into two parts, enlarges each part, and performs OCR again to obtain two more accurate text items.

[0149] S505 uses a visual language model to identify table titles, units, multi-level header structures, and grid structures. For example, the visual language model identifies that the table has fields such as "Region," "Gross Product," "Primary Industry," "Secondary Industry," and "Tertiary Industry," and identifies that "Gross Product" contains several sub-indicators.

[0150] S506, match the table header recognized by the visual language model with the OCR text. If "Primary Industry" matches the OCR text item successfully, then use the coordinates of the OCR text item as the header anchor point of the "Primary Industry" column.

[0151] S507, calculate column boundaries based on multiple header anchor points. If a header is not fully recognized by OCR, the system calculates the column boundary based on the spacing between adjacent header anchor points, the number of columns provided by the visual language model, and the OCR text distribution.

[0152] S508 generates row boundaries based on column boundaries and the vertical distribution of OCR text, forming a complete candidate cell matrix.

[0153] S509 fills the candidate cells with OCR text according to their coordinate overlap. For cells without text, the system retains blank values; for multiple characters within the same cell, the system merges them according to their position order.

[0154] S510 supports statistical semantic post-processing of units, bilingual representation, continuation tables, and parent-child hierarchies. Specifically, it identifies unit text such as "Unit: 10,000 yuan" and determines its scope. If the unit applies to multiple numerical columns, the system appends the unit to the corresponding table header or indicator field; it combines multi-level table headers into complete fields. For example, it combines "Gross Production Value" and "Primary Industry" into "Gross Production Value - Primary Industry" for subsequent database entry and statistical calculations.

[0155] S511 outputs structured data, including table title, units, complete table header, data matrix, cell coordinates, and recognition confidence level.

[0156] S512, user correction and template learning are used to accumulate rules for subsequent reuse.

[0157] Through this embodiment, the present invention can solve the problems of data misalignment, unit loss, and incomplete table headers that are prone to occur in ordinary OCR scenarios with multi-level headers, unit rows, and merged cells.

[0158] The following is an exemplary embodiment of a file processing method provided by another embodiment of this application. It should be noted that this embodiment of the application is used to process continuation tables that span multiple pages or regions in the file to be processed, as well as complex statistical tables containing bilingual (Chinese and English) content.

[0159] The execution subject of this application embodiment is an electronic device (or system). Specifically, the file processing method in this application embodiment includes the following steps: S601 receives a multi-page file to be processed, or receives an image containing multiple table areas on the same page.

[0160] S602, perform image preprocessing and OCR recognition on each page or each table area to obtain their respective OCR text item sets.

[0161] S603 invokes a visual language model to detect the number of tables, table titles, continuation table identifiers, and table boundaries on each page or in each area. If identifiers such as "continuation," "following previous table," or "continued" are detected, the system marks the corresponding table as a candidate continuation table.

[0162] S604 compares the similarity of titles, headers, unit information, number of rows and columns, indicator names, and page order among candidate tables to determine whether multiple tables belong to the same logical statistical table.

[0163] S605, for vertical continuation tables, the system retains the header structure of the previous table, appends the data rows of the next table to the data area of ​​the previous table, and completes the parent path in the continuation table according to the parent-child indicator relationship.

[0164] S606, for horizontal continuation tables, the system merges the newly added columns of the subsequent table into the previous table based on the matching relationship of the indicator row, year column or region column, and unifies the multi-level table headers.

[0165] S607, for bilingual statistical tables, the system identifies the positional relationship between Chinese indicators, English translations, English units, and English regional names. If the English text is below the Chinese indicator and does not contain independent values, it is determined to be a translation of the Chinese indicator; if the English text and the Chinese regional name are in adjacent columns, it is determined to be a bilingual comparison field for the region.

[0166] S608 determines how bilingual content is processed based on user configuration or business needs. When bilingual information needs to be retained, the system merges Chinese and English into the same field; when only Chinese structured data is needed, the system retains the Chinese indicators and saves the English translation as supplementary explanation or metadata.

[0167] S609 performs unit attribution, header completion, parent-child hierarchy completion, and null value retention processing on the merged data to ensure that the merged table has a unified header and a complete data matrix.

[0168] S610 outputs a complete logical table. The output includes the source page number, original coordinates, table continuation and merging relationships, bilingual processing results, and final structured data.

[0169] This embodiment can solve the problem of not being able to automatically merge when the same logical table in the file to be processed is split into multiple pages or multiple regions, and at the same time can reduce the risk of data misalignment and field pollution caused by bilingual content.

[0170] Corresponding to the file processing method in the above embodiments, Figure 6 This is a structural block diagram of a document processing apparatus provided in one embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown. (Refer to...) Figure 6 The device includes: The acquisition module 61 is used to acquire a file to be processed, which includes at least one table. OCR recognition module 62 is used to perform optical character recognition (OCR) on the file to be processed and obtain a set of text items in the file to be processed. The set of text items includes at least one of the following: at least one text block, coordinate information of the text block, and OCR confidence level. The visual language model processing module 63 is used to recognize the file to be processed based on a preset visual language model, and obtain table information corresponding to at least one table. The table information includes at least one of the following: the number of tables, the table title, the unit of measurement in the table, the table header level, and the table grid structure. The visual language model is used to perform first-level recognition and second-level recognition on the file to be processed. The first-level recognition is used to recognize at least one of the following: table title, unit of measurement, number of tables, table header list, multi-level table header relationship, and whether there is a continuation table. The second-level recognition is used to recognize at least one of the following: number of grid rows, number of columns, merged cells, table header area, data area, parent-child index relationship, column examples, and blank cell positions. Module 64 is used to map text blocks in the text item set to the row and column structure defined by the table information according to their coordinate information, based on the text item set and table information, to construct the structured data corresponding to the file to be processed.

[0171] In one possible implementation of the first aspect, the acquisition module 61 is specifically used for: Get the initial file; The initial file is preprocessed to obtain the file to be processed. The preprocessing includes at least one of the following: image format unification, resolution adjustment, grayscale conversion, noise reduction, tilt correction, orientation detection, edge enhancement, and image cropping.

[0172] In one possible implementation of the first aspect, the acquisition module 61 is further configured to: Get the height of the text box of any text block in at least one text block; Text blocks whose height is greater than or equal to the height threshold are identified as abnormal text blocks; Divide the text box corresponding to the abnormal text block into at least two regions; Perform OCR on at least two regions to obtain the target text blocks of the abnormal text blocks in at least two regions; Replace the abnormal text block in the text item set with the target text block corresponding to the abnormal text block.

[0173] In one possible implementation of the first aspect, a visual language model is used to perform first-level and second-level recognition of the file to be processed. The first layer of identification is used to identify at least one of the following: table title, unit, number of tables, table header list, multi-level header relationship, and whether there is a continuation table; The second layer of recognition is used to identify the grid structure of the table, which includes at least one of the following: number of grid rows, number of columns, merged cells, header area, data area, parent-child index relationship, column examples, and blank cell positions.

[0174] In one possible implementation of the first aspect, the construction module 64 is specifically used to: for any text block in at least one text block, match the header identified by the visual language model with the text block based on the coordinate information of the text block; If the match is successful, the coordinate information of the text block will be used as the header anchor of the text block; Column boundaries are determined based on at least one of the following: the header anchor point corresponding to at least one text block, the spacing between adjacent header anchor points, the number of columns, and the distribution of at least one text block; and row boundaries are generated based on the column boundaries and the vertical distribution of at least one text block. Construct a candidate cell matrix corresponding to the file to be processed based on column and row boundaries; Generate structured data based on the candidate cell matrix.

[0175] In one possible implementation of the first aspect, the building module 64 is specifically used for at least one of the following: Based on the coordinate information of at least one text block, fill at least one text block into the candidate cell matrix; Identify the unit of measurement in at least one text block and append the text corresponding to the unit of measurement to the header or indicator field of the candidate cell matrix; Combine multi-level headers into complete fields and populate the candidate cell matrix; Based on the positional relationship between different languages ​​in the candidate cell matrix, fill the candidate cell matrix with different languages; According to the preset configuration, multiple languages ​​are merged into the same field and filled into the candidate cell matrix, or the target language from multiple languages ​​is filled into the candidate cell matrix.

[0176] In one possible implementation of the first aspect, the building module 64 is specifically used for at least one of the following: When the first language text is below the second language text and does not contain an independent value, the first language text is used as a translation description of the second language text and filled into the candidate cell matrix; When the first language text and the second language text are in adjacent columns, they are merged into a bilingual comparison field and filled into the candidate cell matrix.

[0177] In one possible implementation of the first aspect, the building module 64 is also used for: Based on the table information, at least one candidate table is determined from at least one table, wherein the table information of the candidate table contains a preset identifier for indicating that the candidate table is a continuation table; Based on at least one of the following criteria for candidate tables: title similarity, header similarity, unit information, number of rows and columns, indicator name, and page order relationship, candidate tables belonging to the same table are identified from at least one candidate table; at least one candidate table includes at least one of the following criteria: vertical continuation table and horizontal continuation table. For vertical continuation tables, append the data rows of the vertical continuation table to the data area of ​​the first table and complete the parent path. The first table is the previous table that belongs to the same table as the vertical continuation table and is logically or page-sequentially adjacent to it. For horizontal continuation tables, the newly added columns of the horizontal continuation table are merged into the second table, and the multi-level table headers are unified. The second table is the preceding table that belongs to the same table as the horizontal continuation table and is logically or page-sequentially adjacent to it.

[0178] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device includes: at least one processor 70 ( Figure 7(Only one is shown in the image) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described file processing method embodiments.

[0179] Electronic device 7 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This electronic device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of electronic device 7 and does not constitute a limitation on electronic device 7. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0180] The processor 70 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0181] In some embodiments, memory 71 may be an internal storage unit of electronic device 7, such as a hard disk or memory of electronic device 7. In other embodiments, memory 71 may be an external storage device of electronic device 7, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on electronic device 7. Furthermore, memory 71 may include both internal and external storage units of electronic device 7. Memory 71 is used to store operating system, electronic device, bootloader, data, and other programs, such as program code of computer programs. Memory 71 may also be used to temporarily store data that has been output or will be output.

[0182] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to a device / control device, a recording medium, a computer memory, a read-only memory (ROM), etc. Only memory), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical discs.

[0183] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0184] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0186] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0187] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0188] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, a computer-readable medium cannot be an electrical carrier signal or a telecommunication signal.

[0189] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0190] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0191] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A file processing method, characterized in that, The method includes: Obtain a file to be processed, which includes at least one table; The file to be processed is subjected to optical character recognition (OCR) to obtain a set of text items in the file to be processed. The set of text items includes at least one of the following: at least one text block, coordinate information of the text block, and OCR confidence score. The file to be processed is identified based on a preset visual language model to obtain table information corresponding to at least one table. The table information includes at least one of the following: the number of tables, the table title, the unit of measurement in the table, the table header level, and the table grid structure. The visual language model is used to perform a first-level recognition and a second-level recognition on the file to be processed. The first-level recognition identifies at least one of the following: the table title, unit of measurement, number of tables, header list, multi-level header relationship, and whether a continuation table exists. The second-level recognition identifies at least one of the following: the number of grid rows, the number of columns, merged cells, header area, data area, parent-child indicator relationship, column examples, and blank cell positions. Based on the text item set and the table information, the text blocks in the text item set are mapped to the row and column structure defined by the table information according to their coordinate information, thereby constructing the structured data corresponding to the file to be processed; The step of mapping text blocks in the text item set to the row and column structure defined by the table information according to their coordinate information, based on the text item set and the table information, to construct the structured data corresponding to the file to be processed includes: For any text block in the at least one text block, the header identified by the visual language model is matched with the text block based on the coordinate information of the text block; If the match is successful, the coordinate information of the text block is used as the header anchor point of the text block; The column boundary is determined based on at least one of the following: the header anchor point corresponding to the at least one text block, the spacing between adjacent header anchor points, the number of columns, and the distribution of the at least one text block; and the row boundary is generated based on the column boundary and the vertical distribution of the at least one text block. Based on the column boundaries and the row boundaries, construct a candidate cell matrix corresponding to the file to be processed; The structured data is generated based on the candidate cell matrix.

2. The method according to claim 1, characterized in that, The process of obtaining the file to be processed includes: Get the initial file; The initial file is preprocessed to obtain the file to be processed. The preprocessing includes at least one of the following: image format unification, resolution adjustment, grayscale conversion, noise reduction, tilt correction, orientation detection, edge enhancement, and image cropping.

3. The method according to claim 1, characterized in that, After performing optical character recognition (OCR) on the file to be processed to obtain the set of text items in the file, the method further includes: Obtain the text box height of any text block in the at least one text block; Text blocks whose height is greater than or equal to the height threshold are identified as abnormal text blocks; Divide the text box corresponding to the abnormal text block into at least two regions; Perform OCR on the at least two regions to obtain the target text block of the abnormal text block in the at least two regions; Replace the abnormal text block in the text item set with the target text block corresponding to the abnormal text block.

4. The method according to claim 1, characterized in that, The step of generating the structured data based on the candidate cell matrix includes at least one of the following: Based on the coordinate information of the at least one text block, the at least one text block is filled into the candidate cell matrix; Identify the unit of measurement in the at least one text block, and append the text corresponding to the unit of measurement to the header or index field of the candidate cell matrix; The multi-level headers are combined into complete fields and then filled into the candidate cell matrix; Based on the positional relationship between different languages ​​in the candidate cell matrix, different languages ​​are filled into the candidate cell matrix; According to the preset configuration, multiple languages ​​are merged into the same field and filled into the candidate cell matrix, or the target language among the multiple languages ​​is filled into the candidate cell matrix.

5. The method according to claim 4, characterized in that, The step of filling the candidate cell matrix with different languages ​​according to the positional relationship between different languages ​​in the candidate cell matrix includes at least one of the following: When the first language text is below the second language text and does not contain an independent value, the first language text is used as a translation description of the second language text and filled into the candidate cell matrix; When the first language text and the second language text are in adjacent columns, they are merged into a bilingual comparison field and filled into the candidate cell matrix.

6. The method according to claim 5, characterized in that, When the file to be processed is a single-page file, the structured data includes at least one of the following: table title, unit of measurement, complete table header, data matrix, cell coordinates, and identification confidence level; If the file to be processed is a multi-page file, a complete logical table is output. The complete logical table includes at least one of the following: source page number, original coordinates, continuation table merging relationship, bilingual processing result, and the structured data.

7. The method according to any one of claims 1-6, characterized in that, Also includes: Based on the table information, at least one candidate table is determined from the at least one table, wherein the table information of the candidate table includes a preset identifier for indicating that the candidate table is a continuation table; Based on at least one of the following factors among the at least one candidate tables: title similarity, header similarity, unit information, number of rows and columns, indicator name, and page order relationship, candidate tables belonging to the same table are determined; the at least one candidate table includes at least one of vertical continuation tables and horizontal continuation tables. For the vertical continuation table, the data rows of the vertical continuation table are appended to the data area of ​​the first table, and the parent path is completed. The first table is the previous table that belongs to the same table as the vertical continuation table and is logically or page-orderly adjacent to it. For the horizontal continuation table, the newly added columns of the horizontal continuation table are merged into the second table, and the multi-level table headers are unified. The second table is the previous table that belongs to the same table as the horizontal continuation table and is logically or page-orderly adjacent to it.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images