Combined small model-based electric power rich text document identification method and system

Abstract

The invention discloses an electric power rich text document identification method and system based on a combined small model. The method comprises the steps of obtaining metadata information of a power rich text document, converting the metadata information into a picture format, and performing preprocessing to obtain an image data set; constructing layout analysis model test indexes, and selecting the model with the highest index from the open source layout recognition models as a layout analysis model; constructing a formula recognition model test index, and selecting a formula recognition model with the highest index from the open source layout recognition models as the formula recognition model; constructing table recognition model test indexes, and selecting the table recognition model with the highest index from the open source table recognition models as a table recognition model; and recognizing data in the image data set by using a layout analysis model, a formula recognition model, a table recognition model and an open source character recognition model, and converting an obtained result into a Markdown format to finish the recognition of the power rich text document. According to the method, the multi-stage model evaluation and the document analysis algorithm are combined together to form a complete processing flow, and recognition of the electric power rich text document is achieved.

Description

Technical Field

[0001] This invention belongs to the field of power system document parsing technology, specifically relating to a method and system for recognizing rich text documents related to power systems based on a combined small model. Background Technology

[0002] With the development of economy and technology and the deepening of the digital transformation of the power industry, well-structured, machine-readable data has replaced the "paper + manual" model and become the core fuel for intelligent operation and decision-making in new power systems. However, rich text documents such as drawings, test reports, dispatching procedures, and equipment ledgers of in-service power grid equipment are still scattered across various levels of units and systems in scanned copies, photos, or Word / PDF formats. These documents cannot be directly used as corpus for large-scale model training and fine-tuning, nor can they be retrieved in real time by the RAG knowledge base. Therefore, rapidly and cleanly converting rich text documents in the power industry into standardized Markdown / JSON formats has become the primary bottleneck for the large-scale implementation of artificial intelligence in the power sector.

[0003] Artificial intelligence technology has become a key force driving high-quality development in the power industry. From intelligent inspection of transmission lines and substations to the generation of power supply plans in seconds, and intelligent review of distribution network feasibility studies, algorithm models are deeply embedded in the entire process of generation, transmission, transformation, distribution, and consumption. However, the "data shortage" and "knowledge gaps" remain prominent issues—if the massive amounts of historical documents cannot be converted into machine-readable knowledge in a low-cost and high-precision manner, any higher-level intelligent applications will be like water without a source.

[0004] For a long time, the industry has mainly relied on manual input, traditional OCR with rule templates, or multimodal large models for information extraction. Manual methods are costly, time-consuming, and prone to field omissions due to differences in professional expertise. While traditional OCR solutions can output text, they suffer from soaring error rates when dealing with power industry symbols (such as φ, Ω, kV, and per-unit values ​​of short-circuit capacity), formulas, complex two-dimensional tables, and irregular page layouts. Furthermore, they require extensive regular expression cleaning, resulting in poor portability. Multimodal large models offer acceptable accuracy, but they have high requirements for hardware environments such as GPUs. They are also prone to misinterpreting the structure of rich text documents, leading to inverted or missing text sequences and significant semantic discrepancies with the original document. With the rise of large models and Retrieval Augmentation (RAG) technology, scenarios such as power dispatching, maintenance, and planning have placed higher demands on the timeliness and scale of "clean corpora," which traditional solutions can no longer meet. Summary of the Invention

[0005] To address the shortcomings of existing technologies, one of the objectives of this invention is to provide a method for recognizing rich text documents related to electricity based on a combined small model. By combining multi-stage model evaluation and document parsing algorithms, the accuracy of recognition is improved, enabling the parsing and recognition of rich text documents related to electricity.

[0006] The second objective of this invention is to provide a system for implementing the power rich text document recognition algorithm based on the combined small model.

[0007] This invention provides a method for recognizing rich text documents related to electricity based on a combined small model, comprising the following steps:

[0008] S1. Obtain the metadata information of the rich text document on electricity, convert it into image format, and perform preprocessing to obtain an image dataset;

[0009] S2. Construct test metrics for the layout analysis model, and select the model with the highest metrics from the open-source layout identification models as the layout analysis model;

[0010] S3. Construct test metrics for the formula recognition model, and select the model with the highest metrics from the open-source formula recognition models;

[0011] S4. Construct test metrics for the table recognition model, and select the model with the highest metrics from open-source table recognition models as the table recognition model;

[0012] S5. Using layout analysis models, formula recognition models, table recognition models, and open-source text recognition models, the data in the image dataset is identified, and the results are converted into Markdown format to complete the recognition of power rich text documents.

[0013] In step S1, the rich text document for electricity is in PDF format or a format that can be converted to PDF after processing;

[0014] The preprocessing includes removing watermarks and resizing images;

[0015] The main types of watermarks for rich text documents in the power industry include: text watermarks, background watermarks, and image watermarks. Among them, text watermarks are usually used to identify the owner or copyright information of the document; image watermarks are generally company logos or other images, usually used for brand protection; background watermarks are watermarks added to the background of the page, usually diluted text or images.

[0016] The watermark removal process uses Python's Pillow and PyMuPDF libraries as the basic programming libraries to change the watermark color to white.

[0017] To remove text watermarks, simply set all pixels whose RGB sum exceeds a threshold to white.

[0018] The template matching algorithm is used for removing image watermarks. Template matching means finding the part in the current image A that is most similar to the part in image B, sliding the template image B on image A, and traversing all pixels to complete the matching.

[0019] Background watermark removal is considered as a combination of text watermark and image watermark, and is removed by methods targeting text watermark and image watermark respectively.

[0020] The image resizing adjustment specifically involves uniformly adjusting the size of the input image to a preset fixed pixel size, specifically a 1024*1024 pixel thumbnail.

[0021] In step S2, the layout analysis model marks the page elements of the power rich text document; the page elements include image elements, formula elements, table elements, page number elements, annotation elements, and text elements;

[0022] When the layout analysis model begins document analysis, it crops the original high-resolution image into local regions and then marks the layout elements.

[0023] The construction of the layout analysis test dataset for the layout analysis model specifically includes the following steps:

[0024] Extract the constituent elements from rich text documents such as drawings, test reports, dispatching procedures, and equipment ledgers of in-service power grid equipment. Extract nine types of component elements, including titles, plain text, annotations (including headers, footers, page numbers, footnotes, and side notes), images, figure captions, tables, table notes, independent equations, and formula descriptions.

[0025] The above 9 types of component elements are used as an element pool, and a portion of them are randomly selected from different element pools each time as a candidate set of elements.

[0026] An element is randomly sampled from the candidate set and placed at a certain position on a blank page, with a bias towards larger elements.

[0027] A grid is constructed based on the layout. Invalid grids that overlap with the inserted elements are filtered out, and the remaining grids are used to match candidate elements in subsequent steps.

[0028] The best-fit pair search iterates through all elements that meet the size requirements for each grid and searches for the grid candidate element with the maximum fill rate.

[0029] Remove the element from the candidate set and update the page layout;

[0030] Repeat the above steps until no valid candidate element meets the mesh size requirements;

[0031] Apply random center scaling to all filled elements;

[0032] Export the generated diversity document and the corresponding JSON string as a layout analysis test dataset.

[0033] Layout coverage is used as the test metric for the layout analysis model;

[0034] The layout coverage rate is calculated using the following steps:

[0035] The layout of the power-related rich text document segmented by the layout analysis model is P, which is divided into i regions, each region being... It can be expressed using the following formula: ;

[0036] The actual layout of the rich text document for electricity is G, which has j regions, each region being... It can be expressed using the following formula: ;

[0037] The layout coverage (PC) of this layout analysis model segmentation is expressed by the following formula: ;in The score for layout P; The score for layout G;

[0038] The score for layout G is expressed using the following formula: Where p represents all The number of pixels covered, where M represents the non-background area of ​​the page;

[0039] The score for layout P is expressed using the following formula: ;

[0040] We collected open-source layout recognition models, used the obtained layout analysis test dataset to conduct layout coverage tests, and selected the open-source layout recognition model with the highest layout coverage as the layout analysis model.

[0041] In step S3, the formulas encountered in the rich text parsing are divided into four categories: simple printed expressions, complex printed expressions, screen capture expressions, and handwritten expressions.

[0042] Simple printable expressions are formula images rendered from LaTeX expressions shorter than the preset length; complex printable expressions are formula images rendered from LaTeX expressions longer than the preset length; screen capture expressions are screenshots from rich text documents in the power industry; handwritten expressions are from handwritten test reports, equipment drawings, and substation inspection reports.

[0043] For both simple and complex printed expressions, the Pix2tex public dataset was used. Then, XeLaTeX was used to compile the expressions into several fonts. Uncompilable expressions were discarded. Subsequently, ImageMagic's conversion function was used to convert these images into expressions with multiple DPIs. For screen-captured expressions and handwritten expressions, the cropped formula images were automatically labeled using three formula recognition mini-models. The labeling principle was adopted. If the three formula recognition mini-models predicted the same result, the label was directly generated. If the three formula recognition mini-models failed to output the same result, the sample was verified by two people for the formula and labeling. The output image and label were then used to form the formula recognition test dataset.

[0044] The multi-segment evaluation rate is used as the test metric for the formula recognition model, expressed by the following formula: Where n represents the length of the LaTeX segment that divides the model's formula and the actual formula; BP is the length penalty factor. The weight of the score for each segmentation method; This is the score for this segmentation method when the segment length is n; N is the total number of segments.

[0045] The length penalty factor BP is expressed using the following formula: ;in, Identify the formula length for the model; This is the actual formula length;

[0046] When the segment length is n, this segmentation method scores a certain score. Express it using the following formula: ;in, When the segment length is n, the maximum number of times a certain segment of the model's formula appears in the actual formula; C represents the number of times a certain segment appears in the model's recognition formula when the segment length is n; C is the actual formula.

[0047] Express it using the following formula: ;in, When the segment length is n, the number of times a certain segment of the model recognizes the formula is found in the actual formula;

[0048] We collected open-source formula recognition models, used the obtained formula recognition test dataset to conduct multi-segment evaluation rate tests, and selected the open-source formula recognition model with the highest multi-segment evaluation rate as the formula recognition model.

[0049] In step S4, a table recognition test dataset is constructed by mixing publicly available table datasets with tables from manually annotated rich text documents on electricity. The publicly available table dataset uses a large-scale dataset called PubTabNet, from which tables are randomly extracted. The tables in the table recognition test dataset include four types of tables: ordinary tables, horizontal tables, vertical tables, and nested tables.

[0050] The table is expanded into a tree structure, and the tree edit distance is used as the test metric for the table recognition model. The core of the tree edit distance is to find the minimum number of editing operations required to transform two trees. The editing operations include insertion, deletion, and replacement. Insertion is adding a node to the tree; deletion is deleting a node from the tree; and replacement is replacing one node with another node. The edit distance between the two trees is calculated by the minimum number of editing operations.

[0051] For tables recognized by the table recognition model Compared with the actual table Edit distance between the two Express it using the following formula: ;

[0052] Tree edit distance Normalize the edit distance to The range is expressed using the following formula: ;in, The total number of nodes in the table identified by the table recognition model; This represents the total number of nodes in the actual table.

[0053] We collected open-source table recognition models, used the obtained table recognition test dataset to perform tree edit distance tests, and selected the open-source table recognition model with the tree edit distance closest to 1 as the table recognition model.

[0054] Step S5 specifically includes the following steps:

[0055] For the image dataset obtained in step S1, use the layout analysis model obtained in step S2 to perform image region segmentation and element labeling;

[0056] The obtained formula recognition model and table recognition model are used respectively to identify elements marked as formulas and tables in the image, and the results are exported as Markdown format files using the python-markdownm library;

[0057] An open-source text recognition model was used to identify the original image regions marked with text, and the results were exported as Markdown format files using the python-markdownm library.

[0058] By combining the results of formula recognition model, table recognition model and open source text recognition model, the recognition of power rich text documents was completed.

[0059] The present invention also provides a system for implementing the power rich text document recognition algorithm based on the combined small model, including a power rich text document processing module, a layout analysis model testing module, a formula recognition model testing module, a table recognition model testing module, and a power rich text document recognition module;

[0060] The power rich text document processing module acquires the metadata information of the power rich text document, converts it into image format, performs preprocessing to obtain an image dataset, and uploads the data to the power rich text document recognition module.

[0061] The layout analysis model testing module constructs layout analysis model testing indicators, selects the highest indicator from the open-source layout recognition model as the layout analysis model, and uploads the data to the power rich text document recognition module.

[0062] The formula recognition model testing module constructs test indicators for the formula recognition model, selects the model with the highest indicator from the open-source recognition models, and uploads the data to the power rich text document recognition module.

[0063] The table recognition model testing module constructs test indicators for the table recognition model, selects the highest-performing table recognition model from open-source table recognition models, and uploads the data to the power rich text document recognition module.

[0064] The power industry rich text document recognition module uses layout analysis models, formula recognition models, table recognition models, and open-source text recognition models to recognize the data in the image dataset based on the received data. The results are then converted into Markdown format to complete the power industry rich text document recognition.

[0065] This invention discloses an algorithm and system for recognizing rich text documents related to electricity based on a combined small model. By combining multi-stage model evaluation and document parsing algorithms, a complete processing flow is formed, which realizes the recognition of rich text documents related to electricity. Attached Figure Description

[0066] Figure 1 This is a schematic flowchart of the method of the present invention;

[0067] Figure 2 This is a test result diagram of the layout coverage test of the layout analysis model in an embodiment of the present invention;

[0068] Figure 3 This is a test result diagram of the multi-segment evaluation rate test of the formula recognition model in an embodiment of the present invention;

[0069] Figure 4 This is a test result diagram of the tree edit distance test of the table recognition model in an embodiment of the present invention;

[0070] Figure 5 This is a schematic diagram of the system of the present invention. Detailed Implementation

[0071] This invention provides a method for recognizing rich text documents related to electricity based on a combined small model, the flowchart of which is shown below. Figure 1 As shown, it includes the following steps:

[0072] S1. Obtain the metadata information of the rich text document on electricity, convert it into image format, and perform preprocessing to obtain an image dataset;

[0073] In step S1, the rich text document for electricity is in PDF format or a format that can be converted to PDF after processing;

[0074] The preprocessing includes removing watermarks and resizing images;

[0075] The main types of watermarks for rich text documents in the power industry include: text watermarks, background watermarks, and image watermarks. Among them, text watermarks are usually used to identify the owner or copyright information of the document; image watermarks are generally company logos or other images, usually used for brand protection; background watermarks are watermarks added to the background of the page, usually diluted text or images.

[0076] The watermark removal process uses Python's Pillow and PyMuPDF libraries as the basic programming libraries to change the watermark color to white.

[0077] To remove text watermarks, simply set all pixels whose RGB sum exceeds a threshold to white.

[0078] The template matching algorithm is used for removing image watermarks. Template matching means finding the part in the current image A that is most similar to the part in image B, sliding the template image B on image A, and traversing all pixels to complete the matching.

[0079] Background watermark removal is considered as a combination of text watermark and image watermark, and is removed by methods targeting text watermark and image watermark respectively.

[0080] The image resizing adjustment specifically involves uniformly adjusting the size of the input image to a preset fixed pixel size, specifically a 1024*1024 pixel thumbnail.

[0081] S2. Construct test metrics for the layout analysis model, and select the model with the highest metrics from the open-source layout identification models as the layout analysis model;

[0082] In step S2, the layout analysis model marks the page elements of the power rich text document; the page elements include image elements, formula elements, table elements, page number elements, annotation elements, and text elements;

[0083] When the layout analysis model begins document analysis, it crops the original high-resolution image into local regions and then marks the layout elements.

[0084] The construction of the layout analysis test dataset for the layout analysis model specifically includes the following steps:

[0085] Extract the constituent elements from rich text documents such as drawings, test reports, dispatching procedures, and equipment ledgers of in-service power grid equipment. Extract nine types of component elements, including titles, plain text, annotations (including headers, footers, page numbers, footnotes, and side notes), images, figure captions, tables, table notes, independent equations, and formula descriptions.

[0086] The above 9 types of component elements are used as an element pool, and a portion of them are randomly selected from different element pools each time as a candidate set of elements.

[0087] An element is randomly sampled from the candidate set and placed at a certain position on a blank page, with a bias towards larger elements.

[0088] A grid is constructed based on the layout. Invalid grids that overlap with the inserted elements are filtered out, and the remaining grids are used to match candidate elements in subsequent steps.

[0089] The best-fit pair search iterates through all elements that meet the size requirements for each grid and searches for the grid candidate element with the maximum fill rate.

[0090] Remove the element from the candidate set and update the page layout;

[0091] Repeat the above steps until no valid candidate element meets the mesh size requirements;

[0092] Apply random center scaling to all filled elements;

[0093] Export the generated diversity document and the corresponding JSON string as a layout analysis test dataset.

[0094] Layout coverage is used as the test metric for the layout analysis model;

[0095] The layout coverage rate is calculated using the following steps:

[0096] The layout of the power-related rich text document segmented by the layout analysis model is P, which is divided into i regions, each region being... It can be expressed using the following formula: ;

[0097] The actual layout of the rich text document for electricity is G, which has j regions, each region being... It can be expressed using the following formula: ;

[0098] The layout coverage (PC) of this layout analysis model segmentation is expressed by the following formula: ;in The score for layout P; The score for layout G;

[0099] The score for layout G is expressed using the following formula: Where p represents all The number of pixels covered, where M represents the non-background area of ​​the page;

[0100] The score for layout P is expressed using the following formula: ;

[0101] We collected open-source layout recognition models, used the obtained layout analysis test dataset to conduct layout coverage tests, and selected the open-source layout recognition model with the highest layout coverage as the layout analysis model.

[0102] In the specific implementation, four open-source layout recognition models were collected: Doclayout-yolo, yolo-10, Layoutlmv3, and Dit-cascade-L. They were tested using a layout analysis test dataset, and the specific results are as follows: Figure 2 As shown. From Figure 2 As can be seen, Doclayout-yolo has the best recognition effect, so this model is adopted as the layout analysis model;

[0103] S3. Construct test metrics for the formula recognition model, and select the model with the highest metrics from the open-source formula recognition models;

[0104] In step S3, the formulas encountered in the rich text parsing are divided into four categories: simple printed expressions, complex printed expressions, screen capture expressions, and handwritten expressions.

[0105] Simple printable expressions are formula images rendered from LaTeX expressions shorter than the preset length; complex printable expressions are formula images rendered from LaTeX expressions longer than the preset length; screen capture expressions are screenshots from rich text documents in the power industry; handwritten expressions are from handwritten test reports, equipment drawings, and substation inspection reports.

[0106] For both simple and complex printed expressions, the Pix2tex public dataset was used. Then, XeLaTeX was used to compile the expressions into several fonts. Uncompilable expressions were discarded. Subsequently, ImageMagic's conversion function was used to convert these images into expressions with multiple DPIs. For screen-captured expressions and handwritten expressions, the cropped formula images were automatically labeled using three formula recognition mini-models. The labeling principle was adopted. If the three formula recognition mini-models predicted the same result, the label was directly generated. If the three formula recognition mini-models failed to output the same result, the sample was verified by two people for the formula and labeling. The output image and label were then used to form the formula recognition test dataset.

[0107] The multi-segment evaluation rate is used as the test metric for the formula recognition model, expressed by the following formula: Where n represents the length of the LaTeX segment that divides the model's formula and the actual formula; BP is the length penalty factor. The weight of the score for each segmentation method; This is the score for this segmentation method when the segment length is n; N is the total number of segments.

[0108] The length penalty factor BP is expressed using the following formula: ;in, Identify the formula length for the model; This is the actual formula length;

[0109] When the segment length is n, this segmentation method scores a certain score. Express it using the following formula: ;in, When the segment length is n, the maximum number of times a certain segment of the model's formula appears in the actual formula; C represents the number of times a certain segment appears in the model's recognition formula when the segment length is n; C is the actual formula.

[0110] Express it using the following formula: ;in, When the segment length is n, the number of times a certain segment of the model recognizes the formula is found in the actual formula;

[0111] We collected open-source formula recognition models, used the obtained formula recognition test dataset to conduct multi-segment evaluation rate tests, and selected the open-source formula recognition model with the highest multi-segment evaluation rate as the formula recognition model.

[0112] In the specific implementation, three open-source formula recognition models were collected: Pix2tex, Texify, and Unimernet. Tests were conducted using a formula recognition test dataset, and the results are as follows: Figure 3 As shown. By Figure 3 As can be seen, Unimernet has the best recognition performance, so this model was chosen as the formula recognition model.

[0113] S4. Construct test metrics for the table recognition model, and select the model with the highest metrics from open-source table recognition models as the table recognition model;

[0114] In step S4, a table recognition test dataset is constructed by mixing publicly available table datasets with tables from manually annotated rich text documents on electricity. The publicly available table dataset uses a large-scale dataset called PubTabNet, from which tables are randomly extracted. The tables in the table recognition test dataset include four types of tables: ordinary tables, horizontal tables, vertical tables, and nested tables.

[0115] The table is expanded into a tree structure, and the tree edit distance is used as the test metric for the table recognition model. The core of the tree edit distance is to find the minimum number of editing operations required to transform two trees. The editing operations include insertion, deletion, and replacement. Insertion is adding a node to the tree; deletion is deleting a node from the tree; and replacement is replacing one node with another node. The edit distance between the two trees is calculated by the minimum number of editing operations.

[0116] For tables recognized by the table recognition model Compared with the actual table Edit distance between the two Express it using the following formula: ;

[0117] Tree edit distance Normalize the edit distance to The range is expressed using the following formula: ;in, The total number of nodes in the table identified by the table recognition model; This represents the total number of nodes in the actual table.

[0118] We collected open-source table recognition models, used the obtained table recognition test dataset to perform tree edit distance tests, and selected the open-source table recognition model with the tree edit distance closest to 1 as the table recognition model.

[0119] In the specific implementation, four open-source table recognition mini-models were collected: StructEqTable, RapidTable (SLANet), wired_table_rec v2, and ppstructure_table_master. Tests were conducted using a table recognition test dataset, and the specific results are as follows: Figure 4 As shown. By Figure 4 It can be seen that wired_table_rec v2 has the best recognition effect, so this model was chosen as the table recognition model.

[0120] S5. Using layout analysis models, formula recognition models, table recognition models, and open-source text recognition models, the data in the image dataset is identified, and the results are converted into Markdown format to complete the recognition of power rich text documents.

[0121] Step S5 specifically includes the following steps:

[0122] For the image dataset obtained in step S1, use the layout analysis model obtained in step S2 to perform image region segmentation and element labeling;

[0123] The obtained formula recognition model and table recognition model are used respectively to identify elements marked as formulas and tables in the image, and the results are exported as Markdown format files using the python-markdownm library;

[0124] An open-source text recognition model was used to identify the original image regions marked with text, and the results were exported as Markdown format files using the python-markdownm library.

[0125] By combining the results of formula recognition model, table recognition model and open source text recognition model, the recognition of power rich text documents was completed.

[0126] This invention also provides a system for implementing the power rich text document recognition algorithm based on the combined small model, the schematic diagram of which is shown below. Figure 5 As shown, it includes a power rich text document processing module, a layout analysis model testing module, a formula recognition model testing module, a table recognition model testing module, and a power rich text document recognition module.

[0127] The power rich text document processing module acquires the metadata information of the power rich text document, converts it into image format, performs preprocessing to obtain an image dataset, and uploads the data to the power rich text document recognition module.

[0128] The layout analysis model testing module constructs layout analysis model testing indicators, selects the highest indicator from the open-source layout recognition model as the layout analysis model, and uploads the data to the power rich text document recognition module.

[0129] The formula recognition model testing module constructs test indicators for the formula recognition model, selects the model with the highest indicator from the open-source recognition models, and uploads the data to the power rich text document recognition module.

[0130] The table recognition model testing module constructs test indicators for the table recognition model, selects the highest-performing table recognition model from open-source table recognition models, and uploads the data to the power rich text document recognition module.

[0131] The power industry rich text document recognition module uses layout analysis models, formula recognition models, table recognition models, and open-source text recognition models to recognize the data in the image dataset based on the received data. The results are then converted into Markdown format to complete the power industry rich text document recognition.

Claims

1. A method for recognizing rich text documents related to electricity based on a combined small model, characterized in that, Includes the following steps: S1. Obtain the metadata information of the rich text document on electricity, convert it into image format, and perform preprocessing to obtain an image dataset; S2. Construct test metrics for the layout analysis model, and select the model with the highest metrics from the open-source layout identification models as the layout analysis model; S3. Construct test metrics for the formula recognition model, and select the model with the highest metrics from the open-source formula recognition models; S4. Construct test metrics for the table recognition model, and select the model with the highest metrics from open-source table recognition models as the table recognition model; S5. Using layout analysis models, formula recognition models, table recognition models, and open-source text recognition models, the data in the image dataset is identified, and the results are converted into Markdown format to complete the recognition of power rich text documents.

2. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 1, characterized in that, In step S1, the rich text document for electricity is in PDF format or a format that can be converted to PDF after processing; The preprocessing includes removing watermarks and resizing images; The main types of watermarks for rich text documents in the power industry include: text watermarks, background watermarks, and image watermarks. Among them, text watermarks are usually used to identify the owner or copyright information of the document; image watermarks are generally company logos or other images, usually used for brand protection; background watermarks are watermarks added to the background of the page, usually diluted text or images. The watermark removal process uses Python's Pillow and PyMuPDF libraries as the basic programming libraries to change the watermark color to white. To remove text watermarks, simply set all pixels whose RGB sum exceeds a threshold to white. The template matching algorithm is used for removing image watermarks. Template matching means finding the part in the current image A that is most similar to the part in image B, sliding the template image B on image A, and traversing all pixels to complete the matching. Background watermark removal is considered as a combination of text watermark and image watermark, and is removed by methods targeting text watermark and image watermark respectively. The image resizing adjustment specifically involves uniformly adjusting the size of the input image to a preset fixed pixel size, specifically a 1024*1024 pixel thumbnail.

3. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 1, characterized in that, The layout analysis model marks the page elements of the power rich text document; the page elements include image elements, formula elements, table elements, page number elements, annotation elements, and text elements; Build a layout analysis test dataset; Layout coverage is used as the test metric for the layout analysis model; We collected open-source layout recognition models, used the obtained layout analysis test dataset to conduct layout coverage tests, and selected the open-source layout recognition model with the highest layout coverage as the layout analysis model.

4. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 3, characterized in that, The construction of the layout analysis test dataset specifically includes the following steps: The system extracts the constituent elements from rich text documents such as drawings, test reports, dispatching procedures, and equipment ledgers of in-service power grid equipment. Nine types of component elements are extracted, including titles, plain text, annotations, images, figure captions, tables, table notes, independent equations, and formula descriptions. Annotations include headers, footers, page numbers, footnotes, and side notes. The above 9 types of component elements are used as an element pool, and a portion of them are randomly selected from different element pools each time as a candidate set of elements. Randomly sample an element from the candidate set and place it at a certain position on a blank page; Build a grid based on the layout, filter out invalid grids that overlap with the inserted elements, and the remaining grids participate in matching with candidate elements in subsequent steps; The best-fit pair search iterates through all elements that meet the size requirements for each grid and searches for the grid candidate element with the maximum fill rate. Remove the element from the candidate set and update the page layout; Repeat the above steps until no valid candidate element meets the mesh size requirements; Apply random center scaling to all filled elements; Export the generated diversity document and the corresponding JSON string as a layout analysis test dataset; The layout coverage rate is calculated using the following steps: The layout of the power-related rich text document segmented by the layout analysis model is P, which is divided into i regions, each region being... It can be expressed using the following formula: ; The actual layout of the rich text document for electricity is G, which has j regions, each region being... It can be expressed using the following formula: ; The layout coverage (PC) of this layout analysis model segmentation is expressed by the following formula: ;in The score for layout P; The score for layout G; The score for layout G is expressed using the following formula: Where p represents all The number of pixels covered, where M represents the non-background area of ​​the page; The score for layout P is expressed using the following formula: .

5. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 1, characterized in that, In step S3, the formulas encountered in the rich text parsing are divided into four categories: simple printed expressions, complex printed expressions, screen capture expressions, and handwritten expressions. A simple print expression is a formula image rendered from a LaTeX expression that is shorter than a preset length; Complex printable expressions are formula images rendered from LaTeX expressions of a minimum preset length; The screen capture expression comes from a screenshot image of a rich text document in the power industry. The handwritten expressions are derived from handwritten test reports, equipment drawings and specifications, and substation inspection reports. For both simple and complex print expressions, the Pix2tex public dataset was used, and then XeLaTeX was used to compile them into expressions in several fonts. Uncompilable expressions were discarded. Subsequently, ImageMagic's conversion function was used to convert these images into expressions with multiple DPIs. For screen-captured expressions, handwritten expressions, and cropped formula images, three formula recognition mini-models are used to automatically annotate the existing formula images. The annotation is performed according to the consistency principle. If the three formula recognition mini-models predict the same result at the same time, the annotation is generated directly. If the three formula recognition mini-models fail to output the same result, the sample is verified by two people for formula and annotation, and then the image and annotation are output to form a formula recognition test dataset. Multi-segment evaluation rate is used as the test index for the formula recognition model; We collected open-source formula recognition models, used the obtained formula recognition test dataset to conduct multi-segment evaluation rate tests, and selected the open-source formula recognition model with the highest multi-segment evaluation rate as the formula recognition model.

6. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 5, characterized in that, The multi-segment evaluation rate is expressed using the following formula: Where n represents the length of the LaTeX segment that divides the model's recognition formula and the actual formula; BP is the length penalty factor. The weight of the score for each segmentation method; This is the score for this segmentation method when the segment length is n; N is the total number of segments. The length penalty factor BP is expressed using the following formula: ;in, Identify the formula length for the model; This is the actual formula length; The score for this segmentation method when the segment length is n. Express it using the following formula: ;in, When the segment length is n, the maximum number of times a certain segment of the model's formula appears in the actual formula; C represents the number of times a certain segment appears in the model's recognition formula when the segment length is n; C is the actual formula. Express it using the following formula: ;in, This represents the number of times a certain segment of the model's formula appears in the actual formula when the segment length is n.

7. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 1, characterized in that, In step S4, a table recognition test dataset is constructed by mixing publicly available table datasets with tables from manually annotated rich text documents on electricity. The publicly available table dataset uses a large-scale dataset called PubTabNet, from which tables are randomly extracted. The tables in the table recognition test dataset include four types: ordinary tables, horizontal tables, vertical tables, and nested tables. The table is expanded into a tree structure, and the tree edit distance is used as the test metric for the table recognition model. The core of the tree edit distance is to find the minimum number of editing operations required to transform two trees. The editing operations include insertion, deletion, and replacement. Insertion is adding a node to the tree; deletion is deleting a node from the tree; and replacement is replacing one node with another node. The edit distance between the two trees is calculated by the minimum number of editing operations. We collected open-source table recognition models, used the obtained table recognition test dataset to perform tree edit distance tests, and selected the open-source table recognition model with the tree edit distance closest to 1 as the table recognition model.

8. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 7, characterized in that, For tables recognized by the table recognition model Compared with the actual table Edit distance between the two Express it using the following formula: ; Tree edit distance Normalize the edit distance to The range is expressed using the following formula: ;in, The total number of nodes in the table identified by the table recognition model; This represents the total number of nodes in the actual table.

9. The method for recognizing rich text documents related to electricity based on a combined small model according to claim 1, characterized in that, Step S5 specifically includes the following steps: For the image dataset obtained in step S1, use the layout analysis model obtained in step S2 to perform image region segmentation and element labeling; The obtained formula recognition model and table recognition model are used respectively to identify elements marked as formulas and tables in the image, and the results are exported as Markdown format files using the python-markdownm library; An open-source text recognition model was used to identify the original image regions marked with text, and the results were exported as Markdown format files using the python-markdownm library. By combining the results of formula recognition model, table recognition model and open source text recognition model, the recognition of power rich text documents was completed.

10. A system for implementing the power rich text document recognition algorithm based on a combined small model as described in any one of claims 1 to 9, characterized in that, It includes a power rich text document processing module, a layout analysis model testing module, a formula recognition model testing module, a table recognition model testing module, and a power rich text document recognition module. The power rich text document processing module acquires the metadata information of the power rich text document, converts it into image format, performs preprocessing to obtain an image dataset, and uploads the data to the power rich text document recognition module. The layout analysis model testing module constructs layout analysis model testing indicators, selects the highest indicator from the open-source layout recognition model as the layout analysis model, and uploads the data to the power rich text document recognition module. The formula recognition model testing module constructs test indicators for the formula recognition model, selects the model with the highest indicator from the open-source recognition models, and uploads the data to the power rich text document recognition module. The table recognition model testing module constructs test indicators for the table recognition model, selects the highest-performing table recognition model from open-source table recognition models, and uploads the data to the power rich text document recognition module. The power industry rich text document recognition module uses layout analysis models, formula recognition models, table recognition models, and open-source text recognition models to recognize the data in the image dataset based on the received data. The results are then converted into Markdown format to complete the power industry rich text document recognition.

Images