Relay protection scheduling work file automatic identification method and system

Abstract

This invention belongs to the field of data processing technology. It proposes an automatic identification method and system for relay protection scheduling work documents. The method involves preprocessing the acquired relay protection scheduling work documents to obtain a set of text fragments and potential table areas. Based on the text fragment set, a multi-feature fusion document logical structure parsing model is used to generate a title hierarchy tree and a title-text mapping relationship. Based on the potential table areas, a structured analysis network is used to reconstruct the tables, obtaining structured table data. According to the title hierarchy tree, the structured table data is associated with the corresponding chapter nodes, and the title hierarchy tree, title-text mapping relationship, and structured table data are integrated to form standardized structured document description information. This invention achieves automatic identification of work documents, structured extraction of key data, and automatic generation of standardized templates.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method and system for automatic identification of relay protection scheduling work files. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the process of power system automation and intelligent operation and maintenance, relay protection dispatching documents serve as a crucial carrier of power grid operation instructions, technical specifications, and data acquisition requirements. Their processing efficiency and accuracy directly impact the safety and stability of the power grid. Currently, these documents are generally in the form of unstructured electronic documents such as PDF and Word, integrating hierarchical natural language text with complex professional tables containing multi-level headers and merged cells. While general-purpose optical character recognition (OCR) and natural language processing (NLP) methods can achieve basic text extraction with the development of artificial intelligence, their parsing capabilities remain insufficient when faced with the specialized terminology, rigorous document logic, and complex layouts unique to the power sector, making it difficult to meet the power industry's demand for high-precision structured information.

[0004] The core problem with existing technical solutions for processing unstructured relay protection dispatching documents lies in their inability to achieve coordinated and accurate reconstruction of the overall logical structure of the document and its complex local tables. Specifically, common methods typically separate text parsing from table recognition, resulting in isolated header hierarchy trees, body text, and structured table data. These isolated elements cannot be effectively correlated and integrated based on the original document's chapter logic. This information silo phenomenon leads to a lack of inherent consistency in the final structured output. It fails to accurately reflect the complete semantic relationships between "chapter-body text-table" in the original document and cannot directly generate standardized data templates for seamless integration with downstream power safety management systems. This severely restricts the efficiency and reliability of automated processing and execution of dispatching instructions. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an automatic identification method and system for relay protection dispatching work documents. Relying on layout analysis and semantic parsing technologies, it achieves automatic identification of work documents, structured extraction of key data, and automatic generation of standardized templates. This accurately solves the problems of inconsistent manual parsing, inconsistent template standards, and low template compilation efficiency in existing technologies, and fully adapts to the actual work needs of relay protection dispatching and maintenance management.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for automatic identification of relay protection scheduling work files.

[0007] A method for automatically identifying relay protection dispatching work files includes the following steps: The obtained relay protection scheduling work files are preprocessed to obtain a set of text fragments and a suspected table area; Based on a set of text fragments, a document logical structure parsing model with multi-feature fusion is used to generate a title hierarchy tree and a title-body mapping relationship. Based on suspected table regions, a structured analysis network is used to reconstruct the table and obtain structured table data. Based on the heading hierarchy tree, the structured table data is associated with the corresponding chapter nodes, and the heading hierarchy tree, heading-to-text mapping relationship and structured table data are integrated to form standardized structured document description information.

[0008] In one implementation of the first aspect of the present invention, document preprocessing is performed on the acquired relay protection scheduling work file, including: An OCR engine loaded with a corpus of high-frequency terms for power dispatching was used to perform text recognition on relay protection dispatching work documents; Based on a pre-built dictionary of power dispatch terminology, the edit distance algorithm is used to correct typos in the identified text. Specifically, when the edit distance between the identified text and a word in the dictionary is less than or equal to 2, the identified text is corrected to the word in the dictionary. Based on the consistency between the distribution density of delimiters and the number of fields in adjacent rows, suspected table regions are detected and located, and non-table regions are divided into a set of text fragments by blank lines.

[0009] In one implementation of the first aspect of the present invention, the process of constructing a document logical structure parsing model based on multi-feature fusion includes: Based on a predetermined-size training set of power dispatching work guidance documents, a title numbering pattern library, a domain keyword library, and a layout rule library were constructed. The set of text fragments is input into the language analysis model adjusted by power dispatch text samples, and the semantic information vector is output. The semantic information vector is concatenated with the numbering feature vector from the title numbering pattern library, the TF-IDF vector from the domain keyword library, and the format normalization vector from the layout rule library to form a fused feature vector.

[0010] As a further limitation of the first aspect of the present invention, generating a title hierarchy tree and a title-text mapping relationship includes: The fused feature vectors are input into the language analysis model encoder, the association weights between each text unit are determined through a multi-dimensional association calculation layer, and the probability distribution of the title level is output through a fully connected classification layer to generate a title level tree. The global feature vector of each title is extracted, and the text fragments within the candidate body text range are aggregated to obtain the body text aggregate features. By calculating the cosine similarity between the global feature vector and the body text aggregate features, the range of body text content under each title is determined to establish the title-body text mapping relationship.

[0011] As a further limitation of the first aspect of the present invention, after generating the title hierarchy tree and the title-body text mapping relationship, the method further includes: Mark candidate misspellings that do not match the dictionary of power dispatch terminology; Using a language analysis model calibrated with text from the power industry, we extract the contextual semantic features of candidate misspellings and calculate their semantic coherence scores with candidate words in the dictionary. When the semantic coherence score is greater than or equal to a preset threshold, the corresponding candidate word is selected to replace the misspelled candidate.

[0012] In one implementation of the first aspect of the present invention, table reconstruction is performed using a structured analysis network, including: For suspected table areas, a structured analysis model is constructed that includes header information units, data cell units, and row and column index units, and relationship identifiers are established between each unit; The features of each unit are initialized as a concatenated vector of text embedding vector, position feature vector and power field feature vector; The representation of each unit is learned through a two-layer associative computational network, and the unit features are output.

[0013] As a further limitation of the first aspect of the present invention, the obtained structured tabular data includes: Based on the unit features, the header information units are clustered, and a hierarchical mapping is established for the header units whose feature similarity after clustering is greater than or equal to the first threshold. Calculate the feature similarity between data cell units and header information units, and allocate data based on the data logical order of the power dispatch document table to form initial data rows; The correlation between adjacent cells is calculated based on the characteristics of row and column index units. When the correlation is greater than or equal to the second threshold, the cells are determined to be merged and their original range is recorded.

[0014] In one implementation of the first aspect of the present invention, the title hierarchy tree includes a unique identifier for the title, text, level and position information, the title-body mapping relationship uses the title hierarchy path as the key and the standardized body text as the value, and the structured table data includes a table header, data rows, merged cell information and the document chapter to which it belongs.

[0015] Secondly, the present invention provides an automatic identification system for relay protection scheduling work documents.

[0016] An automatic identification system for relay protection dispatching work documents, comprising: The document preprocessing unit is configured to preprocess the acquired relay protection scheduling work file to obtain a set of text fragments and a suspected table area; The logical parsing unit is configured to generate a title hierarchy tree and a title-body mapping relationship based on a document logical structure parsing model that uses multi-feature fusion, based on a set of text fragments. The table reconstruction unit is configured to: reconstruct the table based on the suspected table area using a structured analysis network to obtain structured table data; The information integration unit is configured to: associate structured tabular data with the corresponding chapter nodes according to the heading hierarchy tree, and integrate the heading hierarchy tree, heading-text mapping relationship and structured tabular data to form standardized structured document description information.

[0017] Thirdly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the automatic identification method for relay protection scheduling work files according to the first aspect of the present invention.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention innovatively proposes an automatic identification method for relay protection dispatching work documents, solving the technical problem of inconsistent semantic parsing of unstructured documents. By constructing a power industry-specific feature library (numbering pattern library, keyword library, and layout rule library), and combining a multi-feature fusion mechanism with an attention-based hierarchical recognition model, it achieves accurate parsing of the relationship between the title level and the body text. The title recognition accuracy is ≥95%, and the body text association accuracy is ≥92%, effectively overcoming the subjective differences caused by manual reading.

[0019] This invention solves the technical problem of inconsistent template standards. It automatically reconstructs complex table structures through structured analysis networks and combines this with a data standardization rule base for the power industry to automatically normalize field value ranges. This ensures that the data collection templates generated for similar work documents are standardized and can be directly integrated into the power safety management system, eliminating data cleaning and verification costs. It also solves the technical problem of poor timeliness in task distribution: the entire process requires no manual intervention, achieving fully automated processing from document preprocessing, text structuring, and table structuring. This significantly improves the efficiency of power dispatch document processing and can fully meet the rapid response requirements of emergency dispatch tasks.

[0020] To address the complex table structures unique to relay protection dispatch documents, such as multi-level headers and merged cells, this invention utilizes graph convolutional networks to learn the relationships between units, achieving accurate table structure reconstruction and adapting to complex layouts. Test results show that the table structure reconstruction accuracy of this method is ≥90%, significantly outperforming general document analysis methods.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 A flowchart illustrating an automatic identification method for relay protection scheduling work files, provided as an exemplary embodiment of the present invention; Figure 2 A flowchart of document preprocessing and region segmentation provided as an exemplary embodiment of the present invention; Figure 3 A flowchart for constructing a title-specific feature library is provided as an exemplary embodiment of the present invention; Figure 4 A flowchart of table structure reconstruction based on graph convolutional networks is provided as an exemplary embodiment of the present invention; Figure 5 A schematic diagram of an automatic identification system for relay protection scheduling work files provided as an exemplary embodiment of the present invention; Figure 6 A schematic diagram of a computer device provided for an exemplary embodiment of the present invention. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] In the field of relay protection dispatching and operation management technology, dispatching management departments need to issue work task instructions to lower-level power supply units and power generation enterprises on a regular or irregular basis and collect feedback data. These work task instructions are usually presented in the form of unstructured electronic documents such as PDF and Word. These documents not only contain work requirements described in natural language, but also nested tables defining data fields, as well as professional technical information such as time constraints, format specifications, and reporting standards. Although existing natural language processing technologies can extract text from general documents, processing professional documents in the relay protection dispatching field still faces unique technical challenges: First, the power industry has complex professional terminology, such as terms like "setting value adjustment," "plate activation / deactivation," and "secondary circuit," which have a high error rate in general OCR technology; second, dispatching work documents have a strict hierarchical heading system and complex table structures (multi-level headers, merged cells), making it difficult for general document analysis methods to accurately reconstruct their logical structure; third, existing technologies lack standardized processing mechanisms tailored to the characteristics of power business data, and the processed results cannot be directly integrated into the power safety management system.

[0027] In view of the problems existing in the current solution, this invention proposes an automatic identification method for relay protection scheduling work files, such as... Figure 1 As shown, after the file acquisition is completed, the document preprocessing, document logical structure parsing, table reconstruction, structured information integration and output steps are used to achieve comprehensive structured recognition of work guidance documents in the field of power dispatching.

[0028] More specifically, it includes the following processes: S101: Document preprocessing and region segmentation.

[0029] The relay protection dispatching and maintenance management work guidance document contains hierarchical headings, semantically related text, and tables with complex structures such as multiple headers and merged cells. Figure 2 This is a flowchart of document preprocessing and region segmentation.

[0030] The core operations of this step are divided into three parts, as follows: S101-1: Basic text recognition is performed using an optimized OCR text recognition tool specifically designed for the power industry. This tool's dedicated corpus covers high-frequency terms and common vocabulary in the power dispatching field, including relay protection, dispatch management, hazard investigation, setting values, secondary circuits, switchboard activation / deactivation, power grid safety and stability, grid-connected power plants, extended service life, on-site operations, and voltage levels. This effectively improves the accuracy of professional text recognition.

[0031] S101-2: Based on a pre-built dictionary of power dispatch terminology, obvious typos in the recognition results are corrected using an edit distance algorithm. The specific formula for the edit distance algorithm is as follows: For two strings S (length m) and T (length n), the edit distance d(S,T) is defined as the minimum number of insertion, deletion, or replacement operations required to transform S into T. Its recursive calculation formula is as follows: When m=0, d(S,T)=n; When n=0, d(S,T)=m; When S[m-1]=T[n-1], d(S,T)=d(S[0..m-2],T[0..n-2]); When S[m-1]≠T[n-1]: d(S,T)=1+min{d(S[0..m-2],T[0..n]),d(S[0..m],T[0..n-2]),d(S[0..m-2],T[0..n-2])}; This invention sets an edit distance threshold of 2. When the edit distance between the identified text and a word in the terminology dictionary is ≤2, the text is determined to be an obvious misspelling and is corrected. For example, the incorrectly identified "secondary loop" is corrected to the correct "quadratic loop".

[0032] S101-3: Automatically detect and locate table regions based on the distribution density of delimiters in the document and the consistency of the number of fields in adjacent lines: Set the threshold for the frequency of single-line occurrence of delimiters to 3 times and the threshold for the number of consecutive consistent lines to 3 lines. When a piece of text in the document meets the condition that "the number of times the single-line delimiter appears is ≥3 times and the number of fields in more than 3 consecutive lines is consistent", it is marked as a suspected table region; divide the non-table region into a set of continuous text segments by blank lines.

[0033] S102: Document logical structure parsing based on multi-feature fusion.

[0034] The core of this step is to construct a hierarchical title recognition model suitable for power dispatch documents. The feature library for this model is automatically extracted by analyzing a training set of power dispatch work guidance documents of a predetermined size. The number of documents in the training set ranges from 100 to 1000 (the specific size can be adjusted according to the actual application scenario). The specific construction process of the feature library is as follows: Figure 3 As shown.

[0035] S102-1: Feature library construction.

[0036] The construction of the feature library involves three core steps, which are described in detail below: (1) Construction of title numbering pattern library. The title numbers in the training set are extracted by regular expression, and the frequency of occurrence and hierarchical differentiation of each numbering format are counted; the frequently occurring fixed numbering formats are stored by hierarchical classification and weighted.

[0037] The formula for calculating the weight is: ; in, To balance the frequency of occurrence with the hierarchical differentiation coefficient, an optimized selection was made. The reasonable range for the value is 0.5-0.8. When calculating the weight, a fixed constant is selected within the above range as the weight. The value to be substituted; The frequency of occurrence of this numbering format. The highest frequency of occurrence among all numbering formats. This is the hierarchical differentiation of the numbering format (i.e., the probability that it is only used for a single level). This represents the highest level of differentiation among all numbering formats. In the weighting rules, first-level numbers have a higher weight than second-level numbers, and attachment numbers have a weight between that of first-level and second-level numbers.

[0038] (2) Construction of the domain keyword database. The TF-IDF algorithm was used to extract core words from the titles of the training set. In this algorithm, the TF value of a word is the ratio of the number of times that word appears in a single document title to the total number of words in that document title, and the IDF value is... (N is the total number of training set documents, and n is the number of documents containing the word). Finally, the core words in the titles of power dispatch documents are selected based on the sorting results of the TF-IDF values.

[0039] (3) Construction of the layout rule base. Based on the common layout characteristics of the training set documents, the first line alignment of the title, the spacing requirements between the title and the preceding and following text, the standard use of punctuation marks, and the text length threshold are statistically analyzed to extract general title format rules; S102-2: Hierarchical heading recognition and association with body text.

[0040] After completing the feature library construction, follow these steps to parse the document's logical structure: (1) Input the set of text fragments into the Distil BERT lightweight language analysis model after the power dispatch text sample is adjusted. During the adjustment process, the pre-trained parameters are retained, and only the title classification module and the semantic association calculation module are trained. The number of iterations of model adjustment ranges from 50 to 200 times, the learning rate is set to 1e-5-5e-5, and the final output dimension is fixed at 768-dimensional semantic information vector.

[0041] (2) The above 768-dimensional semantic information vector is concatenated with the number feature vector (10-dimensional) of the title numbering pattern library, the TF-IDF vector (20-dimensional) of the domain keyword library, and the format normalization vector (5-dimensional) of the layout rule library to form a fusion feature vector with dimensions [seq_len,768+35].

[0042] (3) Combining the above-mentioned fused feature vectors, the association between the numbering and keywords is sorted out through multi-dimensional association analysis methods to automatically identify the hierarchical structure of the title. The specific algorithm is as follows: ① The language analysis model encoder integrates feature vectors as input, and determines the association weights between text units through a multi-dimensional association calculation layer. The calculation formula is as follows: ,in To fuse the linear transformation matrix of the eigenvectors, This is a preset dimension parameter with a value of 64. ② The encoder's output is processed by a fully connected classification layer to output the probability distribution of the title level, as shown in the formula: H n The core identifier output for the text segment is W1, which is a weight matrix of dimension [798, 512], W2 is a weight matrix of dimension [512, 4], b1 is a bias term of dimension 512, and b2 is a bias term of dimension 4. ③ Pre-set category labels, specifically divided into {0: non-heading, 1: first-level heading, 2: second-level heading, 3: attachment heading}. The category labels are set based on the statistical analysis results of the heading levels of the training set documents; ④ Sort candidate titles according to their position in the document, establish hierarchical continuity rules based on the title hierarchy logic of the power dispatch document, eliminate misjudged items that do not conform to the rules, and finally generate a title hierarchy tree.

[0043] (4) The scope of the main text content under each title is determined by semantic similarity calculation. The specific algorithm is as follows: ① Extract the core identifiers of each title and output them. As a global feature vector of the title; ② For text segments within the candidate title text range, extract the core identifiers of each segment and output them, then use mean pooling to obtain the aggregated text features. ; ③ Calculate cosine similarity: The similarity threshold is determined based on the semantic association characteristics of the training set documents (with a value range of 0.7-0.8). When Sim≥threshold, it can be confirmed that the text fragment belongs to the corresponding title. ④ For fuzzy text fragments with similarity within the threshold range, supplementary judgment is made using a domain keyword association dictionary to determine their affiliation.

[0044] S102-3: Semantic-level OCR error correction.

[0045] After linking the title and body text, semantic-level OCR error correction is also required. The specific steps are as follows: (1) Mark and identify candidate misspellings in the text that do not match the dictionary of power dispatch terminology; (2) Using a language analysis model calibrated with power field text, extract the contextual semantic features of the candidate misspellings, calculate the semantic similarity between the context of the candidate misspellings and the candidate words in the terminology dictionary, and select the Top N (N values ​​3-5) semantically similar words; (3) Calculate the semantic coherence score between candidate words and their context: ; in, Language model embedding vectors (768-dimensional) for candidate words. The embedding vector (768-dimensional) of the candidate context for misspelling; (4) Set a score threshold, with a value range of 0.75-0.85, and select candidate words with a score ≥ the threshold to replace the misspelled candidate words to achieve accurate correction; Through the above series of operations, the final output is the document's heading hierarchy tree, as well as the mapping relationship between headings and body text.

[0046] S103: Table reconstruction based on structured analysis networks.

[0047] For the identified suspected table areas, a structured analysis model was constructed to reconstruct the tables. The flowchart for model construction is shown below. Figure 4 The specific algorithm operation is as follows: S103-1: Construction of structured analysis model.

[0048] (1) Header information unit: Based on the power dispatch document table training set, the candidate header fields that frequently appear in the table are automatically extracted through K-means text semantic similarity clustering and domain feature matching. Each independent field corresponds to a header information unit. (2) Data cell unit: The text content of each cell is a data cell unit. For cell units corresponding to multiple header fields, the hierarchical attributes are automatically marked based on the hierarchical characteristics of the training set table. (3) Row and column index cells: Each row and each column has an index cell, which is used to associate the cell cells in the same row / column; (4) Relationship construction: Establish a membership identifier for the header information unit and the corresponding column index unit, the column index unit and the data cell unit in the same column, and the row index unit and the data cell unit in the same row, respectively. The weight of the identifier is set to 1.0; establish a merge association identifier for multiple data cell units corresponding to the merged cell. The weight of the identifier is set to 0.8.

[0049] S103-2: Unit feature initialization.

[0050] The initial unit features are a concatenated vector of text embedding vector, position feature vector and power domain feature vector. The text embedding vector is a 768-dimensional vector output by the language analysis model, the position feature vector is 10-dimensional, the power domain feature vector is 20-dimensional, and the initial feature vector of the unit has a dimension of 798.

[0051] S103-3: Feature learning in relational computation networks.

[0052] The feature learning algorithm for the association computation network, which learns unit representations through a two-layer association computation network, is as follows: (1) Construct an adjacency matrix A and add self-loops, through Normalization is performed (D is the degree matrix, calculated based on the number of connections between units in the structured analysis model, i.e., D[i,i] is the total number of connections between unit i). (2) First-level association calculation: ,in, The initial feature vector of the unit (dimension 798). The weight matrix has a preset dimension [798, 256]. For the bias term of dimension 256; (3) Second-level association calculation: ,in The weight matrix has a preset dimension of [256, 256]. For the bias term of dimension 256; (4) Output the unit features with dimensions [n_units,256] to accurately represent the relationship between the table header and data and the data in the same row / column.

[0053] S103-4: Accurate reconstruction of table structure.

[0054] Based on the learned cell features, the hierarchical relationship of the table header, the correspondence between data cells and the table header, and the original range of merged cells are automatically inferred. The specific operations are as follows: (1) Header hierarchy mapping: mapping the features of header information units Perform K-means clustering, calculate the feature similarity between each header unit after clustering, and establish a hierarchical mapping for header units with similarity ≥ 0.8; (2) Data row matching: Calculate the feature similarity between data cell units and header information units, allocate data based on the data logical order of the power dispatch document table, and form initial data rows; (3) Merged cell processing: Calculate the correlation degree of adjacent cell units based on the characteristics of row / column index units. When the correlation degree is ≥0.75, it is determined to be a merged cell, and the starting row / column and ending row / column information of the merge are recorded; (4) Field standardization: Based on a predefined data standardization rule base in the power sector, the value range of specific types of fields in the table is normalized. This rule base is constructed by analyzing the field expression patterns in the power dispatch document training set, and includes rules such as voltage level completion, rectification status unification, and plant type standardization.

[0055] By following the steps above, machine-readable structured tabular data can be reconstructed.

[0056] S104: Structured information integration and output.

[0057] Based on the parsed document logical structure, the obtained structured table data is associated with the corresponding chapter nodes in the title hierarchy tree through text position matching.

[0058] Integrate the heading hierarchy tree, heading-to-body mapping relationships, and all structured table data to generate a unified, standardized JSON-formatted document structure description file. This file specifically includes: (1) Title hierarchy tree: includes unique title identifier, text, level, start / end position, adapted to the general hierarchical structure of power dispatching work guidance documents; (2) Title-body mapping relationship: The title-body mapping relationship is clearly presented by using the title level path as the key and the standardized body text after cross-segment splicing and redundant character removal as the value; (3) Structured table data: includes table header, data rows, merged cell information and the document chapter to which the table belongs, adapts to the table type specific to the power dispatching field, and can be directly connected to the power safety management system; This standardized and structured result can be widely applied to scenarios such as power dispatch document archiving, business data statistics, and integration with power safety management systems.

[0059] To better understand the solution of the present invention, the following example is provided: The test document selected in this embodiment is in the PDF picture format generated by digitizing a fax, with a total of 8 pages. The document contains a typical power dispatching notice structure: first, multi-level headings, including 4 first-level headings such as "I. General Requirements", "II. Verification Contents", etc., 9 second-level headings, and 2 appendix headings "Appendix 1" and "Appendix 2"; second, semantically related text, which involves the execution standards, time nodes, and responsibility requirements of the攻坚行动 (it seems there is a specific term missing here, which should be filled in according to the actual context); third, three complex structure tables, namely Table 2-1 to Table 2-3, with the characteristics of multi-level table headers and merged cells.

[0060] After preliminary recognition by a general OCR tool, recognition errors such as "March 3rd" and "dispatching unit" appeared in the document.

[0061] A dedicated OCR tool loaded with a power dispatching high-frequency term corpus is used to extract text from the PDF picture. This corpus contains more than 2,000 power professional terms such as "setting value setting", "switching of pressure plates", and "secondary circuit", which can effectively reduce the initial recognition error of professional vocabulary.

[0062] Call the pre-built power dispatching professional term dictionary, use the edit distance algorithm to correct recognition errors, and set the edit distance threshold to 2. Calculate the edit distance between the recognized text "March 3rd" and the words in the dictionary. Its edit distance from "April 3rd" is 1, meeting the threshold requirement, and it is automatically corrected to the correct expression. Division of table area and text area Start the automatic detection process for the table area, and set the judgment rule as "the number of single-line text separators ≥ 3 times, and the number of fields in 3 consecutive lines or more is the same". After scanning the document, the text areas of Table 2-1 to Table 2-3 all meet the judgment rule and are marked as suspected table areas.

[0063] The non-table area is divided into 12 independent text segment sets by blank lines, providing a regular processing object for subsequent logical structure analysis.

[0064] Select 1,000 power dispatching notice documents covering different voltage levels and different task types as the training set, and construct 3 feature libraries respectively: (1) Title number pattern library: Count the occurrence frequency and hierarchical discrimination of title numbers in the training set, and assign weights to different hierarchical numbers. The weights of the first-level numbers "I.", "II." are set to 1.0; the weights of the second-level numbers "1.", "2." are set to 0.7, and the weights of the appendix numbers "Appendix 1", "Table X-X" are set to 0.9. This weight assignment rule is determined based on the statistical results of the hierarchical discrimination of the number formats in the training set.

[0065] (2) Domain keyword library: The core words of the title are extracted using the TF-IDF algorithm. The confidence level of the first-level title words "general requirements" and "verification content" is 0.95, the confidence level of the second-level title word "verification value adjustment status" is 0.9, and the confidence level of the appendix title word "hidden danger rectification progress table" is 0.9. The confidence threshold is back-calibrated based on the accuracy of title recognition in the training set.

[0066] (3) Layout rule library: Extract the common layout characteristics of training set documents and determine the general rules of left alignment of title, blank line before and after, no punctuation before, and text length ≤ 30 characters. These rules cover more than 98% of the title layout styles in the training set.

[0067] Twelve text fragments were input into the adjusted Distil BERT lightweight language analysis model, with 100 iterations and a learning rate of 3e-5, outputting a 768-dimensional semantic information vector.

[0068] The semantic information vector is concatenated with a 10-dimensional numbered feature vector, a 20-dimensional TF-IDF vector, and a 5-dimensional format normalized vector to form a 798-dimensional fused feature vector.

[0069] The input fused feature vector is fed into the language analysis model encoder, which calculates the association weights between text units through the attention mechanism formula. The output hierarchical probability distribution is then passed through a fully connected classification layer, and finally a title hierarchy tree is generated, which includes 4 first-level titles, 9 second-level titles, and 2 attachment titles.

[0070] The cosine similarity algorithm was used to determine the association between the title and the body text, with a similarity threshold of 0.75. For example, the similarity between "II. Verification Content" and the body text segments corresponding to its five subheadings was ≥0.75, thus successfully establishing the attribution relationship.

[0071] The text “dispatch unit” in the unmatched terminology dictionary is marked as a candidate for misspelling. The contextual semantic features of the text are extracted, and its semantic similarity with the words in the dictionary is calculated. The top three candidate words are then selected.

[0072] The semantic coherence score of candidate words and context is calculated, and the score threshold is set at 0.8. The score of "scheduling unit" reaches 0.82, which meets the threshold requirement. The correction is completed automatically, and the accuracy of correcting professional terminology errors reaches 98%.

[0073] Execute the tabular structured analysis model construction, specifically including: (1) Model the three suspected table areas separately and set three types of core units: header information unit, data cell unit, and row / column index unit.

[0074] (2) Establish association identifiers between cells: the membership weight between the header information cell and the corresponding column index cell is set to 1.0, the membership weight between the row index cell and the data cell cell in the same row is set to 1.0, and the association weight between multiple data cell cells in a merged cell is set to 0.8.

[0075] (3) Initialize the unit feature vector, which is composed of a 768-dimensional text embedding vector, a 10-dimensional position feature vector, and a 20-dimensional power field feature vector, with a total dimension of 798.

[0076] Execution unit feature learning and table structure restoration, specifically including: (1) Construct an adjacency matrix and add self-loops, using the formula Symmetric normalization is performed to avoid feature learning bias caused by differences in node degree.

[0077] (2) After processing by a two-layer correlation computing network, a 256-dimensional unit feature is output, which can accurately characterize the correlation between “header-data” and “data in the same row / column” within the table.

[0078] (3) Perform K-means clustering on the features of the header information units, set the similarity threshold to 0.8, and establish a hierarchical mapping for the "major / medium / minor problem categories" in Table 2-1.

[0079] (4) For merged cells, calculate the correlation between adjacent cells, set the correlation threshold to 0.75, and accurately identify and record the starting row / column and ending row / column information of merged cells.

[0080] The table fields were standardized, including: calling the power sector data standardization rule library to normalize the table fields: completing "220" and "110" in the table to "220kV" and "110kV"; uniformly marking "Completed rectification" as "Yes" and "Incomplete rectification" as "No" and adding the planned rectification time. Finally, three machine-readable structured tables were reconstructed.

[0081] The three reconstructed structured tables are then linked to the corresponding chapter nodes of "Appendix 2" through text location matching.

[0082] Integrate the heading hierarchy tree, heading-to-body mapping, and structured table data to generate a standardized JSON document. This document contains three core types of information: first, the unique identifier, text content, and location information of the heading level; second, the standardized body text corresponding to the heading; and third, the table header, data rows, and merged cell information.

[0083] The generated JSON document can be directly imported into the power safety management system without additional format conversion, enabling rapid statistical analysis of business data.

[0084] Figure 5 An automatic identification system for relay protection dispatching work documents is shown, comprising: The document preprocessing unit 501 is configured to: perform document preprocessing on the acquired relay protection scheduling work file to obtain a set of text fragments and a suspected table area; The logical parsing unit 502 is configured to generate a title hierarchy tree and a title-body mapping relationship based on a document logical structure parsing model that uses multi-feature fusion, based on a set of text fragments. The table reconstruction unit 503 is configured to: reconstruct the table based on the suspected table area using a structured analysis network to obtain structured table data; Information integration unit 504 is configured to: associate structured table data with corresponding chapter nodes according to the heading hierarchy tree, and integrate the heading hierarchy tree, heading-text mapping relationship and structured table data to form standardized structured document description information.

[0085] It is understood that the aforementioned units can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of the present invention. The aforementioned units are based on logical functional division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of the present invention, the system may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0086] According to another embodiment of the present invention, the system of this embodiment can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method of the present invention on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the aforementioned computing device through the computer-readable recording medium, and run therein.

[0087] Figure 6A computer device is shown, which includes a processor 601, a communication interface 602, and a computer-readable storage medium 603. The processor 601, communication interface 602, and computer-readable storage medium 603 can be connected via a bus or other means.

[0088] The communication interface 602 is used to receive and send data. The computer-readable storage medium 603 can be stored in the memory of the electronic device. The computer-readable storage medium 603 is used to store computer programs, which include program instructions. The processor 601 is used to execute the program instructions stored in the computer-readable storage medium 603.

[0089] The processor 601 is the computing and control core of an electronic device. It is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to achieve the corresponding method flow or corresponding function.

[0090] Processor 601 is configured to perform the following procedure: The obtained relay protection scheduling work files are preprocessed to obtain a set of text fragments and a suspected table area; Based on a set of text fragments, a document logical structure parsing model with multi-feature fusion is used to generate a title hierarchy tree and a title-body text mapping relationship. Based on suspected table regions, a structured analysis network is used to reconstruct the table and obtain structured table data. Based on the heading hierarchy tree, the structured table data is associated with the corresponding chapter nodes, and the heading hierarchy tree, heading-to-text mapping relationship and structured table data are integrated to form standardized structured document description information.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0092] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital cable) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatically identifying relay protection dispatching work files, characterized in that, The process includes the following: The obtained relay protection scheduling work files are preprocessed to obtain a set of text fragments and a suspected table area; Based on a set of text fragments, a document logical structure parsing model with multi-feature fusion is used to generate a title hierarchy tree and a title-body mapping relationship. Based on suspected table regions, a structured analysis network is used to reconstruct the table and obtain structured table data. Based on the heading hierarchy tree, the structured table data is associated with the corresponding chapter nodes, and the heading hierarchy tree, heading-to-text mapping relationship and structured table data are integrated to form standardized structured document description information.

2. The automatic identification method for relay protection dispatching work documents as described in claim 1, characterized in that, The acquired relay protection scheduling work files undergo document preprocessing, including: An OCR engine loaded with a corpus of high-frequency terms for power dispatching was used to perform text recognition on relay protection dispatching work documents; Based on a pre-built dictionary of power dispatch terminology, the edit distance algorithm is used to correct typos in the identified text. Specifically, when the edit distance between the identified text and a word in the dictionary is less than or equal to 2, the identified text is corrected to the word in the dictionary. Based on the consistency between the distribution density of delimiters and the number of fields in adjacent rows, suspected table regions are detected and located, and non-table regions are divided into a set of text fragments by blank lines.

3. The automatic identification method for relay protection dispatching work documents as described in claim 1, characterized in that, The construction process of a multi-feature fusion document logical structure parsing model includes: Based on a predetermined-size training set of power dispatching work guidance documents, a title numbering pattern library, a domain keyword library, and a layout rule library were constructed. The set of text fragments is input into the language analysis model adjusted by power dispatch text samples, and the semantic information vector is output. The semantic information vector is concatenated with the numbering feature vector from the title numbering pattern library, the TF-IDF vector from the domain keyword library, and the format normalization vector from the layout rule library to form a fused feature vector.

4. The automatic identification method for relay protection dispatching work documents as described in claim 3, characterized in that, Generate a heading hierarchy tree and heading-to-body text mapping relationships, including: The fused feature vectors are input into the language analysis model encoder, the association weights between each text unit are determined through a multi-dimensional association calculation layer, and the probability distribution of the title level is output through a fully connected classification layer to generate a title level tree. The global feature vector of each title is extracted, and the text fragments within the candidate body text range are aggregated to obtain the body text aggregate features. By calculating the cosine similarity between the global feature vector and the body text aggregate features, the range of body text content under each title is determined to establish the title-body text mapping relationship.

5. The automatic identification method for relay protection dispatching work documents as described in claim 4, characterized in that, After generating the heading hierarchy tree and the heading-to-body mapping, the following steps are also included: Mark candidate misspellings that do not match the dictionary of power dispatch terminology; Using a language analysis model calibrated with text from the power industry, we extract the contextual semantic features of candidate misspellings and calculate their semantic coherence scores with candidate words in the dictionary. When the semantic coherence score is greater than or equal to a preset threshold, the corresponding candidate word is selected to replace the misspelled candidate.

6. The automatic identification method for relay protection dispatching work documents as described in claim 1, characterized in that, Table reconstruction using structured analysis networks includes: For suspected table areas, a structured analysis model is constructed that includes header information units, data cell units, and row and column index units, and relationship identifiers are established between each unit; The features of each unit are initialized as a concatenated vector of text embedding vector, position feature vector and power field feature vector; The representation of each unit is learned through a two-layer associative computational network, and the unit features are output.

7. The automatic identification method for relay protection dispatching work documents as described in claim 6, characterized in that, Obtain structured tabular data, including: Based on the unit features, the header information units are clustered, and a hierarchical mapping is established for the header units whose feature similarity after clustering is greater than or equal to the first threshold. Calculate the feature similarity between data cell units and header information units, and allocate data based on the data logical order of the power dispatch document table to form initial data rows; The correlation between adjacent cells is calculated based on the characteristics of row and column index units. When the correlation is greater than or equal to the second threshold, the cells are determined to be merged and their original range is recorded.

8. The automatic identification method for relay protection dispatching work documents as described in claim 1, characterized in that, The heading hierarchy tree contains unique identifiers for headings, text, hierarchy, and position information. The heading-to-body mapping relationship uses the heading hierarchy path as the key and the standardized body text as the value. The structured tabular data includes table headers, data rows, merged cell information, and the document chapter to which it belongs.

9. An automatic identification system for relay protection dispatching work documents, characterized in that, include: The document preprocessing unit is configured to preprocess the acquired relay protection scheduling work file to obtain a set of text fragments and a suspected table area; The logical parsing unit is configured to generate a title hierarchy tree and a title-body mapping relationship based on a document logical structure parsing model that uses multi-feature fusion, based on a set of text fragments. The table reconstruction unit is configured to: reconstruct the table based on the suspected table area using a structured analysis network to obtain structured table data; The information integration unit is configured to: associate structured tabular data with the corresponding chapter nodes according to the heading hierarchy tree, and integrate the heading hierarchy tree, heading-text mapping relationship and structured tabular data to form standardized structured document description information.

10. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the automatic identification method for relay protection scheduling work files as described in any one of claims 1 to 8.

Images