Stabilization rule regularization extraction method and device and storage medium

Abstract

The invention discloses a regularized extraction method and device for stability regulations and a storage medium. The regularized extraction method comprises the following steps: preprocessing a document; converting the stability regulation file into a file in a specified format; table recognition is matched with a title; identifying the content of the table and identifying the content contained in the header of the table; according to the table information and a preset title template, matching title contents corresponding to the table; constructing a prompt project according to a business field demand, and extracting actual corresponding entity content according to the limited field; and the results of rule extraction and large model extraction are fused, and finally automatic input of the stable and specified section quota is realized and the equipment model is associated. The invention provides an information extraction method fusing regularization and a large model, and aims to overcome the defects of an existing section limit manual input electronization technology. A professional field noun range is limited through regularization, and an electric power regularization capability and a large model extraction capability are fused, so that an intelligent information extraction capability without a large amount of manual intervention is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of power information processing, and specifically relates to a method, device and storage medium for regular extraction of stable procedures. Background Art

[0002] The stability limit rules define the operation mode, control equipment and limit values. With the overall acceleration of the construction of ultra-high voltage AC / DC interconnected large power grids, the AC / DC coupling is closer, and the in-depth promotion of power market reform, the grid operation mode and section limit values ​​change more frequently, the section control limit coupling increases, the number and complexity of stability limits are greatly increased, and the workload of manually entering stability limits is becoming more and more arduous. It is urgent to study a method that can reduce the workload of compiling stability limit rules.

[0003] In the prior art, for example, the technical solution described in the patent with publication number CN105140912A is: a method for identifying the control limit quota of a stable section of a power grid taking into account the operating status of a stable control system, by standardizing the structural constraints of the modeled equipment of the stable section control limit of the power grid, and supplementing the description of the unstructured information or the relevant information of the unmodeled equipment through the notes, so as to realize the unified modeling of the stable section control limit. Then, based on the actual operating conditions of the stable control device in the power grid dispatching automation system, the online identification of the operating status of the stable control system related to the stable limit is realized.

[0004] In the prior art, there is a problem that unstructured information that relies on textual supplementation of remarks may be incomplete or inaccurate, thereby affecting the effectiveness of the overall model and the accurate identification of the stable limit. Summary of the invention

[0005] The purpose of the present invention is to provide a method, device and storage medium for regular extraction of stability regulations, so as to solve the problem in the prior art proposed in the background technology that the complexity of compiling stability limit maintenance rules is greatly increased, the workload of manually editing stability limit maintenance rules becomes increasingly heavy, and there is a problem of low editing efficiency.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for automatically identifying documents with stable quota rules includes the following steps:

[0008] Step S1, document preprocessing; implementing format conversion service operations for different types of stable procedure files; converting the stable procedure files into files of a specified format;

[0009] Step S2, table identification and title matching, including identifying the table content and identifying the content contained in the table header in the table;

[0010] Step S3, matching the title content corresponding to the table content according to the identified table content and the preset title template;

[0011] Step S4, constructing a prompt project according to the business field requirements, inputting the preset rules and the original text of the table into the pre-trained language model, and using the pre-trained language model to extract the table content corresponding to the given business field;

[0012] Step S5, integrating the results of the extraction according to the preset rules and the pre-trained language model, and summarizing the content according to the field matching;

[0013] Step S6: convert the extracted information into object-oriented devices or formulas according to the data type.

[0014] According to the above technical solution, in step S1, the stabilization procedure file is converted into a file of a specified format as follows:

[0015] Step S101, by identifying the format of the source file, checking the file header to identify the file type, extracting the file content and re-encoding it into a specified format;

[0016] Step S102, screening the converted file content, cleaning the title content in the file and retaining the text content of the title;

[0017] Step S103, segment the content according to the title level of the document, narrow the extraction range according to the title content, and distinguish the rules of the corresponding areas through title comparison.

[0018] According to the above technical solution, cleaning of the title content includes deleting special characters, blanks and formatting marks in the title.

[0019] According to the above technical solution, in step S2, table recognition is specifically as follows: according to the xml structure configuration file corresponding to the document, by matching the position information of the table header information, the actual position of the table in the document is determined, the context is judged by the position, and the corresponding title range is determined according to the position distance.

[0020] According to the above technical solution, in step S2, the title matching is specifically as follows: matching the title content corresponding to the table according to the table information and the preset title template; analyzing the correlation between the table text content and the context paragraph by common rules based on business data, and screening the contextual suspected remark information that matches the information content in the table.

[0021] According to the above technical solution, in step S3, the title content corresponding to the matching table is specifically: according to the position information of the table in the document, the system stores different tables and their related contents in a structured manner according to the same position logic; the stored content includes the corresponding title, table text and filtered note information.

[0022] According to the above technical solution, step S4, content aggregation according to field matching is specifically as follows:

[0023] Step S401, set the rule extraction result to R and the weight to W R , the extraction result of the pre-trained language model is M, and the weight is W M , the content summary result is S, and the field matching function is F match ; The content summary function is F summarize_weighted ; Then the algorithm formula is:

[0024] S=F summarize_weighted (F match (R, M, W R , W M ))

[0025] Among them, W R and W M are the weight vectors of R and M respectively, each weight corresponds to the confidence of the corresponding field; F match Indicates that weights are taken into account during the matching process and the weighted results after matching are returned; F summarize_weighted Indicates that the content is weighted and summarized according to the weighted results after matching to generate the final summary result S;

[0026] Step S402, field matching and weight adjustment: F match (R, M, W R , W M ) is implemented as follows:

[0027] For each field k, calculate the weighted matching result:

[0028] If both R and M contain field k, the field values ​​are merged and the weight is taken into account; the matching result is:

[0029] R′ k =W R [k]*R[k]+W M [k]*M[k]

[0030] If only R or M contains field k, the weight value of that field is used directly:

[0031] If R contains k and M does not, then: R′ k =W R [k]*R[k];

[0032] If M contains k and R does not contain k, then: R′ k =W M [k]*M[k].

[0033] According to the above technical solution, F summarize_weightedThe implementation is as follows:

[0034] For each field k, according to its weighted matching result R' k and the results of other related fields to generate the summary result S:

[0035]

[0036] Among them, S represents the summary result; R' k Represents the weighted matching result, W' k It represents the weight of the weighted matching result of field k.

[0037] A device for automatically identifying documents with stable quota rules, the identification device includes a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of a method for automatically identifying documents.

[0038] A storage medium for automatic identification of documents with stable quota rules, wherein a computer program is stored on the storage medium, and when the program is executed by a processor, the steps of a method for automatic document identification are implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention provides a method for automatic identification of stable quota regularized documents to solve the shortcomings of the prior art. By limiting the scope of professional field terms through regularization, integrating power regularization capability and extraction capability, intelligent information extraction capability without a lot of manual intervention is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the method for extracting the stable rules according to the present invention;

[0042] Figure 2 It is a block diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Embodiment 1

[0045] like Figure 1 As shown, a method for automatically identifying documents with stable quota rules includes the following steps:

[0046] Step S1, document preprocessing; implementing format conversion service operations for different types of stable procedure files; converting the stable procedure files into files of a specified format;

[0047] Step S2, table identification and title matching, including identifying the table content and identifying the content contained in the table header in the table;

[0048] Step S3, matching the title content corresponding to the table content according to the identified table content and the preset title template;

[0049] Step S4, constructing a prompt project according to the business field requirements, inputting the preset rules and the original text of the table into the pre-trained language model, and using the pre-trained language model to extract the table content corresponding to the given business field;

[0050] Step S5, integrating the results of the matching extraction according to the preset rules and the pre-trained language model, and summarizing the content according to the field matching;

[0051] Step S6: convert the extracted information into object-oriented devices or formulas according to the data type.

[0052] The present invention provides an information extraction method integrating regularization and large models to solve the shortcomings of the prior art. By limiting the scope of professional field terms through regularization, integrating power regularization capabilities and large model extraction capabilities, intelligent information extraction capabilities are achieved without a lot of manual intervention.

[0053] Embodiment 2

[0054] This embodiment provides a specific implementation method, and its application process specifically involves the following steps:

[0055] Step 100: The stable procedure file is converted into a specified file through a format conversion service, for example, converting doc, wps and other formats into docx format.

[0056] Step 101: Clean the first, second, and third level titles in the document, and narrow the extraction scope according to the title content, for example, remove conceptual chapters such as general principles, calculation conditions, and overall operating requirements.

[0057] Use titles such as "Kangding Channel Control Limits and Operation Requirements" to distinguish the rules for corresponding areas.

[0058] Table recognition relies on table header information. According to the XML structure configuration file corresponding to the docx document, the actual position of the table in the document is determined by matching the position information of the table header information (which usually includes key identification information such as the table title and column titles), and the context is determined by the position, and the approximate range of the corresponding title is determined according to the position distance.

[0059] Step 200: table identification and title matching, including: matching the title content corresponding to the table according to the table information and the preset title template; analyzing the correlation between the table text content and the context paragraph through the rule algorithm, and screening the contextual suspected remark information that matches the information content in the table.

[0060] According to the location information of the table in the document, the system stores different tables and their related contents in a structured manner according to the same location logic; the stored content includes the corresponding title, table text, and filtered note information.

[0061] Identify the content of the table, for example, if the title is "Kangding Channel Control Limits and Operation Requirements", identify the content contained in the table header, such as matching "Changkang Double Loop Line" with "Kangding Channel", and determine the approximate range based on the relationship between the position judgment table and the context.

[0062] Step 300: Match the title content corresponding to the table according to the table information and the title template of "Figure XX"; match the suspected remark information below according to the information content in the table. Store the title, table, and remark as a separate table file according to the actual location;

[0063] Step 400: Set up an expert database to explain the specific terms of the actual business, such as the actual name of the abbreviated line, the actual composition of the line section, and the coverage of the wide area section. For example, the Changkang double-circuit line includes the "Changkang I Line" and the "Changkang II Line".

[0064] Step 500: Construct a prompt project according to the business field requirements. These fields are usually key information points in the business process, such as station name, voltage level, sending end name, etc.; input the rules and table original text into the large model, and extract the actual corresponding entity content according to the limited fields.

[0065] Step 600: Integrate the results of rule extraction and large model extraction, and summarize the content according to field matching. Among them, rule extraction is usually based on predefined logic and patterns, with high accuracy and structured degree. Identify the same business fields in the two results, such as "station name", "delivery end", etc.; compare the contents of the same fields to check whether there are differences or conflicts; for content with differences, resolve conflicts according to preset priority rules; merge the field contents that have been confirmed to be correct or resolved to form the final summary result.

[0066] Let the rule extraction result be R and the weight be W R , the extraction result of the large model is M, and the weight is W M , the content summary result is S, and the field matching function is F match ; The content summary function is F summarize_weighted ; Then the algorithm formula is:

[0067] S=F summarize_weighted (Fmatch(R,M,W R , W M ))

[0068] Among them, W R and W M are the weight vectors of R and M respectively, each weight corresponds to the confidence of the corresponding field; F match Indicates that weights are taken into account during the matching process and the weighted results after matching are returned; F summarize_weighted It means that the content is weighted and summarized according to the weighted results after matching to generate the final summary result S.

[0069] Field matching and weight adjustment: F match (R, M, W R , W M ) is implemented as follows:

[0070] For each field k, calculate the weighted matching result:

[0071] If both R and M contain field k, the field values ​​are merged and the weight is taken into account; the matching result is:

[0072] R′ k =W R [k]*R[k]+W M [k]*M[k]

[0073] If only R or M contains field k, the weight value of that field is used directly:

[0074] If R contains k and M does not, then: R′ k =W R [k]*R[k];

[0075] If M contains k and R does not contain k, then: R′ k =W M [k]*M[k].

[0076] F summarize_weighted The implementation is as follows:

[0077] For each field k, according to its weighted matching result R' k and the results of other related fields to generate the summary result S:

[0078]

[0079] Among them, S represents the summary result; R' k Represents the weighted matching result, W' k It represents the weight of the weighted matching result of field k.

[0080] Step 700: Use Python's JSON library to convert the dictionary into JSON format. The extracted results are passed to the object recognition service to define object models, which represent the devices, formulas and the relationships between them in the field; the model should contain the properties and methods of the device and the rules on how to correspond these properties and methods to the description in the text. According to the device features described in the text, the device objects in the object model are instantiated; attribute values ​​are assigned to the device objects, which come from the information extracted from the text. The converted object devices and formulas are output as format content that can be further processed.

[0081] Step 800: According to the incoming JSON format, the extracted information is converted into objectified equipment, formula and other information according to the data type.

[0082] like Figure 2 As shown, a stable procedure regularized extraction device comprises:

[0083] Document preprocessing: preprocess stable procedure documents, convert the file format into the specified format, and perform preliminary screening and cleaning of text paragraph information and table information.

[0084] Identify table information and table title, match table identification with title, identify table information and table title.

[0085] Summarize table information, match tables with corresponding titles and subsequent descriptions based on location information, and store each table in the document as a single file.

[0086] Write rules, based on expert experience, store special place names, equipment names, and section names into a word segmentation vocabulary, and preliminarily extract trusted entities.

[0087] Large model extraction, based on the large model and prompt engineering, extract the remaining entities.

[0088] Fusion results, aggregate regularized entities and large model entities, and fuse the results.

[0089] Format conversion: convert the extracted rules into JSON format for output.

[0090] Objectify the content and send the extracted rules to the objectified recognition service to obtain objectified data.

[0091] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0092] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for automatic identification of stable quota regularized documents, characterized by: The following steps are involved: Step S1, document preprocessing; implementing format conversion service operations for different types of stable procedure files; converting the stable procedure files into files of a specified format; Step S2, table identification and title matching, including identifying the table content and identifying the content contained in the table header in the table; Step S3, matching the title content corresponding to the table content according to the identified table content and the preset title template; Step S4, constructing a prompt project according to the business field requirements, inputting the preset rules and the original text of the table into the pre-trained language model, and using the pre-trained language model to extract the table content corresponding to the given business field; Step S5, integrating the results of the matching extraction according to the preset rules and the pre-trained language model, and summarizing the content according to the field matching; Step S6, converting the extracted information into object-oriented devices or formulas according to the data type.

2. According to claim 1, a method for automatically identifying documents with stable quota rules is characterized by: In step S1, the stabilization procedure file is converted into a file of a specified format as follows: Step S101, by identifying the format of the source file, checking the file header to identify the file type, extracting the file content and re-encoding it into a specified format; Step S102, screening the converted file content, cleaning the title content in the file and retaining the text content of the title; Step S103, segment the content according to the title level of the document, narrow the extraction range according to the title content, and distinguish the rules of the corresponding areas through title comparison.

3. The method for automatic identification of stable quota regularized documents according to claim 2 is characterized by: Title content cleaning includes deleting special characters, spaces, and formatting marks in the title.

4. The method for automatic identification of stable quota regularized documents according to claim 3 is characterized by: In step S2, table recognition is specifically as follows: according to the XML structure configuration file corresponding to the document, by matching the position information of the table header information, the actual position of the table in the document is determined, the context is judged by the position, and the corresponding title range is determined according to the position distance.

5. The method for automatic identification of stable quota regularized documents according to claim 4 is characterized by: In step S2, the title matching is specifically as follows: matching the title content corresponding to the table according to the table information and the preset title template; and screening the contextual suspected remark information that matches the information content in the table by analyzing the correlation between the table text content and the context paragraph.

6. The method for automatic identification of stable quota regularized documents according to claim 5 is characterized by: In step S3, the title content corresponding to the matching table is specifically: according to the position information of the table in the document, the system stores different tables and their related contents in a structured manner according to the same position logic; the stored content includes the corresponding title, table text and filtered note information.

7. The method for automatic identification of stable quota regularized documents according to claim 1 is characterized by: Step S4, summarizing the content according to field matching is as follows: Step S401, set the rule extraction result to R and the weight to W R , the extraction result of the pre-trained language model is M, and the weight is W M , the content summary result is S, and the field matching function is F match ; The content summary function is F summarize_weighted ; Then the algorithm formula is: S=F summarize_weighted (F match (R,M,W R ,W M )) Among them, W R and W M are the weight vectors of R and M respectively, each weight corresponds to the confidence of the corresponding field; F match Indicates that weights are taken into account during the matching process and the weighted results after matching are returned; F summarize_weighted Indicates that the content is weighted and summarized according to the weighted results after matching to generate the final summary result S; Step S402, field matching and weight adjustment: F match (R, M, W R , W M ) is implemented as follows: For each field k, calculate the weighted matching result: If both R and M contain field k, the field values ​​are merged and the weight is taken into account; the matching result is: R′ k =W R [k]*R[k]+W M [k]*M[k] If only R or M contains field k, the weight value of that field is used directly: If R contains k and M does not, then: R′ k =W R [k]*R[k]; If M contains k and R does not contain k, then: R′ k =W M [k]*M[k].

8. The method for automatic identification of stable quota regularized documents according to claim 9 is characterized in that: F summarize_weighted The implementation is as follows: For each field k, according to its weighted matching result R' k and the results of other related fields to generate the summary result S: Among them, S represents the summary result; R' k Represents the weighted matching result, W' k It represents the weight of the weighted matching result of field k.

9. An automatic identification device for stable quota rule documents, characterized in that: The identification device comprises a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 8.

10. A storage medium for automatic identification of stable quota rule documents, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images