Power emergency document table extraction and knowledge generation method
Through pre-training table extraction model and text recognition technology, the problem of insufficient accuracy of power emergency document extraction in traditional technology is solved, and the accurate identification and knowledge generation of non-standardized tables are achieved, which improves the accuracy of power emergency document processing and efficiency.
Patent Information
- Application Number
- CN202411001673.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-12-03
AI Technical Summary
When traditional technology deals with non-standardized and complex tables in the extraction of power emergency document tables, the accuracy is insufficient, and there are knowledge omissions and deviations, which affects the accuracy of the extraction and knowledge generation process of power emergency document tables.
The pre-trained table extraction model is used to identify the images to be processed for text, obtain table structure information and cell position information, and accurately map it into the cells based on the text recognition results, generate knowledge information, and deduplicate and classify storage through the knowledge database.
It improves the accuracy of the power emergency document table extraction and knowledge generation process, avoids the influence of preset rules, ensures the completeness and accuracy of the table content, and supports the rapid response of power emergency treatment.
Smart Images

Figure CN119068490A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for extracting and generating knowledge from power emergency document tables. Background Art
[0002] With the rapid development of the power industry, power emergency response has become an important part of ensuring the stable operation of the power grid. Power emergency response is a series of pre-planned and immediate actions taken when the power system fails, has an accident, or faces other emergencies to control, mitigate, or eliminate potential hazards, restore the normal operation of the power system as soon as possible, and ensure the reliability of power supply. In power emergency response, quickly and accurately obtaining and processing relevant document information is crucial for timely response and problem solving.
[0003] Traditional technology mainly extracts table contents from power emergency documents based on pre-designed simple rule matching. However, traditional technology for table extraction relies on pre-designed domain knowledge templates and rule sets, and has limited processing capabilities for non-standardized and domain-diverse complex tables. In addition, the extracted table content is not complete enough, and there are omissions and deviations in the table content, which is not conducive to improving the accuracy of the table extraction and knowledge generation process of power emergency documents. Summary of the invention
[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for power emergency document table extraction and knowledge generation, which can improve the accuracy of power emergency document table extraction and knowledge generation process in response to the above technical problems.
[0005] In a first aspect, the present application provides a method for extracting and generating knowledge from power emergency document tables, including:
[0006] Performing text recognition on the image to be processed to obtain a text recognition result of the image to be processed and position information corresponding to the text recognition result of the image to be processed; the image to be processed includes a target power emergency document; the target power emergency document includes a table to be processed;
[0007] Acquire table structure information of the table to be processed in the image to be processed by a pre-trained table extraction model, and determine position information of each cell in the table to be processed according to the table structure information;
[0008] For any cell among the cells, according to the position information of the any cell and the position information corresponding to the text recognition result of the image to be processed, a target text recognition result is screened out from the text recognition result of the image to be processed, and the target text recognition result is mapped to the any cell; the position information corresponding to the target text recognition result is matched with the position information of the any cell;
[0009] According to the text recognition results in the cells, knowledge information corresponding to the table to be processed is generated.
[0010] In one embodiment, performing text recognition on the image to be processed to obtain the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed includes:
[0011] Inputting the image to be processed into a pre-trained text detection model to obtain the text detection result; the text detection result is used to characterize the position information of each text in the image to be processed;
[0012] Determining, from the image to be processed according to the text detection result, an image block corresponding to the text detection result;
[0013] Inputting the image block into a pre-trained text recognition model to obtain a text recognition result corresponding to the image block;
[0014] The text recognition result corresponding to the image block is used as the text recognition result of the image to be processed, and the text detection result corresponding to the image block is used as the position information corresponding to the text recognition result of the image to be processed.
[0015] In one embodiment, generating the knowledge information corresponding to the table to be processed according to the text recognition results in each cell includes:
[0016] For any cell among the cells, determining text content information corresponding to the any cell according to a text recognition result in the any cell;
[0017] Determine the entity information corresponding to the table to be processed according to the text content information corresponding to any cell;
[0018] Generate knowledge information corresponding to the table to be processed according to the entity information.
[0019] In one embodiment, after the step of generating the knowledge information corresponding to the table to be processed, the method further includes:
[0020] Comparing the knowledge information corresponding to the table to be processed, determining the knowledge information to be removed, and removing the knowledge information to be removed from the knowledge information corresponding to the table to be processed, to obtain filtered knowledge information;
[0021] Classifying the filtered knowledge information to obtain a knowledge classification result;
[0022] The filtered knowledge information is stored in a preset knowledge database according to the knowledge classification result; the knowledge database is used for power emergency processing.
[0023] In one embodiment, before the step of performing text recognition on the image to be processed, the method further includes:
[0024] Scaling the image to be processed according to a preset size to obtain a preprocessed image to be processed;
[0025] Determining the document information to be removed in the target power emergency document according to the preprocessed image to be processed, and removing the document information to be removed from the target power emergency document to obtain a filtered document;
[0026] The filtered document is standardized according to a preset document format to obtain a preprocessed target power emergency document.
[0027] In one embodiment, the method further comprises:
[0028] Acquire a sample image; the sample image includes a sample document; the sample document includes a sample form;
[0029] Inputting the sample image into a table extraction model to be trained, obtaining table structure information of the sample table in the sample image, and determining position information of each cell in the sample table according to the table structure information of the sample table;
[0030] Obtain table structure reference information and cell position reference information corresponding to the sample table, and train the table extraction model to be trained based on the table structure reference information and the table structure information of the sample table, as well as the cell position reference information and the position information of each cell in the sample table.
[0031] In a second aspect, the present application also provides a power emergency document table extraction and knowledge generation device, comprising:
[0032] A text recognition module, used for performing text recognition on an image to be processed, obtaining a text recognition result of the image to be processed and position information corresponding to the text recognition result of the image to be processed; the image to be processed includes a target power emergency document; the target power emergency document includes a table to be processed;
[0033] A table extraction module, used for obtaining table structure information of the table to be processed in the image to be processed through a pre-trained table extraction model, and determining position information of each cell in the table to be processed according to the table structure information;
[0034] a result matching module, for filtering out a target text recognition result from the text recognition results of the image to be processed according to the position information of the any cell and the position information corresponding to the text recognition result of the image to be processed, and mapping the target text recognition result to the any cell; matching the position information corresponding to the target text recognition result with the position information of the any cell;
[0035] The knowledge generation module is used to generate knowledge information corresponding to the table to be processed according to the text recognition results in each cell.
[0036] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above method are implemented.
[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.
[0038] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0039] The above-mentioned electric power emergency document table extraction and knowledge generation method, device, computer equipment, computer-readable storage medium and computer program product perform text recognition on the image to be processed to obtain the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed, so as to accurately analyze the text content and the position of the text in the image to be processed; the image to be processed includes the target electric power emergency document; the target electric power emergency document includes the table to be processed; the table structure information of the table to be processed in the image to be processed is obtained through a pre-trained table extraction model, and the position information of each cell in the table to be processed is determined according to the table structure information, so as to accurately analyze the table structure of the table in the image to be processed by using the table extraction model, and provide a data basis for the subsequent determination of the table content in combination with the table structure; for any cell in each cell, according to the position information of any cell and the text recognition result of the image to be processed The target text recognition result is selected from the text recognition result of the image to be processed, and the target text recognition result is mapped to any cell, so as to accurately obtain the text recognition result and cell that match the position information; the position information corresponding to the target text recognition result is matched with the position information of any cell; according to the text recognition result in each cell, the knowledge information corresponding to the table to be processed is generated, and the table structure of the table in the power emergency document can be accurately analyzed based on the table extraction model, so as to avoid the influence of preset rules or templates on the accuracy of the table extraction results, and the text content of each cell in the power emergency document table can be completely and accurately obtained by combining the text recognition results and position information for the power emergency document, so as to accurately generate the table knowledge corresponding to the power emergency document table based on the text content of the cell, thereby improving the accuracy of the power emergency document table extraction and knowledge generation process. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 An application environment diagram of a method for extracting and generating knowledge from a power emergency document table in an embodiment;
[0042] Figure 2 A schematic diagram of a flow chart of a method for extracting and generating knowledge from a power emergency document table in one embodiment;
[0043] Figure 3 An architecture diagram of a power emergency document table extraction and knowledge generation in one embodiment;
[0044] Figure 4 A schematic diagram of a flow chart of a method for extracting a table of power emergency documents and generating knowledge in another embodiment;
[0045] Figure 5 It is a structural block diagram of a power emergency document table extraction and knowledge generation device in one embodiment;
[0046] Figure 6 The figure is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0048] The method for extracting and generating knowledge from power emergency document tables provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 performs text recognition on the image to be processed, and obtains the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed; the image to be processed includes the target power emergency document; the target power emergency document includes the table to be processed; the server 104 obtains the table structure information of the table to be processed in the image to be processed through the pre-trained table extraction model, and determines the position information of each cell in the table to be processed according to the table structure information; the server 104, for any cell in each cell, according to the position information of any cell and the position information corresponding to the text recognition result of the image to be processed, selects the target text recognition result from the text recognition result of the image to be processed, and maps the target text recognition result to any cell; the position information corresponding to the target text recognition result matches the position information of any cell; the server 104 generates the knowledge information corresponding to the table to be processed according to the text recognition results in each cell. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart TVs, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.
[0049] In an exemplary embodiment, Figure 2 As shown, a method for extracting and generating knowledge from power emergency document tables is provided, which is described by taking the method applied to a server as an example, and includes the following steps S202 to S208. Among them:
[0050] Step S202, performing text recognition on the image to be processed to obtain the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed; the image to be processed includes the target power emergency document; the target power emergency document includes the table to be processed.
[0051] The image to be processed may refer to an image containing a target power emergency document. In practical applications, the image to be processed may be obtained by photographing the target power emergency document using an image acquisition device (such as a camera, etc.).
[0052] Among them, the target power emergency document can refer to a series of plans, procedures and guidance documents prepared in advance in the power industry to deal with various possible emergency situations.
[0053] The table to be processed may refer to a table included in the target power emergency document.
[0054] The text recognition result of the image to be processed may refer to information representing the text in the image to be processed. In practical applications, the text recognition result may include text content and the confidence level corresponding to the text content.
[0055] Among them, the position information corresponding to the text recognition result of the image to be processed may refer to information characterizing the position of the text recognition result of the image to be processed in the image to be processed. In actual applications, the position information corresponding to the text recognition result of the image to be processed may include the coordinates of the position of the text recognition result of the image to be processed in the image to be processed in a preset coordinate system.
[0056] As an example, power emergency documents can guide relevant personnel to deal with sudden failures and other problems in emergency situations. In order to quickly and accurately obtain information in power emergency documents, relevant personnel can send the image to be processed for the target power emergency document taken by the image acquisition device to the server. The server receives the image to be processed and performs text recognition on the image to be processed to obtain the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed. In actual applications, the server can perform text detection and text recognition on the image to be processed line by line to determine the position and text content of each text area in the image to be processed.
[0057] Step S204, obtaining table structure information of the table to be processed in the image to be processed through the pre-trained table extraction model, and determining the position information of each cell in the table to be processed according to the table structure information.
[0058] Among them, the table extraction model may refer to a model used to extract the table structure and table content from a table. In practical applications, the table extraction model may include but is not limited to convolutional neural networks (CNN), recurrent neural networks (RNN) and SLANet model (a model for table structure recognition).
[0059] The table structure information may refer to information describing the table organization and internal element layout. In practical applications, the table structure information may include table boundary information, table row and column lines, and row and column information.
[0060] The cell may refer to the smallest unit constituting the table to be processed. In practical applications, the cell may refer to the smallest data storage unit formed by the intersection of a row and a column.
[0061] The cell position information may refer to information representing the position of the cell in the image to be processed. In practical applications, the cell position information may include the coordinates of the cell in the image to be processed in a preset coordinate system.
[0062] As an example, the server can input the image to be processed into a pre-trained table extraction model. The pre-trained table extraction model can analyze the table structure of the table to be processed in the image to be processed, obtain the table structure information of the table to be processed in the image to be processed, and determine the position information of each cell in the table to be processed and the boundary information of the cell based on the table structure information, thereby locating each cell in the image to be processed.
[0063] Step S206, for any cell in each cell, based on the position information of any cell and the position information corresponding to the text recognition result of the image to be processed, the target text recognition result is filtered out from the text recognition results of the image to be processed, and the target text recognition result is mapped to any cell; the position information corresponding to the target text recognition result is matched with the position information of any cell.
[0064] The target text recognition result may refer to a text recognition result whose position information matches the position information of any cell.
[0065] As an example, for any cell in each cell, the server can determine whether the position information of the cell matches (or is the same as) the position information corresponding to the text recognition result of the image to be processed by comparing the position information of any cell with the position information corresponding to the text recognition result of the image to be processed. The server can use the text recognition result whose position information matches the position information of any cell as the target text recognition result corresponding to the any cell, and filter out the target text recognition result from the text recognition results of the image to be processed. Then, the server can map the target text recognition result to any cell, thereby obtaining the text information of each cell in the table to be processed.
[0066] Step S208: Generate knowledge information corresponding to the table to be processed according to the text recognition results in each cell.
[0067] The knowledge information may refer to information used for power emergency processing. In practical applications, the knowledge information may include text information describing the corresponding relationship between equipment failures and processing measures.
[0068] As an example, the server can summarize the text recognition results in each cell to generate knowledge information corresponding to the table to be processed. In actual applications, the server can determine the entities associated with each cell (such as power equipment) and the processing measures when the entity fails based on the text recognition results in each cell. The server summarizes the entities associated with each cell (such as power equipment) and the processing measures when the entity fails, and uses the summary results as the knowledge information corresponding to the table to be processed.
[0069] In the above-mentioned electric power emergency document table extraction and knowledge generation method, by performing text recognition on the image to be processed, the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed are obtained, so as to accurately analyze the text content and the text position in the image to be processed; the image to be processed includes the target electric power emergency document; the target electric power emergency document includes the table to be processed; the table structure information of the table to be processed in the image to be processed is obtained by a pre-trained table extraction model, and the position information of each cell in the table to be processed is determined according to the table structure information, so as to accurately analyze the table structure of the table in the image to be processed by using the table extraction model, and provide a data basis for subsequently determining the table content in combination with the table structure; for any cell in each cell, according to the position information of any cell and the position information corresponding to the text recognition result of the image to be processed, the position information of each cell in the table to be processed is extracted from the image to be processed. The target text recognition result is filtered out from the text recognition result of the image, and the target text recognition result is mapped to any cell, so as to accurately obtain the text recognition result and cell that match the position information; the position information corresponding to the target text recognition result is matched with the position information of any cell; according to the text recognition result in each cell, the knowledge information corresponding to the table to be processed is generated, and the table structure of the table in the power emergency document can be accurately analyzed based on the table extraction model, so as to avoid the influence of preset rules or templates on the accuracy of the table extraction results, and the text content of each cell in the power emergency document table can be completely and accurately obtained by combining the text recognition result and position information of the power emergency document, so as to accurately generate the table knowledge corresponding to the power emergency document table based on the text content of the cell, thereby improving the accuracy of the power emergency document table extraction and knowledge generation process.
[0070] In an exemplary embodiment, text recognition is performed on an image to be processed to obtain a text recognition result of the image to be processed and position information corresponding to the text recognition result of the image to be processed, including: inputting the image to be processed into a pre-trained text detection model to obtain a text detection result; the text detection result is used to characterize the position information of each text in the image to be processed; determining an image block corresponding to the text detection result from the image to be processed according to the text detection result; inputting the image block into a pre-trained text recognition model to obtain a text recognition result corresponding to the image block; using the text recognition result corresponding to the image block as the text recognition result of the image to be processed, and using the text detection result corresponding to the image block as the position information corresponding to the text recognition result of the image to be processed.
[0071] The text detection model may refer to a model used to determine the location of each text in the image to be processed.
[0072] Among them, the text detection result may refer to information used to characterize the position of each text in the image to be processed. In practical applications, the text detection result may include coordinate information describing the text area of each text in the image to be processed. The text detection result may also include a text detection box.
[0073] The image block may refer to an image region described by the text detection result in the image to be processed.
[0074] The text recognition model may refer to a model used to recognize text in an image.
[0075] The text recognition result corresponding to the image block may refer to information representing the text in the image block.
[0076] As an example, the server can input the image to be processed into a pre-trained text detection model. The text detection model can detect the text area corresponding to each text in the image to be processed, and output the text detection result used to characterize the position information of each text in the image to be processed. The server can also determine the image block corresponding to the text detection result from the image to be processed based on the text detection result, and input the image block into the pre-trained text recognition model. The text recognition model can recognize the text in each image block and output the text recognition result corresponding to the image block. The server can use the text recognition result corresponding to each image block as the text recognition result of the image to be processed, and use the text detection result corresponding to each image block as the position information corresponding to the text recognition result of the image to be processed.
[0077] In this embodiment, the text detection result is obtained by inputting the image to be processed into a pre-trained text detection model; the text detection result is used to characterize the position information of each text in the image to be processed; the image block corresponding to the text detection result is determined from the image to be processed according to the text detection result; the image block is input into a pre-trained text recognition model to obtain the text recognition result corresponding to the image block; the text recognition result corresponding to the image block is used as the text recognition result of the image to be processed, and the text detection result corresponding to the image block is used as the position information corresponding to the text recognition result of the image to be processed. The image to be processed can be divided into several image blocks based on the text detection model, and the text recognition results corresponding to the image blocks can be obtained one by one through the text recognition model, so as to flexibly perform accurate text detection and text recognition on the image to be processed, reduce the misrecognition rate, and provide a data basis for the subsequent determination of knowledge information based on the text recognition results of each cell, thereby improving the accuracy of the power emergency document table extraction and knowledge generation process.
[0078] In some embodiments, knowledge information corresponding to the table to be processed is generated based on the text recognition results in each cell, including: for any cell in each cell, determining the text content information corresponding to any cell based on the text recognition results in any cell; determining the entity information corresponding to the table to be processed based on the text content information corresponding to any cell; and generating knowledge information corresponding to the table to be processed based on the entity information.
[0079] The text content information may refer to information representing the text contained in each cell in the table to be processed.
[0080] Among them, entity information can refer to things with real structure or real form in the power industry. In practical applications, entity information can include power equipment information, which can be used to describe information such as the model and parameters of the power equipment.
[0081] As an example, for any cell in each cell, the server can summarize the text recognition results in any cell to obtain the text content information corresponding to the any cell, and then the server can determine the entity information corresponding to the table to be processed based on the text content information corresponding to any cell based on the named entity recognition technology. The server can determine the fault type information and treatment measure information corresponding to the entity information based on the text content information of each cell in the table to be processed. The server can generate knowledge information corresponding to the table to be processed based on the entity information and the fault type information and treatment measure information corresponding to the entity information.
[0082] In this embodiment, by targeting any cell in each cell, according to the text recognition result in any cell, the text content information corresponding to any cell is determined; according to the text content information corresponding to any cell, the entity information corresponding to the table to be processed is determined; according to the entity information, the knowledge information corresponding to the table to be processed is generated, and the text content in each cell can be analyzed one by one to determine the entity information corresponding to the table to be processed, and combined with the text content information of each cell in the table to be processed, the fault type information and processing measure information corresponding to the entity information can be accurately obtained, thereby obtaining accurate knowledge information, thereby improving the accuracy of the power emergency document table extraction and knowledge generation process.
[0083] In some embodiments, after the step of generating knowledge information corresponding to the table to be processed, the method further includes: comparing the knowledge information corresponding to the table to be processed, determining the knowledge information to be removed, and removing the knowledge information to be removed from the knowledge information corresponding to the table to be processed to obtain filtered knowledge information; classifying the filtered knowledge information to obtain a knowledge classification result; storing the filtered knowledge information in a preset knowledge database according to the knowledge classification result; and the knowledge database is used for power emergency processing.
[0084] The knowledge information to be removed may refer to information that needs to be removed from the knowledge information corresponding to the table to be processed. In practical applications, the knowledge information to be removed may include repeated knowledge information and redundant knowledge information.
[0085] The filtered knowledge information may refer to information obtained after removing the knowledge information to be removed from the knowledge information corresponding to the table to be processed.
[0086] The knowledge classification result may refer to the information obtained after classifying the filtered knowledge information according to preset classification rules. In practical applications, the classification rules may include but are not limited to classification by entity, classification by fault type, and classification by treatment measures.
[0087] Among them, the knowledge database can refer to a database used for power emergency processing. In actual applications, when the power system encounters problems such as equipment failure, corresponding processing measures can be obtained from the knowledge database to solve the related failure problems and ensure the safe and stable operation of the power system.
[0088] As an example, the server can compare the knowledge information corresponding to the table to be processed, take repeated or redundant knowledge information as knowledge information to be removed, and remove the knowledge information to be removed from the knowledge information corresponding to the table to be processed to obtain filtered knowledge information. The server can then classify the filtered knowledge information according to preset classification rules to obtain knowledge classification results. The server can also store the filtered knowledge information in the corresponding position in the preset knowledge database according to the knowledge classification results to support subsequent power emergency processing.
[0089] In this embodiment, by comparing the knowledge information corresponding to the table to be processed, the knowledge information to be removed is determined, and the knowledge information to be removed is removed from the knowledge information corresponding to the table to be processed to obtain filtered knowledge information; the filtered knowledge information is classified to obtain knowledge classification results; the filtered knowledge information is stored in a preset knowledge database according to the knowledge classification results; the knowledge database is used for power emergency processing, and can deduplicate and classify the knowledge information corresponding to the table to be processed, and reasonably store the deduplicated and classified knowledge information in the preset knowledge database, so as to facilitate the subsequent processing of power emergency problems based on the knowledge database and improve the processing efficiency of power emergency problems.
[0090] In some embodiments, before the step of performing text recognition on the image to be processed, the method further includes: scaling the image to be processed according to a preset size to obtain a preprocessed image to be processed; determining the document information to be removed in the target power emergency document based on the preprocessed image to be processed, and removing the document information to be removed from the target power emergency document to obtain a filtered document; and standardizing the filtered document according to a preset document format to obtain a preprocessed target power emergency document.
[0091] The preset size may refer to pre-set information for limiting the size of the image to be processed.
[0092] The document information to be removed may refer to information in the target power emergency document that is irrelevant to power emergency processing. In practical applications, the document information to be removed may include but is not limited to comments, headers, footers and other information.
[0093] The filtered document may refer to a document obtained after removing the document information to be removed from the target power emergency document.
[0094] Among them, the preset document format may refer to information that is pre-set and used to limit the document format of the target power emergency document. In actual applications, the preset document format may include but is not limited to information such as document font, document font size, document color and alignment.
[0095] As an example, the server can scale the image to be processed according to a preset size to obtain a preprocessed image to be processed. The server can then use regular expressions and natural language processing tools to identify the document information to be removed in the target power emergency document in the preprocessed image to be processed, and remove the document information to be removed from the target power emergency document to obtain a filtered document, thereby preventing the document information to be removed from interfering with the recognition of cell text. The server can standardize the filtered document according to a preset document format to obtain a preprocessed target power emergency document, thereby reducing the impact of the format on text detection and text recognition.
[0096] In this embodiment, the preprocessed image to be processed is obtained by scaling the image to be processed according to a preset size; the document information to be removed in the target power emergency document is determined based on the preprocessed image to be processed, and the document information to be removed is removed from the target power emergency document to obtain a filtered document; the filtered document is standardized according to a preset document format to obtain a preprocessed target power emergency document, which can filter out irrelevant information in the target power emergency document and standardize the target power emergency document, facilitate subsequent text detection and text recognition, improve the accuracy of text recognition results, and provide a data basis for subsequent determination of knowledge information based on the text recognition results of each cell, thereby improving the accuracy of the power emergency document table extraction and knowledge generation process.
[0097] In some embodiments, the above method also includes: obtaining a sample image; the sample image includes a sample document; the sample document includes a sample table; inputting the sample image into a table extraction model to be trained, obtaining the table structure information of the sample table in the sample image, and determining the position information of each cell in the sample table based on the table structure information of the sample table; obtaining the table structure reference information and the cell position reference information corresponding to the sample table, and training the table extraction model to be trained based on the table structure reference information and the table structure information of the sample table, as well as the cell position reference information and the position information of each cell in the sample table.
[0098] The sample image may refer to an image used to train the table extraction model to be trained. In practical applications, the sample image may be obtained by photographing a sample document using an image acquisition device (such as a camera, etc.).
[0099] The sample document may refer to a document used to train the table extraction model to be trained. In practical applications, the sample document may include but is not limited to power emergency documents.
[0100] The sample table may refer to a table used to train the table extraction model to be trained. In practical applications, the sample table may include but is not limited to tables in power emergency documents.
[0101] The table structure reference information may refer to information representing the table structure corresponding to the sample table. In practical applications, the table structure reference information may be obtained by manual pre-annotation.
[0102] The cell position reference information may refer to information representing the position of each cell in the sample table. In practical applications, the cell position reference information may be obtained by manual pre-marking.
[0103] As an example, the server can obtain a sample image containing a sample document, wherein the sample document includes a sample table. The server then inputs the sample image into a table extraction model to be trained, obtains table structure information of the sample table in the sample image, and determines the position information of each cell in the sample table based on the table structure information of the sample table. The server can also obtain table structure reference information and cell position reference information corresponding to the sample table. The server then trains the table extraction model to be trained based on the table structure reference information and the table structure information of the sample table, as well as the cell position reference information and the position information of each cell in the sample table. Specifically, the server can compare the table structure reference information with the table structure information of the sample table, and / or compare the cell position reference information with the position information of each cell in the sample table to determine the loss function value of the table extraction model to be trained, and based on the relationship between the loss function value of the table extraction model to be trained and a preset loss function threshold, modify the model parameters of the table extraction model to obtain a trained table extraction model. The trained table extraction model can be used as a pre-trained table extraction model.
[0104] In this embodiment, a sample image is obtained; the sample image includes a sample document; the sample document includes a sample table; the sample image is input into a table extraction model to be trained, table structure information of a sample table in the sample image is obtained, and location information of each cell in the sample table is determined according to the table structure information of the sample table; table structure reference information and cell location reference information corresponding to the sample table are obtained, and the table extraction model to be trained is trained according to the table structure reference information and the table structure information of the sample table, as well as the cell location reference information and the location information of each cell in the sample table. The table extraction model can be trained based on the training samples, and the model performance of the table extraction model can be optimized, providing a basis for subsequently determining the location information of the cells based on the table extraction model, thereby improving the accuracy of the location information of the cells, and thereby improving the accuracy of the table extraction and knowledge generation process of the power emergency document.
[0105] In some embodiments, Figure 3As shown, an architecture diagram of table extraction and knowledge generation of power emergency documents is provided. The server can pre-process the power emergency documents through data cleaning and format standardization to ensure the purity and consistency of the document content. Among them, the data cleaning step can use regular expressions and natural language processing tools to identify and remove irrelevant information in the document, including comments, headers and footers, etc., to avoid such information interfering with the processing process; format standardization can unify the format of the document, including font, font size, color, alignment, etc., to reduce the impact of format differences on subsequent processing. Through format standardization, format differences are eliminated, so that the documents are unified in format. Then the server can use a deep learning model (such as a table extraction model) to identify and locate the table. The server can use the trained table extraction model to perform boundary detection on the table in the document and extract the position and shape information of the table. It can be understood that the server can select a suitable deep learning model (such as CNN or RNN) for training according to the characteristics of the power emergency document to improve the recognition accuracy of the table. The server can then parse the table and extract the content. Specifically, the server can analyze the structural characteristics of the table by identifying the rows, columns, and cells of the table, providing a basis for subsequent content extraction. The server can use natural language processing technology and regular expressions to extract key content from cells, including text, numbers, symbols, etc. The server can use natural language processing technology to extract knowledge points and automatically generate knowledge. Specifically, the server can use named entity recognition technology to identify key entities from the table content, such as equipment name, fault type, treatment measures, etc., and extract knowledge points in the table based on the relationship between entities, such as the correspondence between equipment failures and treatment measures, to achieve relationship extraction. The server can also generate knowledge information with practical significance based on the extracted knowledge points to support power emergency handling. Furthermore, the server can post-process and optimize the knowledge mechanical energy. Specifically, the server can improve the quality of knowledge by comparing and analyzing the generated knowledge, removing duplicate or redundant information, and classifying and formatting the knowledge according to the type and characteristics of the knowledge to make the knowledge easier to understand and apply, thereby realizing the classification and formatting of knowledge. The server can also build a knowledge base based on the optimized (such as classified and formatted) knowledge or store the optimized (such as classified and formatted) knowledge into the knowledge base to realize the accumulation and sharing of knowledge, thereby providing strong support for subsequent power emergency processing.
[0106] In this embodiment, a deep learning model, especially a convolutional neural network (CNN) and a recurrent neural network (RNN), is used to adaptively learn and recognize tables of different formats and structures, greatly improving the accuracy. Through a large amount of training data, the model can be gradually improved and the cases of misjudgment and missed judgment can be reduced, ensuring the accurate extraction of table information from power emergency documents, and realizing full process automation, from preprocessing, table recognition, parsing to knowledge generation, without human intervention or manual operation. At the same time, the table extraction and knowledge generation models can be adjusted and optimized according to actual needs, thereby providing more efficient and accurate knowledge support for power emergency handling, improving the emergency response capabilities of the power industry, and improving the efficiency and accuracy of power emergency handling.
[0107] In one exemplary embodiment, Figure 4As shown, a flow chart of a method for extracting and generating knowledge from a table of power emergency documents is provided. The server can construct an emergency document table extraction and knowledge generation model to automatically detect and identify single-line text, analyze table structure, locate cells, and integrate the recognition results into table text from document images, so as to facilitate subsequent data processing, analysis, and knowledge generation. The server can perform text detection on the image to be processed. Specifically, the server can preprocess the image: perform preprocessing operations such as scaling and normalization on the input image to meet the input requirements of the model; and perform single-line text detection: use a trained single-line text detection model to scan the preprocessed image, detect the position coordinates of the line text (such as the coordinates of the four corners of the text box), and post-process the detected line text position coordinates, such as denoising, merging overlapping boxes, etc., to obtain the final line text area; the server can perform text recognition on the image to be processed. Specifically, the server can perform single-line text recognition on the image to be processed: for each detected line text area, crop the corresponding image block, send the image block to the trained single-line text recognition model for recognition, and obtain the recognition result output by the recognition model, including text content and confidence. The server can also perform table structure detection on the table to be processed. Specifically, the server can use the SLANet model to perform table structure detection on the original image, extract the table's boundaries, row and column lines and other structural information, and perform table structure analysis on the table to be processed, that is, based on the table's boundaries, row and column lines and other structural information, analyze the boundaries and position coordinates of each cell in the table to be processed (such as the coordinates of the four corners of the cell). It is understandable that the server can flexibly correct and adjust the coordinates of the cell to ensure the accuracy of positioning. The server can match the position coordinates of each cell with the position coordinates of the row text, and map the text recognition results to the corresponding cells according to the matching results, thereby achieving content matching. When there are multiple row text recognition results corresponding to the same cell, the server can merge the multiple row text recognition results to form a complete cell text content information to achieve content integration. For the row text that cannot be accurately matched to the cell, additional processing (such as rule-based allocation or discarding) can be performed.It is understandable that in order to obtain accurate table text output, the server can construct a table structure based on the information obtained from the table structure detection, including the number of rows and columns and border style of the table, and fill the integrated cell content into the corresponding table cell, and format or escape the cell content as needed to ensure the correctness and readability of the generated text. The server can then output the constructed table text, and the output results may include text content information of each cell in the table. The server can use named entity recognition technology to identify key entities from the table content, and based on the relationship between entities, extract knowledge points in the table to achieve relationship extraction. The server can also generate knowledge information with practical significance based on the extracted knowledge points.
[0108] In this embodiment, deep learning and natural language processing technologies are introduced. By constructing a deep learning model, accurate recognition, extraction and automatic generation of knowledge of tables in power emergency documents are achieved. A knowledge generation method based on natural language processing technology is adopted. By using natural language processing technology to deeply understand and parse the parsed table contents, accurate extraction and generation of knowledge points are achieved; by introducing automation and intelligent technologies, an automated and intelligent processing flow of table extraction and knowledge generation is achieved; through post-processing such as data deduplication, classification and formatting, the quality and availability of generated knowledge are further optimized, which can improve the accuracy and flexibility of table extraction, as well as the accuracy and completeness of knowledge generation, and realize automated and intelligent processing flows, providing more efficient and accurate knowledge support for power emergency handling, and enhancing the emergency response capabilities of the power industry. The efficiency and accuracy of power emergency handling are improved, and strong support for emergency response in the power industry is provided.
[0109] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0110] Based on the same inventive concept, the embodiment of the present application also provides a device for extracting and generating knowledge from an electric power emergency document table for implementing the method for extracting and generating knowledge from an electric power emergency document table. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method above, so the specific limitations in one or more embodiments of the device for extracting and generating knowledge from an electric power emergency document table provided below can be found in the above limitations on the method for extracting and generating knowledge from an electric power emergency document table, and will not be repeated here.
[0111] In an exemplary embodiment, Figure 5 As shown, a power emergency document table extraction and knowledge generation device is provided, including: a text recognition module 502, a table extraction module 504, a result matching module 506 and a knowledge generation module 508, wherein:
[0112] The text recognition module 502 is used to perform text recognition on the image to be processed, and obtain the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed; the image to be processed includes a target power emergency document; the target power emergency document includes a table to be processed.
[0113] The table extraction module 504 is used to obtain the table structure information of the table to be processed in the image to be processed through a pre-trained table extraction model, and determine the position information of each cell in the table to be processed according to the table structure information.
[0114] The result matching module 506 is used to filter out a target text recognition result from the text recognition results of the image to be processed for any cell among the cells, based on the position information of the any cell and the position information corresponding to the text recognition result of the image to be processed, and map the target text recognition result to the any cell; the position information corresponding to the target text recognition result is matched with the position information of the any cell.
[0115] The knowledge generation module 508 is used to generate knowledge information corresponding to the table to be processed according to the text recognition results in each cell.
[0116] In one of the exemplary embodiments, the above-mentioned text recognition module 502 is specifically used to input the image to be processed into a pre-trained text detection model to obtain the text detection result; the text detection result is used to characterize the position information of each text in the image to be processed; according to the text detection result, the image block corresponding to the text detection result is determined from the image to be processed; the image block is input into a pre-trained text recognition model to obtain the text recognition result corresponding to the image block; the text recognition result corresponding to the image block is used as the text recognition result of the image to be processed, and the text detection result corresponding to the image block is used as the position information corresponding to the text recognition result of the image to be processed.
[0117] In one of the exemplary embodiments, the above-mentioned knowledge generation module 508 is also specifically used to determine, for any cell among the cells, the text content information corresponding to the any cell according to the text recognition result in the any cell; determine the entity information corresponding to the table to be processed according to the text content information corresponding to the any cell; and generate the knowledge information corresponding to the table to be processed according to the entity information.
[0118] In one of the exemplary embodiments, the above-mentioned device also includes a knowledge processing module, which is specifically used to compare the knowledge information corresponding to the table to be processed, determine the knowledge information to be removed, and remove the knowledge information to be removed from the knowledge information corresponding to the table to be processed to obtain filtered knowledge information; classify the filtered knowledge information to obtain a knowledge classification result; store the filtered knowledge information in a preset knowledge database according to the knowledge classification result; and the knowledge database is used for power emergency processing.
[0119] In one of the exemplary embodiments, the above-mentioned device also includes a preprocessing module, which is specifically used to scale the image to be processed according to a preset size to obtain a preprocessed image to be processed; determine the document information to be removed in the target power emergency document based on the preprocessed image to be processed, and remove the document information to be removed from the target power emergency document to obtain a filtered document; standardize the filtered document according to a preset document format to obtain a preprocessed target power emergency document.
[0120] In one of the exemplary embodiments, the above-mentioned device also includes a training module, which is specifically used to obtain a sample image; the sample image includes a sample document; the sample document includes a sample table; the sample image is input into the table extraction model to be trained, the table structure information of the sample table in the sample image is obtained, and the position information of each cell in the sample table is determined according to the table structure information of the sample table; the table structure reference information and the cell position reference information corresponding to the sample table are obtained, and the table extraction model to be trained is trained according to the table structure reference information and the table structure information of the sample table, as well as the cell position reference information and the position information of each cell in the sample table.
[0121] Each module in the above-mentioned power emergency document table extraction and knowledge generation device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.
[0122] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for extracting and generating knowledge of power emergency document tables is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0123] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0124] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0126] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0130] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for extracting and generating knowledge from power emergency document tables, characterized in that: The method comprises: Performing text recognition on the image to be processed to obtain a text recognition result of the image to be processed and position information corresponding to the text recognition result of the image to be processed; the image to be processed includes a target power emergency document; the target power emergency document includes a table to be processed; Acquire table structure information of the table to be processed in the image to be processed by a pre-trained table extraction model, and determine position information of each cell in the table to be processed according to the table structure information; For any cell among the cells, according to the position information of the any cell and the position information corresponding to the text recognition result of the image to be processed, a target text recognition result is screened out from the text recognition result of the image to be processed, and the target text recognition result is mapped to the any cell; the position information corresponding to the target text recognition result is matched with the position information of the any cell; According to the text recognition results in the cells, knowledge information corresponding to the table to be processed is generated.
2. The method according to claim 1, characterized in that The performing text recognition on the image to be processed to obtain the text recognition result of the image to be processed and the position information corresponding to the text recognition result of the image to be processed includes: Inputting the image to be processed into a pre-trained text detection model to obtain the text detection result; the text detection result is used to characterize the position information of each text in the image to be processed; Determining, from the image to be processed according to the text detection result, an image block corresponding to the text detection result; Inputting the image block into a pre-trained text recognition model to obtain a text recognition result corresponding to the image block; The text recognition result corresponding to the image block is used as the text recognition result of the image to be processed, and the text detection result corresponding to the image block is used as the position information corresponding to the text recognition result of the image to be processed.
3. The method according to claim 1, characterized in that: Generating the knowledge information corresponding to the table to be processed according to the text recognition results in each cell includes: For any cell among the cells, determining text content information corresponding to the any cell according to a text recognition result in the any cell; Determine the entity information corresponding to the table to be processed according to the text content information corresponding to any cell; Generate knowledge information corresponding to the table to be processed according to the entity information.
4. The method according to claim 1, characterized in that: After the step of generating the knowledge information corresponding to the table to be processed, the method further includes: Comparing the knowledge information corresponding to the table to be processed, determining the knowledge information to be removed, and removing the knowledge information to be removed from the knowledge information corresponding to the table to be processed, to obtain filtered knowledge information; Classifying the filtered knowledge information to obtain a knowledge classification result; The filtered knowledge information is stored in a preset knowledge database according to the knowledge classification result; the knowledge database is used for power emergency processing.
5. The method according to claim 1, characterized in that Before the step of performing text recognition on the image to be processed, the method further comprises: Scaling the image to be processed according to a preset size to obtain a preprocessed image to be processed; Determining the document information to be removed in the target power emergency document according to the preprocessed image to be processed, and removing the document information to be removed from the target power emergency document to obtain a filtered document; The filtered document is standardized according to a preset document format to obtain a preprocessed target power emergency document.
6. The method according to claim 1, characterized in that The method further comprises: Acquire a sample image; the sample image includes a sample document; the sample document includes a sample form; Inputting the sample image into a table extraction model to be trained, obtaining table structure information of the sample table in the sample image, and determining position information of each cell in the sample table according to the table structure information of the sample table; Obtain table structure reference information and cell position reference information corresponding to the sample table, and train the table extraction model to be trained based on the table structure reference information and the table structure information of the sample table, as well as the cell position reference information and the position information of each cell in the sample table.
7. A power emergency document table extraction and knowledge generation device, characterized in that: The device comprises: A text recognition module, used for performing text recognition on an image to be processed, obtaining a text recognition result of the image to be processed and position information corresponding to the text recognition result of the image to be processed; the image to be processed includes a target power emergency document; the target power emergency document includes a table to be processed; A table extraction module, used for obtaining table structure information of the table to be processed in the image to be processed through a pre-trained table extraction model, and determining position information of each cell in the table to be processed according to the table structure information; a result matching module, for filtering out a target text recognition result from the text recognition results of the image to be processed according to the position information of the any cell and the position information corresponding to the text recognition result of the image to be processed, and mapping the target text recognition result to the any cell; matching the position information corresponding to the target text recognition result with the position information of the any cell; The knowledge generation module is used to generate knowledge information corresponding to the table to be processed according to the text recognition results in each cell.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
