Electric power knowledge picture content extraction method and system

Abstract

The invention discloses an electric power knowledge picture content extraction method and system, and belongs to the technical field of electric power knowledge extraction, and the method comprises the steps: obtaining an electric power knowledge picture; recognizing the electric power knowledge picture into text information by adopting a pre-trained electric power field content extraction model; and performing quality judgment on the text information through a data quality discriminator, if the quality judgment is excellent, outputting corresponding formatted information, and otherwise, marking the text information as the corresponding formatted information through an electric power expert. According to the method, the electric power field content extraction model can quickly identify the character information, the quality of the character information is judged through the data quality discriminator, the accuracy is higher, and the processing efficiency is higher.

Description

Technical Field

[0001] The present invention belongs to the technical field of power knowledge extraction, and relates to a method and system for extracting content from a power knowledge picture. Background Art

[0002] In the power sector, power knowledge is mainly stored in images. For example, power books and technical standards are usually stored in PDF format. It is difficult and challenging to completely extract text, formulas, icons, text formats and other information from images.

[0003] Optical Character Recognition (OCR) aims to extract words and text information from images and is one of the key technologies in data processing. Traditional OCR technology uses a pipeline structure to extract useful information from images. For example, (a) first extract structured information from a given semi-structured document. (b) Locate the text position through text detection. (c) Input the located text box into the recognizer to identify the specific text. (d) The recognized text and text box position are output to the downstream module to be processed into the required structured information.

[0004] Pipeline OCR tools have high performance, but their model, data construction and computing costs are high; they lack flexibility in language and document type and cannot handle diverse document information; errors accumulate during the recognition process; and the model has poor generalization, so its performance in the power sector is poor;

[0005] Kim et al. proposed an end-to-end OCR model (Document understanding transformer, Donut). The Dount model is built on the Transformer architecture and consists of an encoder and a decoder. During the training process, a two-step training method of pre-training and fine-tuning is adopted. In the pre-training stage, Donut predicts the next word based on the structural image and the previous context. In the fine-tuning stage, Donut learns how to understand the entire document based on downstream tasks. On this basis, Blecher et al. further proposed the Nougat model, which was further optimized through data enhancement methods and showed excellent performance in formula recognition.

[0006] End-to-end OCR tools have the advantages of low computational cost, convenience and speed, but in the actual use process, it is found that OCR tools in general fields have the following problems: 1) poor generalization, and the recognition ability on Chinese data sets needs to be improved; 2) low accuracy, for effective information such as tables and pictures in power data, the model recognition accuracy is poor. 3) Power data processing requirements are more stringent, and general tools will ignore paragraph information in the article during the recognition process, such as not extracting or incorrectly extracting important information such as the title and title number of the article. At the same time, invalid information such as headers, footers, page numbers, and watermarks will be retained. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a method and system for extracting content from electric power knowledge pictures. The electric power field content extraction model of the method can quickly identify text information and perform quality judgment on the text information through a data quality discriminator, with higher accuracy and higher processing efficiency.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] In a first aspect, the present invention provides a method for extracting content from an electric power knowledge picture, comprising:

[0010] Get electricity knowledge pictures;

[0011] Using a pre-trained power field content extraction model to recognize the power knowledge picture into text information;

[0012] The quality of the text information is judged by a data quality discriminator. If the quality is judged to be excellent, the corresponding formatted information is output. Otherwise, the text information is marked as the corresponding formatted information by an electric power expert.

[0013] As a further improvement of the present invention, the structure of the power field content extraction model adopts a Donut model and an encoder-decoder framework.

[0014] As a further improvement of the present invention, the training method of the pre-trained power field content extraction model includes:

[0015] Use recognition tools to pre-process some power knowledge pictures, extract the text information in each power knowledge picture, and convert it into text format data; for table and formula information, convert it into markup language format and output it to obtain power optical character recognition (OCR) pre-training data set;

[0016] The power field content extraction model is pre-trained based on the power OCR pre-training data set to obtain a pre-trained power field content extraction model.

[0017] As a further improvement of the present invention, after the data quality discriminator performs quality judgment on the text information, the method further includes:

[0018] Add high-quality data to the power OCR pre-training dataset to obtain an incremental dataset and start a new round of training.

[0019] As a further improvement of the present invention, the data quality discriminator adopts the model architecture of VILBert. The model encodes the input image into a vector z and the input text into a vector h. The vector z and the vector h are cross-attention calculated to obtain the output vector o, and the vector o will obtain the final probability distribution p through the output layer.

[0020] As a further improvement of the present invention, the data quality discriminator is obtained by using a cross entropy function as a discriminant model and training the text information using a quality discrimination data set;

[0021] The method for obtaining the quality discrimination data set is as follows: label each piece of data with a high or low quality label, complete labeling of all data, and mix the labeled data to obtain the quality discrimination data set.

[0022] As a further improvement of the present invention, before labeling each piece of data with high or low quality, data cleaning is also included. The data cleaning specifically includes:

[0023] Detect language coherence and clean up incoherent or garbled text;

[0024] After cleaning, rewrite the table into a markup language format;

[0025] After marking, extract the title and title serial number information;

[0026] Remove headers and page numbers after extraction;

[0027] After removal, the missing image information in the recognition result is supplemented;

[0028] After supplementation, unify the format and output.

[0029] As a further improvement of the present invention, the quality judgment of the text information by a data quality discriminator includes:

[0030] If the data quality discriminator outputs a probability p greater than a threshold ∈, the data is low-quality data, otherwise it is high-quality data; if the data is low-quality data, the low-quality data is post-processed; the post-processing specifically includes:

[0031] The low-quality data is corrected and re-input into the data quality discriminator. If it is still judged as low-quality data, it is discarded.

[0032] In a second aspect, the present invention provides a system for extracting content from an electric power knowledge picture, comprising:

[0033] An acquisition module is used to acquire power knowledge pictures;

[0034] A recognition module, used to recognize the power knowledge picture into text information using a pre-trained power field content extraction model;

[0035] The judgment module is used to judge the quality of the text information through a data quality discriminator. If the quality is judged to be excellent, the corresponding formatted information is output; otherwise, the text information is marked as the corresponding formatted information by an electric power expert.

[0036] As a further improvement of the present invention, the structure of the power field content extraction model adopts a Donut model and an encoder-decoder framework.

[0037] As a further improvement of the present invention, the training method of the pre-trained power field content extraction model includes:

[0038] Use recognition tools to pre-process some power knowledge pictures, extract the text information in each power knowledge picture, and convert it into text format data; for table and formula information, convert it into markup language format and output it to obtain power optical character recognition (OCR) pre-training data set;

[0039] The power field content extraction model is pre-trained based on the power OCR pre-training data set to obtain a pre-trained power field content extraction model.

[0040] As a further improvement of the present invention, after the data quality discriminator performs quality judgment on the text information, the method further includes:

[0041] Add high-quality data to the power OCR pre-training dataset to obtain an incremental dataset and start a new round of training.

[0042] As a further improvement of the present invention, the data quality discriminator adopts the model architecture of VILBert. The model encodes the input image into a vector z and the input text into a vector h. The vector z and the vector h are cross-attention calculated to obtain the output vector o, and the vector o will obtain the final probability distribution p through the output layer.

[0043] As a further improvement of the present invention, the data quality discriminator is obtained by using a cross entropy function as a discriminant model and training the text information using a quality discrimination data set;

[0044] The method for obtaining the quality discrimination data set is as follows: label each piece of data with a high or low quality label, complete labeling of all data, and mix the labeled data to obtain the quality discrimination data set.

[0045] As a further improvement of the present invention, before labeling each piece of data with high or low quality, data cleaning is also included. The data cleaning specifically includes:

[0046] Detect language coherence and clean up incoherent or garbled text;

[0047] After cleaning, rewrite the table into a markup language format;

[0048] After marking, extract the title and title serial number information;

[0049] Remove headers and page numbers after extraction;

[0050] After removal, the missing image information in the recognition result is supplemented;

[0051] After supplementation, unify the format and output.

[0052] As a further improvement of the present invention, the quality judgment of the text information by a data quality discriminator includes:

[0053] If the data quality discriminator outputs a probability p greater than a threshold ∈, the data is low-quality data, otherwise it is high-quality data; if the data is low-quality data, the low-quality data is post-processed; the post-processing specifically includes:

[0054] The low-quality data is corrected and re-input into the data quality discriminator. If it is still judged as low-quality data, it is discarded.

[0055] As a further improvement of the present invention, it also includes:

[0056] The updating module is used to continuously update the recognition module through the high-quality data output by the discrimination module.

[0057] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method for extracting content from a power knowledge picture is implemented when the processor executes the computer program.

[0058] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting content from a power knowledge picture is implemented.

[0059] In a fifth aspect, the present invention provides a computer program product, wherein the computer program product includes computer instructions, and the computer instructions instruct a computer to execute the method for extracting content from a power knowledge picture.

[0060] The beneficial effects of the present invention compared with the prior art are as follows:

[0061] This solution analyzes the shortcomings of traditional OCR models in the power field and proposes a continuous optimization algorithm. This algorithm optimizes the training data set in combination with large model technology to train the content extraction model in the power field based on end-to-end OCR technology in order to improve the accuracy of the model. At the same time, in order to improve the accuracy of the model, the present invention proposes a data quality discriminator to perform quality detection on the identified data, and uses large models and manual screening to obtain incremental training data sets, and continuously optimizes the content extraction model through continuous learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 A flow chart of extracting content from an electric power knowledge picture provided by the present invention;

[0064] Figure 2 Flowchart of continuous learning algorithm for content extraction model in the power sector;

[0065] Figure 3 A schematic diagram of the application of the power field content extraction model and training algorithm based on continuous learning in downstream tasks;

[0066] Figure 4 A system for extracting content from electric power knowledge pictures provided by the present invention;

[0067] Figure 5 A schematic diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0068] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0069] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0070] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0071] Terminology explanation:

[0072] Mathpix is ​​a powerful mathematical formula recognition tool that uses OCR (optical character recognition) technology to quickly and accurately convert mathematical formulas in images, PDF documents, web pages, and even handwritten mathematical formulas into LaTeX code format.

[0073] OCR (Optical Character Recognition) is a powerful tool that is particularly good at recognizing text in mathematical formulas, scientific symbols, and handwritten notes. Mathpix is ​​more than just a simple OCR software. It can convert complex contents such as scanned or photographed mathematical formulas, diagrams, codes, etc. into editable formats such as LaTeX, MathML, PNG, SVG, etc., greatly improving the work efficiency of scientific researchers, students, teachers, and anyone who needs to deal with mathematical and scientific documents.

[0074] The Donut model (Document Understanding through Universal Transforme) is a new document understanding model consisting of a visual encoder and an NLP language decoder. Both parts are built based on the Transformer. By using the transformer architecture combined with embedding, it can directly process the original input image without going through preprocessing steps such as OCR, with high processing speed and accuracy.

[0075] Swin-Transformer is a new deep learning architecture for computer vision tasks proposed by Microsoft Research Asia in 2021. It combines the powerful capabilities of the Transformer model in sequence data processing with the efficient local feature extraction advantages of convolutional neural networks (CNN) in image recognition, and is particularly suitable for tasks such as image classification, object detection, and semantic segmentation.

[0076] Transformer Decoder is an important component of the Transformer model, and together with Encoder, it constitutes the basic framework of Transformer.

[0077] Markdown is a lightweight markup language that allows people to write documents in a plain text format that is easy to read and write, and then convert it into a valid XHTML (or HTML) document. Markdown has a concise syntax and low learning cost, making it very suitable for writing blog posts, document descriptions, project introductions, etc.

[0078] LaTeX is a typesetting system based on TeX, developed by American computer scientist Leslie Lamport in the early 1980s. LaTeX is widely used in the typesetting of scientific and mathematical documents due to its powerful typesetting functions, especially its excellent performance in processing complex tables and mathematical formulas.

[0079] The VILBert (Vision and Language BERT) model architecture is a multimodal deep learning model designed to process data in two different modalities: images and text. The model extends the BERT (Bidirectional Encoder Representations from Transformers) architecture to enable it to understand and process both image and text information.

[0080] Qwen2-72B is one of the world's most powerful open source models released by Alibaba Cloud. It belongs to the pre-training and instruction fine-tuning model series of the Qwen2 large model.

[0081] PDF (Portable Document Format) is a file format used to present documents, including text, images, tables, charts, forms, etc., in a way that is independent of applications, operating systems, and hardware. PDF files are designed to fully present the format of the source document, including layout, fonts, colors, and images, ensuring that the document remains intact when viewed on any device.

[0082] join Figure 1 The first object of the present invention is to propose a method for extracting content from an electric power knowledge picture, comprising:

[0083] S100, obtaining electric power knowledge pictures;

[0084] S200, using a pre-trained power field content extraction model to recognize the power knowledge picture into text information;

[0085] S300, the text information is subjected to quality judgment by a data quality judgement device. If the quality judgment is excellent, the corresponding formatted information is output. Otherwise, the text information is annotated into the corresponding formatted information by an electric power expert.

[0086] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0087] In view of the shortcomings of existing OCR in the power field, this paper proposes a content extraction model and training method in the power field, and continuously enhances the model performance through continuous learning. The following is a detailed description:

[0088] Aiming at books, standards and other data in the power field, the present invention proposes a content extraction model in the power field based on end-to-end OCR technology, cleans the model training corpus in combination with the big model technology, and proposes a continuous optimization algorithm: for the text information output by the content extraction model, the present invention constructs a quality discriminator that can automatically identify and clean the extracted low-quality text, and continuously use the cleaned high-quality data to optimize the model.

[0089] In the recognition of books, standards and other data in the power field, this model has a higher recognition accuracy than the baseline model. The algorithm flow chart is as follows Figure 2 shown. Figure 2 Flowchart of the continuous learning algorithm for content extraction model in the power sector.

[0090] The content extraction model for the power sector is described as follows:

[0091] The structure of the content extraction model in the power field follows the model structure of Donut and adopts an encoder-decoder framework, in which the encoder uses the Swin-Transformer architecture. Given an input image, the encoder splits it into image blocks of size m×m, and encodes it into vectors through multiple self-attention layers and feedforward neural networks. The decoder uses Transformer Decoder, which receives the input of the input layer and outputs the text sequence x before that moment at each moment. :t-1 =[x0,x1,…,x t-1 ], the decoder aggregates the image and text information through the attention layer, and finally outputs the predicted probability of the next word through the output layer

[0092] The present invention optimizes the performance of the content extraction model in the power field through a large model, and proposes a new method of combining a large model with OCR.

[0093] In the pre-training process, the present invention uses the mathpix tool to pre-process some power books and standard PDF data to obtain the power OCR pre-training data set. The tool will extract the text information in each picture and convert it into text format data. For table and formula information, mathpix converts it into latex format and outputs it.

[0094] The data quality discriminator of the present invention is described as follows:

[0095] In order to improve the quality of the extracted content, the present invention proposes a data quality discriminator and constructs a data quality discriminant dataset. First, the present invention performs data cleaning on the power OCR pre-training dataset. The specific operations include:

[0096] 1) Use the big model to detect the coherence of the language and whether there is garbled text. If there is incoherence or garbled text, use the big model to clean it;

[0097] 2) Rewrite the table into markdown format;

[0098] 3) Extract important information such as title, title serial number, etc.

[0099] 4) Remove the headers and page numbers that exist;

[0100] 5) Supplement the missing image information in the recognition results,

[0101] 6) Unify the output format.

[0102] After the initial cleaning, the big model is used to label the cleaned data. The big model will label each piece of data with 0 or 1 and give corresponding judgment basis, where 0 represents low-quality data and 1 represents high-quality data.

[0103] The present invention mixes the labeled low-quality data, some high-quality data and open source Arxiv (a website that collects preprints of papers in the fields of physics, mathematics, computer science, biology and mathematical economics) data to obtain a quality discrimination data set.

[0104] Among them, the data quality discriminator adopts the VILBert model architecture. The model encodes the input image into a vector z and the input text into a vector h. The vector z and the vector h are calculated through cross-attention to obtain the output vector o. The vector o will pass through the output layer to obtain the final probability distribution p.

[0105] The present invention constructs a quality standard discriminator for filtering low-quality data identified by OCR, and constructs a quality standard discriminator training data set through large models and manual screening and other technologies.

[0106] The present invention uses the cross entropy function to optimize the data quality discriminator:

[0107] LOSS=-[ylogp+(1-y)log(1-p)]

[0108] In the inference stage, if the data quality discriminator outputs a probability p greater than ∈, the data is low-quality data, otherwise it is high-quality data. If the data is low-quality data, the data will be post-processed, including:

[0109] 1) Submit it to power experts for manual correction; 2) Submit it to a large model (such as the Qwen2-72B large model) for correction; 3) Re-input the corrected data into the data quality discriminator. If it is still judged to be low-quality data, it will be discarded.

[0110] The present invention also includes a continuous learning and updating step:

[0111] First, the power field content extraction model will be pre-trained on the power pre-training data set. After obtaining the pre-trained model, the model will be used to identify more power books, technical standards and other data. The obtained data will continue to go through data cleaning, data quality discriminator and data post-processing. For the high-quality data given by the data quality discriminator, it will be added to the training set to obtain an incremental data set to start a new round of training.

[0112] The method proposed in the present invention can be used to extract effective information from various types of books on electricity, technical standards and other books. Given an electricity PDF image, the electricity field content extraction model recognizes it as text information, and passes through a data quality discriminator. If the quality is judged to be excellent, the corresponding formatted information is output, otherwise the text information is annotated into corresponding formatted information by an electricity expert. The output formatted information can be used for subsequent incremental pre-training of large semantic models in the electricity industry, retrieval system construction and other scenarios. Figure 3 The figure shows a schematic diagram of the application of the content extraction model and training algorithm in the power field based on continuous learning in downstream tasks.

[0113] like Figure 4 As shown, the second object of the present invention is to provide a system for extracting content from an electric power knowledge picture, comprising:

[0114] An acquisition module is used to acquire power knowledge pictures;

[0115] A recognition module, used to recognize the power knowledge picture into text information using a pre-trained power field content extraction model;

[0116] The judgment module is used to judge the quality of the text information through a data quality discriminator. If the quality is judged to be excellent, the corresponding formatted information is output; otherwise, the text information is marked as the corresponding formatted information by an electric power expert.

[0117] It also includes an updating module, which is used to continuously update the model corresponding to the recognition module through the high-quality data output by the discrimination module.

[0118] The specific steps are as follows: First, the power field content extraction model will be pre-trained on the power pre-training data set. After obtaining the pre-training model, the model will be used to identify more power books, technical standards and other data. The obtained data will continue to go through data cleaning, data quality discriminator and data post-processing. For the high-quality data given by the data quality discriminator, it will be added to the training set to obtain an incremental data set to start a new round of training.

[0119] Specifically, the structure of the power field content extraction model adopts the Donut model and adopts an encoder-decoder framework.

[0120] The training method of the pre-trained power field content extraction model comprises:

[0121] Use recognition tools to pre-process some power knowledge pictures, extract the text information in each power knowledge picture, and convert it into text format data; for table and formula information, convert it into markup language format and output it to obtain power optical character recognition (OCR) pre-training data set;

[0122] The power field content extraction model is pre-trained based on the power OCR pre-training data set to obtain a pre-trained power field content extraction model.

[0123] This training method focuses on the power field. By preprocessing and constructing a data set for power knowledge pictures, the trained model can more accurately understand and process power-related text, tables, and formula information. This domain-specific training method can provide higher accuracy and efficiency in power field applications compared to general content extraction models. By extracting text, tables, and formulas from power knowledge pictures and converting them into corresponding text and markup language formats, a pretraining data set containing multiple data types is constructed. The richness of this data set helps the model learn more. Using recognition tools to preprocess power knowledge pictures can greatly reduce the burden of manual annotation and improve the efficiency and accuracy of data processing. Automated processing not only saves time and labor costs, but also avoids the impact of human errors on training results. Pre-training the power field content extraction model based on the power OCR pre-training data set can enable the model to have certain power field knowledge in the initial stage. This pre-training process helps the model converge to the optimal solution faster and show better performance in subsequent practical applications.

[0124] As a further improvement, after the data quality discriminator performs quality judgment on the text information, the method further includes:

[0125] Add high-quality data to the power OCR pre-training dataset to obtain an incremental dataset and start a new round of training.

[0126] As an example, the data quality discriminator adopts the model architecture of VILBert. The model encodes the input image into a vector z and the input text into a vector h. The vector z and the vector h are cross-attention calculated to obtain the output vector o, and the vector o will pass through the output layer to obtain the final probability distribution p.

[0127] In the present invention, the VILBert model architecture is used as the design of the data quality discriminator, which has significant advantages. VILBert is a multimodal model specially designed for processing visual and language tasks. By combining image and text information, the model's understanding and discrimination capabilities of complex data are enhanced. This mechanism enhances the model's comprehensive understanding of multimodal data and helps to more accurately judge the quality of the data.

[0128] Furthermore, VILBert encodes the input image into a vector z and the input text into a vector h. This vectorized representation allows the data to be processed in a unified form within the model, improving computational efficiency and accuracy. The vector z and the vector h are calculated through cross-attention to obtain the output vector o. The cross-attention mechanism can capture the potential correlation between images and text, generate richer feature representations, and help the model make more accurate judgments.

[0129] VILBert generates the final probability distribution p through the output layer, which helps to quantify the model's judgment on the data quality. Developers can evaluate the credibility of the data or perform further processing based on the probability distribution.

[0130] Optionally, the data quality discriminator is obtained by using a cross entropy function as a discriminant model and training the text information using a quality discrimination data set;

[0131] The method for obtaining the quality discrimination data set is as follows: label each piece of data with a high or low quality label, complete labeling of all data, and mix the labeled data to obtain the quality discrimination data set.

[0132] As a further improvement, before labeling each piece of data with high or low quality, data cleaning is also included. Data cleaning specifically includes:

[0133] Detect language coherence and clean up incoherent or garbled text;

[0134] After cleaning, rewrite the table into a markup language format;

[0135] After marking, extract the title and title serial number information;

[0136] Remove headers and page numbers after extraction;

[0137] After removal, the missing image information in the recognition result is supplemented;

[0138] After supplementation, unify the format and output.

[0139] Therefore, through the above steps, in the data quality judgment process, the data cleaning step before labeling each piece of data as high or low quality is a key link to ensure the accuracy of subsequent analysis and judgment. Through a series of data cleaning steps, invalid, redundant and erroneous data can be removed to ensure the accuracy and completeness of the data, thereby improving the accuracy of data quality judgment. Removing redundant information and unifying the format during data cleaning can simplify the subsequent data processing process and improve processing efficiency. Cleaned high-quality data helps the model better learn and understand data features, thereby improving the model's discrimination and generalization capabilities. Accurate data quality judgment results can provide users with more valuable information and references, and improve user experience and satisfaction.

[0140] Preferably, the step of judging the quality of the text information by a data quality discriminator includes:

[0141] If the data quality discriminator outputs a probability p greater than a threshold ∈, the data is low-quality data, otherwise it is high-quality data; if the data is low-quality data, the low-quality data is post-processed; the post-processing specifically includes:

[0142] The low-quality data is corrected and re-input into the data quality discriminator. If it is still judged as low-quality data, it is discarded.

[0143] By introducing a data post-processing mechanism and combining strategies such as double verification, data correction and automated processing, the present invention achieves significant advantages in data quality identification and improves the accuracy, efficiency and flexibility of data processing.

[0144] like Figure 5 As shown, the third purpose of an embodiment of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for extracting content from electric power knowledge pictures when executing the computer program.

[0145] A fourth objective of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting content from a power knowledge picture is implemented.

[0146] A fifth objective of an embodiment of the present invention is to provide a computer program product, wherein the computer program product includes computer instructions, and the computer instructions instruct a computer to execute the method for extracting content from an electric power knowledge picture.

[0147] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0149] The present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0150] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting content from an electric power knowledge picture, characterized in that: include: Get electricity knowledge pictures; Using a pre-trained power field content extraction model to recognize the power knowledge picture into text information; The quality of the text information is judged by a data quality discriminator. If the quality is judged to be excellent, the corresponding formatted information is output. Otherwise, the text information is marked as the corresponding formatted information by an electric power expert.

2. The method for extracting content from an electric power knowledge picture according to claim 1, characterized in that: The structure of the power field content extraction model adopts the Donut model and adopts an encoder-decoder framework.

3. The method for extracting content from an electric power knowledge picture according to claim 1 or 2, characterized in that: The training method of the pre-trained power field content extraction model comprises: Use recognition tools to pre-process some power knowledge pictures, extract the text information in each power knowledge picture, and convert it into text format data; for table and formula information, convert it into markup language format and output it to obtain power optical character recognition (OCR) pre-training data set; The power field content extraction model is pre-trained based on the power OCR pre-training data set to obtain a pre-trained power field content extraction model.

4. The method for extracting content from electric power knowledge pictures according to claim 3, characterized in that: After the data quality discriminator performs quality judgment on the text information, the method further includes: Add high-quality data to the power OCR pre-training dataset to obtain an incremental dataset and start a new round of training.

5. The method for extracting content from electric power knowledge pictures according to claim 1, characterized in that: The data quality discriminator adopts the model architecture of VILBert. The model encodes the input image into a vector z and the input text into a vector h. The vector z and the vector h are cross-attention calculated to obtain the output vector o. The vector o will pass through the output layer to obtain the final probability distribution p.

6. The method for extracting content from electric power knowledge pictures according to claim 1, characterized in that: The data quality discriminator is obtained by using the cross entropy function as a discriminant model and training the text information using a quality discrimination data set; The method for obtaining the quality discrimination data set is as follows: label each piece of data with a high or low quality label, complete labeling of all data, and mix the labeled data to obtain the quality discrimination data set.

7. The method for extracting content from electric power knowledge pictures according to claim 6, characterized in that: Before labeling each piece of data as high or low quality, data cleaning is also included. Data cleaning specifically includes: Detect language coherence and clean up incoherent or garbled text; After cleaning, rewrite the table into a markup language format; After marking, extract the title and title serial number information; Remove headers and page numbers after extraction; After removal, the missing image information in the recognition result is supplemented; After supplementation, unify the format and output.

8. The method for extracting content from electric power knowledge pictures according to claim 1, characterized in that: The step of judging the quality of the text information by a data quality discriminator includes: If the data quality discriminator outputs a probability p greater than a threshold ∈, the data is low-quality data, otherwise it is high-quality data; if the data is low-quality data, the low-quality data is post-processed; the post-processing specifically includes: The low-quality data is corrected and re-input into the data quality discriminator. If it is still judged as low-quality data, it is discarded.

9. A system for extracting content from electric power knowledge pictures, characterized in that: include: An acquisition module is used to acquire electric power knowledge pictures; A recognition module, used to recognize the power knowledge picture into text information using a pre-trained power field content extraction model; The judgment module is used to judge the quality of the text information through a data quality discriminator. If the quality is judged to be excellent, the corresponding formatted information is output; otherwise, the text information is marked as the corresponding formatted information by an electric power expert.

10. The power knowledge picture content extraction system according to claim 9, characterized in that: The structure of the power field content extraction model adopts the Donut model and adopts an encoder-decoder framework.

11. The power knowledge picture content extraction system according to claim 9 or 10, characterized in that: The training method of the pre-trained power field content extraction model comprises: Use recognition tools to pre-process some power knowledge pictures, extract the text information in each power knowledge picture, and convert it into text format data; for table and formula information, convert it into markup language format and output it to obtain power optical character recognition (OCR) pre-training data set; The power field content extraction model is pre-trained based on the power OCR pre-training data set to obtain a pre-trained power field content extraction model.

12. The power knowledge picture content extraction system according to claim 11, characterized in that: After the data quality discriminator performs quality judgment on the text information, the method further includes: Add high-quality data to the power OCR pre-training dataset to obtain an incremental dataset and start a new round of training.

13. The power knowledge picture content extraction system according to claim 9, characterized in that: The data quality discriminator adopts the model architecture of VILBert. The model encodes the input image into a vector z and the input text into a vector h. The vector z and the vector h are cross-attention calculated to obtain the output vector o. The vector o will pass through the output layer to obtain the final probability distribution p.

14. The power knowledge picture content extraction system according to claim 9, characterized in that: The data quality discriminator is obtained by using the cross entropy function as a discriminant model and training the text information using a quality discrimination data set; The method for obtaining the quality discrimination data set is as follows: label each piece of data with a high or low quality label, complete labeling of all data, and mix the labeled data to obtain the quality discrimination data set.

15. The power knowledge picture content extraction system according to claim 14, characterized in that: Before labeling each piece of data as high or low quality, data cleaning is also included. Data cleaning specifically includes: Detect language coherence and clean up incoherent or garbled text; After cleaning, rewrite the table into a markup language format; After marking, extract the title and title serial number information; Remove headers and page numbers after extraction; After removal, the missing image information in the recognition result is supplemented; After supplementation, unify the format and output.

16. The power knowledge picture content extraction system according to claim 9, characterized in that: The step of judging the quality of the text information by a data quality discriminator includes: If the data quality discriminator outputs a probability p greater than a threshold ∈, the data is low-quality data, otherwise it is high-quality data; if the data is low-quality data, the low-quality data is post-processed; the post-processing specifically includes: The low-quality data is corrected and re-input into the data quality discriminator. If it is still judged as low-quality data, it is discarded.

17. The power knowledge picture content extraction system according to claim 9, characterized in that: Also includes: The updating module is used to continuously update the recognition module through the high-quality data output by the discrimination module.

18. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the method for extracting content from an electric power knowledge picture as described in any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting content from an electric power knowledge picture as described in any one of claims 1 to 8 is implemented.

20. A computer program product, comprising computer instructions, characterized in that: The computer instructions instruct the computer to execute the method for extracting content from an electric power knowledge picture as described in any one of claims 1-8.

Images