Electric power standard structuring method and system based on image recognition technology
By applying the structured method of power standards based on image recognition technology in the power industry, the challenges in power standard document management and use are solved, automated and efficient structured processing and storage are realized, and the information construction and operation efficiency of the power industry are improved.
Patent Information
- Application Number
- CN202411299562.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2025-03-14
AI Technical Summary
The storage, management and use of standard files in the power industry face challenges, including a large number of complicated documents, diverse formats and rapid update requirements, and lack of methods for information identification of industry-standard PDF files.
The power standard structured method based on image recognition technology is adopted, combined with OCR self-healing mechanism, natural language processing and other technologies, the automation and efficient structured processing of power standard files is achieved, including text extraction, recognition error correction, meta-information extraction and text content entry, and finally stored in the database.
It realizes accurate identification, content extraction and structured storage of power standard documents, improves processing efficiency and accuracy, and provides strong support for the informatization construction and efficient operation of the power industry.
Smart Images

Figure CN119625748A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to an electric power standard structuring method based on image recognition technology. Background Art
[0002] In the current rapidly developing power industry, power standards are an important cornerstone for ensuring the safety of power facilities, promoting the advancement of power technology, and promoting the orderly development of the power market. However, with the continuous updating and improvement of power standards, the storage, management and use of standard documents are also facing new challenges. At present, the status quo of power industry standards is mainly manifested in the following points: First, the number of standard documents is huge and the content is complicated, covering various links such as power generation, transmission, distribution, and power consumption; second, the formats of standard documents are diverse, including paper documents and electronic documents, among which electronic documents are mainly in PDF format; finally, the update speed of standard documents is relatively fast, and they need to be obtained and updated in a timely manner. At present, there is no method for information identification of industry standard PDF files in the power industry.
[0003] Driven by the wave of digitalization, the power industry is undergoing unprecedented changes. With the in-depth advancement of information construction, the digital processing and structured storage of power standard documents have become an important part of the industry's development. As an important basis for guiding the standardized operation of the power industry, the processing efficiency and accuracy of power standard documents directly affect the operating efficiency and service quality of the power industry. Therefore, researching and applying advanced image recognition technology to achieve automated and efficient structured processing of power standard documents is of great significance to promoting the informatization development of the power industry. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for structuring power standards based on image recognition technology. By using advanced image recognition technology, an automated and efficient structured processing method is proposed according to the particularity of power standard documents. By combining image recognition, natural language processing and other technologies, an OCR self-healing mechanism for power standard parsing scenarios is creatively proposed. The OCR self-healing mechanism can automatically correct recognition errors that may occur in the OCR process, such as character confusion, layout parsing errors, etc. It can also continuously optimize the recognition accuracy through context understanding and self-learning, forming a self-repair and improvement capability, realizing accurate recognition, content extraction and structured storage of power standard documents, and providing strong support for the information construction and efficient operation of the power industry.
[0005] To achieve this purpose, the present invention designs a method for structuring power standards based on image recognition technology, which includes the following steps:
[0006] Step 1: Extract text from the power standard file in PDF format based on the OCR self-healing mechanism to obtain all texts in the power standard file;
[0007] Step 2: Apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the text in the power standard document, and obtain all the text in the accurate power standard document;
[0008] Step 3: Identify and extract the power standard meta information of all the texts in the accurate power standard file to obtain the meta information of the power standard file;
[0009] Step 4: Itemize the content of the power standard text for all the texts of the accurate power standard document to obtain itemized information of the power standard document;
[0010] Step 5: Store the obtained meta information of the power standard file and the itemized information of the power standard file into a database.
[0011] An electric power standard structuring system based on image recognition technology includes an OCR text extraction module, an OCR accuracy improvement module, a meta-information extraction module, a text content itemization module and a storage module:
[0012] The OCR text extraction module is used to extract text from the power standard file in PDF format based on the OCR self-healing mechanism to obtain all texts in the power standard file;
[0013] The OCR accuracy improvement module is used to apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the texts in the power standard document, and obtain all the texts in the accurate power standard document;
[0014] The meta information extraction module is used to accurately identify and extract the power standard meta information of all the texts in the power standard file to obtain the meta information of the power standard file;
[0015] The text content itemization module is used to itemize all the texts of the precise power standard file to obtain itemized information of the power standard file;
[0016] The storage module is used to store the obtained meta information of the power standard file and the itemized information of the power standard file in a database.
[0017] Beneficial effects of the present invention:
[0018] A method for structuring power standards based on image recognition technology uses advanced image recognition technology to propose an automated and efficient structured processing method for the particularity of power standard documents. Through in-depth research on the storage form and characteristics of power standard documents, combined with image recognition, natural language processing and other technologies, accurate recognition, content extraction and structured storage of power standard documents are achieved, providing strong support for the information construction and efficient operation of the power industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is the overall architecture diagram of the present invention;
[0020] Figure 2 Structured flow chart for the text;
[0021] Figure 3 It is a structural schematic diagram of the present invention; DETAILED DESCRIPTION
[0022] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0023] Example 1
[0024] like Figure 1 A method for structuring power standards based on image recognition technology is shown, which includes the following steps:
[0025] Step 1: Extract text from the power standard file in PDF format based on the OCR self-healing mechanism to obtain all texts in the power standard file;
[0026] Step 2: Apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the text in the power standard document, and obtain all the text in the accurate power standard document;
[0027] Step 3: Identify and extract the power standard meta information of all the texts in the accurate power standard file to obtain the meta information of the power standard file;
[0028] Step 4: Itemize the content of the power standard text for all the texts of the accurate power standard document to obtain itemized information of the power standard document;
[0029] Step 5: Store the obtained meta information of the power standard file and the itemized information of the power standard file into a database.
[0030] PDF files are a common document format that contains multiple elements such as text, images, tables, etc. In order to extract text from PDF files, you need to convert them into image format first and then use OCR technology to recognize them.
[0031] The step 1: extracting text from the power standard file in PDF format based on the OCR self-healing mechanism, and obtaining all texts in the power standard file is as follows:
[0032] Step 1-1: Use the PDF processing library to convert the power standard file in PDF format into an image sequence;
[0033] Step 1-2: Preprocess the converted image; the preprocessing includes denoising, binarization, and amplification;
[0034] Denoising is performed using a filter. The specific principle of using a Gaussian filter in the present invention is:
[0035]
[0036] Among them, Kernel(i,j) is the value of the Gaussian kernel, K is the normalization factor, x and y represent the coordinates of the pixel in the image, i and j represent the two relative coordinates in the Gaussian kernel (or Gaussian filter), r is used to define the range of the Gaussian kernel, Input(x+i,y+i) is the original image input, and Output(x,y) is the denoising result;
[0037] Binarization is the process of setting the grayscale value of pixels on an image to 0 or 255, that is, converting the image into an image with only two colors, black and white; the purpose of this is to simplify the image data, highlight the outline of the target, and facilitate subsequent image analysis and processing.
[0038] Image magnification refers to increasing the size of an image, which is achieved using bicubic interpolation to improve subsequent recognition accuracy;
[0039] Step 1-3: Perform OCR text recognition on the pre-processed image to extract all text information. During the recognition process, OCR text recognition automatically locates the text area, recognizes the characters, and outputs the text recognition results of the power standard file in PDF format;
[0040] Step 1-4: Apply the OCR self-healing mechanism. For the text recognition results of the power standard documents in PDF format, use the characteristics of the power standard documents to analyze the text context, determine whether the characters with recognition errors conform to the grammatical rules and semantic logic, and then correct them. For the common layout patterns of power standard documents, establish a layout template library. During the parsing process, match the layout of the actual document with the templates in the template library, and re-parse or manually intervene if mismatched parts are found;
[0041] Step 1-5: User checks OCR recognition results, allowing users to provide feedback on OCR recognition results, including marking errors and providing correct text. The system collects this feedback data to train and optimize the OCR model and layout parsing algorithm, and outputs all texts of the power standard file.
[0042] The step 2: applying the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the texts in the power standard document, and the specific method of obtaining all the texts in the accurate power standard document is:
[0043] Step 2-1: Data preparation: Based on the incorrect OCR recognition results encountered in the parsing process of the power standard file in PDF format, they are correctly annotated and organized into an OCR recognition error dataset; the annotation content includes the location of the text in the image, the correct text content, and possible special symbols;
[0044] Step 2-2: Model selection and adjustment: Based on the general PaddleOCR model, adjust the model's network structure and parameter settings according to the characteristics of power standard files to meet the recognition requirements of power standard files;
[0045] Step 2-3: Training and optimization: Use the prepared OCR recognition error dataset to retrain the model; during the retraining phase, the back propagation algorithm is mainly used to update the model parameters;
[0046]
[0047] Among them, θ represents the model parameters, L represents the loss function, η represents the learning rate, represents the gradient of the loss function with respect to the parameter, and t represents the number of iterations;
[0048] Step 2-4: Evaluation and optimization: After training, use the validation set to evaluate the model and check its recognition effect on the power standard file. For the text that was previously misrecognized, check whether its recognition accuracy can reach 95%; if it does not reach 95%, continue to iterate. If the accuracy reaches 95%, stop iterating and output all the text of the accurate power standard file.
[0049] By combining OpenCV and PaddleOCR, we can extract all the text of the power standard document from the PDF file. According to the characteristics of the power standard document, retraining the PaddleOCR model can significantly improve the recognition accuracy and provide data support for the subsequent power standard meta-information extraction and text entry.
[0050] The specific method of step 3: identifying and extracting power standard meta-information from all texts in the power standard PDF file is as follows:
[0051] Step 3-1: Obtain the approximate meta-information range of all texts in the power standard PDF file through a location-based extraction method. The location characteristics of each meta-information are shown in the following table;
[0052] Meta information Page Location Location Features Feature Keywords International Classification Number front page Upper left ICS China Classification Number front page Upper left F / K / L Standard No. front page Top right GB / DL / Q / NB Replace standard number front page Top right replace Standard Chinese name front page Middle position none Standard English name front page Below the Chinese name none Release time front page Lower left release Implementation time front page Bottom right Implementation Drafting Unit Preface none Drafting Unit: Drafted by Preface none Drafted by:
[0053] Step 3-2: For standard meta-information with specific keywords such as drafter and drafting unit, use the keyword-based extraction method to accurately match and extract the content within the approximate meta-information range of all texts in the power standard PDF file. The meta-information keywords are shown in the table above;
[0054] Step 3-3: For standard meta-information with certain patterns such as date and standard number, regular expressions and specific patterns are used to match specific information in the text, so as to accurately extract the meta-information of all texts in the power standard file and obtain the meta-information of all texts in the power standard file. For example:
[0055] 1) Extract standard number: "GB / T\d{4}-\d{4}"
[0056] 2) Extract the release date: "\d{4}-\d{2}-\d{2}"
[0057] In the structured processing of power standards, the identification and extraction of meta-information is crucial. Meta-information is data that describes data. For power standards, meta-information usually includes key information such as standard number, standard name, release date, implementation date, replacement standard, replaced standard, standard status, etc. This information is of great significance for the classification, retrieval, update and management of standards.
[0058] As time goes by, the formats of standard files often differ. This is mainly due to technological updates, changes in industry standards, and different requirements for file management in different periods. In view of the format differences, two different meta-information recognition and extraction models, namely the precise recognition model and the fuzzy recognition model, are proposed. Among them, the precise recognition model is suitable for files with a large number and a relatively fixed format; the fuzzy recognition model is suitable for files with a small number and a non-fixed format.
[0059] The accurate recognition model uses a location-based extraction method, which mainly relies on a deep understanding of the structure of power standard documents. Power standard documents usually have a fixed format and layout, and meta information often appears in specific locations. Therefore, by identifying these locations, meta information can be accurately extracted. For example, the standard number may appear in the title or header of the document, and the release date and implementation date may be located at the beginning or end of the document. By defining these locations, corresponding extraction rules can be written to quickly extract meta information from the document. The keyword-based extraction method is more flexible and versatile. This method performs full-text search and matching in the document by defining keywords or phrases related to meta information. Once a match is found, the text paragraphs containing these keywords can be extracted, and the required meta information can be further analyzed and extracted. This method is particularly suitable for documents where the location of meta information is not fixed or the format is variable. By carefully selecting keywords, the accuracy and completeness of the extraction can be ensured.
[0060] The specific method of step 4: converting all the texts of the power standard document into items, and obtaining the itemized information of the power standard document is as follows:
[0061] Step 4-1: Divide all text of the power standards document into precise chapter contents:
[0062] Divide the text content of all words in the precise power standard document according to the primary directory to form multiple text sets, and obtain the chapter content of all words in the precise power standard document;
[0063] Step 4-2: Itemize the chapter contents of all texts of the precise power standard document:
[0064] All texts under each chapter directory are processed into entries one by one, that is, the chapter number and content of the entry are obtained according to the regular expression; when dividing the entry, the location information of the entry needs to be recorded at the same time; the location information of the chapter entry consists of two parts, which are the page number of the entry and the coordinate position of the entry on the page; the location of the chapter entry needs to merge the coordinates of all the texts belonging to the chapter; the specific calculation formula is as follows:
[0065] X min =Min(Chapter text coordinate X set)
[0066] Y min =Min(Chapter text coordinate Y set)
[0067] X max =Max(Chapter text coordinate X set)
[0068] Y max =Max(Chapter text coordinate Y set)
[0069] Among them, the item coordinates are displayed in rectangular form, with the coordinates of the four corners (Xmin, Ymin), (Xmax, Ymin), (Xmin, Ymax), (Xmax, Ymax);
[0070] Finally, the itemized information of the power standard document is obtained.
[0071] In this process, processing table information is particularly critical because tables often carry a large amount of structured data. The specific steps are as follows:
[0072] Preprocess the image containing the table, such as binarization and denoising, to highlight the boundary features of the table.
[0073] The Hough line detection algorithm is applied to detect possible straight line segments in the image space. These straight line segments often correspond to the borders or separators of the table. By analyzing and combining these straight line segments, the position and shape of the table can be determined. Once the position information of the table is obtained, the data in the table can be further extracted. The basic equation of the Hough transform is:
[0074] ρ=x·cos(α)+y·sin(α)
[0075] Among them, ρ is the distance from the origin to the line, and α is the angle between the line and the horizontal axis.
[0076] The specific method of step 5: storing the obtained meta information of the power standard file and the itemized information of the power standard file into the database is:
[0077] Step 5-1: Identify the type of information to be stored, including power standard file meta information and power standard file entry information;
[0078] Step 5-2: Create a "meta information" table to store the power standard file meta information, and a "standard entry" table to store the power standard file meta information entry information;
[0079] Step 5-3: Set up the association between the tables. Associate the "meta information" table with the "standard entry" table through a foreign key;
[0080] Step 5-4: Store the obtained meta information of the power standard file and the itemized information of the power standard file into a relational database to provide a basis for subsequent data analysis and query.
[0081] Example 2
[0082] An electric power standard structuring system based on image recognition technology includes an OCR text extraction module, an OCR accuracy improvement module, a meta-information extraction module, a text content itemization module and a storage module:
[0083] The OCR text extraction module is used to extract text from the power standard file in PDF format based on the OCR self-healing mechanism to obtain all texts in the power standard file;
[0084] The OCR accuracy improvement module is used to apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the texts in the power standard document, and obtain all the texts in the accurate power standard document;
[0085] The meta information extraction module is used to accurately identify and extract the power standard meta information of all the texts in the power standard file to obtain the meta information of the power standard file;
[0086] The text content itemization module is used to itemize all the texts of the precise power standard file to obtain itemized information of the power standard file;
[0087] The storage module is used to store the obtained meta information of the power standard file and the itemized information of the power standard file in a database.
[0088] A power standard structuring system based on image recognition technology, wherein the OCR text extraction module is used to extract text from power standard files in PDF format based on the OCR self-healing mechanism, and obtain all texts of the power standard files:
[0089] Use the PDF processing library to convert the power standard file in PDF format into an image sequence;
[0090] Preprocess the converted image; the preprocessing includes denoising, binarization, and amplification;
[0091] Denoising is performed using a filter. The specific principle of using a Gaussian filter in the present invention is:
[0092]
[0093] Among them, Kernel(i,j) is the value of the Gaussian kernel, K is the normalization factor, x and y represent the coordinates of the pixels in the image, i and j represent the relative coordinates in the Gaussian kernel (or Gaussian filter), r is used to define the range of the Gaussian kernel, Input(x+i,y+i) is the original image input, and Output(x,y) is the denoising result;
[0094] Binarization is the process of setting the grayscale value of the pixels on the image to 0 or 255, that is, converting the image into an image with only black and white colors;
[0095] Image enlargement refers to increasing the size of an image, which is achieved using bicubic interpolation;
[0096] Perform OCR text recognition on the pre-processed image to extract all text information. During the recognition process, OCR text recognition automatically locates the text area, recognizes the characters, and outputs the text recognition results of the power standard file in PDF format;
[0097] Apply the OCR self-healing mechanism. For the text recognition results of power standard documents in PDF format, use the characteristics of power standard documents to analyze the text context, determine whether the characters with recognition errors conform to grammatical rules and semantic logic, and then correct them. Establish a layout template library for common layout patterns of power standard documents. During the parsing process, match the layout of the actual document with the templates in the template library, and re-parse or manually intervene if mismatched parts are found;
[0098] Users can check OCR recognition results and provide feedback on OCR recognition results, including marking errors and providing correct text. The system collects this feedback data and uses it to train and optimize the OCR model and layout parsing algorithm to output all text in the power standard file.
[0099] A power standard structuring system based on image recognition technology; the OCR accuracy improvement module is used to apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the texts in the power standard file. The specific method for obtaining all the texts in the accurate power standard file is:
[0100] Data preparation: The incorrect OCR recognition results encountered in the parsing process of the power standard files in PDF format are correctly annotated and organized into an OCR recognition error dataset; the annotation content includes the location of the text in the image, the correct text content, and possible special symbols;
[0101] Model selection and adjustment: Based on the general PaddleOCR model, the network structure and parameter settings of the model are adjusted according to the characteristics of power standard files to meet the recognition requirements of power standard files;
[0102] Training and optimization: The model is retrained using the prepared OCR recognition error dataset. During the retraining phase, the back propagation algorithm is mainly used to update the model parameters.
[0103]
[0104] Among them, θ represents the model parameters, L represents the loss function, represents the learning rate, represents the gradient of the loss function with respect to the parameter, and t represents the number of iterations;
[0105] Evaluation and optimization: After training, the model is evaluated using the validation set to check its recognition effect on the power standard file. For the text that was previously misrecognized, check whether its recognition accuracy can reach 95%. If it does not reach 95%, continue to iterate. If the accuracy reaches 95%, stop iterating and output all the text of the accurate power standard file.
[0106] A power standard structuring system based on image recognition technology; the meta-information extraction module is used to accurately identify and extract power standard meta-information of all texts in power standard files, and the specific method for obtaining the meta-information of power standard files is:
[0107] Obtain the approximate meta-information scope of all texts of accurate power standard documents through location-based extraction method;
[0108] For standard meta-information with specific keywords such as drafter and drafting unit, a keyword-based extraction method is used to accurately match and extract the content within the approximate meta-information range of all texts in the power standard PDF file;
[0109] For standard meta-information with certain patterns such as date and standard number, regular expressions and specific patterns are used to match specific information in the text, so as to accurately extract the meta-information of all the text in the power standard file and obtain the meta-information of all the text in the power standard file.
[0110] A power standard structuring system based on image recognition technology; the text content itemization module is used to itemize all the texts of the power standard document, and the specific method for obtaining the itemized information of the power standard document is as follows:
[0111] Divide the precise chapter contents of all texts of the power standard document into:
[0112] Divide the text content of all words in the precise power standard document according to the primary directory to form multiple text sets, and obtain the chapter content of all words in the precise power standard document;
[0113] Itemization of all text chapter contents of precise electrical standard documents:
[0114] All texts under each chapter directory are processed into entries one by one, that is, the chapter number and content of the entry are obtained according to the regular expression; when dividing the entry, the location information of the entry needs to be recorded at the same time; the location information of the chapter entry consists of two parts, which are the page number of the entry and the coordinate position of the entry on the page; the location of the chapter entry needs to merge the coordinates of all the texts belonging to the chapter; the specific calculation formula is as follows:
[0115] X min=Min(Chapter text coordinate X set)
[0116] Y min =Min(Chapter text coordinate Y set)
[0117] X max =Max(Chapter text coordinate X set)
[0118] Y max =Max(Chapter text coordinate Y set)
[0119] Among them, the item coordinates are displayed in rectangular form, with the coordinates of the four corners (Xmin, Ymin), (Xmax, Ymin), (Xmin, Ymax), (Xmax, Ymax);
[0120] Finally, the itemized information of the power standard document is obtained.
[0121] A power standard structuring system based on image recognition technology, wherein the storage module is used to store the obtained meta information of the power standard file and the itemized information of the power standard file into a database.
[0122] Example 3
[0123] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the above method when executed by a processor.
[0124] Example 4
[0125] A computer program product comprises a computer program, wherein the computer program implements the steps of the above method when executed by a processor.
[0126] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.
Claims
1. A method for structuring power standards based on image recognition technology, characterized in that: It includes the following steps: Perform text extraction on the power standard file in PDF format based on the OCR self-healing mechanism to obtain all the text in the power standard file; Apply OCR self-healing mechanism to improve the accuracy of model recognition of all texts in power standard documents and obtain all texts in accurate power standard documents; Perform power standard meta information recognition and extraction on all texts of accurate power standard files to obtain meta information of power standard files; All the texts of the precise electric power standard document are itemized into the electric power standard text content to obtain the itemized information of the electric power standard document.
2. The method for structuring power standards based on image recognition technology according to claim 1, characterized in that: The specific method of extracting text from the power standard file in PDF format based on the OCR self-healing mechanism and obtaining all the text in the power standard file is as follows: Use the PDF processing library to convert the power standard file in PDF format into an image sequence; Preprocess the converted image; the preprocessing includes denoising, binarization, and amplification; Denoising is performed using a filter. The specific principle of using a Gaussian filter in the present invention is: Among them, Kernel(i,j) is the value of the Gaussian kernel, K is the normalization factor, x and y represent the coordinates of the pixel in the image, i and j represent the two relative coordinates in the Gaussian kernel, r is used to define the range of the Gaussian kernel, Input(x+i,y+i) is the original image input, and Output(x,y) is the denoising result; Binarization is the process of setting the grayscale value of pixels on an image to 0 or 255; Image enlargement refers to increasing the size of an image, which is achieved using bicubic interpolation; Perform OCR text recognition on the pre-processed image to extract all text information; during the recognition process, OCR text recognition automatically locates the text area, recognizes the characters, and outputs the text recognition results of the power standard file in PDF format; Applying the OCR self-healing mechanism, for the text recognition results of the power standard documents in PDF format, the text context is analyzed using the characteristics of the power standard documents to determine whether the characters with recognition errors conform to the grammatical rules and semantic logic, so as to make corrections; according to the layout mode of the power standard documents, a layout template library is established; during the parsing process, the layout of the actual document is matched with the templates in the template library, and the mismatched parts are re-parsed or manually intervened; Users can verify OCR recognition results and provide feedback on them, including marking errors and providing correct text. The electric power standard structured system based on image recognition technology collects these feedback data to train and optimize OCR models and layout parsing algorithms, and outputs all text in the electric power standard file.
3. The method for structuring power standards based on image recognition technology according to claim 2 is characterized in that: The OCR self-healing mechanism is applied to improve the accuracy of the model in recognizing all the texts in the power standard document. The specific method for obtaining all the texts in the accurate power standard document is as follows: The incorrect OCR recognition results encountered in the parsing process of the power standard file in PDF format are correctly annotated and organized into an OCR recognition error dataset; the annotation content includes the location of the text in the image, the correct text content, and possible special symbols; Based on the general PaddleOCR model, the network structure and parameter settings of the model are adjusted according to the characteristics of power standard files to meet the recognition requirements of power standard files. The model was retrained using the prepared OCR recognition error dataset. During the retraining phase, the back propagation algorithm was mainly used to update the model parameters. Among them, θ represents the model parameters, L represents the loss function, η represents the learning rate, represents the gradient of the loss function with respect to the parameter, and t represents the number of iterations; After training is completed, the model is evaluated using the validation set to check its recognition effect on the power standard file. For the text that was previously recognized incorrectly, check whether its recognition accuracy can reach the set value. If it does not reach the set value, continue to iterate. If the accuracy reaches the set value, stop iterating and output all the text of the accurate power standard file.
4. The method for structuring power standards based on image recognition technology according to claim 1, characterized in that: The specific method of step 3, identifying and extracting the power standard meta information of all the texts in the accurate power standard file, and obtaining the meta information of the power standard file is as follows: Obtain accurate meta-information scope of all texts of power standard documents through location-based extraction method; For standard meta-information with specific keywords, the keyword-based extraction method is used to accurately match and extract the content within the meta-information range of all texts in the power standard PDF file; For standard meta-information such as date and standard number, regular expressions and specific patterns are used to match specific information in the text, thereby accurately extracting meta-information of all texts in the power standard file and obtaining meta-information of all texts in the power standard file.
5. The method for structuring power standards based on image recognition technology according to claim 1, characterized in that: All the texts of the precise power standard documents are itemized into the power standard text content. The specific method of itemizing the information of the power standard documents is as follows: Divide the precise chapter contents of all texts of the power standard document into: Divide the text content of all words in the precise power standard document according to the primary directory to form multiple text sets, and obtain the chapter content of all words in the precise power standard document; Itemization of all text chapter contents of precise electrical standard documents: All texts under each chapter directory are processed into entries one by one, that is, the chapter number and content of the entry are obtained according to the regular expression; when dividing the entry, the location information of the entry needs to be recorded at the same time; the location information of the chapter entry consists of two parts, which are the page number of the entry and the coordinate position of the entry on the page; the location of the chapter entry needs to merge the coordinates of all the texts belonging to the chapter; the specific calculation formula is as follows: X min =Min(Chapter text coordinate X set) Y min =Min(Chapter text coordinate Y set) X max =Max(Chapter text coordinate X set) Y max =Max(Chapter text coordinate Y set) Among them, the item coordinates are displayed in rectangular form, with the coordinates of the four corners (Xmin, Ymin), (Xmax, Ymin), (Xmin, Ymax), (Xmax, Ymax); Finally, the itemized information of the power standard document is obtained.
6. An electric power standard structuring system based on image recognition technology, characterized in that: It includes OCR text extraction module, OCR accuracy improvement module, meta information extraction module and text content entry module: The OCR text extraction module is used to extract text from the power standard file in PDF format based on the OCR self-healing mechanism to obtain all texts in the power standard file; The OCR accuracy improvement module is used to apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the texts in the power standard document, and obtain all the texts in the accurate power standard document; The meta information extraction module is used to accurately identify and extract the power standard meta information of all the texts in the power standard file to obtain the meta information of the power standard file; The text content itemization module is used to itemize all the texts of the precise power standard file to obtain itemized information of the power standard file.
7. The electric power standard structuring system based on image recognition technology according to claim 6 is characterized by: The OCR text extraction module is used to extract text from the power standard file in PDF format based on the OCR self-healing mechanism. The specific method for obtaining all texts in the power standard file is as follows: Use the PDF processing library to convert the power standard file in PDF format into an image sequence; Preprocess the converted image; the preprocessing includes denoising, binarization, and amplification; Denoising is performed using a filter. The specific principle of using a Gaussian filter in the present invention is: Among them, Kernel(i,j) is the value of the Gaussian kernel, K is the normalization factor, x and y represent the coordinates of the pixel in the image, i and j represent the two relative coordinates in the Gaussian kernel, r is used to define the range of the Gaussian kernel, Input(x+i,y+i) is the original image input, and Output(x,y) is the denoising result; Binarization is the process of setting the grayscale value of the pixels on the image to 0 or 255, that is, converting the image into an image with only black and white colors; Image enlargement refers to increasing the size of an image, which is achieved using bicubic interpolation; Perform OCR text recognition on the pre-processed image to extract all text information; during the recognition process, OCR text recognition automatically locates the text area, recognizes the characters, and outputs the text recognition results of the power standard file in PDF format; Applying the OCR self-healing mechanism, for the text recognition results of the power standard documents in PDF format, the text context is analyzed using the characteristics of the power standard documents to determine whether the characters with recognition errors conform to the grammatical rules and semantic logic, so as to make corrections; according to the layout mode of the power standard documents, a layout template library is established; during the parsing process, the layout of the actual document is matched with the templates in the template library, and the mismatched parts are re-parsed or manually intervened; Users can verify OCR recognition results and provide feedback on them, including marking errors and providing correct text. The electric power standard structured system based on image recognition technology collects these feedback data to train and optimize OCR models and layout parsing algorithms, and outputs all text in the electric power standard file.
8. The electric power standard structuring system based on image recognition technology according to claim 6 is characterized by: The OCR accuracy improvement module is used to apply the OCR self-healing mechanism to improve the accuracy of the model in recognizing all the texts in the power standard document. The specific method for obtaining all the texts in the accurate power standard document is as follows: The incorrect OCR recognition results encountered in the parsing process of the power standard file in PDF format are correctly annotated and organized into an OCR recognition error dataset; the annotation content includes the location of the text in the image, the correct text content, and possible special symbols; Based on the general PaddleOCR model, the network structure and parameter settings of the model are adjusted according to the characteristics of power standard files to meet the recognition requirements of power standard files. The model was retrained using the prepared OCR recognition error dataset. During the retraining phase, the back propagation algorithm was mainly used to update the model parameters. Among them, θ represents the model parameters, L represents the loss function, η represents the learning rate, represents the gradient of the loss function with respect to the parameter, and t represents the number of iterations; After training is completed, the model is evaluated using the validation set to check its recognition effect on the power standard file. For the text that was previously recognized incorrectly, check whether its recognition accuracy can reach the set value. If it does not reach the set value, continue to iterate. If the accuracy reaches the set value, stop iterating and output all the text of the accurate power standard file.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in claim 1 are implemented.
Citation Information
Cited By
Method and system for intelligently extracting data elements for national data standard file
CN120336507A