Setting value list analysis and information extraction method

Abstract

The embodiment of the invention relates to the technical field of power system relay protection, in particular to a constant value list analysis and information extraction method, which comprises the following steps: receiving a constant value list file, identifying the type of the constant value list file, determining a processing mode for the constant value list file based on the type of the constant value list file, and extracting the information of the constant value list file. Comprising the following steps: if the type of a constant value sheet file is a PDF editable format, analyzing a text stream, font attributes and coordinate information of the constant value sheet file, dividing a text region based on font size and coordinate distribution, and generating structured data through template matching; if the type of the constant value single file is a PDF picture format, distortion correction and quality enhancement are carried out through an image preprocessing unit, a text is identified by using an optimized OCR technology, and an identification result is corrected by combining an electric power constant value logic rule; semantic analysis is carried out on the structural data, public area information and fixed value area information are extracted, semantic verification and correction are carried out, and fixed value data are obtained; and outputting the constant value data according to a preset format.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of power system relay protection, in particular to a fixed value single analysis and information extraction method. BACKGROUND

[0002] With the deepening of smart grid construction, the number of relay protection devices increases day by day, and the operation mode of power grid changes frequently, resulting in an increase in protection fixed value switching frequency and a substantial increase in fixed value checking workload. According to the power system operation specification, fixed value checking needs to be carried out after power outage maintenance, during routine inspection and maintenance, and during major holiday power protection. As the standard fixed value issued by the dispatch, the accuracy and efficiency of data extraction directly determine the quality and progress of the checking work.

[0003] At present, the fixed value single issued by the dispatch mainly includes two types: one is in PDF editable format, which contains vector graphics and text elements, and the text can be directly extracted but the complex format needs to be processed; the other is in PDF picture format, which is generated by scanning paper fixed value single, and the text needs to be recognized by OCR technology. The traditional fixed value single data extraction relies on manual work, which is low in efficiency and prone to errors; in the existing automation technology, the extraction tool for PDF editable format (such as Adobe Acrobat text extraction function) cannot recognize the specific area of the fixed value single and the data correlation, and the general OCR tool for PDF picture format (such as Baidu OCR and Tencent OCR) is not adapted to the font and layout of the power industry, and the recognition accuracy cannot meet the requirements, which seriously restricts the automation process of fixed value checking.

[0004] Chinese patent application No. CN202510054966.1 discloses a power system relay protection fixed value single information extraction method and related device. The invention analyzes the global layout of the relay protection fixed value single image content, recognizes the logical partitions in the relay protection fixed value single image, including the title area, the fixed value table area and the signature area, extracts the boundary information of the logical partitions, and generates a partition framework graph; extracts the text area of each logical partition, recognizes the text content in the area, and records the spatial position information of each character in the text; arranges the text content according to the spatial position information of the characters, and constructs the text sequence in the logical partition; combines the context information across the logical partitions, merges the text sequence in the logical partition, and generates a continuous text stream; classifies the text sequence according to the semantics, generates semantic entities, and constructs the hierarchical relationship of the semantic entities; constructs the logical relationship between the semantic entities through a graph neural network, converts it into structured data, and extracts the information of the relay protection fixed value single.

[0005] However, the existing technology still has the following problems:

[0006] The logical partition identification robustness is insufficient, the partition error rate is high, the text space position dependence causes the cross-region information integration failure, the logical modeling of the graph neural network is excessively complicated, the efficiency is low, and the lack of targeted quality enhancement module causes the low-quality image recognition effect to be poor. SUMMARY

[0007] To this end, the application provides a fixed value sheet analysis and information extraction method to overcome the problems of insufficient robustness of logical partition identification, high partition error rate, text space position dependence causing cross-region information integration failure, excessive complication of logical modeling of graph neural network causing low efficiency, and lack of targeted quality enhancement module causing poor low-quality image recognition effect in the prior art.

[0008] To achieve the above-mentioned purpose, the application provides a fixed value sheet analysis and information extraction method. It comprises:

[0009] Step S1, receiving a fixed value sheet file, identifying the type of the fixed value sheet file, including: PDF editable format and PDF picture format;

[0010] Step S2, determining the processing mode for the fixed value sheet file based on the type of the fixed value sheet file, comprising:

[0011] If the type of the fixed value sheet file is the PDF editable format, the text stream, font attribute and coordinate information are parsed, the text area is divided based on the font size and coordinate distribution, and the structured data is generated through template matching;

[0012] If the type of the fixed value sheet file is the PDF picture format, distortion correction and quality enhancement are performed through an image preprocessing unit, the text is recognized by using an optimized OCR technology, and the recognition result is corrected in combination with the power fixed value logic rules;

[0013] Step S3, performing semantic analysis on each of the structured data, extracting public region information and fixed value region information, performing semantic verification and correction, and obtaining fixed value data;

[0014] Step S4, outputting the fixed value data in a preset format.

[0015] Further, in the step S1, it further comprises:

[0016] Step S11, receiving a fixed value sheet file, performing integrity check and storage management, and triggering an alarm for damaged or illegal format files;

[0017] Step S12, preliminarily determining the format type by analyzing the text stream and picture resource proportion in the file metadata;

[0018] Step S13, text extraction test is performed on the mixed format file, and the final format type is determined according to the extraction success rate or manual assistance and distribution.

[0019] Further, in the step S12, the preliminary determination of the format type includes:

[0020] If the specification file contains an extractable text stream, and the picture resource proportion is less than a first preset picture resource proportion, it is preliminarily determined that the type of the specification file is a PDF editable format.

[0021] If the specification file does not contain a text stream, or the picture resource proportion is greater than or equal to a second preset picture resource proportion, it is preliminarily determined that the type of the specification file is a PDF picture format.

[0022] If the specification file contains an unextractable text stream, and the picture resource proportion is greater than or equal to the first preset picture resource proportion and less than the second preset picture resource proportion, the specification file is marked as a mixed format file, and further verification is performed on the mixed format file.

[0023] Further, in the step S13, the determination of the final format type according to the extraction success rate or manual assistance and distribution includes:

[0024] Text extraction test is performed on the mixed format file, the ratio of the number of successfully extracted characters to the total number of extracted characters is calculated, the extraction success rate is obtained, and if the extraction success rate is greater than or equal to a first preset success rate, it is determined that the mixed format file is a PDF editable format.

[0025] If the extraction success rate is less than a second preset success rate, it is determined that the mixed format file is a PDF picture format.

[0026] If the extraction success rate is greater than or equal to the second preset success rate and less than the first preset success rate, manual assistance is triggered to determine the type of the mixed format file.

[0027] Further, in the step S2, the parsing of the text stream, font attribute and coordinate information, the division of the text area based on the font size and coordinate distribution, and the generation of structured data through template matching include:

[0028] Step S211, the page structure of the PDF file is parsed, the text content, font attribute and coordinate information are extracted, and a page number-text content-font type-font size-coordinate correlation data table is generated.

[0029] Step S212, the title area, public area and specification area are identified based on the font size and coordinate distribution, and the table text is divided according to the column coordinates.

[0030] Step S213, complete the missing information by matching the field through the fixed value single template library, and check the value and unit rationality based on the power fixed value rules.

[0031] Further, in step S2, the distortion correction and quality enhancement by the image preprocessing unit, the text recognition using the optimized OCR technology, and the correction of the recognition result combined with the power fixed value logical rules include:

[0032] Step S221, convert the PDF picture format file into a grayscale image, correct the distortion through the perspective transformation algorithm, and improve the image quality by filtering, histogram equalization, and stain repair;

[0033] Step S222, detect the text line region through the YOLOv5 model, and sort the text lines according to the coordinates;

[0034] Step S223, recognize the text content through the CRNN model, and correct similar characters combined with the power special character library;

[0035] Step S224, splice the text lines and divide the fields according to the sequence number-fixed value name-value-unit structure, and mark the abnormal data.

[0036] Further, in step S221, the distortion correction through the perspective transformation algorithm includes:

[0037] Detect the image edge through the Canny operator, and identify the four corner coordinates of the fixed value single;

[0038] Calculate the perspective transformation matrix, correct the tilted image to a horizontal rectangle, and the tilt error is less than or equal to the preset tilt error.

[0039] Further, in step S3, the semantic verification and correction include:

[0040] Based on the power fixed value semantic rule library, logically verify the extracted public region information and fixed value region information;

[0041] The logical verification includes at least one of unit verification, value range verification, and integrity verification;

[0042] If the verification finds data abnormalities, automatically correct the abnormal data or mark it for manual confirmation.

[0043] Further, in step S3, the unit verification includes determining the corresponding standard unit according to the fixed value name, and if the identified unit does not match the standard unit, performing unit correction or marking the unit as abnormal;

[0044] The numerical range check comprises judging whether the numerical value is within a preset reasonable numerical range according to the fixed value type, and marking as a numerical value out of range if the numerical value is out of the reasonable numerical range;

[0045] The integrity check comprises counting a field missing rate in the fixed value data, and triggering a data missing alarm if the missing rate exceeds a preset threshold.

[0046] Further, in step S4, the outputting the fixed value data in a preset format comprises:

[0047] The fixed value data is converted into a plurality of standard formats for output, and the standard formats comprise an XML format, an Excel format and a JSON format.

[0048] The output data is attached with metadata, and the metadata comprises at least one of extraction time, format type, recognition accuracy and check pass state.

[0049] The fixed value data is pushed to a fixed value checking system through a standard data interface.

[0050] Compared with the prior art, the application has the beneficial effects that the application supports both local import and remote transmission modes, adapts to the file acquisition needs of different scenarios in the substation, enhances the operation flexibility, the remote transmission adopts the HTTPS encryption protocol, ensures that the file transmission process between the dispatching center and the substation is not tampered with or stolen, meets the high requirements of the power system on data security, the transmission rate ≥ 10 Mbps ensures the efficiency of file transmission, avoids the influence of transmission delay on work progress, the local import supports 1-50 files at a time, meets the batch processing needs, reduces repeated operations, and improves work efficiency, the integrity check is performed through MD5 hash value comparison, which can accurately identify damaged files, the legal format is limited to PDF, and illegal format files can be quickly screened out, the combination of the two avoids invalid or error files from entering the system from the source, reduces abnormal problems in subsequent processing, triggers a pop-up window alarm for damaged or illegal files and records error logs, which is convenient for staff to handle in time and provides a basis for subsequent tracing of problem causes, forming a closed-loop management, complete and legal files are named according to the format of "device model_value sheet number_receiving time" and stored in the local database, the naming rule is clear and contains key information, which is convenient for staff to quickly locate specific files, greatly improves the efficiency of subsequent query and tracing, and is especially suitable for the management scene of a large number of value sheet files in the substation, the format type is preliminarily determined by analyzing the text flow and picture resource proportion of the metadata, and step S13 further determines the mixed format file through extraction test or manual assistance, ensuring the accuracy of format type identification. This provides a reliable basis for subsequent value sheet file analysis, verification and other processing links, avoids analysis failure or information extraction deviation caused by format determination error, and guarantees the effectiveness of the entire value sheet processing process.

[0051] Further, the present application is based on the core object of ApachePDFBox tool to parse PDF files, and classifies through quantitative indicators instead of subjective judgment, avoids the error of manual recognition, and subdivides the PDF format into three categories of editable format, picture format and mixed format, which covers the diversified forms of the fixed value single file in the actual scene: the editable format file can be directly extracted and automatically processed, reducing manual intervention; the picture format file needs subsequent special processing such as OCR recognition, and the advance marking can avoid invalid text extraction operation; the mixed format is marked for further verification to avoid misjudgment as editable or pure picture format; the different format types are screened through quantitative indicators in advance, so that the subsequent process can be adapted as needed, the editable format directly calls the efficient text extraction tool, the picture format preferentially starts the OCR module, and the mixed format triggers targeted verification, which greatly improves the efficiency of the whole process; for special mixed scenarios, the secondary verification is included through the marking mechanism, which solves the problem that the traditional single determination logic is difficult to cover the edge cases, ensures that even the fixed value single file with irregular format can be recognized and properly processed by the system, and improves the robustness of the system in the actual complex scene.

[0052] Further, the present application extracts pages randomly and combines fixed character extraction test method, uses quantitative extraction success rate as the core determination basis, avoids subjective speculation on mixed format files, triggers manual auxiliary determination in the middle interval, compensates for the limitations of pure machine determination under complex format, and corrects possible deviations through manual experience, which greatly improves the reliability of the final format determination; the files are accurately distributed to the corresponding module according to the determination result, avoiding the waste of resources caused by mismatch processing, and the update of the format type field in the database provides clear labeled data for subsequent tracing, statistics or secondary processing, which is convenient for the system to differentially manage different format files; the mixed format file is often an edge case in fixed value single management, and through the step-by-step determination logic, it is ensured that such files are not simply classified as a certain type, but are selected according to the actual content characteristics to choose the optimal processing path.

[0053] Further, the present application extracts text content, font attributes and coordinate information by parsing page trees, media boxes and content streams, and generates structured data tables, while eliminating graphic elements (table lines, logos) and redundant white space characters, ensuring that subsequent processing focuses on effective text; through device model matching template library, field standardization mapping is realized, avoiding information confusion caused by non-uniform terminology; the finally generated JSON format intermediate data table converts unstructured PDF text into machine recognizable key-value pair data, providing a unified interface for subsequent fixed value comparison, verification, storage and calling.

[0054] Further, the present application improves image clarity from the source through the whole process of gray scale conversion-distortion correction-quality enhancement, clears obstacles for subsequent OCR recognition, Gaussian filter and median filter are respectively for high frequency noise and salt and pepper noise, stain repair eliminates isolated stains, avoids noise points or stains from being misrecognized as characters, CLAHE algorithm through sub-block adaptive equalization greatly improves the gray scale difference between text and background, even low contrast scanning, can also clearly distinguish the text edge, reduce the recognition missed rate, unified conversion to 300DPI gray scale image, through anti-aliasing algorithm to smooth the text edge, ensure that different scanning quality files have consistent image basis when entering the recognition link, reduce the recognition deviation caused by image difference; the detection scheme based on YOLOv5 model solves the problem of irregular distribution and large size difference of single Chinese text line, ensures the consistency of recognition order and actual layout; the CRNN model combines with the power special character library, solves the pain points of power field character recognition, and significantly reduces the misrecognition rate; through regularized segmentation and abnormality detection, the recognized text is converted into standardized data, and the problem data is exposed in advance, reducing the risk of subsequent processing.

[0055] Further, the present application solves the inconsistent expression problem in the fixed value sheet through format unification and synonymous conversion, lays a foundation for subsequent data comparison and management, standardizes the date, variable ratio, device model and other fields, eliminates the format confusion caused by different input habits, and ensures that the data can be uniformly recognized and searched in the system; based on the synonym library, the same value different name problem caused by different terminology expressions is avoided, especially the name consistency with the standard fixed value sheet of the dispatching center is ensured, which clears obstacles for subsequent cross-system data comparison; through three-level verification of unit, value range and integrity, invalid or error data is filtered from multiple dimensions to ensure that the fixed value data meets the professional specifications of the power system; based on the association rules of fixed value type and name, obvious errors are automatically corrected, and the abnormality that cannot be determined is manually confirmed to avoid the misadjustment of the protection device caused by unit error; through semantic logic, scattered data is associated and missing fields are completed, which greatly improves the data integrity and reduces the manual arrangement cost.

[0056] Further, the present application meets the needs of different users and systems by providing three standard formats of XML, Excel and JSON, breaks the data silos; the XML format interfaces professional systems, follows the Q / GDW1161-2013 power industry specification, ensures seamless compatibility with the substation fixed value checking system, provides a standardized data interface for the automation checking and comparison of fixed values, and avoids system interface obstacles caused by inconsistent formats; the Excel format is convenient for manual interaction, divides three worksheets of public information, fixed value data and checking records, clearly presents structured data and checking results, adapts to scenarios such as on-site checking of operation and maintenance personnel, paper archiving and the like, and reduces the threshold of manual checking. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a work flow chart of the fixed value sheet analysis and information extraction method of the embodiment of the present application;

[0058] Figure 2 is a work flow chart of identifying the type of the fixed value sheet file in the fixed value sheet analysis and information extraction method of the embodiment of the present application;

[0059] Figure 3 is a work flow chart of generating structured data through template matching in the fixed value sheet analysis and information extraction method of the embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the objectives and advantages of the present application clearer and more apparent, the present application will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0061] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the protection scope of the present application.

[0062] Please refer to Figures 1-3 as shown, Figure 1 is a work flow chart of the fixed value sheet analysis and information extraction method of the embodiment of the present application; Figure 2 is a work flow chart of identifying the type of the fixed value sheet file in the fixed value sheet analysis and information extraction method of the embodiment of the present application; Figure 3 is a work flow chart of generating structured data through template matching in the fixed value sheet analysis and information extraction method of the embodiment of the present application.

[0063] The technical scheme provided by the embodiment of the present application includes the following steps:

[0064] Step S1, receiving a fixed value single file, identifying the type of the fixed value single file, including: PDF editable format and PDF picture format;

[0065] Step S2, determining the processing mode for the fixed value single file based on the type of the fixed value single file, including:

[0066] If the type of the fixed value single file is PDF editable format, the text stream, font attribute and coordinate information thereof are parsed, the text area is divided based on the font size and coordinate distribution, and the structured data is generated through template matching;

[0067] If the type of the fixed value single file is PDF picture format, distortion correction and quality enhancement are performed through an image preprocessing unit, the text is recognized by using an optimized OCR technology, and the recognition result is corrected in combination with power fixed value logical rules;

[0068] Step S3, performing semantic analysis on each structured data, extracting public area information and fixed value area information, performing semantic verification and correction, and obtaining fixed value data;

[0069] Step S4, outputting the fixed value data in a preset format.

[0070] Specifically, in the step S1, the following steps are further included:

[0071] Step S11, receiving a fixed value single file, performing integrity check and storage management, and triggering an alarm for a damaged or illegal format file;

[0072] Step S12, preliminarily determining the format type by analyzing the text stream and picture resource proportion in the file metadata;

[0073] Step S13, performing text extraction test on a mixed format file, determining the final format type according to the extraction success rate or manual assistance, and distributing.

[0074] In the embodiment of the application, the receiving of the fixed value single file includes supporting two receiving modes, one is local import (reading the fixed value single file in the U disk through the USB interface, supporting importing 1-50 files at a time), and the other is remote transmission (receiving the file issued by the dispatching center through the transformer substation internal network, adopting HTTPS protocol for encrypted transmission, and the transmission rate is greater than or equal to 10 Mbps); the received file is subjected to integrity check by MD5 hash value comparison (consistent with the hash value issued by the dispatching center, then it is determined to be complete), if the file is damaged (the hash values do not match) or the format is illegal (not PDF format), an alarm is triggered immediately (a pop-up window prompts that the file is damaged / has format error, please reacquire), and an error log (including the file name, error type and timestamp) is recorded; the complete and legal PDF file is stored in the local database, the file name is named according to the device model_fixed value single number_receiving time format, so as to facilitate subsequent tracing.

[0075] The application supports both local import and remote transmission modes, adapts to the file acquisition needs of different scenarios in the substation, enhances operation flexibility, and uses HTTPS encryption protocol for remote transmission to ensure that the file transmission process between the dispatch center and the substation is not tampered with or stolen, meeting the high requirements of the power system for data security; the transmission rate ≥ 10 Mbps ensures the efficiency of file transmission, avoiding the impact of transmission delay on work progress; local import supports 1-50 files at a time, meeting the batch processing needs, reducing repetitive operations, and improving work efficiency; integrity checking through MD5 hash value comparison can accurately identify damaged files; clearly limit the legal format to PDF to quickly filter out illegal format files; the combination of the two avoids invalid or incorrect files from entering the system from the source, reduces abnormal problems in subsequent processing, triggers a pop-up alarm for damaged or illegal files and records error logs, which not only facilitates staff to handle in a timely manner, but also provides a basis for subsequent tracing of problem causes, forming a closed-loop management; complete and legal files are named according to the "device model_value sheet number_receiving time" format and stored in the local database, the naming rule is clear and contains key information, which facilitates staff to quickly locate specific files, greatly improving the efficiency of subsequent query and traceability, especially suitable for management scenarios of a large number of value sheet files in substations; by analyzing the text stream and picture resource proportion of the metadata, step S13 further determines the mixed format file through extraction testing or manual assistance to ensure the accuracy of format type identification. This provides a reliable basis for subsequent value sheet file parsing, checking and other processing links, avoids parsing failure or information extraction deviation caused by format determination error, and ensures the effectiveness of the entire value sheet processing process.

[0076] Specifically, in the step S12, the preliminary determination of the format type includes:

[0077] If the value sheet file contains an extractable text stream, and the picture resource proportion is less than a first preset picture resource proportion, it is preliminarily determined that the type of the value sheet file is a PDF editable format;

[0078] If the value sheet file does not contain a text stream, or the picture resource proportion is greater than or equal to a second preset picture resource proportion, it is preliminarily determined that the type of the value sheet file is a PDF picture format;

[0079] If the value sheet file contains an unextractable text stream, and the picture resource proportion is greater than or equal to the first preset picture resource proportion and less than the second preset picture resource proportion, the value sheet file is marked as a mixed format file, and the mixed format file is further verified.

[0080] In the embodiment of the present application, the PDF file analysis tool (based on ApachePDFBox development) is called to read the " / Contents" object (store text stream) and " / Resources / XObject" object (store picture resources) of the file. If the " / Contents" object contains an extractable text stream (text length >= 100 characters) and the picture resource proportion in the " / Resources / XObject" object is < 30% (first preset picture resource proportion), it is preliminarily determined as a PDF editable format. If the " / Contents" object has no text stream, or the picture resource proportion is >= 80% (second preset picture resource proportion), it is preliminarily determined as a PDF picture format. Special case marking: if the text stream length is 10-99 characters (non-extractable text stream) and the picture resource proportion is 30%-79% (mixed format), it is marked as to be further verified.

[0081] The present application is based on the core object of ApachePDFBox tool for analyzing PDF files, and classifies through quantitative indicators instead of subjective judgment, avoiding the error of manual recognition. The PDF format is subdivided into editable format, picture format and mixed format, which covers the diversified forms of value determination single files in actual scenarios: the editable format file can be directly used for text extraction and automatic processing, reducing manual intervention; the picture format file needs subsequent special processing such as OCR recognition, and the advance marking can avoid invalid text extraction operation; the mixed format is marked as to be further verified, avoiding misjudgment as editable or pure picture format; different format types are screened out in advance through quantitative indicators, so that the subsequent process can be adapted as needed. For the editable format, efficient text extraction tools are directly called; for the picture format, the OCR module is preferentially started; for the mixed format, targeted verification is triggered, which greatly improves the efficiency of the whole process; for special mixed scenarios, the marking mechanism is used for secondary verification, solving the problem that traditional single determination logic cannot cover edge cases, ensuring that even the format of the value determination single file is not standardized, it can also be recognized and properly processed by the system, improving the robustness of the system in actual complex scenarios.

[0082] Specifically, in the step S13, the final format type is determined according to the extraction success rate or manual assistance, and the distribution includes:

[0083] For the mixed format file, the ratio of the number of successfully extracted characters to the total number of extracted characters is calculated to obtain the extraction success rate. If the extraction success rate is greater than or equal to the first preset success rate, the mixed format file is determined as a PDF editable format.

[0084] If the extraction success rate is less than the second preset success rate, the mixed format file is determined as a PDF picture format.

[0085] If the extraction success rate is greater than or equal to the second preset success rate and less than the first preset success rate, manual auxiliary determination is triggered to determine the type of the mixed format file.

[0086] In the embodiment of the application, the text extraction test comprises: for the file to be further verified, randomly selecting 3 pages (if less than 3 pages, all pages are selected), extracting 20 continuous characters from each page, and counting the extraction success rate (the number of successfully extracted characters / the total number of extracted characters); the determination rule is: if the extraction success rate is greater than or equal to 80% (the first preset success rate) (for example, 48 or more characters are successfully extracted from 60 characters), it is determined that the PDF is in an editable format (the proportion of editable text in the mixed format is high, and the editable format is preferred); if the extraction success rate is less than 50% (the second preset success rate), it is determined that the PDF is in a picture format; if the success rate is between 50% and 79%, manual auxiliary determination is triggered (a pop-up window displays the extracted text segment, and an operation and maintenance personnel selects "process as an editable format" or "process as a picture format"); format distribution: the file determined to be in the PDF editable format is pushed to the PDF editable format analysis module, the PDF picture format is pushed to the PDF picture format recognition and extraction module, and the format type field of the file in the database is updated.

[0087] The test method of the application extracts pages randomly and extracts fixed characters, uses the quantitative extraction success rate as the core determination basis, avoids subjective speculation of the mixed format file, triggers manual auxiliary determination for the intermediate interval, compensates for the limitations of pure machine determination under complex formats, corrects possible deviations through manual experience, greatly improves the reliability of the final format determination, accurately distributes the file to the corresponding module according to the determination result, avoids resource waste caused by mismatched processing, updates the format type field in the database, provides clear labeled data for subsequent tracing, statistics or secondary processing, and facilitates the system to manage different format files differently; the mixed format file is often an edge case in fixed value single management, and through the step-by-step determination logic, the file is not simply classified as a certain type, but the optimal processing path is selected according to the actual content characteristics.

[0088] Specifically, in the step S2, the text stream, font attribute and coordinate information are analyzed, the text area is divided based on the font size and coordinate distribution, and the structured data is generated through template matching, including:

[0089] In step S211, the page structure of the PDF file is analyzed, the text content, font attribute and coordinate information are extracted, and the page number-text content-font type-font size-coordinate correlation data table is generated.

[0090] In the embodiment of the application, the page tree ( / Pages) of the PDF file is read, the total number of pages (1-20 pages, and if exceeding, the "file page number is abnormal" is prompted), for each page, the media box (MediaBox, defining the page size, such as A4 paper [0, 0, 595, 842]) and the content stream (text and graphic elements are parsed according to the drawing order) are parsed; for the text stream of each page, the text content, font information (font type such as "SimHei" "Arial", font size such as 10pt, 12pt, 16pt), text coordinates (taking the lower left corner of the page as the origin, the x-axis is horizontal to the right, the y-axis is vertical upward, and the accuracy is 0.1pt) are extracted, and a structured data table of page number-text content-font type-font size-x coordinate-y coordinate is generated, the graphic elements (such as table lines, logo icons) and blank characters (text fragments with continuous spaces ≥3) in the page are removed, and the pure text elements are retained to reduce the interference in subsequent processing.

[0091] In step S212, the title area, the public area and the fixed value area are identified based on the font size and the coordinate distribution, and the table text is segmented according to the column coordinates.

[0092] In the embodiment of the application, the title area is identified: the text with font size ≥14pt and y coordinate ≥750pt (top area of A4 paper) is screened, sorted according to x coordinate from small to large, and spliced into a fixed value single title (such as "Jiaxing power grid relay protection setting sheet"), if there are multiple texts meeting the conditions (such as the title being divided into two lines), the texts are spliced according to y coordinate from high to low; the public area is identified: the text with font size 10-12pt and y coordinate 650-740pt is screened, and the public information is extracted according to the "keyword + content" mode (the keywords are such as "protected equipment:", "device model:", "CT ratio:", "preparation date:"), for example, in the text "device model: CSC-211", "device model:" is the keyword, and "CSC-211" is the content, which are stored in the public information data table; the fixed value area is identified: the text with font size 10pt and y coordinate 100-640pt is screened, sorted according to y coordinate from high to low (from top to bottom), and whether there is a table structure (the text x coordinate is regularly distributed, such as the serial number column x = 80 ± 5pt, the fixed value name column x = 120 ± 5pt) is judged, if there is a table, the text is segmented according to the column coordinates (the serial number column x ≤ 100pt, the fixed value name column 100pt < x ≤ 300pt, the numerical value column 300pt < x ≤ 380pt, and the unit column 380pt < x ≤ 420pt); if there is no table (pure text layout), the text is segmented according to the semantic rule of "serial number + fixed value name + numerical value + unit" (such as "1 change amount starting current fixed value 0.08A" is divided into serial number "1", name "change amount starting current fixed value", numerical value "0.08", and unit "A").

[0093] Step S213, complete the missing information by matching the field of the value single template library, and check the value and unit rationality based on the power value rule.

[0094] In the embodiment of the application, template matching: calling a PDF editable format value single template library (containing 20+ mainstream device templates, such as Nanrui PCS-915 and Sifang CSC-211), matching the corresponding template according to the "device model" keyword (such as "CSC-211" matching the Sifang line protection template), and defining the standard name of the public information field (such as "value single number" and "execution date") and the value field (such as "zero sequence starting current value" and "ground distance I section value");

[0095] The application systematically extracts text content, font attribute and coordinate information by analyzing the page tree, media box and content stream, and generates a structured data table, while eliminating graphic elements (table lines, logos) and redundant white space characters, so as to ensure that the subsequent processing focuses on the effective text; the device model is matched with the template library to realize the standardized mapping of the field, so as to avoid the information confusion caused by the non-uniform terms; and finally, the generated JSON format intermediate data table converts the unstructured PDF text into machine recognizable key-value pair data, so as to provide a unified interface for the subsequent value comparison, checking, storage and calling.

[0096] Specifically, in step S2, the distortion correction and quality enhancement by the image preprocessing unit, the text is recognized by using the optimized OCR technology, and the recognition result is corrected in combination with the power value logic rule, including:

[0097] Step S221, converting the PDF picture format file into a gray scale image, correcting the distortion by using a perspective transformation algorithm, and improving the image quality by using filtering, histogram equalization and stain repair;

[0098] In the embodiment of the present application, each page of the PDF picture format file is converted into a PNG image, the resolution is set to 300 DPI (dots per inch), and the color mode is set to grayscale (to avoid color noise interference). In the conversion process, an anti-aliasing algorithm is used to smooth the text edges through pixel interpolation, ensuring that the text pixel transition is uniform, and the formula is as follows:

[0099] Gray value calculation: for the RGB color pixels in the original PDF picture, the gray value is converted according to the sensitivity of the human eye to the three primary colors , and the formula is as follows:

[0100] ;

[0101] Among them, is the red channel pixel value (0-255), is the green channel pixel value (0-255), is the blue channel pixel value (0-255); the converted gray image only retains the single-channel gray value (0-255, 0 for pure black and 255 for pure white).

[0102] In the embodiment of the present application, improving the image quality includes: noise removal: Gaussian noise removal: using a 3x3 Gaussian filter, to filter high-frequency noise; salt and pepper noise removal: using a 3x3 median filter, the gray values of 9 pixels in the neighborhood are sorted, and the median value is taken as the output value, and the formula is as follows:

[0103] ;

[0104] Among them, is the median function, which can effectively remove isolated black and white noise points;

[0105] Contrast enhancement (CLAHE algorithm): for low-contrast images (gray value variance , variance formula , is the gray mean value), the contrast is enhanced using adaptive histogram equalization: first, the image is divided into sub-blocks (tileGridSize=(8,8)), and each sub-block has a size of ( , );

[0106] Second, the gray histogram of each sub-block is calculated ( , indicating the number of pixels with a gray level of );

[0107] ​Third step: clipping histogram, setting clipping threshold The number of pixels exceeding the threshold is evenly distributed to other gray levels, and the formula is:

[0108]

[0109] After distribution, it meets ;

[0110] Fourth step: calculate the cumulative distribution function of each sub-block , and map the gray value ;

[0111] Fifth step: for the overlapping area (overlapping rate 50%) of adjacent sub-blocks, bilinear interpolation is adopted to avoid obvious gray mutation at the sub-block boundary; after processing, the image gray value variance , the text and background gray difference is significantly improved;

[0112] Stain repair (fast marching method): for the stain area (gray value mutation and area pixels²) in the image, the stain area is repaired by the fast marching method:

[0113] First step: mark the stain area , which meets ( is the gray difference threshold, is neighborhood 3*3 mean value);

[0114] Second step: determine the repair boundary , the boundary pixel gray value is kept unchanged;

[0115] Third step: define the energy function ( is the level set function), minimize the energy function by gradient descent method, and iteratively update the pixel value of the stain area:

[0116] ;

[0117] Wherein, is the step size, is the iteration number, until the gray value of the stain area is consistent with the surrounding background (the gray difference ).

[0118] Step S222, detect the text line area by the YOLOv5 model, and sort the text lines according to the coordinates;

[0119] In the embodiment of the application, the YOLOv5 model is used to detect the text line area, including: loading a pre-trained YOLOv5 text line detection model (based on 50,000+ power fixed value single picture training, input size ), the model core parameters are as follows:

[0120] Input image size: (scale the preprocessed PNG image to 640x640, and fill the blank area with a gray value of 128);

[0121] Confidence threshold: (filter low-confidence bounding boxes);

[0122] Non-maximum suppression (NMS) threshold: (merge bounding boxes with an overlap rate of >45%);

[0123] Anchor size: based on the power constant value single text line geometric feature clustering, the size is (widthxheight, unit: pixel), which is suitable for text lines with different lengths and heights;

[0124] The YOLOv5 model adopts a "backbone-neck-head" structure, and the text line detection process is as follows:

[0125] Feature extraction (Backbone: CSPDarknet53): the input image is down-sampled 5 times (step 2) to generate 5 scale feature maps , , , , , , , , , , the number of channels of each feature map is 256, 512, 1024, 2048, and 2048 respectively, and the feature expression ability is improved through residual connection and cross-stage local network (CSP);

[0126] Feature fusion (Neck: PANet): adopt bidirectional feature fusion from top to bottom and from bottom to top: from top to bottom: after up-sampling (step 2) and splicing, generate ; after up-sampling and splicing, generate ; from bottom to top: after down-sampling (step 2) and splicing, generate ; after down-sampling and splicing, generate Finally output 3 fusion feature maps cover the detection requirements of small, medium and large size text lines

[0127] Head (detection layer): each fusion feature map corresponds to 3 detection layers, and each detection layer outputs a prediction tensor , the dimension is ( is the batch size, =18, 3 is the anchor box number, 5 is the boundary box parameter, and 1 is the text line category probability): boundary box parameter: , wherein is an anchor box center offset, is an anchor box width-height scaling factor, is a confidence; boundary box coordinate conversion: convert the prediction parameters into boundary box coordinates in the image coordinate system , the formula is:

[0128] , , , ;

[0129] , , , ;

[0130] wherein is a Sigmoid function (mapping the offset to 0-1), is an anchor box grid center coordinate, is an anchor box original width-height; category probability: calculate the text line category probability through the Sigmoid function , determine as a text line detection box.

[0131] In the embodiment of the application, the text line is sorted according to the coordinates, including: sorting the coordinates (upper left y coordinate) from large to small (from top to bottom, because the image coordinate system takes the upper left corner as the origin, axis downward), if the difference value of two detection boxes is pixels (same horizontal height), the coordinates (upper left x coordinate) are sorted from small to large (from left to right), and the sorting formula is: define the comparison function ( , ): if​​​​​​​ Then When Rank in front; if Then When Rank in front; the order of the text lines is consistent with the actual layout of the fixed value sheet after sorting.

[0132] In the embodiment of the application, the text content is recognized by the CRNN model, and similar characters are corrected in combination with the power fixed value sheet special character library.

[0133] In the embodiment of the application, the text content is recognized by the CRNN model, and similar characters are corrected in combination with the power fixed value sheet special character library. , 32 is the height, and 256 is the width), the model structure and parameters are as follows: a convolution layer (CNN): 5 convolution layers, the output channel numbers are 64, 128, 256, 256 and 512 respectively, the convolution kernel size is 3*3, the step is 1, and padding=1; two layers of maximum pooling (pooling kernel 2*2, step 2), and the final output feature map size is (height 1, width 8, channel 512); a recurrent layer (RNN): two layers of bidirectional LSTM (long short-term memory network), the hidden layer dimension is 256, and the output dimension is 512 (bidirectional splicing); a transcription layer (CTC): a connectionist temporal classifier, the sequence (length 8, dimension 512) output by the LSTM is input, and the character category probability distribution (length 8, dimension 2036) is output, so as to realize alignment-free text recognition; according to the text line detection frame output , the text line region is cropped from the preprocessed image, and the cropping coordinates are (outward expansion of 2 pixels, to avoid the text edge being cut off); size normalization: the cropped text line image is scaled to , if the original width-height ratio ( , ) > 1 , then scale by width 256, and height by proportion, then fill in the middle to 32; if , then scale by height 32, and width by proportion, then fill in the middle to 256; the fill area gray value is 128; pixel normalization: map the gray value (0-255) to , the formula is:

[0134]

[0135] Eliminate the influence of pixel value magnitude difference on model training and inference;

[0136] CNN feature extraction: input the normalized text line image into the CRNN model, and output the feature map with a dimension of , where "1" corresponds to the image height direction (compressed to 1), "8" corresponds to the image width direction (256 / 32=8, 32 is the step after pooling), and "512" is the number of feature channels, each channel corresponds to the local features of the text line (such as stroke edges, character structures); RNN sequence modeling: expand in the width direction to sequence ( , ), input 2-layer bidirectional LSTM: forward LSTM: model sequence dependence from to , output ( ); backward LSTM: model sequence dependence from to , output ( ); concatenate the bidirectional output: ( ), capture the context features of the text line; CTC transcription and probability calculation: fully connected layer: map to character category probability ( is the weight matrix, is the bias, is the normalization function), represents the probability of predicting character at the th time step; CTC decoding: use beam search (beamwidth=10) to decode the probability sequence , eliminate repeated characters and white spaces (placeholders introduced by CTC), output the recognized text sequence ( is the text length), and calculate the recognition confidence (the average maximum probability).

[0137] In the embodiment of the present application, the modification of similar characters in combination with the power special font library includes: for a character with a recognition confidence , calling a similar character feature library for modification, and the flow is as follows: feature extraction: extracting local features of a suspected character (such as the bottom hook area of "1" and the top flat area of "I") from a text line clipping image, and calculating a feature vector (dimension 128, containing edge direction, stroke length, curvature, etc.); similarity calculation: calculating the cosine similarity of each character feature vector in the similar character feature library :

[0138]

[0139] wherein, is the dot product, is the L2 norm;

[0140] modification determination: selecting a character with the maximum similarity ( ), if , the suspected character is modified to ; if , combined with context semantic modification (for example, "grounding distance constant value" followed by "10" is reasonable, followed by "I0" is unreasonable, and "I0" is modified to "10").

[0141] Step S224: splice the text line and divide the field according to the sequence-number-constant name-value-unit structure, and mark the abnormal data.

[0142] In the embodiment of the present application, the splicing of text lines and the segmentation of fields according to the sequence-number-value name-number-unit structure includes: text splicing: according to the sequence of the text line sorting result in step 3.2.3, the identified text of each line is spliced in sequence, and if there is a sequence number correlation between two adjacent lines (for example, the first line "1 change amount starting current value" and the second line "0.08A"), the two lines are spliced into a complete text "1 change amount starting current value 0.08A"; during splicing, blank characters (spaces, line breaks) are removed, and valid text is retained; field segmentation: based on the power value layout rule, the field is segmented according to the "sequence number-value name-number-unit" structure, and a regular expression is used for matching: sequence number: 1-2 digits; value name: string containing Chinese characters, letters and numbers; number: integer or decimal; unit: commonly used power units (A, kV, Ω, s, ms, etc., matching mode: (A|kV|Ω|s|ms)); for example, the text "2 zero sequence starting current value 0.08A" is segmented into: sequence number "2", value name "zero sequence starting current value", number "0.08", and unit "A"; marking abnormal data includes: field missing: if "number" or "unit" is missing after segmentation, it is marked as "field missing, to be checked"; number abnormality: if the number exceeds the conventional range of power value (for example, the "ground distance value" number > 20Ω), it is marked as "number abnormality (exceeding the conventional range)"; unit abnormality: if the unit does not match the value name (for example, the unit of "current value" is "kV"), it is marked as "unit abnormality, to be confirmed"; the segmented value data and the abnormality mark are integrated to generate a JSON format data table, which is stored in the path and stored locally.

[0143] Specifically, in step S221, the distortion correction by the perspective transformation algorithm includes:

[0144] The image edges are detected by a Canny operator to identify the four corner coordinates of the value;

[0145] The perspective transformation matrix is calculated to correct the tilted image to a horizontal rectangle, and the tilt error is less than or equal to a preset tilt error.

[0146] In the embodiment of the present application, for the image distortion caused by scanning tilt, edge detection and perspective transformation are used to correct the distortion, and the specific process is as follows: edge detection (Canny operator): first, Gaussian filter is used to smooth the image and remove noise, and the pixel value of the filtered image is The formula is:

[0147] ;

[0148] wherein, is the pixel value of the original gray image at the coordinate, (kernelsize=3x3), (Gaussian function standard deviation), is a two-dimensional Gaussian function:

[0149] ;

[0150] Second step: Calculate image gradient (horizontal gradient , vertical gradient ), using Sobel operator:

[0151] , ;

[0152] Where, gradient amplitude , gradient direction (taking the value range 0-180°);

[0153] Third step: Non-maximum suppression, keep the local maximum pixel in the gradient direction, and eliminate non-edge pixels;

[0154] Fourth step: Double threshold segmentation, set low threshold , high threshold , gradient amplitude determined as strong edge, determined as weak edge (need to be connected with strong edge to be retained), determined as non-edge; The final output edge image, identify the four corner coordinates of the single value (left upper corner), (right upper corner), (left lower corner), (right lower corner), meet , , , ; Perspective transformation: If the corner line is not vertical (inclination angle meet ), the inclined image is corrected to a horizontal rectangle through the perspective transformation matrix. Set the target corrected four corner coordinates as (0, H), (W, H), (0, 0), (W, 0) (where is the image width, is the image height), the perspective transformation matrix satisfies:

[0155] ( );

[0156] Expand into a linear equation system :

[0157]

[0158] Substitute four groups of coordinates to get 8 equations, solve 8 unknowns , finally realize image correction through inverse matrix transformation, the text line is parallel to the horizontal edge of the image after correction, and the inclination error (Prescribed inclination error).

[0159] The present application improves image definition from the source through the whole process of gray scale conversion-distortion correction-quality enhancement, clears the obstacles for subsequent OCR recognition, Gaussian filter and median filter are respectively aimed at high frequency noise and salt and pepper noise, stain repair eliminates isolated stains, avoids noise points or stains from being misrecognized as characters, CLAHE algorithm realizes self-adaptive equalization through sub-block, greatly improves the gray difference between text and background, even if the scanning piece with low contrast, the text edge can also be clearly distinguished, the recognition omission rate is reduced, and the scanning piece is converted into 300DPI gray scale picture through anti-aliasing algorithm, the text edge is smoothed, so that the files with different scanning qualities have consistent image basis when entering the recognition link, and the recognition deviation caused by image difference is reduced; the detection scheme based on YOLOv5 model solves the problems of irregular distribution of single Chinese text line and large size difference, ensures that the recognition order is consistent with the actual layout; the CRNN model combines with the power special character library, and solves the pain points of character recognition in the power field, and significantly reduces the misrecognition rate; through regularized segmentation and abnormality detection, the recognized text is converted into standardized data, and the problem data is exposed in advance, and the risk of subsequent processing is reduced.

[0160] Specifically, in step S3, the semantic verification and correction comprises:

[0161] Based on the power value semantic rule library, the extracted public region information and value region information are logically verified;

[0162] The logical verification includes at least one of unit verification, numerical range verification and integrity verification;

[0163] If the verification finds data anomaly, the abnormal data is automatically corrected or marked for manual confirmation.

[0164] Specifically, in step S3, the unit verification includes judging the corresponding standard unit according to the value name, if the recognized unit does not match the standard unit, unit correction or unit anomaly marking is performed;

[0165] The numerical range verification includes judging whether the value is within the preset reasonable numerical range according to the value type, if it is out of the reasonable numerical range, it is marked as value out of range;

[0166] The integrity check includes a field missing rate in the statistical fixed value data, and if the missing rate exceeds a preset threshold, a data missing alarm is triggered.

[0167] In the embodiment of the application, the public area semantic template library (classified according to device types, such as line protection and bus protection) is called, for example, the "line protection" template contains the fields of "fixed value single number", "dispatching number", "PT transformation ratio" and the like, and compared with the extracted public information, if the "PT transformation ratio" is missing and the fixed value data contains "line voltage PT primary rated value 220 kV", the "220 kV" is automatically extracted as the "primary rated value" of the "PT transformation ratio" to complete the public information; format standardization: the date field (such as "2024.5.8") is uniformly converted into the "YYYY-MM-DD" format ("2024-05-08"); the transformation ratio field (such as "1250 / 1A") is removed of redundant characters and standardized as "1250 / 1"; the device model (such as "CSC211") is supplemented with a separator and standardized as "CSC-211"; fixed value type division: based on the keywords in the fixed value name, the fixed value type is divided, for example, the fixed value containing "current", "voltage", "impedance" and "time" is classified as "protection fixed value", the fixed value containing "CT", "PT" and "rated value" is classified as "equipment parameter fixed value", and the fixed value containing "control word" and "input / exit" is classified as "control word fixed value"; cross-line data association: if there is text line splitting in step S3 (such as "3 grounding distance I section fixed value" and "3.49 Ω" in two lines), through the serial number matching (the serial numbers of the two lines are both "3") and semantic logic ("grounding distance I section fixed value" should be followed by a value and a unit), the complete field (serial number "3", name "grounding distance I section fixed value", value "3.49", unit "Ω") is associated; synonymous name unification: a fixed value name synonym library (such as "change amount starting current" and "sudden change amount starting current" are synonyms, and "zero sequence voltage blocking fixed value" and "zero voltage blocking fixed value" are synonyms) is established, and the extracted fixed value name is standardized, for example, "sudden change amount starting current fixed value" is uniformly standardized as "change amount starting current fixed value" to ensure consistency with the dispatching fixed value single name; unit verification: based on the fixed value type and name, the unit rationality is verified, for example, the unit of "current fixed value" should be "A", if it is "V", it is automatically corrected to "A"; the unit of "time fixed value" should be "s" or "ms", if it is "Ω", it is marked as "unit abnormal, manual confirmation is required"; value range verification: according to the fixed value type, a value range library is established (such as "110 kV line change amount starting current fixed value range 0.05-0.2 A" and "grounding distance I section fixed value range 1-5 Ω"), if the extracted value is out of range (such as "change amount starting current fixed value 0.3 A"), it is marked as "value out of range, it is recommended to check"; integrity verification: the missing rate of the fields in the fixed value data (missing field number / total field number) is counted, if the missing rate is <5%, it is determined as "qualified"; 5%-10% is determined as "partially missing"; >10% is determined as "serious missing", triggering the alarm prompt "too much fixed value data missing, re-extraction is required".

[0168] The application solves the inconsistent expression problem in the fixed value sheet through format unification and synonym conversion, lays a foundation for subsequent data comparison and management, standardizes the date, variable ratio, device model and other fields, eliminates the format confusion caused by different input habits, ensures that the data can be uniformly identified and retrieved in the system, avoids the same value different name problem caused by different terminology expressions based on the synonym library, especially ensures the name consistency with the standard fixed value sheet of the dispatch center, clears the obstacles for subsequent cross-system data comparison, filters invalid or error data from multiple dimensions through three-level verification of unit, value range and integrity, ensures that the fixed value data meets the professional specifications of the power system, automatically corrects obvious errors based on the association rules of fixed value type and name, manually confirms the abnormal labels that cannot be determined, and avoids the misadjustment of the protection device caused by unit error, and through semantic logic, the scattered data is associated, the missing fields are completed, the data integrity is greatly improved, and the manual sorting cost is reduced.

[0169] Specifically, in step S4, the fixed value data is output in a preset format, including:

[0170] The fixed value data is converted into multiple standard formats, including XML format, Excel format and JSON format;

[0171] The output data is attached with metadata, and the metadata includes at least one of extraction time, format type, identification accuracy and verification passing state.

[0172] The fixed value data is pushed to a fixed value checking system through a standard data interface.

[0173] In the embodiment of the application, the fixed value data after semantic analysis is converted into three standard formats, one is XML format (in line with Q / GDW1161-2013 specification, used for interfacing with the substation fixed value checking system), two is Excel format (including "public information", "fixed value data" and "verification record" three workbooks, used for manual checking), and three is JSON format (used for internal data transmission of the system); format example (XML): local output: store the converted file to the local specified path, and generate an output log (including file name, format, output time, file size); remote push: through IEC61850 protocol or MQTT protocol, the XML format data is pushed to the substation fixed value checking system, and data integrity verification is performed during the pushing process (if the receiving party feedbacks "received complete", it is determined to be successful, otherwise, it is retried for 3 times, and the retry interval is 1s); manual export: support connecting the system through a mobile terminal (tablet / mobile phone), export the Excel format file (transmitted through Wi-Fi or Bluetooth), and facilitate the on-site checking of the operation and maintenance personnel.

[0174] The application meets the needs of different users and systems by providing three standard formats of XML, Excel and JSON, and breaks the data island; the XML format is connected with professional systems and follows the power industry specification Q / GDW1161-2013 to ensure seamless compatibility with the substation fixed value checking system, provides a standardized data interface for the automatic checking and comparison of fixed values, and avoids system connection obstacles caused by inconsistent formats; the Excel format is convenient for manual interaction, has three worksheets of public information, fixed value data and checking records, clearly presents structured data and checking results, is suitable for scenarios such as on-site checking of operation and maintenance personnel and paper archiving, and reduces the threshold of manual checking; and the JSON format optimizes internal circulation, the lightweight feature is suitable for data transmission between internal modules of the system, improves the data processing efficiency, and is convenient for subsequent expansion and secondary development.

[0175] The above examples are only examples for clearly illustrating, but not limitation to the embodiments. Other different forms of changes or variations can be made on the basis of the above description for those skilled in the art. All the embodiments need not and cannot be exhausted here. The obvious changes or variations derived therefrom are still within the protection scope of the application.

Claims

1. A method for value tag analysis and information extraction, characterized by, The method comprises the following steps: Step S1, receiving a fixed value sheet file, identifying the type of the fixed value sheet file, including: PDF editable format and PDF picture format; Step S2, determining the processing mode of the fixed value sheet file based on the type of the fixed value sheet file, including: If the type of the fixed value sheet file is PDF editable format, parse the text stream, font attribute and coordinate information, divide the text area based on the font size and coordinate distribution, and generate structured data through template matching; If the type of the fixed value sheet file is PDF picture format, perform distortion correction and quality enhancement through an image preprocessing unit, identify the text by using an optimized OCR technology, and correct the identification result by combining the power fixed value logic rules; Step S3, performing semantic analysis on each structured data, extracting common area information and fixed value area information, performing semantic verification and correction, and obtaining fixed value data; Step S4, outputting the fixed value data in a preset format.

2. The method of claim 1, wherein, In the step S1, the method further comprises the following steps: Step S11, receiving a fixed value sheet file, performing integrity check and storage management, and triggering an alarm for damaged or illegal format files; Step S12, preliminarily determining the format type by analyzing the text stream and picture resource proportion in the file metadata; Step S13, performing text extraction test on the mixed format file, determining the final format type according to the extraction success rate or manual assistance, and distributing.

3. The method of claim 2, wherein In the step S12, the preliminary determination of the format type comprises: If the fixed value sheet file contains extractable text stream, and the picture resource proportion is less than a first preset picture resource proportion, it is preliminarily determined that the type of the fixed value sheet file is PDF editable format; If the fixed value sheet file does not contain text stream, or the picture resource proportion is greater than or equal to a second preset picture resource proportion, it is preliminarily determined that the type of the fixed value sheet file is PDF picture format; If the fixed value sheet file contains unextractable text stream, and the picture resource proportion is greater than or equal to the first preset picture resource proportion and less than the second preset picture resource proportion, the fixed value sheet file is marked as a mixed format file, and the mixed format file is further verified.

4. The method of claim 3, wherein In the step S13, the determination of the final format type according to the extraction success rate or manual assistance and the distribution comprise: Performing text extraction test on the mixed format file, calculating the ratio of the number of successfully extracted characters to the total number of extracted characters to obtain the extraction success rate, if the extraction success rate is greater than or equal to a first preset success rate, it is determined that the mixed format file is PDF editable format; If the extraction success rate is less than a second preset success rate, it is determined that the mixed format file is PDF picture format; If the extraction success rate is greater than or equal to the second preset success rate and less than the first preset success rate, manual assistance is triggered to determine the type of the mixed format file.

5. The method of claim 1, wherein, In the step S2, the parsing of the text stream, font attribute and coordinate information, the division of the text area based on the font size and coordinate distribution, and the generation of the structured data through template matching comprise: Step S211, parse the page structure of the PDF file, extract the text content, font attributes and coordinate information, and generate a page number-text content-font type-font size-coordinate correlation data table; Step S212, identify the title area, public area and fixed value area based on font size and coordinate distribution, and split the table text according to column coordinates; Step S213, match the fields through the fixed value single template library, complete the missing information, and check the value and unit rationality based on the power fixed value rules.

6. The method of claim 1, wherein, In step S2, the distortion correction and quality enhancement are performed by the image preprocessing unit, the text is recognized by using the optimized OCR technology, and the recognition result is corrected based on the power fixed value logical rules, including: Step S221, convert the PDF picture format file into a gray scale image, correct the distortion by using a perspective transformation algorithm, and improve the image quality by using filtering, histogram equalization and stain repair; Step S222, detect the text line area by using a YOLOv5 model, and sort the text lines according to coordinates; Step S223, recognize the text content by using a CRNN model, and correct similar characters by using a power special character library; Step S224, splice the text lines and split the fields according to the sequence number-fixed value name-value-unit structure, and mark abnormal data.

7. The method of claim 6, wherein In step S221, the distortion correction by using the perspective transformation algorithm includes: Detecting the image edge by using a Canny operator, and identifying the four corner coordinates of the fixed value sheet; Calculating the perspective transformation matrix, correcting the tilted image into a horizontal rectangle, and the tilt error is less than or equal to a preset tilt error.

8. The method of claim 1, wherein, In step S3, the semantic verification and correction include: Based on the power fixed value semantic rule library, the extracted public area information and fixed value area information are logically verified; The logical verification includes at least one of unit verification, value range verification and integrity verification; If the verification finds data abnormalities, the abnormal data is automatically corrected or marked for manual confirmation.

9. The method of claim 8, wherein, In step S3, the unit verification includes determining the standard unit corresponding to the fixed value name, and if the identified unit does not match the standard unit, the unit is corrected or marked as unit abnormal; The value range verification includes determining whether the value is within a preset reasonable value range according to the fixed value type, and if it exceeds the reasonable value range, it is marked as value out of range; The integrity verification includes calculating the field missing rate in the fixed value data, and if the missing rate exceeds a preset threshold, a data missing alarm is triggered.

10. The method of claim 1, wherein, In step S4, the fixed value data is output in a preset format, including: Converting the fixed value data into multiple standard formats, including XML format, Excel format and JSON format; The output data is attached with metadata, including at least one of extraction time, format type, recognition accuracy and verification pass status; The fixed value data is pushed to the fixed value checking system through a standard data interface.

Images