Key information positioning method and device for OCR (Optical Character Recognition) of power supply and consumption contract document, and electronic equipment

Abstract

The invention relates to the technical field of OCR (optical character recognition), in particular to a key information positioning method and device for OCR of power supply and consumption contract receipts and electronic equipment, and the method comprises the following steps: aiming at a table area, performing text recognition and table structure analysis on the table area by using a table structured recognition model group to obtain an HTML (hypertext markup language) character string of a table; performing text recognition on the non-table area by utilizing a character recognition model group aiming at the non-table area to obtain a corresponding character recognition result and coordinates; and extracting key information in the HTML character string / character recognition result. According to the method, bounding box coordinates and categories of key content areas in the power supply and consumption contract document image are determined, complex format division is realized, text recognition is performed by using different models for table areas and non-table areas, key information is extracted, and the text recognition efficiency is improved. Therefore, the problem of low OCR (optical character recognition) rate caused by complicated layout in the existing power supply and consumption contract receipt recognition is solved, and an effective guarantee is provided for automatic contract auditing.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of OCR recognition, and is a key information positioning method and device for OCR recognition of power supply and consumption contract documents and electronic equipment. BACKGROUND

[0002] With the deepening of power market reform and the demand for digital transformation, the power supply and consumption contract, as the core legal document between power enterprises and users, directly relates to the accuracy of billing, service quality and legal risk prevention and control in terms of legality, logic and data consistency of its clauses, so accurate recognition of the power supply and consumption contract document is very important.

[0003] The current recognition method of the power supply and consumption contract document includes: (1) manual review and recognition, which is greatly affected by human factors, low in efficiency and prone to errors, especially in the multi-page contract comparison and table analysis scenarios.

[0004] (2) using OCR technology to recognize the power supply and consumption contract document, the existing OCR technology mostly uses a single model (such as DB+CRNN) for text detection and recognition, but it has poor adaptability to special formats (such as multi-column tables and nested structures), so this method cannot effectively extract structured information of the table and accurately locate the cell fields and associated relationships in the power supply and consumption contract document when facing complex formats in the power supply and consumption contract document (such as multi-column tables and nested structures). SUMMARY

[0005] The application provides a method, which overcomes the shortcomings of the prior art and effectively solves the problem of low OCR recognition rate caused by complex formats in the existing power supply and consumption contract document recognition.

[0006] One of the technical solutions of the application is achieved by the following measures: a key information positioning method for OCR recognition of power supply and consumption contract documents, comprising: determining the bounding box coordinates and categories of each key content area in the power supply and consumption contract document image; for the table area, using a table structured recognition model group to perform text recognition and table structure analysis on the table area to obtain an HTML string of the table, wherein the table structured recognition model group includes a single-line text recognition model and a table structure analysis model, both of which are trained using samples; for the non-table area, using a text recognition model group to perform text recognition on the non-table area to obtain the corresponding text recognition result and coordinates, wherein the text recognition model group includes a text detection model and a text recognition model, both of which are trained using samples; Extract key information in the HTML string / text recognition result, including entities and the association information between entities.

[0007] The following is a further optimization or / and improvement of the above technical solutions: The above determination of the bounding box coordinates and categories of each key content area in the power supply and consumption contract document image includes: Obtain the power supply and consumption contract document image and perform preprocessing; Classify the image direction of the preprocessed power supply and consumption contract document image, and correct the rotation of the power supply and consumption contract document image based on the image direction classification result; Input the rotated power supply and consumption contract document image into the layout analysis model to obtain the bounding box coordinates and categories of each key content area in the power supply and consumption contract document image, wherein the layout analysis model is obtained by training the YOLOv8 network model with a plurality of samples, and each sample includes historical power supply and consumption contract document images and the bounding box coordinates and category identification information of the corresponding key content areas.

[0008] The above extraction of key information from the HTML string / text recognition result includes: Call all recognition methods in the semantic entity recognition library to extract entity information from the HTML string / text recognition result, wherein the semantic entity recognition library includes keyword matching, semantic analysis, regular expressions, and location information. Determine the association information between entities based on syntax rules and table structure analysis.

[0009] For the table area, the table structure recognition model group is used for text recognition and table structure analysis to obtain the HTML string of the table, including: Detect and recognize the single-line text of the table area using the single-line text recognition model to obtain the corresponding text recognition result and coordinates, wherein the single-line text recognition model is obtained by training the initial text recognition network model with a plurality of samples, the initial text recognition network model includes a DB network and a CRNN network, the DB network detects text lines, and the CRNN network performs character sequence recognition, and the samples include the table area of the historical power supply and consumption contract document image and the corresponding text recognition result and coordinates; Use the table structure analysis model to identify the table structure of the table area to obtain the coordinates of each cell in the table, wherein the table structure analysis model is obtained by training the SLANet network model with a plurality of samples, and each sample includes the table area of the historical power supply and consumption contract document image and the corresponding coordinates of each cell; Combine the text recognition result and coordinates and the coordinates of each cell to obtain the HTML string of the table.

[0010] The above text recognition model includes a text detection model and a text recognition model, and the corresponding construction process includes: The text detection model is obtained by training a DB network model through a plurality of samples, and each sample includes a non-table region of a historical power supply and consumption contract document image and corresponding text region segmentation result identification information. The text recognition model is obtained by training a CRNN network model through a plurality of samples, and each sample includes a text region of a non-table region in a historical power supply and consumption contract document image and corresponding text recognition result and coordinates.

[0011] The second technical solution of the application is realized by the following measures: a key information positioning device for OCR recognition of a power supply and consumption contract document, comprising: An image processing unit determines the bounding box coordinates and categories of each key content region in the power supply and consumption contract document image. A table region identification unit, for the table region, uses a table structured identification model group to perform text recognition and table structure analysis on the table region to obtain an HTML string of the table, wherein the table structured identification model group includes a single-line text recognition model and a table structure analysis model, both of which are trained using samples. A non-table region identification unit, for the non-table region, uses a text recognition model group to perform text recognition on the non-table region to obtain corresponding text recognition results and coordinates, wherein the text recognition model group includes a text detection model and a text recognition model, both of which are trained using samples. A key information extraction unit extracts key information from the HTML string / text recognition result, wherein the key information includes entities and associated information between entities.

[0012] The following is a further optimization or / and improvement of the above technical solution of the application: The above image processing unit includes: A preprocessing module acquires the power supply and consumption contract document image and performs preprocessing. An image correction module classifies the image direction of the preprocessed power supply and consumption contract document image, and performs rotation correction on the power supply and consumption contract document image based on the image direction classification result. A layout analysis module inputs the rotation-corrected power supply and consumption contract document image into a layout analysis model to obtain the bounding box coordinates and categories of each key content region in the power supply and consumption contract document image, wherein the layout analysis model is obtained by training a YOLOv8 network model through a plurality of samples, and each sample includes historical power supply and consumption contract document images and corresponding key content region bounding box coordinates and category identification information.

[0013] The above key information extraction unit includes: The entity recognition module extracts entity information in the HTML string / character recognition result by calling all recognition modes in a semantic entity recognition library, wherein the semantic entity recognition library includes keyword matching, semantic analysis, regular expression, and location information. The association recognition module determines the association information between entities based on syntax rules and table structure analysis.

[0014] The table region recognition unit includes: The single-line character recognition module detects and recognizes single-line characters in the table region by using a single-line character recognition model to obtain corresponding character recognition results and coordinates, wherein the single-line character recognition model is obtained by training an initial character recognition network model by using a plurality of samples, the initial character recognition network model includes a DB network and a CRNN network, the DB network detects text lines, and the CRNN network performs character sequence recognition, and the samples include table regions of historical power supply and consumption contract documents and corresponding character recognition results and coordinates. The table structure recognition module performs table structure recognition on the table region by using a table structure analysis model to obtain the coordinates of each cell in the table, wherein the table structure analysis model is obtained by training an SLANet network model by using a plurality of samples, and each sample includes a table region of a historical power supply and consumption contract document and corresponding coordinates of each cell. The fusion module combines the character recognition results and coordinates and the coordinates of each cell to obtain an HTML string of the table.

[0015] The third technical solution of the present application is realized by the following measures: an electronic device, characterized by comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to realize the steps in the positioning of key information for power supply and consumption contract document OCR recognition.

[0016] The present application aims at the problem of low OCR recognition rate caused by complex format in existing power supply and consumption contract document recognition. Firstly, the boundary box coordinates and categories of each key content region in the power supply and consumption contract document image are determined. Then, on the basis of reducing human intervention, different models are used for text recognition in the table region and the non-table region. Finally, the key information is extracted based on the HTML string / character recognition results of the table region and the non-table region, thereby realizing the positioning and structured extraction of the key information, providing effective guarantee for automatic contract auditing, and improving the work order processing efficiency of power enterprises. BRIEF DESCRIPTION OF DRAWINGS

[0017] The present application provides an implementation environment schematic diagram. Figure 1 The present application provides an implementation environment schematic diagram.

[0018] The present application provides an implementation environment schematic diagram.Figure 2 The key information positioning method flowchart provided by the present application.

[0019] The key information positioning method flowchart provided by the present application. Figure 3 The key information positioning method flowchart provided by the present application.

[0020] The key information positioning method flowchart provided by the present application. Figure 4 The key information positioning method flowchart provided by the present application.

[0021] The key information positioning method flowchart provided by the present application. Figure 5 The key information positioning method flowchart provided by the present application. DETAILED DESCRIPTION

[0022] The present application is not limited by the following examples, and the specific implementation can be determined according to the technical solutions of the present application and the actual situation.

[0023] Those skilled in the art can understand that, unless specifically stated, the "module" or "unit" in the embodiments of the present application refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0024] In addition, "multiple" in the embodiments of the present application refers to two or more, and "first" and "second" are used for differentiation and description, and cannot be understood as implying relative importance.

[0025] The embodiments of the present application provide a key information positioning method for power supply and consumption contract document OCR identification, a plurality of neural networks are subjected to deep learning to obtain an image direction classification model, a layout analysis model, a table structure identification model group and a character recognition model group; the image direction classification model performs image direction classification on a preprocessed power supply and consumption contract document image; the layout analysis model analyzes a power supply and consumption contract document image after input rotation correction to obtain boundary box coordinates and categories of each key content area in the power supply and consumption contract document image; for a table area, the table structure identification model group is used to perform text recognition and table structure analysis on the table area to obtain an HTML string of the table; for a non-table area, the character recognition model group is used to perform text recognition on the non-table area to obtain corresponding character recognition results and coordinates; key information in the HTML string / character recognition results is extracted, wherein the key information includes entities and associated information between the entities.

[0026] The method provided by the embodiment of the application can involve an artificial intelligence (AI) technology and can be implemented based on the artificial intelligence technology, for example, a deep learning manner is used to train a corresponding model.

[0027] Machine learning (ML) is a multi-field interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other subjects. Machine learning is a core of artificial intelligence and a fundamental approach to enabling a computer to have intelligence, and is applied to various fields of artificial intelligence.

[0028] Deep learning (DL) specifically refers to machine learning based on a deep neural network model and method. Deep learning is developed based on statistical machine learning, artificial neural network and other algorithm models, combined with the development of current big data and big computing power, and the most important technical feature of deep learning is the ability to automatically extract features.

[0029] The machine learning and deep learning described above usually include neural network, belief network, reinforcement learning, transfer learning, inductive learning, and rule teaching learning.

[0030] As shown in the accompanying Figure 1 The implementation environment provided by the embodiment of the application is shown in the accompanying drawings. The implementation environment can include a training device and a use device.

[0031] The training device and the use device are computer devices; optionally, the computer device is a terminal device, such as a mobile phone, a tablet computer, a personal computer (PC), and the like; or, the computer device is a server, which can be a single server or a server cluster composed of multiple servers, or a cloud computing service center, and the embodiment of the application does not limit the computer device.

[0032] The training device refers to a computer device with neural network training and learning capability. Optionally, the training device has neural network acquisition capability and is trained and learned according to application requirements. For example, the training device acquires a neural network from other devices through a network, and then trains the neural network through training samples according to application requirements, so that the neural network has corresponding analysis capability; optionally, the training device has neural network construction capability, which can construct a neural network according to application requirements, and then train and learn the neural network.

[0033] Use various models built using the equipment training device to determine the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image; for the table area, use the table structured recognition model group to perform text recognition and table structure analysis on the table area to obtain the HTML string of the table; for the non-table area, use the text recognition model group to perform text recognition on the non-table area to obtain the corresponding text recognition results and coordinates; extract key information from the HTML string / text recognition results, where the key information includes the association information between entities.

[0034] Based on this, the technical solution of the present invention will be introduced and explained with reference to several examples below.

[0035] Example 1: As shown in the attached Figure 2 As shown, an embodiment of the present invention discloses a method for locating key information for OCR recognition of electricity supply and use contract documents, comprising: Step S110, determining the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image; Step S120: Using a table structure recognition model group to perform text recognition and table structure analysis on the table area to obtain the HTML string of the table. The table structure recognition model group includes a single-line text recognition model and a table structure analysis model, both of which are trained using samples. Step S130 , performing text recognition on the non-table area using a text recognition model group to obtain corresponding text recognition results and coordinates, wherein the text recognition model group includes a text detection model and a text recognition model, both of which are trained using samples; Step S140 , extracting key information from the HTML character string / text recognition result, wherein the key information includes entities and association information between entities.

[0036] An embodiment of the present invention discloses a key information positioning method for optical character recognition (OCR) of electricity supply and utilization contract documents. To address the problem of low OCR recognition rate due to complex layout in existing electricity supply and utilization contract document recognition, the method first determines the bounding box coordinates and categories of each key content area in the electricity supply and utilization contract document image. Then, on the basis of reducing human intervention, different models are used for table areas and non-table areas to perform text recognition within the area. Finally, key information is extracted based on the HTML string / text recognition results of table areas and non-table areas. This achieves the positioning and structured extraction of key information, provides effective guarantees for automated contract review, and improves the efficiency of work order processing in power companies.

[0037] In step S140, key information is extracted from the HTML string / text recognition result, including: (1) Call semantic entity recognition library to extract entity information in HTML string / text recognition results, wherein the semantic entity recognition library includes keyword matching, semantic analysis, regular expression, and position information, and the specific recognition process is as described below: Keyword matching: match key fields such as "name", "date", "ID number" and the like through a predefined business dictionary; Semantic analysis: intelligently analyze by using context information to improve recognition accuracy; Regular expression: perform rule matching on date, amount, telephone number and other specific format data; Position information: determine the relative position of key information in combination with layout analysis results to improve extraction accuracy.

[0038] (2) Determine the association information between entities based on grammar rules and table structure analysis, and specifically: Grammar rule analysis: analyze the corresponding relationship between entities, such as name-ID number, amount-currency, and the like; Table structure analysis: analyze the HTML string of the table area, accurately extract the cell field and its corresponding value (such as "user name-ID number", "electricity fee amount-payment method"), and preferentially extract the table header (column name) and the first line of text of the contract clause to improve the positioning accuracy of key fields.

[0039] Embodiment 2: As shown in the accompanying Figure 3 , the embodiment of the present application is a further optimization of the above-mentioned embodiment, wherein the boundary box coordinates and categories of each key content area in the power supply and use contract document image are determined, including: Step S210: Obtain the power supply and use contract document image and perform preprocessing.

[0040] The above preprocessing specifically includes: (1) Denoising: use Gaussian filtering and bilateral filtering to remove image noise and reduce background interference; (2) Binarization: use adaptive thresholding to convert the image to black and white, improving the OCR detection effect; (3) Inclination correction: detect text lines by Hough transform and perform affine transform to correct the inclination angle; (4) Contrast enhancement: apply CLAHE (Contrast Limited Adaptive Histogram Equalization) to enhance the contrast of the image, making the text edges clearer.

[0041] In the actual application scenarios, such as document scanning and license shooting, the supply and demand electricity contract document image directions may be inconsistent due to shooting angles, device rotation, etc. In order to ensure that the input supply and demand electricity contract document image direction is correct and improve the accuracy of OCR recognition, the embodiment classifies the image direction of the pre-processed supply and demand electricity contract document image, and corrects the rotation of the supply and demand electricity contract document image based on the image direction classification result.

[0042] In actual application scenarios, such as document scanning, license shooting, etc., the supply and demand electricity contract document image directions may be inconsistent due to shooting angles, device rotation, etc. In order to ensure that the input supply and demand electricity contract document image direction is correct and improve the accuracy of OCR recognition, the embodiment classifies the image direction of the pre-processed supply and demand electricity contract document image, and corrects the rotation of the supply and demand electricity contract document image based on the image direction classification result.

[0043] Specifically, the image direction classification can be performed by a model constructed by machine learning, for example, using CNN (Convolutional Neural Network) modeling, and the modeling process is as follows: Obtain samples, divide them into training sample set and test sample set according to the proportion, wherein each sample includes supply and demand electricity contract document image and corresponding image direction identification information; Train the CNN network using the training sample set, introduce a loss function during training, and end training when the value of the loss function is stable, obtaining an image direction classification model, wherein the convolutional layer in the CNN network uses a convolution kernel to extract local features of the image, such as edges, lines, and textures, the pooling layer reduces the size of the feature map, reduces the amount of calculation, and improves the robustness of the model, and the fully connected layer integrates feature information and completes the classification task; Test the trained image direction classification model using the test sample set, optimize the model parameters of the image direction classification model, and output the image direction classification model that meets the test evaluation requirements.

[0044] Step S230, input the rotated supply and demand electricity contract document image to the layout analysis model to obtain the boundary box coordinates and categories of each key content area in the supply and demand electricity contract document image, wherein the layout analysis model is obtained by training the YOLOv8 network model using a plurality of samples, and each sample includes historical supply and demand electricity contract document image and corresponding key content area boundary box coordinates and category identification information.

[0045] The above uses the layout analysis model to divide the supply and demand electricity contract document image into regions, accurately identifies the key content areas such as text, title, table, and picture, and then processes them in regions.

[0046] The above process of training the YOLOv8 network model to obtain the layout analysis model is the same as the construction process of the image direction classification model, and will not be repeated here. It should be noted that the YOLOv8 network model has the following characteristics: Global analysis: YOLOv8 network takes the entire power supply contract document image as input, predicts multiple categories of bounding boxes at once, improving detection speed; Anchor mechanism: detects targets at different scales, improving small target detection capability; Single-stage detection: compared with two-stage detection, YOLOv8 network directly regresses bounding boxes and categories, improving real-time performance.

[0047] Embodiment 3: as shown in the accompanying Figure 4 The embodiment of the present application is a further optimization of the above-mentioned embodiment, wherein for the table region, a table structure recognition model group is used for text recognition and table structure analysis of the table region to obtain the HTML string of the table, including: Step S310, the single-line text recognition model is used to detect and recognize the single-line text of the table region to obtain the corresponding text recognition result and coordinates, wherein the single-line text recognition model is obtained by training the initial text recognition network model with a plurality of samples, the initial text recognition network model includes a DB network and a CRNN network, the DB network detects text lines, and the CRNN network performs character sequence recognition, and the samples include the table region of the historical power supply contract document image and the corresponding text recognition result and coordinates; The above-mentioned initial text recognition network model includes a DB network and a CRNN network, the DB network detects text regions through a segmentation network and can adapt to different shapes and sizes of text, the CRNN network combines the characteristics of CNN and RNN to perform character sequence recognition, and CTC (Connectionist Temporal Classification) decoding is used.

[0048] Step S320, the table structure recognition model is used to recognize the table structure of the table region to obtain the coordinates of each cell in the table, wherein the table structure recognition model is obtained by training the SLANet network model with a plurality of samples, and each sample includes the table region of the historical power supply contract document image and the corresponding coordinates of each cell; The above-mentioned SLANet network model is provided with a basic network based on PP-LCNet, which provides efficient feature extraction capability, adopts CSP-PAN for feature fusion to enhance feature interaction at different levels, introduces an Attention mechanism to enhance the decoding capability of key structures, and realizes accurate prediction of table structure and cell position information.

[0049] It should be further pointed out that the construction steps of the above-mentioned single-line text recognition model and table structure recognition model are the same as those of embodiment 2, and therefore will not be repeated.

[0050] Step S330: Combine the text recognition result with the coordinates and the coordinates of each cell to obtain the HTML string of the table. Specifically, the coordinates of the text recognition are matched with the coordinates of each cell to obtain the HTML string of the table.

[0051] Example 4: This embodiment of the present invention is a further optimization of the above embodiment, wherein the text recognition model includes a text detection model and a text recognition model, and the corresponding construction process includes: The text detection model is obtained by training the DB network model with several samples. Each sample includes the non-table area of ​​the historical power supply and use contract document image and the corresponding text area segmentation result identification information; The DB network model can predict the probability map of the text area and use a differentiable binarization method to make the text boundaries clearer. The DB network model is adaptable to complex scenes and various OCR application scenarios. It can detect curved text, small text, and dense text and has strong robustness.

[0052] The text recognition model is obtained by training the CRNN network model through several samples. Each sample includes the text area in the non-table area of ​​the historical electricity supply and use contract document image and the corresponding text recognition results and coordinates.

[0053] The text recognition model built based on the CRNN network model is an end-to-end text recognition model that combines CNN, RNN, and CTC. The CNN part is used to extract image features, and the RNN part models character sequences, improving the ability to recognize continuous text for end-to-end training. It does not require character-level alignment labels and is adaptable to variable-length text.

[0054] Example 5: As shown in the attached Figure 5 As shown, an embodiment of the present invention discloses a key information positioning device for OCR recognition of electricity supply and use contract documents, comprising: An image processing unit, which determines the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image; The table area recognition unit uses a table structure recognition model group to perform text recognition and table structure analysis on the table area to obtain the HTML string of the table. The table structure recognition model group includes a single-line text recognition model and a table structure analysis model, both of which are trained using samples. The non-table area recognition unit uses a text recognition model group to perform text recognition on the non-table area to obtain corresponding text recognition results and coordinates. The text recognition model group includes a text detection model and a text recognition model, both of which are trained using samples. A key information extraction unit extracts key information from the HTML string / text recognition result, wherein the key information includes entities and association information between the entities.

[0055] Embodiment 5: The embodiment of the application is a further optimization of the above-mentioned embodiments, in particular: The image processing unit comprises: The preprocessing module acquires the power supply and consumption contract document image and performs preprocessing. The image correction module classifies the image direction of the preprocessed power supply and consumption contract document image and performs rotation correction on the power supply and consumption contract document image based on the image direction classification result. The layout analysis module inputs the rotation-corrected power supply and consumption contract document image into a layout analysis model to obtain the bounding box coordinates and categories of each key content region in the power supply and consumption contract document image, wherein the layout analysis model is obtained by training a YOLOv8 network model with a plurality of samples, and each sample includes a historical power supply and consumption contract document image and identification information of the bounding box coordinates and categories of the corresponding key content region.

[0056] The key information extraction unit comprises: The entity recognition module extracts entity information from the HTML string / text recognition result by calling all recognition methods in a semantic entity recognition library, wherein the semantic entity recognition library includes keyword matching, semantic analysis, regular expressions, and location information. The association recognition module determines the association information between entities based on grammar rules and table structure analysis.

[0057] The table region identification unit comprises: The single-line text recognition module detects and recognizes the single-line text in the table region using a single-line text recognition model to obtain the corresponding text recognition result and coordinates, wherein the single-line text recognition model is obtained by training an initial text recognition network model with a plurality of samples, the initial text recognition network model includes a DB network and a CRNN network, the DB network detects text lines, and the CRNN network performs character sequence recognition, and the samples include the table region of the historical power supply and consumption contract document image and the corresponding text recognition result and coordinates. The table structure recognition module identifies the table structure of the table region using a table structure analysis model to obtain the coordinates of each cell in the table, wherein the table structure analysis model is obtained by training a SLANet network model with a plurality of samples, and each sample includes the table region of the historical power supply and consumption contract document image and the corresponding coordinates of each cell. The fusion module combines the text recognition result and coordinates and the coordinates of each cell to obtain the HTML string of the table.

[0058] Embodiment 6: The embodiment of the present application discloses an electronic device, comprising a processor and a memory, the memory stores a computer program, the computer program is loaded and executed by the processor to realize the power grid weak link identification method based on extreme ice disaster.

[0059] The processor can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. It can also be a combination of computing functions, such as one or more microprocessor combinations, DSP and microprocessor combinations, etc. The memory can include, but is not limited to, a U disk, a read-only memory, a mobile hard disk, a magnetic disk or an optical disk, and various computer program storage media.

[0060] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as object-oriented programming language Java and interpreted scripting language JavaScript.

[0061] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one block or multiple blocks.

[0062] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1the function specified in the one or more blocks.

[0063] The above merely provides the specific implementation of the present application, which has strong adaptability and implementation effect, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application, thus the equivalent changes made according to the claims of the present application are still covered within the scope of the present application.

Claims

1. A key information positioning method for OCR recognition of power supply and use contract documents, characterized in that: include: Determine the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image; For the table area, the table structure recognition model group is used to perform text recognition and table structure analysis on the table area to obtain the HTML string of the table. The table structure recognition model group includes a single-line text recognition model and a table structure analysis model, both of which are trained using samples; For non-table areas, a text recognition model group is used to perform text recognition on the non-table areas to obtain corresponding text recognition results and coordinates. The text recognition model group includes a text detection model and a text recognition model, both of which are trained using samples. Extract key information from HTML string / text recognition results, where key information includes entities and association information between entities.

2. The key information positioning method for OCR recognition of power supply and use contract documents according to claim 1 is characterized in that: Determining the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image includes: Obtain electricity supply and use contract document images and perform pre-processing; Performing image orientation classification on the pre-processed power supply and use contract document image, and performing rotation correction on the power supply and use contract document image based on the image orientation classification result; The rotation-corrected electricity supply and use contract document image is input into the layout analysis model to obtain the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image. The layout analysis model is obtained by training the YOLOv8 network model through several samples. Each sample includes the historical electricity supply and use contract document image and the bounding box coordinates and category identification information of the corresponding key content area.

3. The key information positioning method for OCR recognition of power supply and use contract documents according to claim 1 or 2 is characterized in that: The key information extracted from the HTML string / text recognition result includes: Call all recognition methods in the semantic entity recognition library to extract entity information from the HTML string / text recognition results, where the semantic entity recognition library includes keyword matching, semantic analysis, regular expressions, and location information; Determine the association information between entities based on grammatical rules and table structure analysis.

4. The key information positioning method for OCR recognition of power supply and use contract documents according to any one of claims 1 to 3, characterized in that: The table area is targeted, and the table structure recognition model group is used to perform text recognition and table structure analysis on the table area to obtain the HTML string of the table, including: A single-line text recognition model is used to detect and recognize single-line text in a table area, obtaining corresponding text recognition results and coordinates. The single-line text recognition model is obtained by training an initial text recognition network model using several samples. The initial text recognition network model includes a DB network and a CRNN network. The DB network detects text lines, and the CRNN network recognizes character sequences. The samples include the table area of ​​a historical electricity supply and utilization contract document image and the corresponding text recognition results and coordinates. The table structure analysis model is used to identify the table area and obtain the coordinates of each cell in the table. The table structure analysis model is trained on a SLANet network model using several samples. Each sample includes the table area and the corresponding cell coordinates of the historical power supply and use contract document image. Combine the text recognition results and coordinates with the coordinates of each cell to obtain the HTML string of the table.

5. The key information positioning method for OCR recognition of power supply and use contract documents according to any one of claims 1 to 4, characterized in that: The text recognition model includes a text detection model and a text recognition model, and the corresponding construction process includes: The text detection model is obtained by training the DB network model with several samples. Each sample includes the non-table area of ​​the historical power supply and use contract document image and the corresponding text area segmentation result identification information; The text recognition model is obtained by training the CRNN network model through several samples. Each sample includes the text area in the non-table area of ​​the historical electricity supply and use contract document image and the corresponding text recognition results and coordinates.

6. A key information locating device for OCR recognition of power supply and use contract documents using the method according to any one of claims 1 to 5, characterized in that: include: An image processing unit, which determines the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image; The table area recognition unit uses a table structure recognition model group to perform text recognition and table structure analysis on the table area to obtain the HTML string of the table. The table structure recognition model group includes a single-line text recognition model and a table structure analysis model, both of which are trained using samples. The non-table area recognition unit uses a text recognition model group to perform text recognition on the non-table area to obtain corresponding text recognition results and coordinates. The text recognition model group includes a text detection model and a text recognition model, both of which are trained using samples. The key information extraction unit extracts key information from the HTML string / text recognition result, wherein the key information includes entities and association information between entities.

7. The key information locating device for OCR recognition of power supply and use contract documents according to claim 6 is characterized in that: The image processing unit includes: The pre-processing module obtains the electricity supply and use contract document image and performs pre-processing; An image correction module is used to perform image orientation classification on the pre-processed power supply and use contract document image, and to perform rotation correction on the power supply and use contract document image based on the image orientation classification result; The layout analysis module inputs the rotation-corrected electricity supply and use contract document image into the layout analysis model to obtain the bounding box coordinates and categories of each key content area in the electricity supply and use contract document image. The layout analysis model is obtained by training the YOLOv8 network model through several samples. Each sample includes the historical electricity supply and use contract document image and the bounding box coordinates and category identification information of the corresponding key content area.

8. The key information locating device for OCR recognition of power supply and use contract documents according to claim 6 or 7, characterized in that: The key information extraction unit includes: The entity recognition module uses all recognition methods in the semantic entity recognition library to extract entity information from the HTML string / text recognition results. The semantic entity recognition library includes keyword matching, semantic analysis, regular expressions, and location information. The association recognition module determines the association information between entities based on grammatical rules and table structure analysis.

9. The key information locating device for OCR recognition of power supply and use contract documents according to any one of claims 6 to 8, characterized in that: The table area identification unit includes: The single-line text recognition module uses a single-line text recognition model to detect and recognize single-line text in a table area, obtaining the corresponding text recognition results and coordinates. The single-line text recognition model is obtained by training an initial text recognition network model using several samples. The initial text recognition network model includes a DB network and a CRNN network. The DB network detects text lines, and the CRNN network recognizes character sequences. The samples include the table area of ​​a historical electricity supply and utilization contract document image and the corresponding text recognition results and coordinates. The table structure recognition module uses a table structure analysis model to identify the table structure of the table area and obtain the coordinates of each cell in the table. The table structure analysis model is trained on a SLANet network model using several samples. Each sample includes the table area of ​​a historical power supply and use contract document image and the corresponding cell coordinates. The fusion module combines the text recognition results with the coordinates and the coordinates of each cell to obtain the HTML string of the table.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the steps in the method according to any one of claims 1 to 5.

Images