A table information updating method and system based on intelligent text recognition

Abstract

The present invention relates to the technical field of table information update, and in particular to a table information update method and system based on intelligent text recognition; the method comprises the following steps: associating at least one table to be updated, and preprocessing an input table image; using OCR recognition technology and a deep learning convolutional neural network to recognize the preprocessed table image, and extracting table content in the table image; understanding the semantic relationship in the table content through a pre-trained language model, and associating the table content; inputting the table content into a table to be updated according to the semantic relationship, and updating accordingly. By combining OCR, CNN, a pre-trained language model and image processing technology, the entire process can be automated, and the boundary detection, content extraction, semantic analysis and data association of the table are all automatically completed through an algorithm, which greatly improves the processing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of table information updating, and in particular to a table information updating method and system based on intelligent text recognition. Background Art

[0002] People often need to fill out various forms in their daily work, study and life: such as information registration forms, financial statements, questionnaires, bank deposit / remittance slips, etc., which have become an important way to record information; with the popularization of paperless office, a large number of paper documents need to be entered into computers, and manual information extraction and accuracy verification are required, which is both time-consuming and labor-intensive; with the rapid development of digital construction, how to efficiently and accurately extract key information from paper documents is particularly important; in the prior art, OCR recognition technology is generally used to process form information. OCR can scan images or documents of forms, identify the characters therein, and convert these characters into editable text data; however, the OCR recognition accuracy is limited by the quality of the image, the complexity of the form, the font style, etc. Especially for complex tables containing multiple formats, OCR may find it difficult to accurately identify the row and column relationship of the table; in addition, OCR has a low recognition accuracy for non-standardized fonts (such as handwriting) or images with low resolution; China Patent Publication No. CN110796031B discloses an artificial intelligence-based table recognition method, artificial intelligence-based table recognition device and electronic equipment, mainly involving computer vision, natural language processing and machine learning technologies. The method includes: performing image segmentation on the image to be recognized based on image semantics to obtain a foreground line image; determining multiple table cells according to the line relationship in the foreground line image, and establishing an electronic table according to the multiple table cells; performing text recognition on the image to be recognized to obtain text information in the image to be recognized; filling the text information into the electronic table according to the position of the text information in the image to be recognized and the position of the table cell in the electronic table. The method can avoid mutual interference between the two in the recognition process by segmenting, recognizing and recombining the table structure and the table content, thereby improving the recognition accuracy of the table. It uses image recognition technology to determine the cells of the table and fill them, but there are still some defects:

[0003] Rule-based systems are not very adaptable to tables with slightly changed formats, and are difficult to expand to different types of tables and information. It is difficult to ensure the accuracy of different types of data when they are being updated;

[0004] There are still deficiencies in the semantic understanding and contextual association of table information, and the consistency requirements cannot be met during the information update process.

[0005] Therefore, the present application proposes a table information updating method and system based on intelligent text recognition. Summary of the invention

[0006] The present invention aims at solving the technical problems in the prior art and provides a table information updating method and system based on intelligent text recognition.

[0007] The technical solution of the present invention to solve the above technical problem is as follows: a table information updating method and system based on intelligent text recognition; the method comprises the following steps:

[0008] S1: Associating at least one table to be updated and preprocessing the input table image;

[0009] S2: The preprocessed table image is recognized using OCR recognition technology and deep learning convolutional neural network to extract the table content in the table image;

[0010] S3: Understand the semantic relationship in the table content through the pre-trained language model and associate the table content;

[0011] S4: Input the table content into the table to be updated according to the semantic relationship, and update it accordingly.

[0012] Furthermore, the table information updating method based on intelligent text recognition includes in S1: based on line detection in the image, using Hough transform to detect straight lines and extract row and column boundaries of the table;

[0013] Hough transform detects horizontal and vertical lines in an image:

[0014] ;

[0015] in, Represents the distance from the line to the origin, represents the angle of the line, x and y are coordinates;

[0016] Through detection Divide the lines into horizontal and vertical lines, horizontal lines , vertical line ;

[0017] Get the line collection:

[0018] ;

[0019] is the set of detected lines, represents the Hough transform function;

[0020] Table cells in the table image are obtained according to the horizontal lines and the vertical lines, and the table cells are represented by diagonal coordinate points.

[0021] Furthermore, the table information updating method based on intelligent text recognition includes in S2:

[0022] Character recognition: The table image is detected through the sliding window technology to generate candidate regions containing characters. The candidate regions are represented by matrix R, and each candidate region is represented by position and size:

[0023] ;

[0024] in It is The upper left corner coordinates of the candidate regions, and Respectively expressed as width and height;

[0025] Feature extraction: CNN is used to extract features for each candidate region, and each character region is used as an input matrix , after the convolution layer, pooling layer and activation function processing to generate feature maps, the convolution operation can be expressed as:

[0026] ;

[0027] in Indicates The feature map of the layer, is the convolution kernel, is the convolution operation, is the bias term, is the activation function;

[0028] Character classification: After the feature map passes through several fully connected layers, it will be mapped to the character classification space, and the softmax layer will be used to output the probability of character classification:

[0029] ;

[0030] It is a candidate area Classified as a character The probability of Is the corresponding character The activation value of the class, N is the number of character classes.

[0031] Furthermore, the table information updating method based on intelligent text recognition includes in S3:

[0032] Constructing a graph model: Each cell is considered as a node in the graph model , the relationships between cells are used as edges in the graph model , the edge weight Calculation is based on the location similarity and content similarity between cells:

[0033] ;

[0034] in is the position similarity, is the content similarity, and is the weight coefficient;

[0035] ;

[0036] ;

[0037] in is the distance between the geometric center points of two cells, and It is a cell and cells Text embedding vector of ;

[0038] Calculate its edges:

[0039] ;

[0040] in, It is a similarity function based on position and content.

[0041] Furthermore, in the table information updating method based on intelligent text recognition, the pre-trained language model includes:

[0042] A multi-head attention mechanism is used to generate context-dependent embedding representations for cell text embedding vectors. , whose context embedding is expressed as:

[0043] ;

[0044] in is the query matrix, K is the key matrix, V is the value matrix, is the transpose of the key matrix, is the dimension of the bond matrix;

[0045] Generate semantic labels for cells based on contextual embedding representations and distinguish between different levels of titles and associated data.

[0046] Furthermore, the table information updating method based on intelligent text recognition defines positive and negative samples through the contrastive learning method in self-supervised learning to optimize the association relationship of cells. The goal of self-supervised learning is to maximize the similarity between cells at the same level and minimize the similarity between cells at different levels. The loss function of contrastive learning is:

[0047] .

[0048] Furthermore, the table information updating method based on intelligent text recognition maps the extracted table content to the target table to be updated according to the semantic relationship generated by the pre-trained language model.

[0049] A table information updating system based on intelligent text recognition is applied to any one of the table information updating methods based on intelligent text recognition, and the updating system comprises:

[0050] Preprocessing unit: preprocess the input table image;

[0051] Content extraction unit: The pre-processed table image is recognized using OCR recognition technology and deep learning convolutional neural network to extract the table content in the table image;

[0052] Association unit: understands the semantic relationship in the table content through the pre-trained language model and associates the table content;

[0053] Update unit: Input the table content into the table to be updated according to the semantic relationship and update it accordingly.

[0054] The beneficial effects of the present invention are:

[0055] By combining OCR, CNN, pre-trained language models and image processing technology, this technology can automate the entire process, from table image preprocessing, content extraction, semantic understanding to data verification and update, with almost no human intervention required; table boundary detection, content extraction, semantic analysis and data association are all automatically completed through algorithms, greatly improving processing efficiency;

[0056] Using pre-trained language models, the technology can understand the semantic relationships in the table and automatically match the table content with the corresponding fields. Through the multi-head attention mechanism, the system can accurately distinguish between titles and data, identify titles and related data items at different levels, and complete automatic classification and mapping of content;

[0057] By building a graph model, this technology can effectively handle multi-level titles. By calculating the content similarity and position similarity between cells, the system can automatically determine the titles of different levels and their corresponding data items, and correctly associate them. This enables the system to maintain efficient and accurate parsing and updating when processing tables with complex hierarchies. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flowchart of a table information updating method based on intelligent text recognition according to the present invention;

[0059] Figure 2 The present invention is a schematic diagram of a table information updating system based on intelligent text recognition. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0061] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0062] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0063] like Figure 1 As shown in the figure: In one embodiment, a table information updating method based on intelligent text recognition includes the following steps:

[0064] S1: Associating at least one table to be updated and preprocessing the input table image;

[0065] S2: Use OCR recognition technology and deep learning convolutional neural network to recognize the preprocessed table image and extract the table content in the table image; Specifically: Extract the text content in the table through OCR recognition technology; OCR will convert the scanned text into processable digital text, such as recognizing "October 1, 2023" in the image as readable date text. Combined with convolutional neural network (CNN), the system can more accurately recognize the characters and cell boundaries in the table, especially for complex tables (such as multi-level headers, merged cells); CNN extracts the feature information of the table through convolution operations, assisting OCR to improve the accuracy of character recognition.

[0066] S3: Understand the semantic relationship in the table content through the pre-trained language model, and associate the table content; Specifically: The pre-trained language model (such as GPT) is used to understand the text content in the table; The model understands the meaning of each cell through context, for example, it recognizes that "income" belongs to the financial category, and "5,000 yuan" is related to the income amount; According to the contextual semantics generated by the language model, the various data items in the table are associated with their corresponding categories; For example, the system can associate the "product name" in the table with its corresponding quantity and unit price.

[0067] S4: Input the table content into the table to be updated according to the semantic relationship, and update it accordingly.

[0068] In this embodiment, in order to extract information from the table image, the system first needs to process the image and associate it with the table to be updated in the target database or system; the input table image may be a scanned invoice, contract, purchase order, financial statement, etc.; the updated target table may be a database field in the ERP system or a table template in the business system; image preprocessing: preprocess the input table image, and use image processing technology (such as grayscale, binarization, and noise removal) to improve the OCR recognition effect; for example, use Hough transform to detect the row and column boundaries of the table to ensure that the structure of the table can be accurately identified; OCR technology is used to recognize text content in images, but in complex table structures, the use of OCR alone is often not accurate enough, so it is combined with deep learning models (such as convolutional neural networks CNN) to further optimize the parsing of the table structure; at the same time, the data in the table often requires semantic understanding, especially when the table contains multi-level titles and multiple classification items, the system needs to determine which data is related to which title and category; through a pre-trained language model (such as GPT), the system can automatically understand the semantic relationship in the table and associate the data with the corresponding fields.

[0069] In a specific embodiment: if an enterprise receives a large number of financial invoices, the formats of the invoices may be different. Manually entering these invoice information into the financial system is not only time-consuming but also prone to errors. In order to solve this problem, an automated invoice processing system based on OCR and deep learning can be used to automatically update the data in the invoice to the financial management system;

[0070] Table association and preprocessing: When an enterprise uploads a scanned invoice to the system, the system first preprocesses the invoice, uses Hough transform to detect the row and column boundaries in the invoice, and associates it with the invoice template in the financial system; for example, it identifies the boundaries of the invoice number, date, total amount, tax amount and other columns in the invoice.

[0071] OCR and CNN identify invoice content: The system uses OCR technology to extract text information from invoices, such as "Invoice number: 123456", "Date: October 1, 2023", and "Total amount: 5,000 yuan". CNN identifies the structure in the invoice to ensure that this information is accurately separated.

[0072] The GPT model understands the semantic relationship of invoices: The GPT model analyzes the content of the invoice, understands the relationship between the "invoice number" and the value "123456", recognizes that the "total amount" corresponds to "5,000 yuan", and ensures that each data item is correctly associated with its parent category (such as "invoice information" and "amount details").

[0073] Data matching and updating: Based on the semantic analysis results of GPT, the system matches the extracted information such as "invoice number", "date", and "amount" to the corresponding fields in the financial system, and automatically proofreads the data format; finally, the system saves the data to the financial system to ensure that the invoice information is updated.

[0074] In one embodiment, S1 includes: based on line detection in the image, using Hough transform to detect straight lines and extract row and column boundaries of the table;

[0075] Hough transform detects horizontal and vertical lines in an image:

[0076] ;

[0077] in, Represents the distance from the line to the origin, It represents the angle of the line, X and Y are the coordinates;

[0078] Through detection Divide the lines into horizontal and vertical lines, horizontal lines , vertical line ;

[0079] Get a collection of lines:

[0080] ;

[0081] is the set of detected lines, represents the Hough transform function;

[0082] Table cells in the table image are obtained according to the horizontal lines and the vertical lines, and the table cells are represented by diagonal coordinate points.

[0083] In this embodiment, the image is preprocessed first. In the table image preprocessing stage, the system first performs a preprocessing operation on the input image, for example:

[0084] Grayscale: Convert color images into grayscale images to facilitate subsequent line detection.

[0085] Edge detection: Use edge detection algorithms (such as Canny edge detection) to detect edges in the image and obtain possible table lines.

[0086] After edge detection, the table lines in the image are highlighted, and then Hough transform is used to detect these lines.

[0087] Each pixel is mapped into parameter space according to the Hough transform , different pixels (especially pixels on continuous lines) will map intersecting points in the parameter space. These intersecting points represent a straight line. The detected lines can be divided into horizontal lines and vertical lines. The value of , vertical line ,Through this classification, the row and column lines in the table can be ,identified respectively, which represent the row and column boundaries of the ,table.

[0088] According to the detected intersections of horizontal and vertical lines, the image can be divided into multiple table cells. Table cells are represented by two coordinate points of the diagonal line, and all table cells can be divided by the intersections of horizontal and vertical lines.

[0089] In one embodiment, S2 includes:

[0090] Character recognition: The table image is detected through the sliding window technology to generate candidate regions containing characters. The candidate regions are represented by matrix R, and each candidate region is represented by position and size:

[0091] ;

[0092] in It is The upper left corner coordinates of the candidate regions, and Respectively expressed as width and height;

[0093] Feature extraction: CNN is used to extract features for each candidate region, and each character region is used as an input matrix , after the convolution layer, pooling layer and activation function processing to generate feature maps, the convolution operation can be expressed as:

[0094] ;

[0095] in Indicates The feature map of the layer, is the convolution kernel, is the convolution operation, is the bias term, is the activation function;

[0096] Character classification: After the feature map passes through several fully connected layers, it will be mapped to the character classification space, and the softmax layer will be used to output the probability of character classification:

[0097] ;

[0098] It is a candidate area Classified as a character The probability of Is the corresponding character The activation value of the class, N is the number of character classes.

[0099] In this embodiment, the characters in the table are usually located in cells, and different characters have different positions and sizes in the image. In order to accurately locate these character areas in the image, the system uses sliding window technology to gradually scan the table image and detect areas that may contain characters. Sliding window technology is a commonly used method in computer vision. By moving a fixed-size window on the image, objects in different areas are gradually detected.

[0100] In one embodiment, S3 includes:

[0101] Constructing a graph model: Each cell is considered as a node in the graph model , the relationships between cells are used as edges in the graph model , the edge weight Calculation is based on the position similarity and content similarity between cells:

[0102] ;

[0103] in is the position similarity, is the content similarity, and is the weight coefficient;

[0104] ;

[0105] ;

[0106] in is the distance between the geometric center points of two cells, and It is a cell and cells Text embedding vector of ;

[0107] Calculate its edges:

[0108] ;

[0109] in, It is a similarity function based on position and content.

[0110] In one embodiment, the pre-trained language model includes:

[0111] A multi-head attention mechanism is used to generate context-dependent embedding representations for cell text embedding vectors. , whose context embedding is expressed as:

[0112] ;

[0113] in is the query matrix, K is the key matrix, V is the value matrix, is the transpose of the key matrix, is the dimension of the bond matrix;

[0114] Generate semantic labels for cells based on contextual embedding representations and distinguish between different levels of titles and associated data.

[0115] The positive and negative samples are defined by the contrastive learning method in self-supervised learning to optimize the association relationship between cells. The goal of self-supervised learning is to maximize the similarity between cells at the same level and minimize the similarity between cells at different levels. The loss function of contrastive learning is:

[0116] .

[0117] According to the semantic relations generated by the pre-trained language model, the extracted table content is mapped to the target table to be updated.

[0118] A table information updating system based on intelligent text recognition is applied to any one of the table information updating methods based on intelligent text recognition, and the updating system comprises:

[0119] Preprocessing unit: preprocess the input table image;

[0120] Content extraction unit: The pre-processed table image is recognized using OCR recognition technology and deep learning convolutional neural network to extract the table content in the table image;

[0121] Association unit: understands the semantic relationship in the table content through the pre-trained language model and associates the table content;

[0122] Update unit: Input the table content into the table to be updated according to the semantic relationship and update it accordingly.

[0123] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0124] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A table information updating method based on intelligent text recognition, characterized in that: The following steps are involved: S1: Associating at least one table to be updated and preprocessing the input table image; S2: The preprocessed table image is recognized using OCR recognition technology and deep learning convolutional neural network to extract the table content in the table image; S3: Understand the semantic relationship in the table content through the pre-trained language model and associate the table content; S4: input the table content into the table to be updated according to the semantic relationship and update it accordingly; Included in S3: Constructing a graph model: Each cell is considered as a node in the graph model , the relationships between cells are used as edges in the graph model , the edge weight Calculation is based on the location similarity and content similarity between cells: ； in is the position similarity, is the content similarity, and is the weight coefficient; ； ； in is the distance between the geometric center points of two cells, and It is a cell and cells Text embedding vector of ; Calculate its edges: ； in, It is a similarity function based on position and content; Pre-trained language models include: A multi-head attention mechanism is used to generate context-dependent embedding representations for cell text embedding vectors. , whose context embedding is expressed as: ； in is the query matrix, is the key matrix, is the value matrix, is the transpose of the key matrix, is the dimension of the bond matrix; Generate semantic labels for cells based on contextual embedding representations and distinguish between different levels of titles and associated data; The positive and negative samples are defined by the contrastive learning method in self-supervised learning to optimize the association relationship of cells. The goal of self-supervised learning is to maximize the similarity between cells at the same level and minimize the similarity between cells at different levels. The loss function of contrastive learning is: ; According to the semantic relations generated by the pre-trained language model, the extracted table content is mapped to the target table to be updated.

2. The table information updating method based on intelligent text recognition as described in claim 1 is characterized in that: S1 includes: based on line detection in the image, using Hough transform to detect straight lines and extracting row and column boundaries of the table; Hough transform detects horizontal and vertical lines in an image: ； in, Represents the distance from the line to the origin, Indicates the angle of the line, and are coordinates; Through detection Divide the lines into horizontal and vertical lines, horizontal lines , vertical line ; Get a collection of lines: ； is the set of detected lines, represents the Hough transform function; Table cells in the table image are obtained according to the horizontal lines and the vertical lines, and the table cells are represented by diagonal coordinate points.

3. The table information updating method based on intelligent text recognition as described in claim 1 is characterized in that: Included in S2: Character recognition: The table image is detected through the sliding window technology to generate candidate regions containing characters. The candidate regions are represented by matrix R, and each candidate region is represented by position and size: ； in It is The upper left corner coordinates of the candidate regions, and Respectively expressed as width and height; Feature extraction: CNN is used to extract features for each candidate region, and each character region is used as an input matrix , after the convolution layer, pooling layer and activation function processing to generate feature maps, the convolution operation can be expressed as: ； in Indicates The feature map of the layer, is the convolution kernel, is the convolution operation, is the bias term, is the activation function; Character classification: After the feature map passes through several fully connected layers, it will be mapped to the character classification space, and the softmax layer will be used to output the probability of character classification: ； It is a candidate area Classified as a character The probability of Is the corresponding character The activation value of the class, N is the number of character classes.

4. A table information updating system based on intelligent text recognition, characterized in that: A table information updating method based on intelligent text recognition as described in any one of claims 1 to 3, wherein the updating system comprises: Preprocessing unit: preprocess the input table image; Content extraction unit: The pre-processed table image is recognized using OCR recognition technology and deep learning convolutional neural network to extract the table content in the table image; Association unit: understands the semantic relationship in the table content through the pre-trained language model and associates the table content; Update unit: Input the table content into the table to be updated according to the semantic relationship and update it accordingly.

Images