Certificate information automatic identification and extraction method and related device thereof

Abstract

The embodiment of the application belongs to the technical field of image information processing, is applied to the scene of automatic extraction of certificate information, and relates to a certificate information automatic identification and extraction method and a related device thereof, which comprises the following steps: obtaining a target certificate image to be subjected to information extraction; identifying text information and image information through a text information identification template and an image information identification template in a certificate information unified extraction model; extracting text data and image data; screening out text content to be corrected; correcting the text content to be corrected to obtain corrected text content; and outputting the image data, the text data and the corrected text content, thereby completing information extraction on the target certificate image. Through learning and training on the certificate information unified extraction model, the information identification template and the text correction strategy are introduced, so that high-accuracy information extraction can be realized on dozens of certificate types and formats of certificates without knowing the types of the certificates, manpower is effectively saved, and the method is more intelligent and automatic.

Description

Technical Field

[0001] This application relates to the field of image information processing technology and is applied to the scenario of automated extraction of document information, and in particular to a method for automated identification and extraction of document information and related equipment. Background Technology

[0002] In financial transactions such as insurance applications, claims, and bank loans, customers often need to provide various information. This frequently involves identifying images of customer-submitted documents. Since document types are not standardized—potentially dozens of types, each with multiple versions—and considering the vast number of products, scenarios, and the sheer volume of business, relying solely on manual review would be extremely time-consuming. Furthermore, the lack of standardized document submissions could lead to chaotic customer information management and serious business problems such as duplicate insurance applications.

[0003] Existing solutions typically develop a targeted extraction method for a specific type of document. However, since there are many types of documents that can provide information, and customers do not specify the document type when submitting document images, this approach may not be suitable. Therefore, there is currently a lack of a solution that can automatically extract information from a target document without prior identification of the document type. Summary of the Invention

[0004] The purpose of this application is to propose an automated identification and extraction method for document information and related equipment, so as to solve the problem that the existing technology lacks a solution that can automatically extract information of target documents without prior differentiation of document types.

[0005] To address the aforementioned technical problems, this application provides an automated method for identifying and extracting document information, employing the following technical solution:

[0006] A method for automatically identifying and extracting document information includes the following steps:

[0007] Acquire the image of the target document from which information is to be extracted;

[0008] The target document image is input into a preset unified document information extraction model, wherein the unified document information extraction model contains information recognition templates corresponding to different versions and categories of document images as prior knowledge.

[0009] The unified extraction model for document information is used to identify the text features, image features, and location features of the target document image.

[0010] Based on the text features, image features, and location features, the text information recognition template and image information recognition template corresponding to the target document image are selected;

[0011] Based on the text information recognition template, text data is identified and extracted from the target document image;

[0012] Based on the image information recognition template, image data is identified and extracted from the target document image;

[0013] The text content to be corrected contained in the text data is filtered out using a preset filter list.

[0014] The text content to be corrected is corrected based on a preset correction strategy to obtain the corrected text content;

[0015] Output the image data, the uncorrected text content in the text data, and the corrected text content to complete the information extraction of the target document image.

[0016] To address the aforementioned technical problems, this application also provides an automated document information recognition and extraction device, which employs the following technical solution:

[0017] An automated identification and extraction device for document information includes:

[0018] The target document image acquisition module is used to acquire the target document image from which information is to be extracted;

[0019] The target document image input module is used to input the target document image into a preset unified document information extraction model, wherein the unified document information extraction model contains information recognition templates corresponding to different versions and categories of document images as prior knowledge.

[0020] The feature recognition module is used to identify the text features, image features, and location features of the target document image through a unified extraction model based on the document information.

[0021] The information recognition template filtering module is used to filter out the text information recognition template and image information recognition template corresponding to the target document image based on the text features, image features and location features;

[0022] The text data extraction module is used to identify and extract text data from the target document image based on the text information recognition template;

[0023] The image data extraction module is used to identify and extract image data from the target document image based on the image information recognition template.

[0024] The text content filtering module is used to filter out the text content to be corrected contained in the text data through a preset filtering list.

[0025] The text content correction module is used to correct the text content to be corrected based on a preset correction strategy to obtain the corrected text content.

[0026] The information output module is used to output the image data, the uncorrected text content in the text data, and the corrected text content, thereby completing the information extraction of the target document image.

[0027] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0028] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the above-described method for automatically identifying and extracting document information.

[0029] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0030] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the document information automatic identification and extraction method described above.

[0031] Compared with the prior art, the embodiments of this application have the following main advantages:

[0032] The document information automated recognition and extraction method described in this application involves: acquiring a target document image for information extraction; using text and image information recognition templates in a unified document information extraction model; extracting text and image data; filtering out text content to be corrected; correcting the text content to obtain corrected text content; and outputting image data, text data, and corrected text content to complete the information extraction from the target document image. This application, through learning and training a unified document information extraction model and introducing information recognition templates and text extraction correction strategies, achieves high-accuracy information extraction from dozens of document types and formats without needing to know the document type, effectively saving manpower and making it more intelligent and automated. Attached Figure Description

[0033] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0035] Figure 2 This is a flowchart of an embodiment of the document information automatic identification and extraction method according to this application;

[0036] Figure 3 This is a flowchart of an embodiment of the method for automatically identifying and extracting document information described in this application, which involves learning and training the unified extraction model for document information.

[0037] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 305 shown;

[0038] Figure 5 yes Figure 3 A flowchart of a specific embodiment of step 306 shown;

[0039] Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 203 shown;

[0040] Figure 7 This is a schematic diagram of the structure of an embodiment of the document information automatic identification and extraction device according to this application;

[0041] Figure 8 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0043] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0044] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0045] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0046] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0047] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0048] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0049] It should be noted that the automated identification and extraction method for document information provided in this application embodiment is generally executed by a server, and correspondingly, the automated identification and extraction device for document information is generally located in the server.

[0050] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0051] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the document information automated identification and extraction method according to this application. The document information automated identification and extraction method includes the following steps:

[0052] Step 201: Obtain the image of the target document from which information is to be extracted.

[0053] In this embodiment, the target document image for information extraction includes document images uploaded by the user terminal. These target document images are frequently required in various business scenarios within the financial sector, such as insurance applications, claims, and bank loan processing. Examples of such images include: ID card images, business license images, property ownership certificate images, marriage certificate images, vehicle registration certificates, and driver's license images. The automated document information recognition and extraction method described in this application, by acquiring the target document image for information extraction and combining it with a preset unified document information extraction model, can automatically extract image information from the target document image, making it more intelligent and assisting relevant financial institutions in efficiently extracting image information.

[0054] Step 202: Input the target document image into a preset unified document information extraction model, wherein the unified document information extraction model contains information recognition templates corresponding to different versions and categories of document images as prior knowledge.

[0055] In this embodiment, the information recognition template includes a text information recognition template and an image information recognition template. Since different types of documents, such as ID card images, business license images, property ownership certificate images, marriage certificate images, vehicle registration certificates, and driver's license images, contain different text data, image data, and text location information, and even documents of the same category may differ in their format (e.g., second-generation ID card images and first-generation ID card images), deploying the information recognition templates corresponding to different formats and categories of document images as prior knowledge into the unified document information extraction model facilitates document image information extraction and improves data extraction efficiency.

[0056] In this embodiment, the unified extraction model for document information includes an input image tilt angle prediction component, a text feature extraction component, an image feature extraction component, and a feature encoding component.

[0057] In this embodiment, before performing the step of inputting the target document image into the preset unified document information extraction model, the method further includes a step of training the unified document information extraction model. Essentially, the purpose of training the unified document information extraction model is to learn the knowledge mapping relationship between document categories and formats and image feature information, as well as the knowledge mapping relationship between document categories and formats and text feature information, thereby constructing information recognition templates corresponding to different categories and formats of document images, and deploying them into the unified document information extraction model.

[0058] Continue to refer to Figure 3 , Figure 3 This is a flowchart of an embodiment of the document information unified extraction model learning and training in the document information automated identification and extraction method described in this application, including:

[0059] Step 301: Obtain a batch of document images, wherein the batch of document images includes document images of different categories and formats;

[0060] Step 302: Input the batch of document images into the unified document information extraction model;

[0061] Step 303: Train the input image tilt angle prediction component using the batch of document images to obtain the trained input image tilt angle prediction component, and obtain the tilt angle corresponding to each image in the batch of document images.

[0062] Specifically, since the acquisition of ID card images is mostly done by users uploading them themselves, which requires users to take photos before uploading, or by financial industry employees to take photos and upload them, the uploaded ID card images often have misalignment with the shooting frame. In this embodiment, before extracting information from the ID card image, an input image tilt angle prediction component is set up to quickly predict the tilt angle of the input ID card image in subsequent use, and to perform orientation correction in time before information extraction, so as to facilitate efficient extraction of data information from the ID card image.

[0063] In this embodiment, the step of training the input image tilt angle prediction component specifically includes: identifying the line where the text is located, calculating the angle between the line where the text is located and a preset horizontal line, and using the angle as the tilt angle.

[0064] Step 304: Correct the orientation of the input batch of document images according to the tilt angles corresponding to all images in the batch of document images to obtain the orientation-corrected batch of document images.

[0065] By performing orientation correction on the input batch of document images, orientation-corrected batch document images are obtained, which facilitates the subsequent extraction of data information from the document images.

[0066] Step 305: The text feature extraction component is used to extract text features from the batch of document images after orientation correction to obtain text feature information. The text feature information includes text features and position features. The text features include the content of text fields, and the position features include the position information of each text field content.

[0067] In this embodiment, the text feature extraction component includes a text extraction component based on OCR text recognition technology. Specifically, the location information can be obtained by calculating the coordinates of the four vertices of the frame based on the frame diagram corresponding to each text field content, and the location information can be represented by the coordinates of the four vertices of the frame. Similarly, the location information can also be represented by the coordinates of the center point of the frame based on the frame diagram corresponding to each text field content, and the location information can be represented by the coordinates of the center point of the frame. Alternatively, the location information can be represented by the distance information of the text field content from the top, bottom, left, and right sides of the four sides within the entire document image frame.

[0068] Continue to refer to Figure 4 , Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 305 shown includes:

[0069] Step 401: Input the orientation-corrected batch of document images into the text extraction component based on OCR text recognition technology;

[0070] Step 402: Extract the text field content from all document images using the text extraction component based on OCR text recognition technology;

[0071] Step 403: The text extraction component based on OCR text recognition technology identifies the location information of the text field content in all document images in the corresponding document images.

[0072] Step 404: Group and organize the text field content and the location information to obtain the content of each text field and the location information corresponding to each text field content;

[0073] In this embodiment, the text extraction component based on OCR text recognition technology is only used to extract the content of each text field and the location information corresponding to each text field content.

[0074] Step 405: Input the grouping results one by one into the preset embedding layer, and transform them through the embedding layer to obtain the word vectors, one-dimensional position vectors of words, and two-dimensional position vectors of words corresponding to different grouping results.

[0075] Step 406: Accumulate the word vectors, one-dimensional position vectors, and two-dimensional position vectors corresponding to the sorting results of the same group to obtain the text vector;

[0076] Step 407: Use the text vector as the text feature information of the text field content in the corresponding grouping and sorting results.

[0077] Specifically, the content of each text field and its corresponding positional information are extracted and input into a preset embedding layer. Through the embedding layer, the corresponding word vectors, one-dimensional word position vectors, and two-dimensional word position vectors are obtained. In other words, feature vector encoding is performed on the extracted text field content and positional information to obtain the text vector representation corresponding to each text field content. This text vector representation serves as the text feature information of the text field content. Vectorizing the text content and positional information and introducing feature vector encoding facilitates subsequent direct decoding to obtain the corresponding text vector representation.

[0078] Step 306: Extract image features from the batch of orientation-corrected document images using the image feature extraction component to obtain image feature information;

[0079] Continue to refer to Figure 5 , Figure 5 yes Figure 3 A flowchart of a specific embodiment of step 306 shown includes:

[0080] Step 501: Scale the batch of orientation-corrected document images to the size of the extraction box of the image feature extraction component.

[0081] Step 502: The scaled batch of document images are segmented and the segmented image blocks are position-encoded. The position encoding of the segmented image blocks follows the principle of gradually increasing encoding characters from top to bottom and from left to right. The encoding characters are from 1 to N, where N is a positive integer and the maximum value of N is the number of image blocks.

[0082] Step 503: Input the segmented image blocks into a preset linear mapping layer for linear mapping, and output the linear mapping sequence according to the position encoding;

[0083] Step 504: The linear mapping sequence is vectorized to obtain the vectorization result as the image vector of the corresponding document image;

[0084] Step 505: Use the image vector as the image feature information of the corresponding document image.

[0085] Similarly, the extracted image patches are linearly mapped to obtain a linear mapping sequence, and then vectorized. A feature vector encoding method is introduced to encode the image features, which makes it easier to obtain the corresponding image vector representation directly through subsequent decoding.

[0086] Step 307: The text feature information and image feature information of the same type and format of document images are grouped and sorted to obtain the grouping and sorting results of the batch of document images. The grouping and sorting results of the batch of document images vary depending on the type and format of the document images.

[0087] Step 308: Input the homogenization and sorting results of the batch of document images into the feature encoding component to be trained, train the feature encoding component, and obtain the trained feature encoding component.

[0088] In this embodiment, the feature encoding component includes a Transformer-based feature encoding component, which is capable of multimodal feature encoding, that is, it can encode both text features and image features.

[0089] Specifically, a random masking method is used to input the homogenized sorting results of the batch of document images into the feature encoding component to be trained, and the feature encoding component is trained to obtain the trained feature encoding component. The random masking method includes randomly masking a certain proportion of text feature information and / or randomly masking a certain proportion of image feature information in the homogenized sorting results when training the feature encoding component.

[0090] In this embodiment, a random masking method is used to randomly mask a certain proportion of text feature information and / or a certain proportion of image feature information in the homogenized sorting result. For example, 30% of the text feature information and 40% of the image feature information are randomly masked. This achieves masked text modeling and masked image modeling. Then, through training, the corresponding multimodal feature encoding and decoding components are obtained, so that even if the input target document image is missing, the fixed image data and fixed text data in the document can be identified as much as possible during subsequent actual use.

[0091] Step 309: Obtain the feature encoding results corresponding to different categories and different formats of document images output by the trained feature encoding component;

[0092] Step 310: By decoding the feature encoding results corresponding to the same type and format of document images, the image information and text information corresponding to the same type and format of document images are obtained.

[0093] In this embodiment, after performing the step of decoding the feature encoding results corresponding to document images of the same category and format to obtain image information and text information corresponding to document images of the same category and format, the method further includes: constructing a knowledge mapping relationship between document category and format and image feature information based on the image information corresponding to document images of the same category and format; generating image information recognition templates corresponding to document images of different formats and categories based on the knowledge mapping relationship between document category and format and image feature information; constructing a knowledge mapping relationship between document category and format and text feature information based on the text information corresponding to document images of the same category and format; and generating text information recognition templates corresponding to document images of different formats and categories based on the knowledge mapping relationship between document category and format and text feature information.

[0094] In essence, by processing the batch of document images, the system learns the knowledge mapping relationship between document categories and formats and image feature information, as well as the knowledge mapping relationship between document categories and formats and text feature information. This allows for the construction of information recognition templates corresponding to different categories and formats of document images, which are then deployed into the unified document information extraction model. In practical applications, after obtaining the text and image feature information of the target document image, the system can directly select the corresponding information recognition templates for text and image data recognition, facilitating rapid extraction based on the recognition results.

[0095] Step 203: Identify the text features, image features, and location features of the target document image using the unified document information extraction model.

[0096] Continue to refer to Figure 6 , Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 203 shown includes:

[0097] Step 601: Predict the tilt angle of the target document image using the trained input image tilt angle prediction component;

[0098] Step 602: Perform orientation correction on the target document image based on the tilt angle to obtain the orientation-corrected target document image;

[0099] Step 603: The text feature extraction component is used to extract text features from the orientation-corrected target document image to obtain text feature information. The text feature information includes text features and position features. The text features include text field content, and the position features include the position information of each text field content.

[0100] Step 604: The image feature extraction component is used to extract image features from the orientation-corrected target document image to obtain image feature information.

[0101] The text feature extraction component extracts text features from the orientation-corrected target document image to obtain text feature information, and the image feature extraction component extracts image features from the orientation-corrected target document image to obtain image feature information. This facilitates subsequent information recognition template selection, enabling the selection of text information recognition templates and image information recognition templates corresponding to the target document image.

[0102] Step 204: Based on the text features, image features, and location features, filter out the text information recognition template and image information recognition template corresponding to the target document image.

[0103] In this embodiment, the step of filtering the text information recognition template and image information recognition template corresponding to the target document image based on the text features, image features, and location features specifically includes: identifying the text information recognition template corresponding to the target document image based on the knowledge mapping relationship between the text feature information, document category, document format, and text feature information; and identifying the image information recognition template corresponding to the target document image based on the knowledge mapping relationship between the image feature information, document category, document format, and image feature information.

[0104] Specifically, based on the text and image features of the target document image, the corresponding information recognition templates are selected for text and image data recognition, facilitating rapid extraction based on the recognition results.

[0105] In this embodiment, after performing the step of filtering out the text information recognition template and image information recognition template corresponding to the target document image based on the text features, image features, and location features, the method further includes: if the text information recognition template and image information recognition template corresponding to the target document image cannot be filtered out based on the text features, image features, and location features, then the target document image is amplified using image amplification technology to obtain a batch of target document images; the batch of target document images is input into the unified document information extraction model; steps 303 to 310 are continued to obtain the image information and text information corresponding to the batch of target document images; based on the image information and text information corresponding to the batch of target document images, a knowledge mapping relationship is constructed between the category and layout of the target document image and the image feature information and text feature information, respectively; based on the knowledge mapping relationship between the category and layout of the target document image and the image feature information, an image information recognition template corresponding to the target document image is generated; and based on the knowledge mapping relationship between the category and layout of the target document image and the text feature information, a text information recognition template corresponding to the target document image is generated.

[0106] In essence, an incremental update method is adopted. When extracting information from a new document image, corresponding image information recognition templates and text information recognition templates can be constructed for the new document image. This makes it easier to directly select the image information recognition templates and text information recognition templates for information recognition and extraction when extracting information from the new document image in the future.

[0107] Step 205: Based on the text information recognition template, identify and extract the text data in the target document image.

[0108] Specifically, the text information recognition template can be understood as follows: based on the processing in step 202, and according to the text information extraction results, text information recognition templates are constructed for different versions and categories of document images. These templates can identify the location of text in the document image, thereby enabling rapid recognition and extraction of text data.

[0109] Step 206: Identify and extract image data from the target document image based on the image information recognition template.

[0110] Accordingly, the image information recognition template can be understood as, based on the processing in step 202, constructing image information recognition templates for different versions and categories of document images according to the image information extraction results. It can identify the location of the image in the document image, thereby achieving rapid recognition and extraction of image data.

[0111] Step 207: Filter out the text content to be corrected contained in the text data using a preset filter list.

[0112] In this embodiment, the preset filter list includes identification information for the text content to be corrected.

[0113] Specifically, the step of filtering out the text content to be corrected contained in the text data through a preset filtering list includes: determining whether the text data contains identification information of the text content to be corrected; if the text data contains the identification information, then identifying the text content to be corrected corresponding to the identification information; if the text data does not contain the identification information, then the text content to be corrected contained in the text data is a NULL value.

[0114] Specifically, for example, regarding company names and legal representative information, since changes to company names or legal representative information are involved, this application provides a revised solution when extracting information from previous business licenses. The purpose is to ensure that the extracted text content is the changed information, thereby improving the accuracy of text information extraction.

[0115] Step 208: Correct the text content to be corrected based on a preset correction strategy to obtain the corrected text content.

[0116] In this embodiment, the step of correcting the text content to be corrected based on a preset correction strategy to obtain the corrected text content specifically includes: determining whether the text content to be corrected is a NULL value; if the text content to be corrected is a NULL value, then outputting the NULL value as the corrected text content; if the text content to be corrected is not a NULL value, then obtaining the authoritative text content corresponding to the text content to be corrected from a preset authoritative information source, and correcting the text content to be corrected according to the authoritative text content to obtain the corrected text content.

[0117] Specifically, the step of correcting the text content to be corrected based on the authoritative text content to obtain the corrected text content includes: calculating the edit distance between the authoritative text content and the text content to be corrected using the Levenshtein distance algorithm; determining whether the edit distance exceeds a preset distance threshold by comparison; if the edit distance exceeds the preset distance threshold, directly replacing the text content to be corrected with the authoritative text content to obtain the corrected text content; if the edit distance does not exceed the preset distance threshold, identifying the difference text between the authoritative text content and the text content to be corrected, and using the difference text to correct the text content to obtain the corrected text content.

[0118] Specifically, the Levenshtein distance algorithm is an edit distance algorithm that calculates the edit distance between two comparison strings based on their differences. It can be understood as the correction step size required to convert a string into the string to be compared with it. If the step size is large, the authoritative text content is directly used to replace the text content to be corrected. If the step size is small, the difference text content can be used to correct the text content to be corrected.

[0119] Step 209: Output the image data, the uncorrected text content in the text data, and the corrected text content to complete the information extraction of the target document image.

[0120] This application can extract information from dozens of document types and formats with high accuracy without needing to know the document type, effectively saving manpower and making it more intelligent and automated.

[0121] This application completes the information extraction from the target document image by acquiring the target document image for information extraction; using text and image information recognition templates from a unified document information extraction model; extracting text and image data; filtering out text content to be corrected; correcting the text content to obtain the corrected text content; and outputting image data, text data, and the corrected text content. This application, through training a unified document information extraction model and introducing information recognition templates and text extraction correction strategies, achieves high-accuracy information extraction from dozens of document types and formats without needing to know the document type, effectively saving manpower and making it more intelligent and automated.

[0122] The embodiments of this application can acquire and process relevant data based on image processing technology. Image processing (Artificial Intelligence, AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0123] Fundamental image processing technologies generally include sensors, dedicated image processing chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. Image processing software technologies mainly include computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0124] In this embodiment, the process involves: acquiring a target document image for information extraction; using text and image information recognition templates from a unified document information extraction model; extracting text and image data; filtering out text content to be corrected; correcting the text content to obtain the corrected text content; and outputting image data, text data, and the corrected text content to complete the information extraction from the target document image. This application, through training a unified document information extraction model and introducing information recognition templates and text extraction correction strategies, achieves high-accuracy information extraction from dozens of document types and formats without needing to know the document type, effectively saving manpower and making the process more intelligent and automated.

[0125] Further reference Figure 7 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an automated document information recognition and extraction device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0126] like Figure 7 As shown, the document information automated recognition and extraction device 700 described in this embodiment includes: a target document image acquisition module 701, a target document image input module 702, a feature recognition module 703, an information recognition template filtering module 704, a text data extraction module 705, an image data extraction module 706, a text content filtering module 707, a text content correction module 708, and an information output module 709. Wherein:

[0127] The target document image acquisition module 701 is used to acquire the target document image from which information is to be extracted.

[0128] The target document image input module 702 is used to input the target document image into a preset unified document information extraction model, wherein the unified document information extraction model includes information recognition templates corresponding to different versions and categories of document images as prior knowledge.

[0129] The feature recognition module 703 is used to recognize the text features, image features, and position features of the target document image through the unified extraction model of the document information;

[0130] The information recognition template filtering module 704 is used to filter out the text information recognition template and image information recognition template corresponding to the target document image based on the text features, image features and location features;

[0131] The text data extraction module 705 is used to identify and extract text data from the target document image based on the text information recognition template.

[0132] Image data extraction module 706 is used to identify and extract image data from the target document image based on the image information recognition template;

[0133] The text content filtering module 707 is used to filter out the text content to be corrected contained in the text data through a preset filtering list.

[0134] The text content correction module 708 is used to correct the text content to be corrected based on a preset correction strategy to obtain the corrected text content.

[0135] The information output module 709 is used to output the image data, the uncorrected text content in the text data, and the corrected text content to complete the information extraction of the target document image.

[0136] This application completes the information extraction from the target document image by acquiring the target document image for information extraction; using text and image information recognition templates from a unified document information extraction model; extracting text and image data; filtering out text content to be corrected; correcting the text content to obtain the corrected text content; and outputting image data, text data, and the corrected text content. This application, through training a unified document information extraction model and introducing information recognition templates and text extraction correction strategies, achieves high-accuracy information extraction from dozens of document types and formats without needing to know the document type, effectively saving manpower and making it more intelligent and automated.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0138] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0139] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0140] The computer device 8 includes a memory 8a, a processor 8b, and a network interface 8c that are interconnected via a system bus. It should be noted that only the computer device 8 with components 8a-8c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital digital processors (DSPs), embedded devices, etc.

[0141] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0142] The memory 8a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 8a may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 8a may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), flash card, etc. Of course, the memory 8a may include both the internal storage unit and the external storage device of the computer device 8. In this embodiment, the memory 8a is typically used to store the operating system and various application software installed on the computer device 8, such as computer-readable instructions for an automated identification and extraction method for document information. In addition, the memory 8a can also be used to temporarily store various types of data that have been output or will be output.

[0143] In some embodiments, the processor 8b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 8b is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is used to execute computer-readable instructions stored in the memory 8a or to process data, such as executing computer-readable instructions for the automated identification and extraction method of document information.

[0144] The network interface 8c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 8 and other electronic devices.

[0145] The computer device proposed in this embodiment belongs to the field of image information processing technology and is applied to the scenario of automated extraction of document information. This application acquires the target document image from which information is to be extracted; uses text information recognition templates and image information recognition templates in a unified document information extraction model; extracts text data and image data; filters out text content to be corrected; corrects the text content to be corrected to obtain the corrected text content; and outputs image data, text data, and the corrected text content to complete the information extraction of the target document image. This application, through learning and training the unified document information extraction model and introducing information recognition templates and text extraction correction strategies, achieves high-accuracy information extraction for dozens of document types and formats without needing to know the document type, effectively saving manpower and making it more intelligent and automated.

[0146] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the document information automatic identification and extraction method described above.

[0147] The computer-readable storage medium proposed in this embodiment belongs to the field of image information processing technology and is applied to the scenario of automated extraction of document information. This application acquires the target document image from which information is to be extracted; uses text information recognition templates and image information recognition templates in a unified document information extraction model; extracts text data and image data; filters out text content to be corrected; corrects the text content to be corrected to obtain the corrected text content; and outputs image data, text data, and the corrected text content to complete the information extraction of the target document image. This application, through learning and training the unified document information extraction model and introducing information recognition templates and text extraction correction strategies, achieves high-accuracy information extraction for dozens of document types and formats without needing to know the document type, effectively saving manpower and making it more intelligent and automated.

[0148] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0149] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for automatically identifying and extracting document information, characterized in that, Includes the following steps: Acquire the image of the target document from which information is to be extracted; The target document image is input into a preset unified document information extraction model. This model includes information recognition templates corresponding to different formats and categories of document images, serving as prior knowledge. The model includes an input image tilt angle prediction component, a text feature extraction component, an image feature extraction component, and a feature encoding component. Before performing the step of inputting the target document image into the preset unified document information extraction model, the method further includes: Acquire a batch of document images; specifically, the batch of document images includes document images of different categories and formats. The batch of document images are input into the unified document information extraction model; The input image tilt angle prediction component is trained using the batch of document images to obtain a trained input image tilt angle prediction component, and the tilt angles corresponding to each image in the batch of document images are obtained. The input batch of document images are oriented according to the tilt angles corresponding to all images in the batch of document images, so as to obtain the oriented oriented batch of document images. The text feature extraction component extracts text features from the orientation-corrected batch of document images to obtain text feature information. Specifically, the text feature information includes text features and position features. The text features include the content of text fields, and the position features include the position information of each text field. The image feature extraction component is used to extract image features from the batch of orientation-corrected document images to obtain image feature information. Textual and image feature information of document images of the same category and format are grouped and organized to obtain the grouping and organization results of the batch of document images. Specifically, the grouping and organization results of the batch of document images vary depending on the category and format of the document images. The homogenization and sorting results of the batch of document images are input into the feature encoding component to be trained, and the feature encoding component is trained to obtain the trained feature encoding component. Obtain the feature encoding results corresponding to different categories and different formats of document images output by the trained feature encoding component; By decoding the feature encoding results corresponding to document images of the same category and format, image information and text information corresponding to document images of the same category and format can be obtained. The unified extraction model for document information is used to identify the text features, image features, and location features of the target document image. Based on the text features, image features, and location features, the text information recognition template and image information recognition template corresponding to the target document image are selected; Based on the text information recognition template, text data is identified and extracted from the target document image; Based on the image information recognition template, image data is identified and extracted from the target document image; The text content to be corrected contained in the text data is filtered out using a preset filter list. The text content to be corrected is corrected based on a preset correction strategy to obtain the corrected text content; Output the image data, the uncorrected text content in the text data, and the corrected text content to complete the information extraction of the target document image.

2. The method for automated identification and extraction of document information according to claim 1, characterized in that, The text feature extraction component includes a text extraction component based on OCR text recognition technology. The step of extracting text features from a batch of orientation-corrected document images using the text feature extraction component to obtain text feature information specifically includes: The orientation-corrected batch of document images are input into the text extraction component based on OCR text recognition technology; The text extraction component based on OCR text recognition technology is used to extract the text field content from all document images respectively; The text extraction component based on OCR text recognition technology identifies the location information of the text field content in the corresponding document images of all document images; The text field content and the location information are grouped and organized to obtain the content of each text field and the location information corresponding to each text field content; The grouping and sorting results are input into the preset embedding layer one by one, and the word vectors, one-dimensional position vectors of words, and two-dimensional position vectors of words are obtained through the transformation of the embedding layer. The text vector is obtained by summing the word vectors, one-dimensional position vectors, and two-dimensional position vectors corresponding to the same grouping results. The text vector is used as the text feature information of the text field content in the corresponding grouping and sorting results.

3. The method for automated identification and extraction of document information according to claim 1, characterized in that, The step of extracting image features from the orientation-corrected batch of document images using the image feature extraction component to obtain image feature information specifically includes: The batch of document images after orientation correction are uniformly scaled to the size of the extraction box of the image feature extraction component; The scaled batch of document images are segmented, and the segmented image blocks are position-encoded. The position encoding of the segmented image blocks follows the principle of gradually increasing encoding characters from top to bottom and from left to right. The encoding characters are from 1 to N, where N is a positive integer and the maximum value of N is the number of image blocks. The segmented image blocks are input into a preset linear mapping layer for linear mapping, and a linear mapping sequence is output according to the position encoding. The linear mapping sequence is vectorized to obtain the vectorized result as the image vector of the corresponding document image; The image vector is used as the image feature information of the corresponding document image.

4. The method for automated identification and extraction of document information according to claim 1, characterized in that, After performing the step of decoding the feature encoding results corresponding to document images of the same category and format to obtain image information and text information corresponding to document images of the same category and format, the method further includes: Based on the image information corresponding to document images of the same category and format, construct a knowledge mapping relationship between document category and document format and image feature information; Based on the knowledge mapping relationship between document type and document format and image feature information, image information recognition templates corresponding to different document formats and categories are generated. Based on the text information corresponding to document images of the same category and format, construct a knowledge mapping relationship between document category and document format and text feature information; Based on the knowledge mapping relationship between document type and format and text feature information, text information recognition templates are generated for document images of different formats and categories.

5. The method for automated identification and extraction of document information according to claim 1, characterized in that, The step of identifying the text features, image features, and location features of the target document image using the unified extraction model of the document information specifically includes: The tilt angle of the target document image is predicted using the trained input image tilt angle prediction component. The target document image is oriented based on the tilt angle to obtain the oriented oriented target document image. The text feature extraction component extracts text features from the orientation-corrected target document image to obtain text feature information, wherein the text feature information includes text features and position features, the text features include text field content, and the position features include the position information of each text field content; The image feature extraction component is used to extract image features from the orientation-corrected target document image to obtain image feature information.

6. The method for automated identification and extraction of document information according to claim 4, characterized in that, The step of filtering out the text information recognition template and image information recognition template corresponding to the target document image based on the text features, image features, and positional features specifically includes: Based on the knowledge mapping relationship between the text feature information, document type and document format and text feature information, the text information recognition template corresponding to the target document image is identified; Based on the knowledge mapping relationship between the image feature information, document type and document format and image feature information, the image information recognition template corresponding to the target document image is identified.

7. The method for automated identification and extraction of document information according to claim 1, characterized in that, The preset filter list contains identifiers for the text content to be corrected. The step of filtering out the text content to be corrected from the text data using the preset filter list specifically includes: Determine whether the text data contains identification information of the text content to be corrected; If the text data contains the identification information, then the text content to be corrected corresponding to the identification information is identified; If the text data does not contain the identification information, then the text content to be corrected contained in the text data is NULL. The step of correcting the text content to be corrected based on a preset correction strategy to obtain the corrected text content specifically includes: Determine whether the text to be corrected is a NULL value; If the text to be corrected is NULL, then output NULL as the corrected text content; If the text content to be corrected is not NULL, then the authoritative text content corresponding to the text content to be corrected is obtained from a preset authoritative information source, and the text content to be corrected is corrected according to the authoritative text content to obtain the corrected text content.

8. An automated identification and extraction device for document information, characterized in that, The document information automated identification and extraction device implements the steps of the document information automated identification and extraction method as described in any one of claims 1 to 7, wherein the document information automated identification and extraction device comprises: The target document image acquisition module is used to acquire the target document image from which information is to be extracted; The target document image input module is used to input the target document image into a preset unified document information extraction model, wherein the unified document information extraction model contains information recognition templates corresponding to different versions and categories of document images as prior knowledge. The feature recognition module is used to identify the text features, image features, and location features of the target document image through a unified extraction model based on the document information. The information recognition template filtering module is used to filter out the text information recognition template and image information recognition template corresponding to the target document image based on the text features, image features and location features; The text data extraction module is used to identify and extract text data from the target document image based on the text information recognition template; The image data extraction module is used to identify and extract image data from the target document image based on the image information recognition template; The text content filtering module is used to filter out the text content to be corrected contained in the text data through a preset filtering list. The text content correction module is used to correct the text content to be corrected based on a preset correction strategy to obtain the corrected text content. The information output module is used to output the image data, the uncorrected text content in the text data, and the corrected text content, thereby completing the information extraction of the target document image.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the document information automatic identification and extraction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the document information automatic identification and extraction method as described in any one of claims 1 to 7.

Images