Business document image content recognition method and device

Abstract

The present invention provides a method and device for identifying the content of a business document image, which is applicable to the field of the Internet of Things. The method includes: performing text recognition on the business document image to be identified to generate text information of the business document including identified text and text position information; generating candidate target paragraphs based on the identified text, text position information and a preset semantic discrimination model; and generating a structured business document content recognition result based on the identified text and the candidate target paragraphs using a pre-trained text classification model. The present invention can be widely used in business scenarios based on text recognition results, and is particularly suitable for images with non-standardized image organization structures and a variety of styles. It can realize structured extraction of image data based on text recognition results in the same way. It can effectively realize structured extraction of complex image data on the basis of a general implementation that does not rely on specific text recognition, and ultimately improve the accuracy of the overall task of automated processing of text recognition.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to a method and device for identifying the image content of a business document. Background Art

[0002] In international businesses such as letters of credit, customers need to provide documents such as invoices, insurance policies and bills of lading for bank review. These cross-border or regional transactions and settlements often involve document images of different companies in different formats, such as tabular, columnar or open.

[0003] In the existing technology, although OCR technology can effectively identify the content of images, it cannot effectively organize the results after identification. Therefore, currently related international business is still mainly processed manually by business personnel. Summary of the invention

[0004] In order to overcome at least one defect of business document image recognition in the prior art, the present invention provides a business document image content recognition method, comprising:

[0005] Performing text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information;

[0006] Generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model;

[0007] A structured business document content recognition result is generated based on the recognized text and candidate target paragraphs using a pre-trained text classification model.

[0008] In an embodiment of the present invention, the step of generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model includes:

[0009] Determine a paragraph candidate area according to the text position information;

[0010] The preset semantic discrimination model is used to perform semantic discrimination on the recognized text in the paragraph candidate area to generate a candidate target paragraph.

[0011] In an embodiment of the present invention, the text information further includes: information about the font size of the text, information about the spacing between the text and information about the height of the text; and determining the candidate paragraph area according to the text position information includes:

[0012] Determining an initial paragraph candidate region according to the text position information;

[0013] The paragraph candidate region is determined according to the font size, the distance between characters and the height of characters in the initial paragraph candidate region.

[0014] In the embodiment of the present invention, the method of generating a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model includes:

[0015] Determining keywords present in the recognized text according to preset keywords;

[0016] Determine a candidate target paragraph corresponding to the keyword according to the determined position information of the keyword;

[0017] The pre-trained text classification model is used to match and parse keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

[0018] In the embodiment of the present invention, the step of generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model further includes:

[0019] Obtain pre-stored business text data;

[0020] The business text data is used to perform model training on a preset initial language model to generate a preset semantic discrimination model.

[0021] In the embodiment of the present invention, the method of generating a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model includes:

[0022] Acquire pre-stored business key-value pair data; wherein the business key-value pair data includes: a business keyword and paragraph data corresponding to the keyword;

[0023] The preset initial text classification model is trained using the keywords of the business and the paragraph data corresponding to the keywords to generate a preset text classification model.

[0024] At the same time, the present invention also provides a business document image content recognition device, comprising:

[0025] A text recognition module, used to perform text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information;

[0026] A paragraph generation module, used to generate candidate target paragraphs according to the recognized text, text position information and a preset semantic discrimination model;

[0027] The matching module is used to generate a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model.

[0028] In an embodiment of the present invention, the paragraph generation module includes:

[0029] A paragraph candidate region determining unit, configured to determine a paragraph candidate region according to the text position information;

[0030] The discrimination unit is used to perform semantic discrimination on the recognized text in the paragraph candidate area using a preset semantic discrimination model to generate a candidate target paragraph.

[0031] In the embodiment of the present invention, the matching module includes:

[0032] A keyword search unit, which determines the keywords present in the recognized text according to preset keywords;

[0033] A candidate target paragraph determining unit, which determines a candidate target paragraph corresponding to the keyword according to the determined position information of the keyword;

[0034] The classification and matching unit is used to use a pre-trained text classification model to match and parse keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

[0035] At the same time, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the above method is implemented when the processor executes the computer program.

[0036] At the same time, the present invention also provides a computer-readable storage medium, which stores a computer program for executing the above method.

[0037] The business document image content recognition method and device provided by the present invention perform text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text location information; generate candidate target paragraphs based on the recognized text, text location information and a preset semantic discrimination model; and generate structured business document content recognition results using a pre-trained text classification model based on the recognized text and candidate target paragraphs. A device for iterative structured extraction of image recognition results based on path analysis and semantic understanding is proposed, which can be widely used in business scenarios based on OCR recognition results, and is particularly suitable for images with non-standardized image organization structures and various styles, and can achieve structured extraction of image data based on OCR recognition results in the same way. The present invention is a beneficial supplement to existing OCR technology. Through this method, it is possible to effectively achieve structured extraction of complex image data on the basis of a general method that does not rely on specific OCR implementations, and ultimately improve the accuracy of the overall task of automated processing based on OCR technology.

[0038] In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of the business document image content recognition method provided by the present invention;

[0041] Figure 2 A block diagram of a system for structured extraction of image recognition results provided by an embodiment of the present invention;

[0042] Figure 3 is a screenshot of the image to be recognized in the embodiment of the present invention;

[0043] Figure 4 is a block diagram in an embodiment of the present invention;

[0044] Figure 5 is a screenshot of the image to be recognized in the embodiment of the present invention;

[0045] Figure 6 is a screenshot of the image to be recognized in the embodiment of the present invention;

[0046] Figure 7 is a screenshot of the image to be recognized in the embodiment of the present invention;

[0047] Figure 8 is a block diagram in an embodiment of the present invention;

[0048] Fig. 9 is a screenshot of the image to be recognized in the embodiment of the present invention;

[0049] Fig.10 is a flow chart in an embodiment of the present invention;

[0050] Fig.11 A block diagram of a business document image content recognition device provided by the present invention;

[0051] Fig.12 is a block diagram in an embodiment of the present invention;

[0052] Fig.13 is a block diagram in an embodiment of the present invention;

[0053] Fig.14 A schematic diagram of an electronic device embodiment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In order to solve the problems in the prior art, overcome the defects in the recognition of business document images in the prior art, improve business processing efficiency, and realize automatic entry of transaction data and intelligent document review, the present invention provides a general image content structuring organization technology based on OCR recognition results.

[0056] like Figure 1 As shown, the business document image content recognition method provided by the present invention includes:

[0057] Step S10, performing text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information;

[0058] Step S20, generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model;

[0059] Step S30, generating a structured business document content recognition result using a pre-trained text classification model according to the recognized text and candidate target paragraphs.

[0060] The business document image content recognition method provided by the present invention completes the paragraph composition path analysis by analyzing the spatial relative position between the recognized text areas and the NLP (Natural Language Processing) word order discrimination, and aggregates the valid text areas to form paragraphs. According to the recognized text and the candidate target paragraphs, a pre-trained text classification model is used to complete the aggregation of key-value pairs by analyzing the relative position relationship between the paragraph and the target key and text classification, thereby realizing the structuring of the image recognition content.

[0061] In an embodiment of the present invention, the step of generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model includes:

[0062] Determine a paragraph candidate area according to the text position information;

[0063] The preset semantic discrimination model is used to perform semantic discrimination on the recognized text in the paragraph candidate area to generate a candidate target paragraph.

[0064] In the embodiment of the present invention, by analyzing the relative position relationship between text regions and a preset trained semantic continuity discrimination model, it is identified and determined whether the connected text regions belong to the same paragraph, and the originally scattered recognition results are aggregated into semantically continuous and complete paragraphs.

[0065] Through the above processing, potential text areas belonging to the same paragraph are found, which can reduce the amount of data used in the subsequent semantic discrimination of the recognized text in the candidate paragraph area using a preset semantic discrimination model, and further improve the efficiency of text recognition.

[0066] In an embodiment of the present invention, the text information further includes: information about the font size of the text, information about the spacing between the text and information about the height of the text; and determining the candidate paragraph area according to the text position information includes:

[0067] Determining an initial paragraph candidate region according to the text position information;

[0068] The paragraph candidate region is determined according to the font size, the distance between characters and the height of characters in the initial paragraph candidate region.

[0069] Specifically, in the embodiment of the present invention, the relative position relationship of the text regions where the text is located is determined through the text information, so as to judge whether two text regions potentially belong to the same paragraph, and find the text regions that potentially belong to the same paragraph.

[0070] In a specific embodiment of the present invention, potential text regions belonging to the same paragraph are screened out based on the font size information, the spacing distance between the texts, and the text height information. Specifically, in this embodiment, two text regions belonging to the same paragraph meet one or more of the following characteristics:

[0071] 1. The size of individual characters in two text areas should be basically the same. For example, in the following figure, 1 and 2 belong to the same paragraph, but 2 and 3 do not, because the size of individual characters in 2 and 3 text areas is too different.

[0072] 2. Two text areas should be on the same horizontal line, and the distance between their horizontal axes should be about the width of a single word, and generally not more than two word widths. For example, 1 and 2 in the figure below should belong to the same paragraph, while 3 and 4 in the lower right figure do not belong to the same paragraph because the distance between 3 and 4 is too large, which is greater than the width of a single text.

[0073] 3. The vertical distance between two text areas should be about the height of a single character, generally not more than two characters, and the starting horizontal coordinates of the areas should be basically the same. For example, in the figure below, areas 1 and 2 belong to the same paragraph, while areas 2 and 3 should not belong to the same paragraph because the vertical distance between the two areas is too large.

[0074] In the embodiment of the present invention, the method of generating a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model includes:

[0075] Determining keywords present in the recognized text according to preset keywords;

[0076] Determine a candidate target paragraph corresponding to the keyword according to the determined position information of the keyword;

[0077] The pre-trained text classification model is used to match and parse keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

[0078] In the embodiment of the present invention, the target key and its position in the image are found by keyword search.

[0079] Specifically, the target value (paragraph) should be immediately to the right, below or in the lower right area of ​​the target key, and the line connecting the target key and the target value will not pass through another text paragraph area. The candidate paragraphs belonging to the target key are found in the above manner. Thus, the pre-trained text classification model is used to match and parse the keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

[0080] In the embodiment of the present invention, the step of generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model further includes:

[0081] Obtain pre-stored business text data;

[0082] The business text data is used to perform model training on a preset initial language model to generate a preset semantic discrimination model.

[0083] Specifically, two pieces of text that potentially constitute the same paragraph are connected, and the connected text is judged by a preset trained semantic discrimination model to determine whether the text is valid and continuous. If it is valid, the two texts are combined into a new text region. The above text region aggregation and discrimination process is iterated until no new text that meets the conditions for aggregation is found, and paragraph aggregation is completed.

[0084] Specifically, in the embodiment of the present invention, a semantic continuity judgment model is trained by collecting enough valid texts in the processed field.

[0085] The preset semantic continuity judgment model in this embodiment, namely the semantic discrimination model, includes: a statistical language model based on the Bayesian principle or a language model based on deep learning. Any language model that can realize the functions of the semantic discrimination model involved in this embodiment is acceptable, and is not limited to this embodiment.

[0086] The semantic discrimination model inputs a text and outputs whether the text is a continuous and valid expression. In addition, in the embodiment of the present invention, the model generation unit is executed once when the system is initialized, and is executed in a timely manner while the system is running and continuously accumulating training samples.

[0087] In the embodiment of the present invention, the method of generating a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model includes:

[0088] Acquire pre-stored business key-value pair data; wherein the business key-value pair data includes: a business keyword and paragraph data corresponding to the keyword;

[0089] The preset initial text classification model is trained using the keywords of the business and the paragraph data corresponding to the keywords to generate a preset text classification model.

[0090] Use enough valid key-value pairs in the business field to be processed to train a text classification model. In this embodiment, a deep learning text classification model such as Fasttext, Transformer, or BERT can be selected.

[0091] The model input is a piece of text, and the output is the key to which the text belongs. It is executed once at the beginning, and training samples are accumulated and executed in due time.

[0092] The candidate paragraphs found are passed through the text classification model in turn to check whether the value output by the model is consistent with the target key. If they are consistent, the paragraph of text is the value of the target key. The above process is iterated to complete the parsing of all key-value pairs.

[0093] The business document image content recognition method provided by the present invention can be widely used in business scenarios based on OCR recognition results, and is particularly suitable for images with non-standardized image organization structures and various styles, and can achieve structured extraction of image data based on OCR recognition results in the same way. The present invention is a beneficial supplement to existing OCR technology. Through this method, it is possible to effectively achieve structured extraction of complex image data on a general basis that does not rely on specific OCR implementations, and ultimately improve the accuracy of the overall automated processing task based on OCR technology. OCR (Optical Character Recognition) technology is a technology that directly converts the text content on images and photos into editable text.

[0094] The technical solution of the present invention is further described in detail below in conjunction with specific embodiments:

[0095] This embodiment proposes an iterative structured extraction device for image recognition results based on path analysis and semantic understanding, which analyzes and identifies the spatial relative positions between text regions, completes the path analysis of paragraph composition with NLP word order discrimination, and aggregates valid text regions to form paragraphs. Then, by analyzing the relative position relationship between paragraphs and target keys, and completing the aggregation of key-value pairs by text classification, the image recognition content is structured.

[0096] like Figure 2 , is a block diagram of a system for structured extraction of image recognition results provided by an embodiment of the present invention, comprising: an OCR positioning and recognition device 1, a text paragraph aggregation device 2, and a target key matching device 3.

[0097] The OCR positioning and recognition device 1 is connected to the text paragraph aggregation device 2; the OCR positioning and recognition device 1 and the text paragraph aggregation device 2 are both connected to the target key matching device 3.

[0098] OCR positioning and recognition device 1: used to perform OCR recognition on the image to obtain the recognized text and the position of the text in the image.

[0099] Text paragraph aggregation device 2: Through the relative position relationship analysis between text regions and the semantic continuity discrimination model, it identifies and determines whether the connected text regions belong to the same paragraph, and aggregates the originally scattered recognition results into a semantically continuous and complete paragraph.

[0100] Target key matching device 3: find the position of the target key in the recognition result by searching for keywords, and then find the paragraph belonging to the target key, that is, the value of the target key, by analyzing the relative position of the target key and the candidate paragraphs and using the text classification method. Iterate the above process to complete the matching of all target key-value pairs and realize the structuring of OCR recognition results.

[0101] The embodiment of the present invention proposes a method for structurally organizing image recognition results based on relative spatial position analysis, NLP semantic discrimination and text classification, which mainly includes the following steps:

[0102] Step 1): Apply the OCR positioning and recognition device to the image to be analyzed to obtain the recognition result text and the position information of the text area corresponding to the recognition result in the image. In this embodiment, the position information of the image of the rectangular text area can be represented by the left vertex pixel coordinates and the length and width pixel values ​​of the area.

[0103] Step 2): The OCR recognition result containing the position information outputted in step 1 is passed through a text paragraph aggregation device to complete paragraph aggregation, thereby combining the originally scattered text areas into semantically continuous paragraphs.

[0104] Step 3): The output results of step 1 and step 2 are passed through the target key matching device to complete the matching of the target key-value pairs and complete the structuring of the OCR recognition results.

[0105] The system and method of the present invention will be described in detail below with reference to the accompanying drawings.

[0106] The OCR positioning and recognition device 1 is used to recognize the image text and the coordinate position of the text in the image. In this embodiment, the OCR recognition engine can be selected from Tesseract. Figure 3 As shown, the recognition results are as follows, including the recognized text and the position coordinates of the corresponding text area in the image.

[0107]

[0108]

[0109] Figure 4 is a block diagram of a text paragraph aggregation device 2 in an embodiment of the present invention, such as Figure 4 As shown, the text paragraph aggregation device 2 includes: a paragraph candidate region search unit 21, a semantic discrimination model generation unit 22 and a paragraph composition semantic discrimination unit 23, wherein:

[0110] The paragraph candidate region searching unit 21 determines whether two text regions potentially belong to the same paragraph based on their relative position relationship in the result output by the OCR positioning and recognition device 1, and finds a pair of text regions potentially belonging to the same paragraph.

[0111] In this embodiment, two text regions belonging to the same paragraph satisfy one or more of the following characteristics:

[0112] Feature 1: The size of each character in two text areas should be roughly the same. Figure 5 Box 1 and box 2 belong to the same paragraph, but box 2 and box 3 do not belong to the same paragraph because the size of individual characters in the text area of ​​box 2 and box 3 is too different.

[0113] Feature 2: Two text areas should be on the same horizontal line, and the distance between their horizontal axes should be about the width of a single character, and generally not more than two character widths. Figure 6 1 and 2 should belong to the same paragraph, and Figure 6 Box 3 and Box 4 do not belong to the same paragraph because the space between Box 3 and Box 4 is too large, which is greater than the width of their single text.

[0114] Feature 3: The vertical distance between two text areas should be about the height of a single character, generally not more than two characters, and the starting horizontal coordinates of the areas should be basically the same. For example, Figure 7 Box 1 and Box 2 belong to the same paragraph, while Box 2 and Box 3 should not belong to the same paragraph because the vertical distance between the two areas is too large.

[0115] Semantic discrimination model generation unit 22: collects valid texts that meet the preset number of processed fields and trains a semantic continuity judgment model. In this embodiment, a statistical language model based on the Bayesian principle or a language model based on deep learning can be selected. The input of the language model is a piece of text, and the output is whether the text is a continuous and valid expression. The model generation unit is executed once at the initialization of the system, and is executed in a timely manner as the training samples are continuously accumulated during the operation of the system.

[0116] The paragraph composition semantic discrimination unit 23 connects the two texts that potentially constitute the same paragraph output by the paragraph candidate region search unit 21, and passes the connected text through the semantic discrimination model to determine whether the text is a valid and continuous text. If it is a valid text, the two texts are combined into a new text region.

[0117] The above text region aggregation and discrimination process is iterated until no new text that meets the conditions for aggregation is found, and paragraph aggregation is completed.

[0118] Figure 8 is a block diagram of the target key matching device 3, such as Figure 8 As shown, the target key matching device 3 includes: a candidate target paragraph search unit 31, a text classification model generation unit 32 and a target key value classification matching unit 33, wherein:

[0119] The candidate target paragraph searching unit 31 finds the target key and its position in the image in the result output by the OCR positioning and recognition device 1 by means of keyword searching.

[0120] In this embodiment, the target value (paragraph) should be immediately to the right, below or in the lower right area of ​​the target key, and the line connecting the target key and the target value will not pass through another text paragraph area. The candidate paragraph belonging to the target key is found in the above manner.

[0121] Examples include Fig. 9As shown, for the target key "Notify Party" (area 3), text area 4 is its candidate target value, while text area 2 is not its candidate target value because area 2 is above "Notify Party"; for the target key "Consignee" (area 1), text area 2 is its candidate target value, while text area 4 is not its candidate target value because the line connecting area 1 and area 4 passes through another text area 2.

[0122] Text classification model generation unit 32: Use enough valid key-value pairs in the processing field to train a text classification model. Fasttext, Transformer, BERT and other deep learning text classification models can be selected. The model input is a piece of text, and the output is the key to which the text belongs. The model generation unit is executed once at the initialization of the system, and is executed in a timely manner as training samples are continuously accumulated during the operation of the system.

[0123] Target key value classification matching unit 33: The candidate paragraphs found by the candidate target paragraph search unit 31 are sequentially passed through the text classification model to check whether the value output by the model is consistent with the target key. If consistent, the paragraph of text is the value of the target key. The above process is iterated to complete the parsing of all key-value pairs.

[0124] like Fig.10 As shown, the processing flow of the image content recognition by the image recognition result structured extraction system provided in this embodiment includes the following steps:

[0125] Step S101: initializing the application system;

[0126] Step S102: generating an initial semantic discrimination model;

[0127] Step S103: generating an initial text classification model;

[0128] Step S104: the application system starts to start;

[0129] Step S105: obtaining an image to be processed;

[0130] Step S106: Perform OCR positioning recognition on the image to obtain recognition content and location information;

[0131] Step S107: finding potential candidate text regions belonging to the same paragraph based on the relative position relationship between the text regions;

[0132] Step S108: After connecting the two candidate text blocks, the semantic discrimination model is used to determine whether the connected text is coherent and valid. If not, the process returns to step S107 to search for new candidate regions.

[0133] Step S109: If valid, they are assigned to the same paragraph and aggregated into a new text region. Iterate steps S107 to S109 until all text regions are aggregated to form a valid paragraph;

[0134] Step S110: Find the target key and its position in the image in the recognition result by keyword search;

[0135] Step S111: Find the candidate target paragraph of the target key through the relative position relationship between the key and the value;

[0136] Step S112: Using the text classification model, determine whether the candidate target paragraph belongs to the target key. If not, return to step S111 to search for a new candidate target paragraph.

[0137] Step S113: If yes, the matching of the target key-value pairs is completed. Then the training data is expanded and steps S110 to S113 are iterated until all the target key-value pairs are matched. The structuring of the image recognition results is completed.

[0138] At the same time, the present invention also provides a business document image content recognition device, such as Fig.11 As shown, including:

[0139] A text recognition module 111 is used to perform text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information;

[0140] A paragraph generation module 112, for generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model;

[0141] The matching module 113 is used to generate a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model.

[0142] In the embodiment of the present invention, Fig.12 As shown, the paragraph generation module 112 includes:

[0143] A paragraph candidate region determining unit 1121, configured to determine a paragraph candidate region according to the text position information;

[0144] The discrimination unit 1122 is used to perform semantic discrimination on the recognized text in the paragraph candidate area by using a preset semantic discrimination model to generate a candidate target paragraph.

[0145] In the embodiment of the present invention, Fig.13 As shown, the matching module 113 includes:

[0146] A keyword search unit 1131 determines keywords existing in the recognized text according to preset keywords;

[0147] A candidate target paragraph determining unit 1132 determines a candidate target paragraph corresponding to the keyword according to the determined position information of the keyword;

[0148] The classification and matching unit is used to use a pre-trained text classification model to match and parse keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

[0149] For those skilled in the art, the implementation method of the business document image content recognition device provided by the present invention can be clearly known through the description of the above embodiments, and will not be repeated here.

[0150] It should be noted that the business document image content recognition method and device provided by the present invention can be used for business document image content recognition in the field of artificial intelligence, can also be used for business document image content recognition in the field of Internet of Things, can also be used for business document image content recognition in the financial field, and can also be used in any field except the financial field. The application field of the method and device for business document image content recognition disclosed in the present invention is not limited.

[0151] This embodiment also provides an electronic device, which may be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited thereto. In this embodiment, the electronic device may refer to the embodiments of the aforementioned method and device, the contents of which are incorporated herein, and the repeated parts are not repeated.

[0152] Fig.14 FIG. 6 is a schematic block diagram of a system structure of an electronic device 600 according to an embodiment of the present invention. Fig.14 As shown, the electronic device 600 may include a central processor 100 and a memory 140; the memory 140 is coupled to the central processor 100. It should be noted that this figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0153] In one embodiment, the business document image content recognition function may be integrated into the central processor 100. The central processor 100 may be configured to perform the following control:

[0154] Performing text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information;

[0155] Generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model;

[0156] A structured business document content recognition result is generated based on the recognized text and candidate target paragraphs using a pre-trained text classification model.

[0157] In another embodiment, the business document image content recognition device can be configured separately from the central processor 100. For example, the business document image content recognition device can be configured as a chip connected to the central processor 100, and the business document image content recognition function is realized through the control of the central processor.

[0158] like Fig.14 As shown, the electronic device 600 may further include: a communication module 110, an input unit 120, an audio processing unit 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily have to include Fig.14 In addition, the electronic device 600 may also include Fig.14 For components not shown, reference may be made to the prior art.

[0159] like Fig.14 As shown, the central processor 100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The central processor 100 receives inputs and controls the operations of various components of the electronic device 600.

[0160] The memory 140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 100 may execute the program stored in the memory 140 to implement information storage or processing.

[0161] The input unit 120 provides input to the CPU 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0162] The memory 140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 140 may also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 600 through the central processor 100.

[0163] The memory 140 may also include a data storage unit 143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0164] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processor 100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0165] Based on different communication technologies, multiple communication modules 110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide an audio output via the speaker 131 and receive an audio input from the microphone 132, thereby realizing a common telecommunication function. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 130 is also coupled to the central processor 100, so that the sound can be recorded on the local machine through the microphone 132, and the sound stored on the local machine can be played through the speaker 131.

[0166] An embodiment of the present invention further provides a computer-readable program, wherein when the program is executed in an electronic device, the program enables a computer to execute the business document image content recognition method as described in the above embodiment in the electronic device.

[0167] An embodiment of the present invention further provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the business document image content recognition described in the above embodiment in an electronic device.

[0168] The embodiment of the present invention proposes an iterative structured extraction device for image recognition results based on path analysis and semantic understanding, which can be widely used in business scenarios based on OCR recognition results, and is particularly suitable for images with non-standardized image organization structures and various styles, and can achieve structured extraction of image data based on OCR recognition results in the same way. The present invention is a beneficial supplement to the existing OCR technology. Through this method, it is possible to effectively achieve structured extraction of complex image data on the basis of a general method that does not rely on specific OCR implementations, and ultimately improve the accuracy of the overall task of automated processing based on OCR technology.

[0169] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings. Many features and advantages of these embodiments are clear from this detailed description, and therefore the appended claims are intended to cover all such features and advantages of these embodiments that fall within their true spirit and scope. In addition, since many modifications and changes are readily apparent to those skilled in the art, the embodiments of the present invention are not intended to be limited to the precise structure and operation illustrated and described, but all suitable modifications and equivalents that fall within their scope may be included.

[0170] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying content of business document images, characterized in that: The method includes: Performing text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information; Generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model; Generate a structured business document content recognition result using a pre-trained text classification model according to the recognized text and candidate target paragraphs; The generating of candidate target paragraphs according to the recognized text, text position information and a preset semantic discrimination model includes: Determining an initial paragraph candidate region according to the text position information; According to the font size, character spacing and character height of the characters in the initial paragraph candidate regions, the multiple initial paragraph candidate regions are classified into the same paragraph candidate region; Using a preset semantic discrimination model to perform semantic discrimination on the recognized text in the paragraph candidate area to generate a candidate target paragraph; Wherein, generating a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model further includes: Obtain pre-stored business text data; Using the business text data to perform model training on a preset initial language model to generate a preset semantic discrimination model; The structured business document content recognition results generated by using the pre-trained text classification model according to the recognized text and candidate target paragraphs include: Acquire pre-stored business key-value pair data; wherein the business key-value pair data includes: a business keyword and paragraph data corresponding to the keyword; The preset initial text classification model is trained using the keywords of the business and the paragraph data corresponding to the keywords to generate a preset text classification model.

2. The method for identifying business document image content according to claim 1, characterized in that: According to the font size, character spacing and character height of the characters in the initial paragraph candidate area, one or more of the following characteristics are satisfied when a plurality of initial paragraph candidate areas are classified as the same paragraph candidate area: the font sizes of the characters in the plurality of initial paragraph candidate areas are the same; the plurality of initial paragraph candidate areas are located on the same horizontal line, and the horizontal axis distance between two adjacent initial paragraph candidate areas does not exceed the width of two characters; the vertical axis distance between two adjacent initial paragraph candidate areas does not exceed the height of two characters.

3. The method for identifying business document image content according to claim 1, characterized in that: The method of generating a structured business document content recognition result based on the recognized text and candidate target paragraphs using a pre-trained text classification model includes: Determining keywords present in the recognized text according to preset keywords; Determine a candidate target paragraph corresponding to the keyword according to the determined position information of the keyword; The pre-trained text classification model is used to match and parse keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

4. A business document image content recognition device, characterized in that: The device comprises: A text recognition module, used to perform text recognition on the business document image to be recognized to generate text information of the business document including recognized text and text position information; A paragraph generation module is used to generate a candidate target paragraph according to the recognized text, text position information and a preset semantic discrimination model; wherein the paragraph generation module is further used to obtain pre-stored business text data; and use the business text data to perform model training on a preset initial language model to generate a preset semantic discrimination model; A matching module is used to generate a structured business document content recognition result using a pre-trained text classification model according to the recognized text and candidate target paragraphs; the matching module is further used to obtain pre-stored business key-value pair data; wherein the business key-value pair data includes: business keywords and paragraph data corresponding to the keywords; the preset initial text classification model is trained using the business keywords and paragraph data corresponding to the keywords to generate a preset text classification model; The paragraph generation module is further used to determine an initial paragraph candidate area based on the text position information; classify multiple initial paragraph candidate areas into the same paragraph candidate area based on the font size, text spacing and text height of the text in the initial paragraph candidate area; and use a preset semantic discrimination model to perform semantic discrimination on the recognized text in the paragraph candidate area to generate a candidate target paragraph.

5. The business document image content recognition device according to claim 4, characterized in that: The paragraph generation module classifies multiple initial paragraph candidate regions as the same paragraph candidate region based on the font size, character spacing and character height of the characters in the initial paragraph candidate regions, when one or more of the following characteristics are met: the font size of the characters in the multiple initial paragraph candidate regions is the same; The multiple initial paragraph candidate regions are located on the same horizontal line, and the horizontal axis distance between two adjacent initial paragraph candidate regions does not exceed the width of two characters; the vertical axis distance between two adjacent initial paragraph candidate regions does not exceed the height of two characters.

6. The business document image content recognition device according to claim 4, characterized in that: The matching module comprises: A keyword search unit, which determines the keywords present in the recognized text according to preset keywords; A candidate target paragraph determining unit, which determines a candidate target paragraph corresponding to the keyword according to the determined position information of the keyword; The classification and matching unit is used to use a pre-trained text classification model to match and parse keywords and their corresponding candidate target paragraphs to generate structured business document content recognition results.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method according to any one of claims 1 to 3.

Images