Key information extraction method and device for document image recognition

Abstract

This invention provides a method and apparatus for extracting key information from document images. The method includes: determining a preset number of candidate key values ​​for each key based on OCR recognition results; determining a total score for candidate paths for all keys based on the scores of the candidate key values ​​selected for each key; and determining the target key value selected for each key when the total candidate path score is the highest, to obtain the corresponding information extraction result. The score for each key selecting a corresponding candidate key value includes at least a score determined by the positional relationship between the key and the candidate key value. This method effectively avoids the error-prone problems caused by fixed keywords in rule-based information extraction methods. Furthermore, this method comprehensively considers all candidate key value selections, effectively utilizing the interrelationships and mutual exclusions between key information, thus achieving high accuracy and solving the global optimization problem of similar entity information extraction.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method and apparatus for extracting key information for document image recognition. Background Technology

[0002] After text or document image recognition, key information needs to be extracted from the identified content. Key information extraction methods fall into two categories: rule-based methods, which use regular expressions or edit distance to find corresponding key information; and machine learning or deep learning methods, which perform entity recognition and output key values ​​and location information corresponding to keywords.

[0003] Rule-based information extraction methods often perform key-value extraction when keywords are fixed or features are easily described. While simple, these methods are prone to errors. Methods based on deep learning models like BERT or Transformer provide a prediction for each key-value pair, but they lack a comprehensive measurement at the page or document level, ignoring the interrelationships and mutual exclusions between key information. They also cannot directly solve the problem of distinguishing entities of the same category at the model level, such as the payer and payee, who belong to the same entity but require separate extraction. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides a method and apparatus for extracting key information for document image recognition.

[0005] This invention provides a method for extracting key information for document image recognition, comprising: determining a preset number of candidate key values ​​for each key based on the OCR (Optical Character Recognition) recognition result; determining the total score of candidate paths for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value; and determining the target key value selected by each key when the total score of the candidate path is the highest, so as to obtain the corresponding information extraction result; wherein, the score when each key selects the corresponding candidate key value includes at least the score value determined by the positional relationship between the key and the candidate key value.

[0006] According to the present invention, a method for extracting key information for document image recognition, wherein determining a preset number of candidate key values ​​for each key includes: for each key, finding a preset number of candidate key values ​​whose rectangular frame distance is closest to the key around each key based on the rectangular frame distance between the key and the key value; wherein the rectangular frame distance is calculated by determining the distance based on the smaller of the difference between the horizontal and vertical coordinates of the two points whose rectangular frame distance between the key and the key value is closest.

[0007] According to the present invention, a method for extracting key information for document image recognition, wherein determining the total candidate path score for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value includes: determining the score when each key selects the corresponding candidate key value based on geometric feature score, OCR recognition score, and entity recognition score; and determining the total candidate path score for all keys selecting candidate key values ​​based on the scores when all keys select the corresponding candidate key values. Wherein, the geometric feature score is a score determined based on the positional relationship between the key and the candidate key value; the OCR recognition score is a score determined based on the posterior confidence of OCR recognition; and the entity recognition score is a score determined based on the posterior probability output by the entity recognition model.

[0008] According to the present invention, a method for extracting key information for document image recognition, wherein determining the total score of candidate paths for all keys by selecting candidate key values ​​based on the score of each key selecting corresponding candidate key values ​​includes: for each key without shared candidate key values, calculating the score when selecting the corresponding candidate key value; for each key with shared candidate key values, selecting candidate key values ​​according to the fixed priority order of the keys, and not selecting candidate key values ​​that have already been selected by other keys during the selection process; wherein, for a key with shared candidate key values, if no candidate key value is available, the candidate key value score is 0.

[0009] According to the present invention, a method for extracting key information for document image recognition, before determining the score for selecting the corresponding candidate key value for each key based on geometric feature score, OCR recognition score, and entity recognition score, further includes:

[0010] The geometric characteristic fraction is determined according to the following formula;

[0011]

[0012] Where d is the distance between the selected candidate key value rectangle and the key rectangle, d1, d2…d i h1 represents the distance between the rectangle for each candidate key value and the rectangle for the key; h2 represents the height of the rectangle corresponding to the key and the selected candidate key value text, respectively.

[0013] According to the present invention, a method for extracting key information for document image recognition, before determining the score for selecting the corresponding candidate key value for each key based on geometric feature score, OCR recognition score, and entity recognition score, further includes:

[0014] The OCR recognition score is determined according to the following formula;

[0015]

[0016] Where R is the OCR recognition score of the selected candidate key value, m is the number of characters in the selected candidate key value, and f is the number of characters in the selected candidate key value. j The posterior confidence score for identifying the j-th character of the selected candidate key value.

[0017] The present invention also provides a key information extraction device for document image recognition, comprising: a preliminary extraction module for determining a preset number of candidate key values ​​for each key based on the optical character recognition (OCR) recognition result; a scoring processing module for determining the total score of candidate paths for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value; and a precise extraction module for determining the target key value selected by each key when the total score of the candidate path is the highest, so as to obtain the corresponding information extraction result; wherein, the score when each key selects the corresponding candidate key value includes at least the score value determined by the positional relationship between the key and the candidate key value.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the key information extraction method for document image recognition as described above.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the key information extraction method for document image recognition as described above.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the key information extraction method for document image recognition as described above.

[0021] The present invention provides a method and apparatus for extracting key information for document image recognition. By setting different combinations of keys and candidate key values ​​through candidate paths, and determining the target key value for each key when the total score of the candidate path is the highest, it effectively avoids the error-prone problem caused by fixed keywords in similar rule-based information extraction methods. Furthermore, this method does not require a prediction result for each key value, but comprehensively considers all candidate key value selections, effectively utilizing the interrelationships and mutual exclusions between key information, thus achieving high accuracy and solving the global optimization problem of similar entity information extraction. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the key information extraction method for document image recognition provided by the present invention.

[0024] Figure 2 This is a schematic diagram illustrating the relationship between keys and candidate key values ​​provided by the present invention;

[0025] Figure 3 This is a schematic diagram illustrating the two cases of shared key values ​​provided by the present invention;

[0026] Figure 4 This is a schematic diagram of the key information extraction device for document image recognition provided by the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The following is combined Figures 1-5 The present invention describes a method and apparatus for extracting key information for document image recognition. Figure 1 This is a flowchart illustrating the key information extraction method for document image recognition provided by the present invention, as shown below. Figure 1 As shown, the present invention provides a method for extracting key information for document image recognition, comprising:

[0030] 101. Based on the optical character recognition (OCR) results, determine a preset number of candidate key values ​​for each key.

[0031] This invention relates to the extraction of key information for document or text image recognition, such as key information extraction based on OCR recognition results. The OCR output includes the position rectangle of text lines and the confidence score of text line recognition. This invention achieves this through a phased approach, from coarse information extraction to precise information extraction.

[0032] Step 101 is the coarse information extraction process. The purpose of coarse information extraction is to enable subsequent steps to accurately extract and construct the candidate information matching network. Coarse information extraction can employ any information extraction method. For example, based on regular expressions, for each key, find the nearest preset number of possible key values ​​to its left, bottom, and top in sequence. For example, the preset number is 3, and the following examples will all use 3.

[0033] 102. Based on the score of the corresponding candidate key value selected for each key, determine the total score of the candidate path for all keys.

[0034] The score for each key when selecting a corresponding candidate key value includes at least the score determined by the positional relationship between the key and the candidate key value. In other words, the score for each key when selecting a corresponding candidate key value can be a composite score from multiple parts, but it must at least include the score determined by the positional relationship between the key and the candidate key value. The aforementioned positional relationship includes the height, distance, and positional relationship between the key and the key value text lines.

[0035] After coarse information extraction, the candidate key values ​​corresponding to the key form a small subset. Figure 2 This is a schematic diagram illustrating the relationship between keys and candidate key values ​​provided by the present invention, such as... Figure 2 As shown. Suppose there are N key pieces of information to be extracted in an information extraction task. Each key can choose any one of the candidate key values. All keys and their corresponding selected key values ​​can form a candidate path i, and the evaluation score of this candidate path is denoted by S. When all possible key-value combinations are traversed, there are a total of 3 N There are several paths, and each path actually records the candidate index corresponding to each key, so that after finding the best information extraction combination, we can backtrack to get the key value corresponding to each key.

[0036] The total score for candidate paths, selecting candidate key values ​​for all keys, is calculated as follows:

[0037]

[0038] Among them, snode j Let N be the evaluation score corresponding to the j-th key, and N be the total number of keys.

[0039] 103. If the total score of the candidate path is the highest, determine the target key value for each key to obtain the corresponding information extraction result.

[0040] This invention uses dynamic programming to determine the candidate path with the best total score:

[0041]

[0042] Among them, S i p is the evaluation score corresponding to a subsequent candidate extraction path. * This represents the optimal combination for information extraction.

[0043] After obtaining the candidate path with the highest total score, the key value selection of each key under the candidate path is used as the final information extraction result, that is, a definite key-value pair is formed, and the information extraction process is completed.

[0044] The key information extraction method for document image recognition of the present invention allows for different combinations of keys and candidate key values ​​to be set through candidate paths. When the total score of the candidate path is the highest, the target key value selected for each key is determined, effectively avoiding the error-prone problem caused by fixed keywords in similar rule-based information extraction methods. Furthermore, this method does not require a prediction result for each key value, but comprehensively considers all candidate key value selections, effectively utilizing the interrelationships and mutual exclusions between key information, thus achieving high accuracy and solving the global optimization problem of similar entity information extraction.

[0045] In one embodiment, determining a preset number of candidate key values ​​for each key includes: for each key, finding a preset number of candidate key values ​​whose rectangular frame distance is closest to the key value around each key, based on the rectangular frame distance between the key and the key value; wherein the rectangular frame distance is calculated by determining the distance based on the smaller of the difference between the horizontal and vertical coordinates of the two points whose rectangular frame distance is closest to the key and the key value.

[0046] For example, based on regular expressions, for each key, find the three closest possible key values ​​to its left, bottom, and top sides in sequence, according to the distance between the bounding boxes. The distance between the bounding boxes is defined as follows: assuming the closest Euclidean distance between the key and its corresponding key value in the bounding box is (x1, y1) and (x2, y2) respectively, then the closest distance between two bounding boxes is defined as:

[0047] d=min(abs(x1-x2),abs(y1-y2))

[0048] In this embodiment of the invention, by determining a preset number of candidate key values ​​that are closest to the rectangle, the probability that all the candidate key values ​​extracted in the coarse extraction are true key values ​​is increased, thereby improving the overall information extraction accuracy.

[0049] In one embodiment, determining the total candidate path score for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value includes: determining the score when each key selects the corresponding candidate key value based on the geometric feature score, OCR recognition score, and entity recognition score; and determining the total candidate path score for all keys selecting candidate key values ​​based on the scores when all keys select the corresponding candidate key values. Wherein, the geometric feature score is a score determined by the positional relationship between the key and the candidate key value; the OCR recognition score is a score determined based on the posterior confidence of the OCR recognition; and the entity recognition score is a score determined based on the posterior probability output by the entity recognition model.

[0050] Specifically, this invention selects a score for each candidate key value, meaning the calculation of the snode in the above formula mainly includes a comprehensive score of geometric features, OCR recognition score, and semantic score. For example, the comprehensive score of the above three aspects is obtained through weighted average:

[0051] snode=λG+αR+βL

[0052] Here, λ, α, and β are weighting parameters used to balance the roles of different features in the evaluation. The geometric score G mainly considers the height, distance, and positional relationship between the key and the key-value text lines. The OCR recognition score R can output the confidence level for the recognition model. The recognition score L can be the posterior probability output by deep entity recognition models based on BERT, Tansformer, etc.

[0053] In this embodiment of the invention, information extraction is performed by integrating geometric information, OCR recognition information, and semantic entity recognition information, which can effectively improve the accuracy of the final extracted information.

[0054] In one embodiment, determining the total candidate path score for all keys by selecting candidate key values ​​based on the score of each key's corresponding candidate key value selection includes: for each key without shared candidate key values, calculating the score when selecting the corresponding candidate key value; for each key with shared candidate key values, selecting candidate key values ​​according to the fixed priority order of the keys, and not selecting candidate key values ​​that have already been selected by other keys during the selection process; wherein, for a key with shared candidate key values, if there are no candidate key values ​​to choose from, the candidate key value score is 0.

[0055] Figure 3 This is a schematic diagram illustrating the two cases of shared key values ​​provided by the present invention, as shown below. Figure 3 As shown, in this embodiment of the invention, based on the relationship between the key and its candidate key values, these candidate key value subsets can be divided into the following two types: a) N1 subsets have no common candidate key values; b) N2 subsets have common candidate key values. Therefore, the information extraction combination score can be expressed as:

[0056]

[0057] For case a), there is no intersection between candidate subsets, and each candidate corresponding to each key can be traversed through permutations and combinations. For case b), the key-value candidates are determined according to the fixed order priority of the keys. That is, when keyn1 and keyn2 (n1 < n2) share a certain key value and keyn1 has already selected this key value when generating the extraction path, then keyn2 can only select other key values. When keyn2 has no key value to select, the score of the keyn2 node is recorded as 0.

[0058] In one embodiment, before determining the score when each key selects the corresponding candidate key value according to the geometric feature score, the OCR recognition score, and the entity recognition score, it further includes:

[0059] Determine the geometric feature score according to the following formula;

[0060]

[0061] Where d is the distance between the selected candidate key value rectangle and the key rectangle, and d1, d2... d i are the distances between each candidate key value rectangle and the key rectangle; h1 and h2 are the heights of the rectangles corresponding to the key and the selected candidate key value text respectively. Among them, d is one of d1, d2... d i in.

[0062] In one embodiment, before determining the score when each key selects the corresponding candidate key value according to the geometric feature score, the OCR recognition score, and the entity recognition score, it further includes:

[0063] Determine the OCR recognition score according to the following formula;

[0064]

[0065] Where R is the OCR recognition score of the selected candidate key value, m is the number of characters of the selected candidate key value, and f j is the posterior confidence of the recognition of the jth character of the selected candidate key value.

[0066] Next, the key information extraction device for document image recognition provided by the present invention will be described. The key information extraction device for document image recognition described below can be mutually corresponding and referred to with the key information extraction method for document image recognition described above.

[0067] Figure 4 is a schematic structural diagram of the key information extraction device for document image recognition provided by the present invention, as Figure 4As shown, the key information extraction device for document image recognition includes: a preliminary extraction module 401, a scoring processing module 402, and a precise extraction module 403. The preliminary extraction module 401 determines a preset number of candidate key values ​​for each key based on the Optical Character Recognition (OCR) results. The scoring processing module 402 determines the total candidate path score for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value. The precise extraction module 403 determines the target key value selected by each key when the total candidate path score is the highest, thus obtaining the corresponding information extraction result. The score for each key selecting a corresponding candidate key value includes at least the score determined by the positional relationship between the key and the candidate key value.

[0068] In one embodiment, the preliminary extraction module 401 is specifically used to: for each key, find a preset number of candidate key values ​​whose rectangular frame distance is closest to the key value around each key, based on the rectangular frame distance between the key and the key value; wherein, the rectangular frame distance is calculated by determining the smaller of the difference between the horizontal and vertical coordinates of the two points whose rectangular frame distance is closest to the key and the key value.

[0069] In one embodiment, the scoring processing module 402 is specifically used to: determine the score for each key when selecting a corresponding candidate key value based on the geometric feature score, the OCR recognition score, and the entity recognition score; and determine the total candidate path score for all keys when selecting candidate key values ​​based on the scores for all keys when selecting corresponding candidate key values; wherein the geometric feature score is a score value determined based on the positional relationship between the key and the candidate key value; the OCR recognition score is a score value determined based on the posterior confidence of the OCR recognition; and the entity recognition score is a score value determined based on the posterior probability output by the entity recognition model.

[0070] In one embodiment, the scoring processing module 402 is specifically used to: calculate the score when selecting the corresponding candidate key value for each key that has no common candidate key value; select candidate key values ​​according to the fixed priority order of the keys for each key that has common candidate key values, and not select candidate key values ​​that have been selected by other keys during the selection process; wherein, for a key that has common candidate key values, if there are no candidate key values ​​to choose from, the candidate key value score is 0.

[0071] The apparatus embodiments provided in this invention are for implementing the above-described method embodiments. For specific processes and details, please refer to the above-described method embodiments, which will not be repeated here.

[0072] The key information extraction device for document image recognition provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned key information extraction method embodiment for document image recognition. For the sake of brevity, any parts not mentioned in the key information extraction device embodiment for document image recognition can be referred to the corresponding content in the aforementioned key information extraction method embodiment for document image recognition.

[0073] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include a processor 501, a communications interface 502, a memory 503, and a communication bus 504, wherein the processor 501, communications interface 502, and memory 503 communicate with each other via the communication bus 504. The processor 501 can call logical instructions in the memory 503 to execute a key information extraction method for document image recognition. This method includes: determining a preset number of candidate key values ​​for each key based on the OCR recognition result; determining the total candidate path score for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value; and determining the target key value selected by each key when the total candidate path score is the highest, to obtain the corresponding information extraction result. The score for each key selecting a corresponding candidate key value includes at least a score determined by the positional relationship between the key and the candidate key value.

[0074] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the key information extraction method for document image recognition provided by the above methods. The method includes: determining a preset number of candidate key values ​​for each key based on the OCR recognition result; determining the total score of candidate paths for all keys to select candidate key values ​​based on the score of each key selecting the corresponding candidate key value; and determining the target key value selected by each key when the total score of the candidate path is the highest, so as to obtain the corresponding information extraction result. The score when each key selects the corresponding candidate key value includes at least the score value determined by the positional relationship between the key and the candidate key value.

[0076] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a key information extraction method for document image recognition provided by the above methods. The method includes: determining a preset number of candidate key values ​​for each key based on the OCR recognition result; determining a total candidate path score for all keys selecting candidate key values ​​based on the score of each key selecting the corresponding candidate key value; and determining the target key value selected by each key when the total candidate path score is the highest, so as to obtain the corresponding information extraction result. The score when each key selects the corresponding candidate key value includes at least a score value determined by the positional relationship between the key and the candidate key value.

[0077] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting key information for document image recognition, characterized in that, include: Based on the optical character recognition (OCR) results, a preset number of candidate key values ​​are determined for each key; Based on the score of the corresponding candidate key value selected for each key, determine the total score of the candidate paths for all keys. If the total score of the candidate path is the highest, the target key value for each key is determined to obtain the corresponding information extraction result; The score for each key when selecting a corresponding candidate key value includes at least the score determined by the positional relationship between the key and the candidate key value. The step of determining the total candidate path score for all keys by selecting candidate key values ​​based on the score of each key's corresponding candidate key value selection includes: Based on the geometric feature score, OCR recognition score, and entity recognition score, determine the score for each key when selecting the corresponding candidate key value; Based on the scores when selecting corresponding candidate key values ​​for all keys, determine the total score of candidate paths for all keys when selecting candidate key values. Wherein, the geometric feature score is a score determined based on the positional relationship between the key and the candidate key value; the OCR recognition score is a score determined based on the posterior confidence of the OCR recognition; and the entity recognition score is a score determined based on the posterior probability output by the entity recognition model.

2. The method for extracting key information for document image recognition according to claim 1, characterized in that, The step of determining a preset number of candidate key values ​​for each key includes: For each key, based on the rectangle distance between the key and the key value, find a preset number of candidate key values ​​whose rectangle distance is closest to the key around each key; The method for calculating the rectangular frame distance includes determining the distance based on the smaller of the difference between the x-coordinate and y-coordinate of the two points that are closest to the Euclidean distance between the key and the key value.

3. The method for extracting key information for document image recognition according to claim 1, characterized in that, The step of determining the total candidate path score for all keys by selecting candidate key values ​​based on the score of each key's corresponding candidate key value selection includes: For each key that has no common candidate key values, calculate the score when selecting the corresponding candidate key value; For each key that has common candidate key values, candidate key values ​​are selected according to the fixed order priority of the keys, and candidate key values ​​that have been selected by other keys are not selected during the selection process. For keys with shared candidate key values, if no candidate key value is available, the candidate key value score is 0.

4. The method for extracting key information for document image recognition according to claim 1, characterized in that, Before determining the score for selecting the corresponding candidate key value for each key based on the geometric feature score, OCR recognition score, and entity recognition score, the method further includes: The geometric characteristic fraction is determined according to the following formula; ； Where d is the distance between the selected candidate key value rectangle and the key rectangle, d1, d2...di are the distances between each candidate key value rectangle and the key rectangle, and h1 and h2 are the heights of the rectangles corresponding to the key and the selected candidate key value text, respectively.

5. The method for extracting key information for document image recognition according to claim 1, characterized in that, Before determining the score for selecting the corresponding candidate key value for each key based on the geometric feature score, OCR recognition score, and entity recognition score, the method further includes: The OCR recognition score is determined according to the following formula; ； Where R is the OCR recognition score of the selected candidate key value, and m is the number of characters in the selected candidate key value. The posterior confidence score for identifying the j-th character of the selected candidate key value.

6. A key information extraction device for document image recognition, characterized in that, include: The preliminary extraction module is used to determine a preset number of candidate key values ​​for each key based on the optical character recognition (OCR) results. The scoring processing module is used to determine the total score of candidate paths for all keys by selecting candidate key values ​​based on the score of each candidate key value. The precise extraction module is used to determine the target key value for each key when the total score of the candidate path is the highest, so as to obtain the corresponding information extraction result; The score for each key when selecting a corresponding candidate key value includes at least the score determined by the positional relationship between the key and the candidate key value. The scoring processing module 402 is specifically used for: determining the score for each key when selecting a corresponding candidate key value based on the geometric feature score, OCR recognition score, and entity recognition score; and determining the total candidate path score for all keys when selecting candidate key values ​​based on the scores for all keys when selecting corresponding candidate key values. The geometric feature score is a score determined based on the positional relationship between the key and the candidate key value; the OCR recognition score is a score determined based on the posterior confidence of the OCR recognition; and the entity recognition score is a score determined based on the posterior probability output by the entity recognition model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the key information extraction method for document image recognition as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the key information extraction method for document image recognition as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the key information extraction method for document image recognition as described in any one of claims 1 to 5.

Images