Information processing device, character recognition method, and program

Abstract

To further improve the recognition accuracy of semi-structured OCR.SOLUTION: An information processing device comprises: an operation receiving unit which receives user-specified coordinates enclosing a first value to be subjected to character recognition in document image data; a learning data acquisition unit which acquires a first keyword around the user-specified coordinates and stores the keyword in association with the user-specified coordinates as learning data; an acquisition unit which acquires the document image data; a value extraction unit which extracts a second keyword and one or more second values defined by item definitions on the basis of preset item definitions from full-text recognition results for the image data acquired by the acquisition unit; and a value determination unit which determines a second value to serve as an item value of the second keyword on the basis of a first distance between coordinates of the second value and the user-specified coordinates associated with a first keyword which is the same as the second keyword associated with the second value in the learning data.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a character recognition method, and a program. [Background technology]

[0002] OCR (Optical Character Recognition / Reader) technology for extracting values ​​from forms can be divided into two main types: standard OCR and semi-standard OCR. Standard OCR is a technology that uses OCR to recognize characters in fixed areas of forms with a fixed layout. Semi-standard OCR does not perform character recognition in fixed areas, but extracts pre-defined item names from the full-text recognition results of the entire form, and extracts item values ​​near the items according to certain rules. Semi-standard OCR can also recognize semi-standard forms with different layouts. However, recognition accuracy tends to be lower compared to standard OCR, which defines the areas that need to be read with certainty.

[0003] There is known a technique for improving the recognition accuracy of OCR (see, for example, Patent Document 1). Patent Document 1 discloses a technique for detecting the coordinate values ​​of a specified location when an arbitrary frame in a table in form image data is specified by a drag operation, and extracting the frame area from the coordinate values. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-250927 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional techniques may not be able to improve the recognition accuracy of OCR. For example, conventional techniques cannot extract a recognition result if there is no frame area at the specified coordinates.

[0006] In view of the above problems, an object of the present invention is to further improve the recognition accuracy of OCR. [Means for solving the problem]

[0007] The present invention is an information processing device that performs character recognition using OCR, and includes an operation receiving unit that receives user-specified coordinates that surround a first value to be subjected to character recognition in image data of a form, a learning data acquisition unit that acquires first keywords surrounding the user-specified coordinates and stores them as learning data in association with the user-specified coordinates, an acquisition unit that acquires image data of the form, a value extraction unit that extracts a second keyword and one or more second values ​​defined by the item definition from the full-text recognition result of the image data acquired by the acquisition unit based on a preset item definition, and a value determination unit that determines the second value to be set as an item value of the second keyword based on a first distance between the coordinates of the second value and the user-specified coordinates that are associated with the first keyword in the learning data. [Effects of the Invention]

[0008] This can further improve the recognition accuracy of OCR. [Brief explanation of the drawings]

[0009] [Figure 1] This is a diagram comparing the advantages and disadvantages of standard OCR and semi-standard OCR. [Figure 2] FIG. 10 is a diagram showing an example of a form in which inconvenience may occur. [Figure 3] FIG. 10 is a diagram illustrating an improvement of a normal item. [Figure 4] 10 is a diagram illustrating an example of user-specified coordinates set by a user. [Figure 5] FIG. 5 is a diagram showing an example of a full-text recognition result obtained by semi-fixed OCR for the form of FIG. 4. [Figure 6] 10 is a diagram illustrating an example of user-specified coordinates and detailed items extracted from a full-sentence recognition result. [Figure 7] An example diagram showing two forms with the same layout but different printing positions. [Figure 8] FIG. 1 is a diagram illustrating an example of the overall configuration of an OCR processing system. [Figure 9] FIG. 1 is a diagram illustrating a hardware configuration of an example of an OCR processing device. [Figure 10] FIG. 10 is a diagram illustrating an example of information contained in an item definition. [Figure 11] FIG. 1 is a diagram illustrating semi-standard OCR recognition. [Figure 12] FIG. 10 is a diagram illustrating confirmation, correction, and learning of a recognition result. [Figure 13] FIG. 2 is an example of a functional block diagram illustrating functions of an OCR processing device divided into blocks. [Figure 14] 10 is an example of a flowchart illustrating the processing performed by the OCR processing device 2, including the improvement of normal items. [Figure 15] FIG. 10 is a diagram illustrating processing (1). [Figure 16] 10 is a diagram illustrating an example of information excerpted from keywords, Value extraction conditions, and extraction rules for result candidates in an item definition. [Figure 17] FIG. 10 is a diagram illustrating extracted keywords and values. [Figure 18] FIG. 10 is a diagram illustrating a method for extracting recognition result candidates. [Figure 19] 10A and 10B are diagrams illustrating a method for calculating a keyword feature score and a position score. [Figure 20] FIG. 2 is an example of a functional block diagram illustrating functions of an OCR processing device divided into blocks. [Figure 21] 10A and 10B are examples of diagrams showing a result correction screen for a schedule and a user-specified coordinate registration screen; [Figure 22] FIG. 10 is a diagram illustrating an example of learning data. [Figure 23] FIG. 10 is a flowchart illustrating an example of a process for acquiring detail items by the OCR processing device. [Figure 24] FIG. 10 is a diagram illustrating extraction of detail lines. [Figure 25] FIG. 10 is a diagram illustrating the combination of coordinates of rectangular frames whose y coordinates overlap. [Figure 26] 10 is a diagram illustrating an example of an extracted detail string. [Figure 27] 10A and 10B are diagrams illustrating a method for extracting lines and headers in the comparative technique. [Figure 28] FIG. 2 is an example of a functional block diagram illustrating functions of an OCR processing device divided into blocks. [Figure 29] 10 is a diagram illustrating an example of user-specified coordinates specified by a user for an input document. [Figure 30] FIG. 10 is a diagram illustrating an example of an identification dictionary. [Figure 31] FIG. 10 is a diagram illustrating an example of learning data. [Figure 32] FIG. 10 is a flowchart illustrating an example of a process in which the OCR processing device improves a form recognition process by using an identification dictionary and learning data. [Figure 33] FIG. 10 is a diagram illustrating an example of a full-text recognition result. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an OCR processing device and a character recognition method performed by the OCR processing device will be described as an example of an embodiment of the present invention with reference to the drawings.

[0011] <About standard OCR and semi-standard OCR> First, let's take a look at Figure 1 and other figures to understand standard OCR and semi-standard OCR. Figure 1 compares the advantages and disadvantages of standard OCR and semi-standard OCR. While semi-standard OCR has the advantage of being "easy to create definitions because it is an OCR definition that does not depend on the layout," it also has the disadvantage of "low recognition accuracy because the position at which characters are recognized is not fixed."

[0012] The applicant of this embodiment has improved recognition accuracy by incorporating a learning function into semi-standard OCR. The learning function, which will be described in detail later, is a function that trains the OCR processing device to recognize characters in a fixed area like standard OCR, for recognition results that could not be obtained with conventional semi-standard OCR mechanisms. When recognizing forms with the same layout, the OCR processing device can improve the recognition accuracy of semi-standard OCR by performing character recognition in the learned reading area (specified by user-specified coordinates, as will be described later).

[0013] However, when the OCR processing device performs character recognition on the learned reading area, there are cases where the correct recognition result cannot be extracted due to misalignment of the print position or fluctuation in the number of lines or other data. Specific examples of forms where this problem occurs include the following: a. Even with the same layout, the learning recognition area and the printing position are misaligned b. Even with the same layout, the number of detail lines varies depending on the data Figure 2 shows an example of a form that may cause problems. Figure 2(a) shows a form whose reading areas have been learned. Since reading areas 301 to 305 have been learned, the next time the user has the same form recognized, a correct recognition result will be obtained.

[0014] FIG. 2(b) shows the same form but with defects a or b. The positions of the reading areas 301 and 305 are different from those in FIG. 2(a). Such deviations can occur due to editing by the user who creates the form. Furthermore, the positions of the reading areas 302 to 304 have not changed, but the number of detail lines has increased. With such a form, the OCR processing device will not recognize the detail lines that fall outside the learning reading area, and correct recognition results will not be obtained.

[0015] <Outline of Character Recognition in This Embodiment> As such, there are some forms for which it is difficult to obtain correct recognition results when specifying a reading area, and in such cases recognition accuracy tends to be low, meaning that users have to spend more time manually entering data.

[0016] Therefore, in this embodiment, by using user-specified coordinates specified by the user and the full-sentence recognition result, recognition accuracy is improved and data input work is reduced. Below, an outline of the three main features of this embodiment will be described.

[0017] (1) Improvement of regular items Figure 3(a) shows an example of the defined position of the reading area. Even if the reading area is defined as in Figure 3(a), if the printing position is shifted from the defined position of the reading area, the OCR processing device will not be able to obtain the recognition results correctly.

[0018] Therefore, in this embodiment, the OCR processing device accepts settings of user-specified coordinates 311 received from the user, as shown in FIG. 3(b). The OCR processing device determines an appropriate item value for the keyword 312 using the distance between the user-specified coordinates 311 and the item value. In FIG. 3(b), of the item values ​​313 and 314, the item value 313 has a smaller distance from the user-specified coordinates 311, so it is expected that the item value 313 will be determined as the item value for the keyword 312. In more detail, a score is calculated not only from this distance but also from other information, and the item value for the keyword 312 is determined.

[0019] (2) Improved statement recognition The OCR processing device automatically extracts the itemized items from the user-specified coordinates received from the user and the full-text recognition results.

[0020] FIG. 4 shows user-specified coordinates set by the user. The user specifies each item value in the first line of a statement in an input document (a statement with the same layout as the statement to be read) using user-specified coordinates 321 to 324. Next, the OCR processing device 2 performs semi-standard OCR on the statement to be read. FIG. 5 is an example of the full-text recognition result obtained by semi-standard OCR for the statement in FIG. 4. The OCR processing device extracts statement items using the user-specified coordinates in FIG. 4 and the full-text recognition result in FIG. 5. FIG. 6 shows the user-specified coordinates and the statement items extracted from the full-text recognition result. As will be described in detail later, the OCR processing device determines the top end of the user-specified coordinates 321 to 324 to be the top end 331 of the statement and the bottom end of the horizontal ruled line to be the bottom end 332 of the statement, and extracts each line by utilizing the fact that character strings of the same line between the top end 331 and the bottom end 332 overlap in the height (y direction). The OCR processing device divides the extracted rows into columns based on the coordinates in the width direction (x direction) of the user-specified coordinates.

[0021] In this way, the OCR processing device can automatically extract all the detail items of a schedule by the user specifying the user-specified coordinates for one row of the schedule, even if the number of rows in the schedule varies.

[0022] (3) Improvement of document identification processing With conventional semi-standard OCR technology, even if two documents appear to have the same layout when viewed visually, if the printing position or other factors are different, they may be identified as different documents, and the OCR processing device may not be able to use the learning data and item definitions that have been prepared in advance for the document.

[0023] Figures 7(a) and (b) show two forms with the same layout but different printing positions. The layout of the detailed table is the same, but the printing positions of the billing amounts 341 and 343 and the addresses 342 and 344 are different in Figures 7(a) and (b). Therefore, the OCR processing device of this embodiment uses the full-text recognition results and user-specified coordinates to determine whether the forms have the same layout even if the printing positions are different. As will be described in detail later, the OCR processing device 2 creates an identification dictionary and learning data based on user-specified coordinates for forms with the same layout as the form to be read. The coordinates of the billing amount 341 and the address 342 are registered in the identification dictionary.

[0024] Next, the OCR processing device obtains full-text recognition results by performing character recognition on the form to be read, and calculates the similarity between the coordinates of each character string (excluding variable values ​​such as numerical values) in the result and the coordinates of the character string in the identification dictionary. For example, the distance between billing amount 341 in the identification dictionary and billing amount 343 in the input document, and the distance between address 342 in the identification dictionary and address 344 in the input document are calculated. Similarity is obtained for each character string from the distance, and the similarity is tallied for the entire input document. The OCR processing device uses training data created for forms whose tallied similarity is above a threshold for character recognition. This allows the OCR processing device to accurately select training data, improving recognition accuracy using training data.

[0025] <System configuration example> Fig. 8 is a diagram illustrating an example of the overall configuration of the OCR processing system 1. As shown in Fig. 8, the OCR processing system 1 includes an OCR processing device 2 and scanner devices 4A and 4B, which are capable of communicating with each other via a network 7. In this embodiment, any one of the scanner devices 4A and 4B will be referred to as the "scanner device 4."

[0026] The network 7 may be, for example, an in-house LAN. The network 7 may also be configured using wireless communication such as Wi-Fi (registered trademark, omitted below). Furthermore, if the OCR processing device 2 exists on a cloud, the network 7 may include a WAN or the Internet. For example, a user terminal can transmit image data read by the scanner device 4 to the OCR processing device 2.

[0027] The OCR processing device 2 may be directly connected to the scanner device 4 in a one-to-one wired manner via a USB cable or the like. Even in the case of one-to-one connection, the OCR processing device 2 and the scanner device 4 may communicate wirelessly. Known communication methods for this purpose include Wi-Fi Direct and Bluetooth (registered trademark).

[0028] The OCR processing device 2 may be a general-purpose information processing device. The OCR processing device 2 performs character recognition on image data of an input document received from the scanner device 4, and allows the user to confirm or correct the recognition result. The OCR processing device 2 of this embodiment performs character recognition using semi-standard OCR technology on image data of an input document received from the scanner device 4, and also performs the above processes (1) to (3).

[0029] Specifically, the OCR processing device 2 may be any information processing device equipped with a communication function, such as a PC (Personal Computer), a server device, a smartphone, or a tablet PC.

[0030] The scanner device 4 is an optical reading device that reads a document and generates image data, which it then transmits to the OCR processing device 2. In this embodiment, the scanner device 4 can scan various types of input documents. Although FIG. 8 illustrates the scanner device 4, image data for character recognition may also be captured by a digital camera or the like. The image data captured by the digital camera may be transmitted via the network 7 or may be stored in a removable storage medium. When the user inserts the storage medium into the OCR processing device 2, the OCR processing device 2 can acquire the image data.

[0031] The scanner device 4 may also be a device called an MFP (Multifunction Peripheral) or a multifunction device, which means that the scanner device 4 may have a printer function, a copy function, a facsimile function, etc. in addition to a scanner function.

[0032] In addition, in FIG. 8, the scanner device 4 and the OCR processing device 2 are separate devices, but the scanner device 4 and the OCR processing device 2 may be integrated into one device (such as an MFP).

[0033] <Hardware configuration example> The hardware configuration of the OCR processing device 2 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the hardware configuration of the OCR processing device 2 according to this embodiment. As shown in Fig. 9, the OCR processing device 2 is constructed by a computer 500, and includes a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0034] Of these, the CPU 501 controls the overall operation of the computer 500. The ROM 502 stores programs, such as an IPL, used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The display 506 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories or printers. The network I / F 509 is an interface for data communication using the network 7. The bus line 510 is an address bus, a data bus, or the like, for electrically connecting the components, such as the CPU 501, shown in FIG. 9.

[0035] The keyboard 511 is a type of input means having multiple keys used to input characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513, which is an example of a removable recording medium. The DVD-RW drive 514 is not limited to a DVD-RW, and may be a DVD-R or the like. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515, such as a flash memory.

[0036] <About semi-standard OCR> Next, we will explain semi-standard OCR in detail. Semi-standard OCR is performed in the following manner. (i) Creating an item definition (ii) Semi-standardized OCR recognition (iii) Checking, correcting, and learning the recognition results Figure 10 shows an example of the information contained in an item definition. The user registers the item definition as information about the item name to be read from the layout of the form. The item definition contains, for each form (each layout), an identification name, read items, format, keywords, value extraction conditions, extraction rules for result candidates, and recognition result conditions. These will be explained below. The identifier name is the identification information that identifies the item definition. Item definitions are usually created for each form (i.e., for each layout), so the identifier name is often the identification information for the form. However, item definitions can be shared between different forms if they have the same layout. The read item is the general name of the item you want to read from the form. We say general because the actual item name varies depending on the form. · format is the data format of the read item. The keyword is a character string that serves as a key to be searched from the full-text recognition result. The OCR processing device 2 searches the full-text recognition result for the keyword to find the read item. The Value extraction condition is the extraction condition corresponding to the keyword. Value is the string to be recognized, and is the item value that is associated with the item. Value is sometimes called the "recognition result." The result candidate extraction rules define where the recognition candidate is located relative to the keyword. For example, it defines whether the candidate is located to the right or below the keyword. Numbers such as "1, 2" indicate the priority. If candidates that meet the recognition result conditions are found both to the right and below the keyword, the candidate with the smaller number is used. The recognition result conditions are conditions for determining whether or not the recognition was successful, thereby enabling the OCR processing device 2 to improve the recognition accuracy.

[0037] FIG. 11 is a diagram for explaining semi-standard OCR recognition. FIG. 11 shows a mechanism for obtaining a recognition result using semi-standard OCR recognition. The OCR processing device 2 performs OCR recognition (semi-standard OCR recognition) using the created item definition. Here, an example will be described in which the "total amount" is recognized using the item definition shown in FIG. 10.

[0038] (1) The OCR processing device 2 searches for the keyword "total amount" from the full-text recognition result (the recognition result of the entire document output by the recognition engine) and identifies the location of the item. In FIG. 11(a), the detected character string is enclosed in a box 351. According to the item definition, the keywords to search for "total amount" are, for example, "total," "total amount," and "grand total," and the OCR processing device 2 detected the keyword "total" 356 from the full-text recognition result.

[0039] (2) Next, the OCR processing device 2 searches for a recognition result candidate for "total amount" near the keyword 356. In FIG. 11(b), based on the position of the detected keyword 356, "total," the OCR processing device 2 searches for an item value in the area 353 to the right or below the keyword based on the extraction rule.

[0040] (3) Next, the OCR processing device 2 identifies recognition result candidates 354 that match the recognition result conditions from the character strings in the area 353 and outputs them as the recognition result. In Figure 11(c), "5,400,000 yen" is a candidate value and satisfies the minimum number of digits, so "5,400,000 yen" is adopted as the "Total" item value (recognition result). Note that if there are multiple recognition result candidates 354, the OCR processing device 2 determines the most likely one as the recognition result based on the internal score of the recognition result candidate or the priority of the result candidate extraction rule.

[0041] <Check, correct, and learn recognition results> FIG. 12 is a diagram illustrating confirmation, correction, and learning of recognition results. The user checks the recognition results for each recognized item and corrects them if there are any errors. If the reading position or range is incorrect or if reading is not possible, the user manually specifies a rectangular frame to correct the reading area. The coordinates of the rectangular frame or reading area are user-specified coordinates. FIG. 12(a) shows the first form, which is used to confirm, correct, and learn the recognition results. If the values ​​for "billing amount" and "payment destination" cannot be obtained, the user specifies two reading areas, 361 and 362, and the reading area is learned. For example, the user instructs the OCR processing device 2 that the value of the billing amount is in box 361. The coordinates of the reading area specified by the user are called user-specified coordinates.

[0042] 12(b) shows the second and subsequent forms. Since the user-specified coordinates have been learned, when the OCR processing device 2 recognizes a form with the same layout, it can recognize it from the learned user-specified coordinates 363, 364.

[0043] [First embodiment] Next, the functional configuration of the OCR processing device 2 according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a functional block diagram illustrating the functions of the OCR processing device 2 in separate blocks.

[0044] The OCR processing device 2 includes an acquisition unit 10, a display control unit 11, an operation reception unit 12, a character recognition unit 13, a learning data acquisition unit 14, a score calculation unit 15, a value extraction unit 16, a value determination unit 17, an item definition storage unit 31, and a learning data storage unit 32. Each of these units included in the OCR processing device 2 is a function or a means for performing the function, which is realized when any of the components shown in Fig. 9 operates in response to an instruction from the CPU 501 in accordance with a program loaded in the RAM 503. Note that this program may be a native application dedicated to the scanner device or a web application.

[0045] The acquisition unit 10 acquires, via a network, for example, image data generated by the scanner device 4 by reading an input document such as a form. The acquisition unit 10 communicates with the scanner device 4 on the network using a communication protocol such as SNMP (Simple Network Management Protocol). Alternatively, the acquisition unit 10 may acquire image data generated by the digital camera 8, download the image data from the network, or read the image data from a storage medium.

[0046] The display control unit 11 displays various screens on the display. For example, the display control unit 11 displays image data and user-specified coordinates specified by the user. In addition, the display control unit 11 can also display recognition results, etc.

[0047] The operation reception unit 12 receives various operations for the OCR processing device 2. For example, the operation reception unit 12 receives a designation of a frame (rectangle) indicating user-designated coordinates.

[0048] The character recognition unit 13 performs character recognition processing on the image data and obtains the recognition result. In this embodiment, the character recognition unit 13 performs semi-standard OCR, but it is also possible to perform standard OCR.

[0049] The learning data acquisition unit 14 acquires learning data including user-specified coordinates specified by the user, and stores the data in the learning data storage unit 32. Details of the learning data will be described later in this embodiment.

[0050] The score calculation unit 15 calculates a score for determining an item value (a value to be described later) to be associated with a keyword by the value determination unit 17. The method of calculating the score will be described in detail later.

[0051] The value extraction unit 16 extracts keywords and one or more item values ​​(values ​​to be described later) associated with the keywords based on preset item definitions from the full-sentence recognition result of the image data acquired by the acquisition unit 10.

[0052] The value determination unit 17 determines an item value (a value to be described later) to be associated with a keyword based on the score. A method for determining an item value based on the score will be described later.

[0053] <Improvements to regular items> Next, we will explain in detail the improvements to the normal items. By improving the normal items, even if the print position is shifted from the user-specified coordinates, the OCR processing device 2 can search the periphery of the specified area based on the user-specified coordinates and obtain correct recognition results.

[0054] First, the overall flow of the processing performed by the OCR processing device 2 of this embodiment will be described with reference to Fig. 14, and each step will be described in detail later. Fig. 14 is a flowchart illustrating the processing performed by the OCR processing device 2, including the improvement of normal items. The explanation of Fig. 14 is based on the following assumptions 1 and 2. ·Premise 1 The character recognition unit 13 performs semi-standard OCR on the image data generated by the scanner device 4 using item definitions that are defined in advance by the user. ·Premise 2 The user checks the recognition results on the correction screen, and if there are any items for which recognition results have not been obtained or for which there are errors, the user specifies the user-specified coordinates for these correction items (generating learning data).

[0055] 14, the main steps are as follows: Steps (1) to (5) correspond to steps (1) to (5) in FIG. Processing (1) The OCR processing device 2 acquires the user-specified coordinates and the keywords in the vicinity thereof, and stores them as learning data. Processing (2) The OCR processing device 2 extracts recognition result candidates from the full-sentence recognition result and determines the recognition result for each item. Processing (3) If there is learning data, the OCR processing device 2 determines whether to correct the recognition result. Processing (4) If correction is necessary, the OCR processing device 2 searches again for a recognition result from the recognition result candidates. Processing (5) If the OCR processing device 2 cannot extract the recognition result, it searches again for the recognition result from other than the recognition result candidates.

[0056] <<Process (1)>> When a user sets user-specified coordinates, the learning data acquisition unit 14 acquires the user-specified coordinates (S1-1). The learning data acquisition unit 14 acquires keywords around the user-specified coordinates and the coordinates of the keywords from the full-text recognition result of the form, and stores them as learning data in the learning data storage unit 32 (S1-2).

[0057] FIG. 15 is a diagram illustrating process (1). As shown in FIG. 15(a), a user sets one user-specified coordinate 371 (an example of a first value) by enclosing "22,000 yen" in a rectangle. The learning data acquisition unit 14 acquires keywords 372 to 376 (an example of a first keyword) from around the user-specified coordinate (for example, within a radius of 5 cm). These keywords 372 to 376 are the same as the keywords in the item definition. In FIG. 15(a), five keywords 372 to 376 are found. The number of five is just an example, and one or more keywords may be found.

[0058] 15(b) is an example of the learning data, in which five keywords 372 to 376 and the coordinates are associated with a user-specified coordinate 371 and saved.

[0059] <<Process (2)>> After acquiring the learning data, the user has the scanner device 4 read the input document (form) that the user wants to character recognize. That is, the user scans the form for which learning data has been registered and performs full-text recognition. The scanner device 4 transmits image data of the input document to the OCR processing device 2. The acquisition unit 10 acquires the image data. The character recognition unit 13 acquires the full-text recognition result for the learning data of the input document (S2-1).

[0060] Next, the value extraction unit 16 detects keywords of the item definition from the full-sentence recognition result, and extracts values ​​according to the value extraction conditions corresponding to the keywords (S2-2).

[0061] 16 shows an excerpt of keywords, value extraction conditions, and result candidate extraction rules in an item definition. The value extraction unit 16 searches for keywords from the full-text recognition result. The value extraction unit 16 also obtains values ​​that match the value extraction conditions from the full-text recognition result.

[0062] Fig. 17 is a diagram illustrating extracted keywords 381 to 385 and values ​​391 to 395. In Fig. 17, five keywords 381 to 385 (examples of second keywords) and five values ​​391 to 395 (examples of second values) are acquired. Note that, at the time of step S2-1, the extraction rule for result candidates is not used. In other words, the keywords 381 to 385 and the values ​​391 to 395 are not linked in any way.

[0063] Returning to Fig. 14, the value extraction unit 16 extracts recognition result candidates (S2-3). Since the value extracted from the item definition for one keyword is not necessarily one, the value extraction unit 16 associates one or more values ​​with one keyword and sets them as recognition result candidates.

[0064] 18 shows a method for extracting recognition result candidates. The rule for extracting result candidates for keyword 381, "billing amount," is to the right or below the keyword, so two Values ​​391 and 392, "22,000" and "2,200," are associated with keyword 381, "billing amount," as recognition result candidates.

[0065] The value extraction unit 16 determines whether or not a recognition result candidate has been extracted (S2-4). If the determination in step S2-4 is Yes, the process proceeds to step S2-5, and if No, the process proceeds to step S101.

[0066] In step S2-5, the value extraction unit 16 associates keywords with values ​​one-to-one from the recognition result candidates (S2-5). When multiple values ​​are associated with one keyword, the value extraction unit 16 determines the associated value based on at least one of the priority of the extraction rule of the result candidate (for example, which is given priority, right or bottom) and the distance. In the example of FIG. 18, according to the item definition of keyword 381 "billing amount," the right has a higher priority, so Value 391 "¥22,000" on the right is determined as the recognition result candidate. For keywords for which no priority is set, the Value with the smallest distance from the keyword is selected.

[0067] In step S101, the character recognition unit 13 determines whether or not there is learning data (S101), and if there is no learning data, outputs the recognition result as null (S102). If there is learning data, the process proceeds to step S5-1.

[0068] <<Process (3)>> In step S3-1, the character recognition unit 13 determines whether or not there is training data (S3-1). If the determination in step S3-1 is Yes, the process proceeds to step S3-2; if the determination is No, the process proceeds to step S3-3. The training data is registered in step 1-1, and as will be described in the third embodiment, the training data registered for the input document according to the third embodiment is selected. Therefore, this determination should be made using the determination in the third embodiment. If the determination in the third embodiment is not used, the determination may be made based on whether or not the processing in step 1-1 has been executed. For example, one method is to associate the ID of a form with the training data, and input the ID when reading the input document.

[0069] In step S3-3, since there is no learning data, the character recognition unit 13 outputs the recognition result of step S2-5 (S3-3).

[0070] In step S3-2, the character recognition unit 13 determines whether or not the recognition result needs to be corrected (S3-2). Conditions under which the recognition result needs to be corrected include the following. The recognition result score is below the reference value, and The recognition result coordinates obtained are outside the range of coordinates specified by the user. The score will be explained in process (4). The score in step S3-2 is calculated for the value associated with the keyword on a one-to-one basis. Therefore, the distance between the keyword and the value, and the distance between the user-specified coordinates associated with this keyword in the learning data and the value can be specified, so both the keyword feature score and the position score can be calculated.

[0071] If the determination in step S3-2 is Yes, the process proceeds to step S4-1, and if No, the process proceeds to step S103. In step S103, since no correction is necessary, the character recognition unit 13 outputs the recognition result of step S2-5 (S103).

[0072] <<Process (4)>> Since the recognition result needs to be corrected, the value determining unit 17 uses the learning data to search again for a more likely recognition result from the recognition result candidates of step S2-3 (S4-1). First, the score calculating unit 15 calculates the necessary score.

[0073] The score is a combined value of two scores: the keyword feature score and the location score. The keyword feature score increases as the distance (an example of the second distance) between the coordinates of one or more values ​​associated with the keyword of the recognition result candidate and the coordinates of the keyword included in the recognition result candidate decreases. The position score is calculated based on the distance (an example of a first distance) between the coordinates of the Value associated with the keyword of the recognition result candidate and the user-specified coordinates, and the difference between the size of the Value associated with the keyword of the recognition result candidate (the size of the circumscribing rectangle) and the size of the user-specified coordinates. The position score increases as the distance and difference decrease. The user-specified coordinates used to calculate the position score are the user-specified coordinates associated with the keyword of the recognition result candidate in the training data. The score calculation method will be described with reference to FIG. 19.

[0074] FIG. 19(a) is a diagram for explaining a method for calculating a keyword feature score. In FIG. 19(a), the explanation focuses on Values ​​391, 392, "¥22,000" and "¥2,200", which are recognition result candidates for keyword 381, "billing amount". The score calculation unit 15 calculates the distance between each Value 391, 392 and the keyword 381. The score calculation unit 15 converts the distance into a keyword feature score so that the smaller the distance, the larger the keyword feature score. In this way, a keyword feature score is calculated for each Value 391, 392.

[0075] FIG. 19(b) is a diagram illustrating a method for calculating a position score. In FIG. 19(b), the description focuses on Values ​​391, 392, "¥22,000" and "¥2,200," which are recognition result candidates for the keyword 381, "billing amount." The score calculation unit 15 calculates the distance between each of Values ​​391, 392 and user-specified coordinates 401, indicated by the dotted line. The score calculation unit 15 also calculates the size of each of Values ​​391, 392 and the size of the user-specified coordinates 401, indicated by the dotted line, and calculates the difference between them. This user-specified coordinates 401 is associated with the keyword 381, "billing amount," in the learning data. The size may be, for example, the area of ​​a rectangle. The score calculation unit 15 converts the difference and distance into a position score for each of Values ​​391, 392, so that the smaller the distance and the smaller the difference, the larger the position score.

[0076] In this way, a keyword feature score and a position score are calculated for each of Values ​​391, 392 associated with a keyword 381 of a recognition result candidate. The score calculation unit 15 appropriately weights the keyword feature score and the position score to calculate the score in step S2-1. The value determination unit 17 outputs the recognition result with the highest score among the recognition result candidates of step S2-3, and whose score is equal to or greater than a reference value. A desirable recognition result is one in which the item value of keyword 381, "billing amount," is determined to be Value 391, for example, "¥22,000."

[0077] The score calculation unit 15 may calculate the score based on at least one of the distance between each Value 391, 392 and the keyword 381, the distance between each Value 391, 392 and the user-specified coordinates 401 shown by the dotted line, and the difference between the size of each Value 391, 392 and the size of the user-specified coordinates 401 shown by the dotted line.

[0078] Also, known functions that output larger output values ​​as input values ​​become smaller include an inverse proportional equation, a proportional equation with a negative slope, logistic decay, etc. Alternatively, a table may be prepared in which larger output values ​​correspond to smaller input values.

[0079] Returning to FIG. 14, the character recognition unit 13 determines whether or not a recognition result can be extracted by re-searching (S4-2). "Unable to extract a recognition result" means that there is no Value whose score exceeds the reference value. If the determination in step S4-2 is "Yes," the process proceeds to step S4-3; if the determination is "No," the process proceeds to step S5-1.

[0080] In step S4-3, the value determination unit 17 outputs the recognition result re-searched in step S4-1 (S4-3). That is, the Value with the highest score among the recognition result candidates and whose score is equal to or greater than the reference value is set as the item value of the keyword of the recognition result candidate.

[0081] <<Process (5)>> If the character recognition unit 13 cannot extract a recognition result even after re-searching in step S4-1, it outputs the recognition result with the highest position score from the Values ​​in step S2-2, whose position score is equal to or greater than the reference value (S5-1). The calculation method for the position score may be the same. The Values ​​in step S2-2 are not associated with keywords. Therefore, the character recognition unit 13 calculates the position scores for all Values ​​and all user-specified coordinates registered in the learning data, and determines whether there is a Value with the highest score and whose score is equal to or greater than the reference value. Therefore, one set of Value and user-specified coordinates is identified.

[0082] Taking FIG. 19(b) as an example, the score calculation unit 15 calculates Value391 and User-specified coordinates 401 associated with keywords 381 User-specified coordinates associated with keywords 382; User-specified coordinates associated with keywords 383; User-specified coordinates associated with the keyword 384, and The position score is calculated for all combinations of the user-specified coordinates associated with the keyword 385. The score calculation unit 15 calculates the position score for the values ​​392 to 395 in the same manner.

[0083] The character recognition unit 13 determines whether or not a recognition result has been extracted (S5-2). That is, the character recognition unit 13 determines whether or not a recognition result (Value) with the highest position score, which is equal to or greater than a reference value, has been found. If the determination in step S5-2 is Yes, the process proceeds to step S5-3; if the determination is No, the process proceeds to step S5-4.

[0084] In step S5-3, the character recognition unit 13 outputs the recognition result extracted in step S5-1 (S5-3). That is, the character recognition unit 13 determines that the recognition result (Value) having the highest position score and whose position score is equal to or greater than the reference value is the item value of the keyword associated with the user-specified coordinates registered in the learning data. The character recognition unit 13 determines that Value is the value of the keyword associated with this user-specified coordinates. If there are multiple keywords associated with the user-specified coordinates in the learning data, the character recognition unit 13 determines that Value is the value of the keyword having the smallest distance from Value or the user-specified coordinates.

[0085] In step S5-4, since the value could not be determined by the re-search, the character recognition unit 13 outputs the recognition result extracted in step S2-5 (S5-4).

[0086] <Major Effects> The OCR processing device 2 of this embodiment associates multiple values ​​with keywords and determines the appropriate keyword from among them using a score based on user-specified coordinates, etc., so even if the printing position is shifted from the user-specified coordinates, it can search the area around the specified area based on the user-specified coordinates and obtain the recognition result.

[0087] [Second embodiment] In this embodiment, an OCR processing device 2 capable of accurately acquiring detail items will be described. When a user specifies the coordinates of one line in a detail sheet, the OCR processing device 2 automatically extracts detail sheet items, thereby improving the accuracy of detail sheet recognition. Therefore, if the number of detail sheet lines included in a form changes after learning, the OCR processing device 2 can extract the item values ​​of the increased detail sheet.

[0088] <About the function> Figure 20 is a functional block diagram explaining the functions of the OCR processing device 2 of this embodiment by dividing them into blocks. The explanation of Figure 20 will mainly focus on the differences from Figure 13. The OCR processing device 2 of this embodiment has a detailed item extraction unit 18.

[0089] The detailed item extraction unit 18 automatically extracts detailed items using the full-text recognition result obtained by semi-standard OCR and the coordinates specified by the user, as will be described in detail later.

[0090] <Automatic extraction of schedule items> In describing this embodiment, the following assumptions 1 and 2 are assumed. ·Premise 1 The OCR processing device 2 executes semi-standard OCR on image data using item definitions defined by the user. ·Premise 2 The user checks the recognition results on the correction screen, and if the recognition results for the statement are not obtained correctly, they register the learning data by specifying the coordinates of one line of statement items in the statement.

[0091] Premise 2 will be explained with reference to Fig. 21. Fig. 21 shows a detailed table result correction screen 410 and a user-specified coordinate registration screen 420. As shown in Fig. 21(a), the recognition results obtained by semi-standard OCR are extracted on the detailed table result correction screen 410. Item values ​​are extracted for each of the item names 411 to 415, namely "item name," "quantity," "unit price," "subtotal," and "remarks." The user compares these item values ​​with the item values ​​in the input document and determines whether or not it is necessary to specify the coordinates of one line of detailed items in the detailed table.

[0092] As shown in Fig. 21(b), the user specifies the coordinates of one line for each item in the schedule for which a correct recognition result has not been obtained. In Fig. 21(b), the user defines user-specified coordinates 421 to 425 that surround the item values ​​in the first line (top level) for each of the items "item name," "quantity," "unit price," "subtotal," and "remarks" on the result correction screen 410. This saves the learning data.

[0093] FIG. 22 shows an example of training data. The training data includes item names and their corresponding user-specified coordinates. The item names are extracted using the item definitions of semi-standard OCR. The training data acquisition unit 14 stores the training data, which associates the user-specified coordinates with the item names of the schedule, in the training data storage unit 32.

[0094] Next, the overall flow of the process performed by the OCR processing device 2 of this embodiment to acquire detailed items will be described with reference to Fig. 23, and each step will be described in detail later. Fig. 23 is a flowchart illustrating the process performed by the OCR processing device 2 to acquire detailed items. The process in Fig. 23 is executed after Prerequisites 1 and 2 are prepared.

[0095] The detailed item extraction unit 18 determines whether or not learning data (user-specified coordinates and item names in FIG. 22) have been set (S11-1). The determination method may be the same as step S3-1 in FIG. 14. If the determination in step S11-1 is Yes, the process proceeds to step S11-2, and if the determination is No, the process proceeds to step S11-3.

[0096] In step S11-2, the detailed item extraction unit 18 acquires the user-specified coordinates for one line and the corresponding item names from the learning data storage unit 32 (S11-2).

[0097] In step S11-3, since there is no learning data, the character recognition unit 13 performs the conventional line extraction process and header extraction process (S11-3), which will be described in detail later.

[0098] Next, the detailed item extraction unit 18 performs row and column extraction processing (layout reanalysis) of the detailed table from the full-sentence recognition result based on the user-specified coordinates of the learning data (S11-4), as will be described in detail below.

[0099] 24 is a diagram for explaining extraction of detail lines. The detail item extraction unit 18 first extracts detail lines from the detailed table 430. The detail item extraction unit 18 groups the coordinates of character strings that are considered to be on the same line from the full-sentence recognition result. (1) The detailed item extraction unit 18 detects the top end 431 and bottom end 432 of the statement from the user-specified coordinates and ruled line information. First, the top end 431 may be the coordinate of the top side of the user-specified coordinates 421 to 425. If the user-specified coordinates 421 to 425 are different, the user-specified coordinate with the smallest value in the y-axis direction is used. Next, the bottom end 432 is the horizontal ruled line with the largest y-coordinate (the lowest) among the horizontal ruled lines whose length is equal to or greater than the reference value. Therefore, the bottom end 432 of the statement is the space between the row for "Parts H" and the row for "Payment destination." (2) The detailed item extraction unit 18 acquires rectangular frames 433 of the character strings within the range between the top end 431 and bottom end 432 detected in (1). In FIG. 24, 8 lines x 5 items = 40 rectangular frames 433 are acquired. The detailed item extraction unit 18 acquires the coordinates of each rectangular frame 433. (3) The detailed item extraction unit 18 extracts the first line and combines the coordinates of rectangular frames 433 whose y coordinates overlap. The five character strings in the first line are item values, and are specified by user-specified coordinates 421 to 425. The detailed item extraction unit 18 considers "Part A," "1000," "10," "10000," and "Memo 1" to be one line, combines these coordinates, and regards them as one line.

[0100] FIG. 25 is a diagram illustrating the combination of coordinates of rectangular boxes 433 whose y coordinates overlap. As shown in FIG. 25(a), rectangular boxes 441 to 443 for "Part A," "1000," and "10" may be slightly offset vertically. For example, the top ends of rectangular boxes 442 and 443 for "1000" and "10" are lower than the top end of rectangular box 441 for "Part A." Because the y coordinates (height direction of the rectangular boxes) of rectangular boxes 441 to 443 overlap, the detailed item extraction unit 18 considers these rectangular boxes 441 to 443 to be on the same line and combines them as one line of coordinates, as shown in FIG. 25(b). Note that combining means considering the y coordinate of the top end of the rectangular box with the smallest y coordinate and the y coordinate of the bottom end of the rectangular box with the largest y coordinate as the height of one line. The detailed item extraction unit 18 repeats this process for all rectangular boxes 433 until there are no more overlapping y coordinates.

[0101] Next, the detailed item extraction unit 18 extracts detailed columns. Extracting detailed columns means separating the extracted rows and extracting columns. (1) The detailed item extraction unit 18 vertically divides the extracted detailed lines using the left and right x coordinates of each item obtained from the user-specified coordinates. For example, the detailed item extraction unit 18 focuses on two adjacent items and determines the center between the rightmost x coordinate of the left-side user-specified coordinates (e.g., the rightmost x coordinate of user-specified coordinates 421) and the leftmost x coordinate of the right-side user-specified coordinates (e.g., the leftmost x coordinate of user-specified coordinates 422) as the column separator. Even if the user sets the user-specified coordinates such that the user-specified coordinates of two adjacent items overlap in the x direction, the center is determined to be the column separator.

[0102] 26 shows the extracted detailed columns 451 to 455. The rows and columns of the detailed table are obtained as described above. In other words, the coordinate area of ​​each item value of the detailed table is obtained.

[0103] As a reference example, a line extraction and header extraction method, which is a comparative technique, will be described with reference to Fig. 27. Fig. 27 is a diagram for explaining a line extraction and header extraction method in the comparative technique. Line extraction: Based on the extraction results of the area surrounded by horizontal and vertical lines, horizontal items that are close to the midpoints 461 to 464 in the y-axis direction are considered to be on the same line. Header extraction: If the extracted rows contain one or more keywords corresponding to the items in the detailed table, starting from the top, they are considered to be headers.

[0104] The comparison technology could not extract items from a schedule unless each area was surrounded by horizontal and vertical lines.

[0105] <Major Effects> The OCR processing device 2 of this embodiment can improve the recognition accuracy of a schedule by automatically extracting schedule items when the user specifies the coordinates of one line in the schedule. Therefore, if the number of schedule lines included in a form changes after learning, it can extract schedule item values ​​included in the increased number of lines.

[0106] [Third embodiment] In this embodiment, an improvement to the form identification process will be described. In this embodiment, the OCR processing device 2 uses character strings and coordinates included in the full-text recognition result of the entire form to determine which form (layout) for which learning data has been saved corresponds to the input document, thereby identifying the appropriate learning data for the input document. This makes it possible to determine that forms that appear to have different layouts due to misaligned printing positions or changes in the number of detail lines also have the same layout.

[0107] If the identification of appropriate learning data in this embodiment is performed before the first and second embodiments, the effects of the first and second embodiments are further enhanced.

[0108] <About the function> 28 is a functional block diagram illustrating the functions of the OCR processing device 2 of this embodiment, divided into blocks. The explanation of FIG. 28 will mainly focus on the differences from FIG. 13. The OCR processing device 2 of this embodiment has a dictionary creation unit 19, a layout identification unit 20, and an identification dictionary storage unit 33.

[0109] The dictionary creation unit 19 associates the item names acquired from the full-text recognition result of the image data of the form with the coordinates of the item names, and stores them in the identification dictionary storage unit 33 as an identification dictionary for the layout.

[0110] The layout identification unit 20 identifies the layout of a form using the results of full-text recognition by semi-standard OCR. The layout identification unit 20 can identify the learning data associated with the form by identifying the layout. Identifying the layout means determining whether the input document has the same layout as a form for which an identification dictionary has been registered. The identification dictionary will be described later, and it associates the character string content of item names with coordinates.

[0111] <Improvement of document identification processing> The following premise 1 is assumed for improving the document identification process. ·Premise 1 The OCR processing device 2 creates an identification dictionary and corresponding learning data.

[0112] First, the creation of an identification dictionary and corresponding learning data will be described with reference to Figs. 29 to 31. Fig. 29 shows user-specified coordinates specified by the user for an input document. The user checks the recognition result of the input document and specifies a reading area (an example of an area of ​​a first value) with user-specified coordinates 471 to 473. When the user-specified coordinates 471 to 473 are specified, the OCR processing device 2 creates an identification dictionary and corresponding learning data.

[0113] An example of an identification dictionary is shown in Fig. 30. The identification dictionary is a dictionary that stores information for identifying layouts. Identification name: A unique name (identification information) that is automatically determined by the dictionary creation unit 19 as a combination of alphanumeric characters when the dictionary creation unit 19 creates an identification dictionary. Identification information: The character string content (an example of a first item name) obtained by character recognition and the coordinates of the bounding rectangle of this character string. The character string content is the character string included in the full-text recognition result. This character string does not include variable character strings such as amounts and dates. The character string coordinates are the coordinates (four points or two diagonal points) of the rectangle surrounding the character string. The dictionary creation unit 19 creates an identification dictionary by associating the character string content with the character string coordinates for each character string.

[0114] Fig. 31 shows an example of learning data. The learning data includes user-specified coordinates and item names. Image data is associated with an identification dictionary by an identification name. In other words, the identification dictionary and learning data are a set. ·Distinguished name: The distinguished name is the same as the distinguished name in the distinguished dictionary. Identification information: The identification information is the user-specified coordinates and the item name. The user-specified coordinates are the user-specified coordinates of the value (an example of a first value) specified by the user for the input document in Figure 29. The item name (an example of a second item name) is determined from the read item that is determined to be the item of the value at the user-specified coordinates, as identified by semi-standard OCR, or the format of the value at the user-specified coordinates. The item name may be a character string surrounding the user-specified coordinates. The learning data acquisition unit 14 creates learning data by associating the user-specified coordinates with the item name for each user-specified coordinate.

[0115] Next, a method for improving the form identification process will be described with reference to Fig. 32. Fig. 32 is a flowchart illustrating the process in which the OCR processing device 2 improves the form identification process by using an identification dictionary and learning data.

[0116] First, the layout identification unit 20 determines whether or not there is an identification dictionary (S21-1). Since an identification dictionary is associated with the type of form, in step S21-1, it is determined whether or not there is an identification dictionary that has not yet been subjected to the process of Fig. 32 until an identification dictionary for the same form as the current input document is found. If the determination in step S21-1 is Yes, the process proceeds to step S21-2, and if No, the process proceeds to step S21-3.

[0117] In step S21-3, the character recognition unit 13 performs semi-standard OCR processing (S21-3).

[0118] In step S21-2, the layout identification unit 20 acquires the full-text recognition result of the input document (S21-2). The full-text recognition result may also be obtained by semi-standard OCR.

[0119] FIG. 33 is an example of a full-text recognition result. FIG. 33(a) shows an input document, and FIG. 33(b) shows the full-text recognition result. As shown in FIG. 33(a), a rectangle 481 is shown in the input document surrounding the character string recognized in the full-text recognition result. The character string within this rectangle 481 is the character string content in FIG. 33(b). The coordinates of this rectangle 481 are the character string coordinates in FIG. 33(b). In this way, each character string recognized in the input document and its coordinates are made clear.

[0120] Returning to FIG. 32, the layout identification unit 20 then compares the full-text recognition result with the information stored in the identification dictionary to determine whether the form (layout) is registered in the identification dictionary (S21-4). The layout identification unit 20 calculates the similarity based on the distance between the coordinates of the character string included in the full-text recognition result, which is a character string that partially matches a character string included in the full-text recognition result, and the character string coordinates in the identification dictionary. This is based on the knowledge that, in the same form, identical or similar character strings exist in nearby locations (coordinates). In determining whether there is a partial match, the user may be able to specify the number of matching characters. The similarity is calculated so that it increases as the distance decreases. It is preferable that the layout identification unit 20 multiplies the distance by a smaller weight as the number of partially matching characters increases. Note that variable character strings such as amounts and dates, or itemized line items that increase or decrease depending on the number of data, are not included in the comparison.

[0121] The similarity is calculated for each character string included in the full-text recognition result. The layout identification unit 20 aggregates (for example, sums, averages, or averages the top N similarities) the similarities calculated for all character strings included in the full-text recognition result to calculate the similarity for the entire full-text recognition result (i.e., the form).

[0122] If this similarity is equal to or greater than a threshold value, the layout identification unit 20 determines that the form that is the input document is a form (layout) for which an identification dictionary has been registered.

[0123] If the determination in step S21-4 is Yes, the process proceeds to step S21-5, and if the determination is No, the process proceeds to step S21-6.

[0124] In step S21-5, the character recognition unit 13 performs character recognition using learning data with the same identification name as the identification name in the identification dictionary with the matching layout (S21-5). That is, since the learning data associates item names with user-specified coordinates, the character recognition unit 13 recognizes the item value corresponding to the item name from the user-specified coordinates.

[0125] Then, the character recognition unit 13 uses the learning data of the first embodiment or the second embodiment as the learning data used in step S21-5, thereby further improving the recognition accuracy of the first embodiment or the second embodiment.

[0126] In step S21-6, the character recognition unit 13 performs semi-standard OCR processing using the item definitions set for the input document (S21-6).

[0127] <Major Effects> In this embodiment, the OCR processing device 2 identifies the appropriate learning data for the input document by using the character strings and coordinates included in the full-text recognition result of the entire form to determine which form (layout) has saved learning data. This makes it possible to determine that forms that appear to have different layouts due to misaligned printing positions or changes in the number of detail lines also have the same layout.

[0128] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0129] For example, the processes described in the first to third embodiments may be executed by a server device. In this case, a user terminal such as a PC and the server device communicate via a network, and the user terminal executes a web application. When the user terminal sends image data of a form to the server device using the web application, the server device performs character recognition and sends the recognition result to the user terminal.

[0130] In addition, the configuration examples in Figure 13 and the like are divided according to main functions to make it easier to understand the processing by the OCR processing device 2. The method of dividing the processing units and their names do not limit the present invention. The processing by the OCR processing device 2 can be divided into even more processing units depending on the processing content. Also, it can be divided so that one processing unit includes even more processes.

[0131] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0132] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, the way information is stored and presented in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful, and robust user interfaces. Such user interfaces provide better human-machine interaction. [Explanation of symbols]

[0133] 1. OCR processing system 2. OCR processing device 4. Scanner device

Claims

1. An information processing device that performs character recognition by OCR, an operation receiving unit that receives user-specified coordinates surrounding a first value to be subjected to character recognition in image data of a form; a learning data acquisition unit that acquires a first keyword around the user-specified coordinates, associates the first keyword with the user-specified coordinates, and stores the first keyword as learning data; an acquisition unit that acquires image data of a form; a value extraction unit that extracts a second keyword and one or more second values ​​defined by a preset item definition from the full-sentence recognition result of the image data acquired by the acquisition unit, based on the preset item definition; a value determination unit that determines the second value to be an item value of the second keyword based on a first distance between a coordinate of the second value and the user-specified coordinate of the first keyword that is the same as the second keyword associated with the second value and that is associated with the first keyword in the learning data; An information processing device having the above.

2. The information processing device according to claim 1 , wherein the value determination unit determines the second value to be the item value of the second keyword based on a second distance between the coordinates of the second value and the coordinates of the second keyword associated with the second value.

3. The value determination unit determines a size of a rectangle that encloses the second value; The information processing device according to claim 1, wherein the second value to be used as the item value of the second keyword is determined based on the difference between the size of the first value surrounded by the user-specified coordinates associated with the first keyword in the learning data and the size of the first value that is the same as the second keyword associated with the second value.

4. a first distance between the coordinates of the second value and the user-specified coordinates of the first keyword, which is the same as the second keyword associated with the second value and is associated with the first keyword in the learning data; a second distance between the coordinates of the second value and the coordinates of the second keyword associated with the second value; a score calculation unit that calculates a score related to the likelihood that the second value will be used as an item value of the second keyword based on at least one of a difference between a size of a rectangle surrounding the second value and a size of the first value surrounded by the user-specified coordinates that is the same as the second keyword that is associated with the second value and that is associated with the first keyword in the learning data, If there is no second value whose score satisfies a criterion among the second values ​​extracted by the value extraction unit, the value determination unit determines distances between the coordinates of all the second values ​​included in the full-sentence recognition result and the user-specified coordinates of all the first values ​​included in the training data, and identifying a set of the second values ​​and the user-specified coordinates based on a difference between a size of a rectangle enclosing all of the second values ​​included in the full-sentence recognition result and a size of an area of ​​all of the first values ​​included in the training data; The information processing apparatus according to claim 1 , wherein the item value of the keyword associated with the identified user-specified coordinates in the learning data is determined to be the identified second value.

5. An information processing device that performs character recognition by OCR, a character recognition unit that performs OCR on image data including a statement and acquires item names of the statement based on preset item definitions; an operation receiving unit that receives user-specified coordinates surrounding each item value in the first row of the specification table; a learning data acquisition unit that stores the user-specified coordinates and item names of the detailed table in association with each other; an acquisition unit that acquires image data including a schedule, the character recognition unit detects horizontal lines on the schedule, determining the top end of the user-specified coordinates in the recognition results of the image data acquired by the acquisition unit as the top end of the detailed table, and the lowest horizontal ruled line among the horizontal ruled lines as the bottom end of the detailed table; a detail item extraction unit that divides the area between the top and bottom of the specification into lines based on a character string between the top and bottom of the specification in the recognition result; An information processing device having the above.

6. 6. The information processing device according to claim 5, wherein, when character strings between the top and bottom ends of the spreadsheet overlap in the height direction of the spreadsheet, the detailed item extraction unit regards the overlapping character strings from the topmost character string to the bottommost character string as one line, thereby dividing the space between the top and bottom ends of the spreadsheet into one or more lines.

7. 7. The information processing apparatus according to claim 5, wherein the detailed item extraction unit divides the detailed table into columns based on the width coordinates of the user-specified coordinates accepted for each item value in the first row of the detailed table.

8. An information processing device that performs character recognition by OCR, a dictionary creation unit that associates a first item name obtained from a full-sentence recognition result of image data of a form with a coordinate of the first item name and stores the result as an identification dictionary of a layout; an operation receiving unit that receives user-specified coordinates of a first value area to be subjected to character recognition in the image data; a learning data acquisition unit that associates second item names around the user-specified coordinates with the user-specified coordinates and stores the coordinates as learning data; an acquisition unit that acquires image data of a form; a layout identification unit that identifies the identification dictionary that matches the layout of the image data acquired by the acquisition unit based on the distance between the coordinates of a character string that partially matches the first item name included in the identification dictionary, among character strings included in the recognition result of the image data acquired by the acquisition unit, and the coordinates of the first item name; a character recognition unit that performs character recognition using the learning data associated with the identification dictionary that matches the layout of the image data; An information processing device having the above.

9. The information processing device according to claim 8, wherein the layout identification unit calculates the distance by using the number of characters in a character string included in the recognition result of the image data acquired by the acquisition unit that match the first item name included in the identification dictionary as a weight for the distance.

10. A recognition method in which an information processing device performs character recognition using OCR, an operation receiving unit receiving user-specified coordinates surrounding a first value to be subjected to character recognition in image data of the form; a learning data acquiring unit acquiring first keywords around the user-specified coordinates, and storing the first keywords as learning data in association with the user-specified coordinates; An acquisition unit acquires image data of a form; a value extraction unit extracting a second keyword and one or more second values ​​defined by a preset item definition from the full-sentence recognition result of the image data acquired by the acquisition unit, based on the preset item definition; a value determination unit determining the second value to be set as an item value of the second keyword based on a first distance between a coordinate of the second value and the user-specified coordinate of the first keyword that is the same as the second keyword associated with the second value and that is associated with the first keyword in the learning data; A recognition method having the following.

11. A recognition method in which an information processing device performs character recognition using OCR, a step in which a character recognition unit performs OCR on image data including a statement, and acquires item names of the statement based on preset item definitions; an operation receiving unit receiving user-specified coordinates surrounding each item value in the first row of the specification table; a learning data acquisition unit storing the user-specified coordinates and the item names of the detailed table in association with each other; an acquiring unit acquiring image data including a schedule; a step in which the character recognition unit detects horizontal lines on the detailed table; determining the top end of the user-specified coordinates in the recognition results of the image data acquired by the acquisition unit as the top end of the detailed table, and the lowest horizontal ruled line among the horizontal ruled lines as the bottom end of the detailed table; a step in which a detailed item extraction unit divides the area between the top and bottom of the detailed table into lines based on character strings between the top and bottom of the detailed table in the recognition result; A recognition method having the following.

12. A recognition method in which an information processing device performs character recognition using OCR, a dictionary creation unit associating a first item name obtained from a full-sentence recognition result of image data of a form with the coordinates of the first item name and storing the result as an identification dictionary of a layout; an operation receiving unit receiving user-specified coordinates of a first value area to be subjected to character recognition in the image data; a learning data acquiring unit storing, as learning data, second item names around the user-specified coordinates in association with the user-specified coordinates; An acquisition unit acquires image data of a form; a layout identification unit specifying the identification dictionary that matches the layout of the image data acquired by the acquisition unit based on a distance between the coordinates of a character string that partially matches the first item name included in the identification dictionary, among character strings included in the recognition result of the image data acquired by the acquisition unit, and the coordinates of the first item name; a step in which a character recognition unit performs character recognition using the learning data associated with the identification dictionary that matches the layout of the image data; A recognition method having the following.

13. An information processing device that performs character recognition by OCR, an operation receiving unit that receives user-specified coordinates surrounding a first value to be subjected to character recognition in image data of a form; a learning data acquisition unit that acquires a first keyword around the user-specified coordinates, associates the first keyword with the user-specified coordinates, and stores the first keyword as learning data; an acquisition unit that acquires image data of a form; a value extraction unit that extracts a second keyword and one or more second values ​​defined by a preset item definition from the full-sentence recognition result of the image data acquired by the acquisition unit, based on the preset item definition; a value determination unit that determines the second value to be an item value of the second keyword based on a first distance between a coordinate of the second value and the user-specified coordinate of the first keyword that is the same as the second keyword associated with the second value and is associated with the first keyword in the learning data; A program to function as a

14. An information processing device that performs character recognition by OCR, a character recognition unit that performs OCR on image data including a statement and acquires item names of the statement based on preset item definitions; an operation receiving unit that receives user-specified coordinates surrounding each item value in the first row of the specification table; a learning data acquisition unit that stores the user-specified coordinates and item names of the detailed table in association with each other; an acquisition unit that acquires image data including a schedule, the character recognition unit detects horizontal lines on the schedule, determining the top end of the user-specified coordinates in the recognition results of the image data acquired by the acquisition unit as the top end of the detailed table, and the lowest horizontal ruled line among the horizontal ruled lines as the bottom end of the detailed table; a detail item extraction unit that divides the area between the top and bottom of the specification into lines based on a character string between the top and bottom of the specification in the recognition result; A program to function as a

15. An information processing device that performs character recognition by OCR, a dictionary creation unit that associates a first item name obtained from a full-sentence recognition result of image data of a form with a coordinate of the first item name and stores the result as an identification dictionary of a layout; an operation receiving unit that receives user-specified coordinates of a first value area to be subjected to character recognition in the image data; a learning data acquisition unit that associates second item names around the user-specified coordinates with the user-specified coordinates and stores the coordinates as learning data; an acquisition unit that acquires image data of a form; a layout identification unit that identifies the identification dictionary that matches the layout of the image data acquired by the acquisition unit based on the distance between the coordinates of a character string that partially matches the first item name included in the identification dictionary, among character strings included in the recognition result of the image data acquired by the acquisition unit, and the coordinates of the first item name; a character recognition unit that performs character recognition using the learning data associated with the identification dictionary that matches the layout of the image data; A program to function as a

Images