Information Processing Apparatus and Method, Data Structure, Recognition System, and Readable Medium

By displaying characters or symbols and their strings they refer to during optical character recognition, the problem of users needing to manually confirm the same string is solved, and more efficient character recognition results are achieved.

CN113496224BActive Publication Date: 2025-07-25FUJIFILM BUSINESS INNOVATION CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202011397775.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-04-08
Filing Date
2020-12-03
Publication Date
2025-07-25
Estimated Expiration
2040-12-03

AI Technical Summary

Technical Problem

During optical character recognition, when the same string exists in the file, the user needs to manually confirm the string referred to by the character or symbol, resulting in an increase in burden.

Method used

An information processing device and method are provided, which obtains character recognition results through a processor, and displays characters or symbols and their referenced character strings on the confirmation screen, allowing the user to replace the referenced character strings and reduces the user's confirmation burden.

Benefits of technology

This reduces the burden on users to confirm character recognition results and improves confirmation efficiency, so users can more easily judge and replace reference strings.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113496224B_ABST
    Figure CN113496224B_ABST
Patent Text Reader

Abstract

The present invention provides an information processing apparatus and method, a data structure, an identification system, and a readable medium. Even if there are characters or symbols that mean including a string identical to the filled string, the burden on the user for confirming the result of character recognition is not likely to increase. The information processing apparatus includes a processor that obtains the result of character recognition for a handwritten string, and when the string for which character recognition has been performed is a character or symbol that means including a string identical to the filled string, replaces the character or symbol with the string represented by the character or symbol.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, a data structure, a character recognition system, and a computer-readable medium. Background Art

[0002] Conventionally, there has been a technique of performing character recognition on handwritten characters described in a filling field in a document using optical character recognition (OCR). Further, a process of prompting a user to confirm whether the result of character recognition is correct is sometimes performed.

[0003] Japanese Patent Laid-Open No. 9-134357 discloses a character processing apparatus. In the character processing apparatus, document data input from an input unit is stored in a data storage unit. When a leading matching string partially exists among the data stored in the data storage unit by a matching determination unit, a specific string is read from a specific string storage unit by a string replacement unit for replacement, and further, blanks are added by a blank insertion unit through the replacement. In addition, a specific string in the data is restored to the original string. Summary of the Invention

[0004] When the same string is included in a plurality of filling fields, characters or symbols such as "same as above", "same as left", and "〃" indicating that the string is the same as the filled string are sometimes described. However, if OCR directly performs character recognition on the characters or symbols, the user has to find out the string referred to by these characters or symbols from the document and confirm it. In this case, the burden on the user increases.

[0005] An object of the present disclosure is to provide an information processing apparatus or the like in which the burden on a user who confirms the result of character recognition is not easily increased even when there are characters or symbols indicating that the string is the same as the filled string.

[0006] According to a first aspect of the present disclosure, there is provided an information processing apparatus including a processor that acquires a result of character recognition of a handwritten string, and when the string for which character recognition has been performed is a character or symbol indicating that the string is the same as the filled string, replaces the character or symbol with the string referred to by the character or symbol.

[0007] According to a second aspect of the present disclosure, the processor displays a confirmation screen for confirming the replaced string to the user.

[0008] According to the third aspect of the present disclosure, as the confirmation screen, the processor arranges and displays a handwritten string that is the character or symbol and the string represented by the character or symbol, and the result of character recognition of the string represented by the character or symbol.

[0009] According to the fourth aspect of the present disclosure, in addition to displaying the confirmation screen, the processor further displays an image of a document with a handwritten string, and in the image of the document, the positions of the part of the character or symbol and the part of the string represented by the character or symbol are displayed.

[0010] According to the fifth aspect of the present disclosure, when the user selects any one of the confirmation screens, the processor displays the positions of the part of the character or symbol and the part of the string represented by the character or symbol in the image of the document.

[0011] According to the sixth aspect of the present disclosure, the processor searches for a string corresponding to the direction determined by the character or symbol.

[0012] According to the seventh aspect of the present disclosure, the processor determines the direction based on the position of the box for filling the string.

[0013] According to the eighth aspect of the present disclosure, the processor compares the information of the box in which the character or symbol is recorded with the information of the box existing in the direction, and when the attributes of the content recorded in the box are the same, it is regarded as a candidate for the string represented by the character or symbol.

[0014] According to the ninth aspect of the present disclosure, as the attribute, when the dictionaries used for character recognition are the same, the processor regards the attributes of the content as the same.

[0015] According to the tenth aspect of the present disclosure, when there are multiple candidates, the processor selects the candidate closest to the character or symbol in position.

[0016] According to the eleventh aspect of the present disclosure, a data structure is provided, including image information of a document in which a string is filled in multiple boxes by handwriting, and a table for storing the result of character recognition of the string in the box. The table has records, and the records include: information representing the content of the string; the result of character recognition; the name of the dictionary used for character recognition, which is prepared corresponding to the information representing the content of the string; and the position and size of the box in the document.

[0017] According to a twelfth aspect of the present disclosure, there is provided a character recognition system, including: a reading device that reads a document with a handwritten string in the form of an image; a character recognition device that performs character recognition on the handwritten string; a display device that displays a confirmation screen of the result of character recognition by the character recognition device confirmed by a user; and an image production device that produces an image of the confirmation screen; the image production device includes a processor, and the processor obtains the result of character recognition from the character recognition device, and when the string for which character recognition has been performed is a character or symbol that means including a string identical to a filled string, replaces the character or symbol with the string denoted by the character or symbol.

[0018] According to a thirteenth aspect of the present disclosure, there is provided a computer-readable medium which is a computer-readable medium storing a program for causing a computer to execute processing, and the processing includes the following steps: obtaining a result of character recognition for a handwritten string; and when the string for which character recognition has been performed is a character or symbol that means including a string identical to a filled string, replacing the character or symbol with the string denoted by the character or symbol.

[0019] According to a fourteenth aspect of the present disclosure, there is provided an information processing method, including the following steps: obtaining a result of character recognition for a handwritten string; and when the string for which character recognition has been performed is a character or symbol that means including a string identical to a filled string, replacing the character or symbol with the string denoted by the character or symbol.

[0020] (Effect)

[0021] According to the first aspect, there can be provided an information processing device in which the burden on a user for confirming the result of character recognition is not easily increased even when there are characters or symbols that mean including a string identical to a filled string.

[0022] According to the second aspect, a user can confirm the string denoted by a character or symbol that means including a string identical to a filled string.

[0023] According to the third aspect, it is possible to compare a character or symbol that means including a string identical to a filled string and the string denoted by these characters or symbols for confirmation.

[0024] According to the fourth aspect, it is possible to confirm the positions of a character or symbol that means including a string identical to a filled string and the string denoted by these characters or symbols in the original paper document.

[0025] According to the fifth aspect, a user can select a string to be confirmed.

[0026] According to the sixth or seventh solution, it is possible to more accurately explore a string denoted by a character or symbol that means a string including the same string as the filled string.

[0027] According to the eighth solution, it is possible to perform exploration in consideration of the attributes of the string filled in the box.

[0028] According to the ninth solution, it is possible to perform exploration in consideration of the content of the string filled in the box.

[0029] According to the tenth solution, it is possible to extract the most accurate string that is a string denoted by a character or symbol that means a string including the same string as the filled string.

[0030] According to the eleventh solution, it becomes easier to display a confirmation screen or to explore a string denoted by a character or symbol that means a string including the same string as the filled string.

[0031] According to the twelfth solution, it is possible to provide an information processing system in which even if there are characters or symbols that mean a string including the same string as the filled string, the burden on the user for confirming the character recognition result is not likely to increase.

[0032] According to the thirteenth solution, it is possible to implement, via a computer-readable medium, a function in which even if there are characters or symbols that mean a string including the same string as the filled string, the burden on the user for confirming the character recognition result is not likely to increase.

[0033] According to the fourteenth solution, it is possible to provide an information processing method in which even if there are characters or symbols that mean a string including the same string as the filled string, the burden on the user for confirming the character recognition result is not likely to increase. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a diagram showing a structural example of the character recognition system in the present embodiment.

[0035] Figure 2 is a diagram showing a hardware structural example of the image forming apparatus.

[0036] Figure 3 is a diagram showing a schematic operation of the character recognition system.

[0037] Figure 4 is a diagram showing an example of a paper document in the present embodiment.

[0038] Figure 5 of (a) to Figure 5 of (c) is a diagram showing an example of a conventional operation as a confirmation operation of the OCR result by Figure 3 1G.

[0039] Figure 6 is a block diagram showing a functional structure example of the information processing system in the present embodiment.

[0040] Figure 7 is a diagram showing the data structure of the character recognition data stored in the storage unit.

[0041] Figure 8 (a) to Figure 8 (b) is a diagram showing a first example of the confirmation screen.

[0042] Figure 9 (a) to Figure 9 (b) is a diagram showing a second example of the confirmation screen.

[0043] Figure 10 is for showing the production of Figure 9 (a) to Figure 9 (b) is a diagram showing the order of the display list for displaying the list of recognition results of the confirmation screen.

[0044] Figure 11 is a diagram showing the process of the search unit retrieving the string referred to as "the same above" from a paper document.

[0045] Figure 12 is a diagram showing the process of the search unit retrieving the string referred to as "the same left" from a paper document.

[0046] Figure 13 (a) is a flowchart for explaining the process of step 210 of Figure 11 in more detail. Figure 13 (b) is a flowchart for explaining the process of step 310 of Figure 12 in more detail.

[0047] Figure 14 is a diagram showing the operation of the terminal device when the user selects a part of the recognition result in the list of recognition results using an input element such as a mouse.

[0048] Figure 15 (a) to Figure 15 (c) is a diagram showing the operation of the replacement unit. Detailed implementation mode

[0049] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0050] <Explanation of the entire character recognition system>

[0051] Figure 1 is a diagram showing a structural example of the character recognition system 1 in the present embodiment.

[0052] As shown in the figure, the character recognition system 1 of the present embodiment is configured by connecting an image forming apparatus 10, a management server 20, and a terminal apparatus 30 via a network 70 and a network 80.

[0053] In addition, in Figure 1 only one image forming apparatus 10 and one terminal apparatus 30 are shown respectively, but a plurality of them may be provided respectively.

[0054] The image forming apparatus 10 is an apparatus that forms an image on a recording medium such as paper and outputs it as a printing medium. The image forming apparatus 10 has a printer function, and in addition, for example, other image processing functions such as a scanner function and a facsimile function.

[0055] Figure 2 It is a diagram showing an example of the hardware configuration of the image forming apparatus 10.

[0056] As shown in the figure, the image forming apparatus 10 includes: a Central Processing Unit (CPU) 11, a Random Access Memory (RAM) 12, a Read Only Memory (ROM) 13, a Hard Disk Drive (HDD) 14, an operation panel 15, an image reading unit 16, an image forming unit 17, and a communication interface (hereinafter referred to as “communication I / F (Interface)”) 18. Moreover, these components exchange necessary data via a bus B.

[0057] The CPU 11 loads various programs stored in the ROM 13 or the like into the RAM 12 and executes them, thereby implementing the respective functions described later.

[0058] The RAM 12 is a memory used as a working memory of the CPU 11 or the like.

[0059] The ROM 13 is a memory that stores various programs executed by the CPU 11 or the like.

[0060] The HDD 14 is, for example, a disk device that stores image information read by the image reading unit 16 or image information for image formation in the image forming unit 17.

[0061] The operation panel 15 is, for example, a touch panel that displays various information and accepts operation inputs from the user. Therefore, the display mechanism includes a display unit such as a liquid crystal panel that displays content (information content) in a predetermined area in the form of an image, and has a function of detecting the position where a contact object such as a human finger or a stylus touches the liquid crystal panel. In the present embodiment, the touch panel is not particularly limited, and various types of touch panels such as a resistive film method or a capacitive method can be used.

[0062] The image reading unit 16 is an example of a reading device that reads an image recorded on a document. In the present embodiment, a document with a handwritten string as the document is read in the form of an image.

[0063] The image reading unit 16 is, for example, a scanner, and a scanner using the following method can be used: a CCD method in which the reflected light of the light irradiated from a light source to the document is reduced by a lens and then received by a Charge Coupled Device (CCD), or a CIS method in which the reflected light of the light irradiated from a Light Emitting Diode (LED) light source to the document in sequence is received by a Contact Image Sensor (CIS).

[0064] The image forming unit 17 is an example of a printing mechanism that forms an image on a recording medium. Here, the image forming unit 17 is, for example, a printer, and a printer using the following method can be used: an electrophotographic method in which toner attached to a photoreceptor is transferred to a recording medium such as paper to form an image, or an inkjet method in which ink is ejected onto the recording medium to form an image.

[0065] The communication I / F 18 transmits and receives various information to and from other devices via the networks 70 and 80.

[0066] The management server 20 is a server computer that manages the entire character recognition system 1. In addition, when the management server 20 is set up on the cloud, the management server 20 can also be said to be a cloud server. The management server 20, for example, authenticates users who operate the image forming device 10 or the terminal device 30, and transmits and receives electronic information to and from the image forming device 10 or the terminal device 30. Here, the so-called "electronic information" refers to information obtained by digitizing certain information. The "electronic information" is not particularly limited in terms of the type, form, data structure, etc. of the information as long as it can be processed by the image forming device 10, the management server 20, and the terminal device 30 respectively. However, as the information processed by the character recognition system 1 of the present embodiment, for example, character information or image information, etc.

[0067] In addition, although details will be described later, the management server 20 is an example of a character recognition device, and performs character recognition on a handwritten character string read by the image reading unit 16 of the image forming apparatus 10.

[0068] The management server 20 includes a CPU as an arithmetic component, a main memory as a storage component, and a storage device such as an HDD or a solid state drive (SSD). The CPU executes various software such as an operating system (OS) (basic software) or application software. In addition, the main memory is a storage area for storing various software or data for its execution, and the storage device is a storage area for storing input data for various software or output data from various software.

[0069] Furthermore, the management server 20 includes a communication I / F for communicating with the outside, and input elements such as a keyboard, a mouse, a touch panel, and a stylus. In addition, the management server 20 may include a display element such as a liquid crystal display.

[0070] The terminal device 30 is a device for a user to confirm the result of character recognition performed by the management server 20. The terminal device 30 is, for example, a general-purpose personal computer (PC). Moreover, under the management of an OS (operating system), the terminal device 30 runs various application software, thereby performing production of electronic information. In addition, the result of the character recognition may sometimes be referred to as an "OCR result" hereinafter.

[0071] The terminal device 30 includes a CPU, a main memory, a storage device, a communication I / F, and input elements in the same manner as the management server 20. These are examples of an information processing device or an image production device, and produce a confirmation screen for a user to confirm the OCR result obtained by the management server 20. Moreover, the CPU is an example of a processor. In addition, the terminal device 30 includes a display element as an example of a display device for displaying the confirmation screen.

[0072] The network 70 is a communication component for information communication between the image forming apparatus 10, the management server 20, and the terminal device 30, and is, for example, the Internet.

[0073] Similar to the network 70, the network 80 is also a communication component for information communication between the image forming apparatus 10, the management server 20, and the terminal device 30, and is, for example, a local area network (LAN).

[0074] <Overview of the operation of the character recognition system 1>

[0075] Figure 3 This is a diagram showing the general operation of the character recognition system 1.

[0076] In addition, in this example, the user operating the image forming apparatus 10 is set as User A, and the user operating the terminal apparatus 30 is set as User B. Further, User A and User B are users who have been authenticated by the management server 20 in advance.

[0077] First, User A scans the paper document Bk using the image forming apparatus 10 (1A). The scanning is performed using the image reading unit 16 of the image forming apparatus 10, and the paper document Bk is transmitted to the management server 20 in the form of electronic information including image information (1B).

[0078] The paper document is an example of a document, and is a document in which User A and others have filled in a printed document Bp printed in a predetermined official document format by handwriting. The paper document Bk is not particularly limited, and for example, it is a bill, a survey form, a summons, an application form, a declaration form, a proposal form, etc.

[0079] Figure 4 This is a diagram showing an example of the paper document Bk of the present embodiment.

[0080] The illustrated paper document Bk is a bill, showing a case where User A has filled in a printed document Bp printed in a predetermined official document format in the form of a handwritten character string Mt. In the printed document Bp, for example, filling-in fields Ra such as name and address are printed in advance, and User A fills in his / her name and address in the filling-in field Ra in the form of a handwritten character string Mt. The filling-in field Ra can also be understood as a frame filled in by User A or an identification frame that is an object for character recognition.

[0081] Return to Figure 3 , when the management server 20 receives the scanned electronic information, it performs character recognition using OCR (1C). This is, for example, registering the filling-in field Ra in the printed document Bp in the management server 20 in advance (1D). Then, based on the registered filling-in field Ra, the part of the handwritten character string Mt is extracted, and character recognition using OCR is performed on the extracted handwritten character string Mt.

[0082] Then, the management server 20 sends the OCR result to the terminal apparatus 30 (1E).

[0083] In the terminal apparatus 30, although the details will be described later, a confirmation screen Ga for confirming whether the OCR result is correct is displayed to User B (1F). That is, in this case, User B is the person in charge or the manager for confirming whether the OCR result is correct.

[0084] User B makes a confirmation through the confirmation screen Ga. When there are corrections, the user corrects the OCR result (1G). Moreover, after User B's confirmation, it is output as the confirmed correction result (1H). In addition, the confirmed correction result is sent to the management server 20 for storage (1I).

[0085] Figure 5 of (a) to Figure 5 of (c) is a diagram showing an example of a previous operation of the confirmation operation of the OCR result by Figure 3 the 1G of

[0086] Among them, Figure 5 of (a) shows the paper document Bk scanned by Figure 3 the 1A of

[0087] The paper document Bk is a summons, and as the destination address and sender information, an address column Ra1 and an address column Ra2 are respectively set. Moreover, it shows the case where User A fills in the destination address in the form of a string Mt1 in the filling column Ra1. In addition, it shows the case where the sender's address is filled in the form of a handwritten string Mt2. However, in the illustrated example, the destination address and the sender's address are the same, so the former fills in the actual address in the form of the string Mt1, and the latter fills in "same as above" in the form of the string Mt2, indicating the same as the filled address.

[0088] Figure 5 of (b) shows Figure 5 the OCR result of the paper document Bk shown in (a) of

[0089] This is a part of the confirmation screen Ga displayed by Figure 3 the 1F of

[0090] In addition, Figure 5 of (c) shows an example of the confirmed correction result saved by Figure 3 the 1H of

[0091] In this example, the confirmed correction result is output in the form of comma-separated values (CSV). Moreover, the output data represents "destination_address, sender_address, ○○ County △△ City ◇◇ Town 1-2-34, same as above".

[0092] However, as shown in (b) of Figure 5 when the text "Same as above" of the string Mc2 is directly displayed as the OCR result on the confirmation screen Ga, the user B who makes the confirmation has to find the string Mt1 corresponding to the "Same as above" from the original paper document Bk and manually input its content. In this case, when the user B finds the "Same as above" of the string Mc2 on the confirmation screen Ga, the user B finds the string Mt2 in the video image of the paper document Bk being displayed. Moreover, thereafter, the user B further finds the string Mt1 in the video image, and corrects the content of the string Mc2 from "Same as above" to "1-2-34, ◇◇ Town, △△ City, ○○ County". In this case, the burden on the user B who confirms the OCR result increases.

[0093] In addition, in Figure 5 (a) to Figure 5 (c) of

[0094] the case where "Same as above" is shown, but the cases where characters or symbols such as "Same as the left" and "〃" are filled in the paper document Bk are the same.

[0095] <Explanation of the structure of the character recognition system 1>

[0096] Figure 6 is a block diagram showing a functional structure example of the character recognition system 1 in the present embodiment.

[0097] In addition, among the various functions of the character recognition system 1, the functions related to the present embodiment are selected and illustrated here.

[0098] The image forming apparatus 10 of the character recognition system 1 includes: a transceiver unit 110, a printing unit 120, a reading unit 130 that performs scanning, and a control unit 140.

[0099] The transceiver unit 110 performs the transmission and reception of electronic information with the management server 20. The transceiver unit 110 transmits the image information of the scanned paper document Bk to the management server 20. The transceiver unit 110 corresponds to, for example, the communication I / F 18.

[0100] The printing unit 120 is a printing mechanism that prints on a recording medium such as paper. The printing unit 120 corresponds to, for example, the image forming unit 17.

[0101] The reading unit 130 scans and reads the paper document Bk to make it into image information. The reading unit 130 corresponds to, for example, the image reading unit 16.

[0102] The control unit 140 controls the entire image forming apparatus 10. The control unit 140 corresponds to, for example, the CPU 11, the RAM 12, and the ROM 13.

[0103] The management server 20 includes: a transceiver unit 210, a character recognition unit 220, and a storage unit 230.

[0104] The transceiver unit 210 transmits and receives electronic information between the image forming apparatus 10 and the terminal device 30. The transceiver unit 210 receives the image information of the paper document Bk scanned by the image forming apparatus 10, and in addition, transmits the OCR result of the paper document Bk to the terminal device 30.

[0105] The character recognition unit 220 is an OCR that performs character recognition of the handwritten string Mt in the paper document Bk. In addition, the storage unit 230 stores the image information of the paper document Bk received by the transceiver unit 210, the OCR result as the result of character recognition, and the definition information of the filling field Ra.

[0106] The definition information includes information on the attributes, positions, and sizes of the respective filling fields Ra in the paper document Bk. As the information on the attributes, there are the recognition frame name that is the name of the frame that becomes the object of character recognition as the name of each filling field Ra, and the dictionary name that is the name of the dictionary used in each filling field Ra. The dictionary is a dictionary used in character recognition, and the applicable dictionary is determined in advance according to the content of the string Mt filled in each filling field Ra. For example, when the filling field Ra is a field for filling in a name, a name dictionary is applied. That is, the name dictionary is, for example, a dictionary that collects people's names and is suitable for performing character recognition of names. In addition, when the filling field Ra is a filling field Ra for filling in an address, an address dictionary is applied. In addition, as the dictionary, there can be cited a date dictionary used for the filling field Ra for filling in a date, etc. In addition, the information on the positions and sizes of the respective filling fields Ra is represented as coordinate information of each filling field Ra in the paper document Bk.

[0107] Moreover, the character recognition unit 220 performs character recognition of the handwritten string Mt according to the definition information.

[0108] The storage unit 230 associates the image information of the paper document Bk, the definition information of the filling field Ra, and the OCR result obtained by the character recognition unit 220, and stores them as character recognition data.

[0109] Figure 7 It is a diagram showing the data structure of the character recognition data stored in the storage unit 230.

[0110] The illustrated data structure can be represented as a combination of the image information of the paper document Bk received by the transceiver unit 210 and the table T for storing the results of character recognition. The table T contains records for each column of "recognition box name", "OCR result", "dictionary name", "coordinate x", "coordinate y", "width w", and "height h".

[0111] The "recognition box name" is the name of each filling field Ra. Additionally, the recognition box name is also the information representing the content of the string filled in the filling field Ra. That is, it can be seen that names such as name, address 1, date of birth, etc. are assigned, and the name, address, and date of birth are filled in respectively. Further, the "OCR result" is the OCR result of the string Mt filled in each filling field Ra. Furthermore, the "dictionary name" is the name of the dictionary used in each filling field Ra. In this case, the dictionary name can also be said to be the name of the dictionary prepared corresponding to the information representing the content of the string and used in character recognition. Moreover, "coordinate x", "coordinate y", "width w", and "height h" are the coordinate information of each filling field Ra in the paper document Bk, corresponding to the x coordinate, y coordinate, width, and height of the filling field Ra respectively. That is, the coordinate information represents the position and size of the filling field Ra in the paper document Bk. Additionally, the x coordinate and y coordinate take the upper left corner of the paper document Bk as the origin (0, 0) and represent the position of the upper left corner of the filling field Ra.

[0112] In addition, the storage unit 230 stores the dictionary used in character recognition.

[0113] The terminal device 30 includes: a transceiver unit 310 that transceives electronic information with an external device; a reception unit 320 that receives an instruction from a user; a determination unit 330 that determines the presence or absence of a string such as "the same above"; a search unit 340 that searches for the string referred to by "the same above" etc.; a display control unit 350 that creates display data; a display unit 360 that displays a confirmation screen Ga; and a replacement unit 370 that replaces a string.

[0114] The transceiver unit 310 transceives electronic information with the management server 20. The transceiver unit 310 obtains the image information of the paper document Bk, the definition information of the filling field Ra, and the OCR result obtained by the character recognition unit 220, which are the data stored in the storage unit 230, from the management server 20.

[0115] The reception unit 320 receives an instruction from a user. The instruction from the user is, for example, carried out through the confirmation screen Ga, such as the selection of the paper document Bk or the correction of the OCR result.

[0116] The determination unit 330 determines whether the string for which character recognition has been performed is a character or symbol that means a string containing the same string as the filled string. That is, the determination unit 330 determines, based on the OCR result, whether the string Mt filled in the filling field Ra is a character or symbol such as "same as above", "same as left", "〃".

[0117] The search unit 340 searches for the string referred to by a character or symbol such as "same as above", "same as left", "〃". That is, in the paper document Bk, the string indicated by these characters or symbols is found. Additionally, this can also be said to be searching for the string referred to by characters or symbols such as "same as above", "same as left", "〃".

[0118] The display control unit 350 creates the display data of the screen to be displayed by the display unit 360. Additionally, the display unit 360 displays an image based on the display data created by the display control unit 350.

[0119] The image displayed here is the confirmation screen Ga.

[0120] The replacement unit 370 replaces a character or symbol such as "same as above", "same as left", "〃" with the string referred to by these characters or symbols. That is, the characters or symbols such as "same as above", "same as left", "〃" are corrected to the string indicated by these characters or symbols.

[0121] <Detailed description of the confirmation screen Ga>

[0122] Subsequently, a detailed description of the confirmation screen Ga is given.

[0123] Figure 8 of (a) to Figure 8 (b) of is a diagram showing the first example of the confirmation screen Ga.

[0124] Among them, Figure 8 (a) of is a diagram showing the entire confirmation screen Ga. In the illustrated confirmation screen Ga, the video image Ga1 formed from the image information of the paper document Bk is displayed on the left side of the figure. That is, the video image Ga1 is the image of the paper document Bk scanned by the image forming apparatus 10.

[0125] Additionally, in the illustrated confirmation screen Ga, the OCR result is displayed as the recognition result list Ga2 on the right side of the figure.

[0126] Figure 8Part (b) is a magnified view of a part of the recognition result list Ga2. In the illustrated example, it shows the recognition results Nk1 to Nk3 for three filling fields Ra arranged vertically in the figure. Among them, the first recognition result Nk1 from the top and the second recognition result Nk2 indicate that characters or symbols such as "same as above", "same as left", "〃" are not filled in the filling field Ra. In this case, a handwritten string Mt11 is displayed above the recognition result Nk1. Moreover, a string Mc11, which is the OCR result of the string Mt11, is displayed below. Similarly, a handwritten string Mt21 is displayed above the recognition result Nk2. Moreover, a string Mc21, which is the OCR result of the string Mt21, is displayed below.

[0127] On the other hand, the third recognition result Nk3 indicates the case where the character "same as above" is filled in the filling field Ra. In this case, a handwritten string Mt31 is displayed in the middle of the recognition result Nk3. Here, it indicates the string "same as above" filled in the filling field Ra. Moreover, a handwritten string Mt32, which is the string referred to by "same as above", is displayed above. Here, the handwritten string Mt32 referred to by "same as above" represents "Aichi City 〇-△-□". Furthermore, a string Mc32, which is the OCR result of the string Mt32, is displayed below.

[0128] That is, here, as the confirmation screen Ga, the handwritten strings (in this case, the strings Mt31 and Mt32 respectively), which are characters or symbols and the strings referred to by the characters or symbols, and the OCR result of the string referred to by the characters or symbols (in this case, the string Mc32) are arranged and displayed.

[0129] Thus, it is easy for the user B who confirms the OCR result to judge whether the OCR result is correct.

[0130] In addition, in this embodiment, the user B can use an input element such as a mouse to select the part of the recognition result Nk3. In this case, in the video image Ga1, the position Pj31 of the filling field Ra where the string Mt31 with "same as above" is filled and the position Pj32 of the filling field Ra where the string Mt32 (recorded as "reference source" in the figure) referred to by "same as above" are highlighted. In this case, the string Mt32 referred to by the string Mt31 with "same as above" is located above the string Mt31. In addition, when the string Mt31 is the symbol "〃", the positional relationship between the string Mt31 and the string Mt32 is the same.

[0131] This can also be said to be, in addition to displaying the confirmation screen Ga, further displaying an image of the paper document Bk on which the handwritten character strings Mt31 and Mt32 are written, and in the image of the paper document Bk, the positions of the part where the character string Mt31 of "same as above" is displayed and the part where the character string Mt32 referred to by "same as above" is located.

[0132] Thereby, the user B who confirms the OCR result can confirm the positions of the character strings Mt31 and Mt32 in the paper document Bk, and it is easier to judge whether the OCR result is correct.

[0133] In addition, when the user B selects the part of the recognition result Nk1, in the video image Ga1, the position of the filling column Ra filled with the character string Mt11 is highlighted. Similarly, when the user B selects the part of the recognition result Nk2, in the video image Ga1, the position of the filling column Ra filled with the character string Mt21 is highlighted.

[0134] Figure 9 of (a) to Figure 9 (b) is a diagram showing a second example of the confirmation screen Ga.

[0135] Among them, Figure 9 the example shown in (b) is the same as Figure 8 the example shown in (b) except that the character string Mt31 changes from "same as above" to "same as the left".

[0136] In addition, in Figure 9 (a), it shows the case where in the video image Ga1, the position Pj31 of the filling column Ra filled with the character string Mt31 of "same as the left" and the position Pj32 of the filling column Ra on which the character string Mt32 (recorded as "citation source" in the figure) referred to by "same as the left" are highlighted. In this case, the character string Mt32 referred to by the character string Mt31 of "same as the left" is located to the left of the character string Mt31.

[0137] <Detailed description of the operation of the terminal device 30>

[0138] Next, the operation of the terminal device 30 will be described in detail.

[0139] Figures 10 to 14 is a flowchart showing the operation of the terminal device 30.

[0140] Among them, Figure 10 is a diagram showing the order of the display list of the recognition result list Ga2 for producing the confirmation screen for displaying Figure 9 of (a) to Figure 9 (b).

[0141] First, the transceiver unit 310 obtains the first record in the table T of character recognition data from the storage unit 230 of the management server 20 (step 101). In Figure 7 In the example shown in, the first record becomes "Name, Test Taro, Given Name, 30, 50, 100, 30".

[0142] Subsequently, the determination unit 330 determines whether the OCR result of the obtained record is a character or symbol such as "Same as above", "Same as left", "〃", etc. (step 102).

[0143] Moreover, when the OCR result is a character or symbol such as "Same as above", "Same as left", "〃", etc. (Yes in step 102), the search unit 340 retrieves the string referred to by these characters or symbols from the paper document Bk, and the display control unit 350 updates the display list. (step 103). The processing of step 103 will be described in detail later.

[0144] In contrast, when the OCR result is not a character or symbol such as "Same as above", "Same as left", "〃", etc. (No in step 102), the display control unit 350 updates the display list accordingly. Specifically, the handwritten string Mk and the OCR result of the string Mk are added to the display list (step 104). That is, the setting for displaying like the recognition result Nk1 or the recognition result Nk2 in (b) of Figure 8 is added.

[0145] After steps 103 and 104, the transceiver unit 310 determines whether there is a next record (step 105).

[0146] As a result, when there is a next record (Yes in step 105), the transceiver unit 310 obtains the next record (step 106) and returns to step 102.

[0147] In contrast, when there is no next record (No in step 105), the series of processes ends.

[0148] Figure 11 And Figure 12 is Figure 10 a flowchart for explaining the processing of step 103 in more detail.

[0149] Among them, Figure 11 is a diagram showing the process in which the search unit 340 retrieves the string referred to by "Same as above" from the paper document Bk. In addition, the process of retrieving the string referred to by "〃" from the paper document Bk is the same.

[0150] First, the transceiver unit 310 obtains the first record in the table T of character recognition data from the storage unit 230 of the management server 20 (step 201).

[0151] Then, the exploration unit 340 determines whether the coordinate x of the record obtained in step 201 is the same as the coordinate x of the record filled with "same as above" (step 202). That is, the exploration unit 340 determines whether the x - coordinate of the filling field Ra obtained in step 201 is the same as the x - coordinate of the filling field Ra filled with "same as above".

[0152] As a result, in the case of being the same (yes in step 202), the exploration unit 340 determines whether the coordinate y of the record obtained in step 201 is less than the coordinate y of the record filled with "same as above" (step 203). That is, the exploration unit 340 determines whether the filling field Ra obtained in step 201 is located above the filling field Ra filled with "same as above".

[0153] Moreover, in the case of being less (yes in step 203), that is, in the case of being located above, the exploration unit 340 determines whether the dictionary name of the record obtained in step 201 is the same as the dictionary name of the record filled with "same as above" (step 204). That is, here, it is determined whether the content of the filling field Ra is the same according to the dictionary used. Specifically, when the dictionary names are, for example, the same name dictionary for both, it can be considered that the filling fields Ra of both contain the same kind of content, such as names. In contrast, when the dictionary names are different, it can be considered that they contain different contents such as names and addresses.

[0154] As a result, in the case of being the same (yes in step 204), the exploration unit 340 determines whether the OCR result of the record obtained in step 201 is "same as above", "same as left", or "〃" (step 205). That is, the exploration unit 340 determines whether the string represented by "same as above" is "same as above", "same as left", or "〃".

[0155] Moreover, in the case of not being "same as above", "same as left", or "〃" (yes in step 205), the record obtained in step 201 is added to the display candidate list (step 206).

[0156] Therefore, here, the exploration unit 340 first determines in steps 202 - 203 whether the filling field Ra of the record obtained in step 201 is directly above the filling field Ra of the record filled with "same as above". Then, the exploration unit 340 further determines in step 204 that in the case of containing the same kind of content, the string Mt of the filling field Ra obtained in step 201 is a candidate for the string represented by "same as above".

[0157] Moreover, after step 206, the transceiver unit 310 determines whether there is a next record (step 207). Additionally, when it is "the same as above", "same as the left", or "〃" in step 205 (no in step 205), the process of step 207 is performed, and when it is no in steps 202, 203, and 204, the process of step 207 is also performed.

[0158] As a result, when there is a next record (yes in step 207), the transceiver unit 310 acquires the next record (step 208) and returns to step 202.

[0159] In contrast, when there is no next record (no in step 207), the display control unit 350 determines whether there is one or more candidate records in the display candidate list (step 209).

[0160] As a result, when there is (yes in step 209), the display control unit 350 updates the display list using a pre-determined method (step 210). This will be described later.

[0161] In contrast, when there is none (no in step 209), that is, when no fill-in field Ra filled with the string Mt referring to "the same as above" is found, the display control unit 350 updates the settings of the display list accordingly. Specifically, the handwritten string Mk and the OCR result of the string Mk are added to the display list (step 211). That is, the setting for displaying without the string Mt32 in the recognition result Nk3 of (b) in Figure 8 is added.

[0162] Additionally, Figure 12 is a diagram showing the process in which the search unit 340 retrieves the string referring to "same as the left" from the paper document Bk.

[0163] In Figure 12 steps 301 and 304 to 311 are the same as steps 201 and 204 to 211 in Figure 11 .

[0164] Therefore, here mainly the different steps 302 to 303 from Figure 11 will be described.

[0165] In step 302, the search unit 340 determines whether the coordinate y of the record acquired in step 301 is the same as the coordinate y of the record filled with "same as the left" (step 302). That is, the search unit 340 determines whether the y coordinate of the fill-in field Ra acquired in step 301 is the same as the y coordinate of the fill-in field Ra filled with "same as the left".

[0166] As a result, in the same case (Yes in step 302), the exploration unit 340 determines whether the coordinate x of the record obtained in step 301 is less than the coordinate x of the record filled with "Same as the left" (step 303). That is, the exploration unit 340 determines whether the filling field Ra of the record obtained in step 301 is located on the left side compared to the filling field Ra of the record filled with "Same as the left".

[0167] Moreover, in the case of being less (Yes in step 303), that is, in the case of being located on the left side, the exploration unit 340 determines whether the dictionary name of the record obtained in step 301 is the same as the dictionary name of the record filled with "Same as the left" (step 304).

[0168] Therefore, here, the exploration unit 340 first determines in steps 302 to 303 whether the filling field Ra of the record obtained in step 301 is located on the left side of the filling field Ra of the record filled with "Same as the left". Then, the exploration unit 340 further determines in step 304 that in the case of including the same content, the string Mt of the filling field Ra obtained in step 301 is a candidate for the string referred to by "Same as the left".

[0169] As described in Figure 11 and Figure 12 , the exploration unit 340 explores the strings determined by the directions corresponding to characters or symbols such as "Same as the above", "Same as the left", and "〃". That is, in the case of "Same as the above" or "〃", above the filling field Ra filled with "Same as the above" or "〃", there is a filling field Ra filled with the string Mt referred to by "Same as the above" or "〃". Therefore, the exploration unit 340 retrieves upward. In addition, in the case of "Same as the left", to the left of the filling field Ra filled with "Same as the left", there is a filling field Ra filled with the string Mt referred to by "Same as the left". Therefore, the exploration unit 340 retrieves to the left. At this time, the exploration unit 340 uses the x coordinate and y coordinate of the filling field Ra filled with the string Mt, and determines the direction based on the position of the filling field Ra.

[0170] In addition, the exploration unit 340 compares the information of the filling field Ra recording characters or symbols such as "Same as the above", "Same as the left", and "〃" with the information of the filling field Ra existing in these directions. In the case where the attributes of the content recorded in the filling field Ra are the same, it is regarded as a candidate for the string referred to by the character or symbol. In this case, as the attribute, when the dictionaries used for character recognition are the same, the exploration unit 340 regards the attributes of the content as the same.

[0171] Figure 13 of (a) is a flowchart for explaining the processing of step 210 of Figure 11 in more detail.

[0172] Here, the display control unit 350 first rearranges the records existing in the display candidate list in descending order of the y coordinate (step 401).

[0173] Next, the display control unit 350 sets the foremost record among the rearranged records as an item to be added to the display list (step 402). That is, when there are multiple records in the display candidate list, the record closest to the entry field Ra filled with "same as above" is selected.

[0174] Furthermore, the display control unit 350 reflects the values in the record in the display list and updates it (step 403).

[0175] Then, the display control unit 350 synthesizes an image based on the display list in order to display the image of the recognition result Nk3 shown in (b) of Figure 8 (step 404).

[0176] In addition, the display control unit 350 adds the synthesized image to the recognition result list Ga2 (step 405).

[0177] In addition, Figure 13 (b) of Figure 12 is a flowchart that explains the processing of step 310 of

[0178] Here, the display control unit 350 rearranges the records existing in the display candidate list in descending order of the x coordinate (step 501).

[0179] Moreover, the subsequent steps 502 to 505 are the same as steps 402 to 405 in Figure 13 (a). Therefore, here, when there are multiple records in the display candidate list, the record closest to the entry field Ra filled with "same as left" is also selected.

[0180] Figure 14 is a diagram showing the operation of the terminal device 30 when the user selects a part of the recognition result in the recognition result list Ga2 using an input element such as a mouse.

[0181] First, the position information in the recognition result list Ga2 selected by the user is obtained (step 601).

[0182] Then, the display control unit 350 determines whether a recognition result is displayed at the said part (step 602).

[0183] Then, in the case where the recognition result is displayed (Yes in step 602), in the video image Ga1, the filling-in fields Ra filled with characters or symbols such as "Same as above", "Same as left", "〃", etc. are highlighted (step 603). Furthermore, the filling-in fields Ra referred to by these characters or symbols are highlighted (step 604).

[0184] In contrast, in the case where the recognition result is not displayed (No in step 602), the series of processes is terminated.

[0185] Figure 15 of (a) to Figure 15 of (c) is a diagram showing the operation of the replacement unit 370.

[0186] After user B confirms the character recognition result in the confirmation screen Ga, the replacement unit 370 replaces characters or symbols such as "Same as above", "Same as left", "〃", etc. with the strings referred to by these characters or symbols.

[0187] Among them, Figure 15 of (a) shows the paper document Bk scanned by Figure 3 1A of.

[0188] The paper document Bk is Figure 5 the same as the paper document Bk shown in (a) of, and as the destination and sender information, an address field Ra1 and an address field Ra2 are respectively set. Moreover, it shows the case where user A fills the address of the destination in the filling-in field Ra1 in the form of a string Mt1.

[0189] Figure 15 of (b) shows Figure 15 the OCR result of the paper document Bk shown in (a) of.

[0190] The illustrated OCR result indicates that for the address of the destination, the handwritten string Mt1 filled by user A has been character-recognized into a string Mc1. In addition, it indicates that for the address of the sender, the handwritten string Mt2 filled by user A has also been character-recognized into a string Mc3. However, the string Mc3 is replaced from the string Mc2 by the replacement unit 370. That is, as shown in Figure 5 of (b), the string Mt2 is first character-recognized as "Same as above" as the string Mc2, but is replaced by the replacement unit 370 with the string Mc3 referred to by "Same as above". In this case, "Same as above" as the string Mc2 is replaced by "○○ County △△ City ◇◇ Town 1-2-34" as the string Mc3.

[0191] In addition, Figure 15 of (c) shows an example of confirming the correction result.

[0192] In this example, the confirmed correction result is output in CSV format. Moreover, the output data represents "Delivery destination_address, Sender_address, 1-2-34, ◇◇ Town, △△ City, ○○ County, 1-2-34, ◇◇ Town, △△ City, ○○ County".

[0193] According to the example described in detail above, an information processing apparatus can be provided in which even if there are characters or symbols that mean including a string identical to the filled string, the burden on the user for confirming the result of character recognition is not likely to increase.

[0194] In addition, in the above example, the cases of using Japanese to record characters or symbols such as "Same as above", "Same as left", and "〃" are described, but other languages can also be applied. For example, when strings such as "Same as above" meaning "Same as above" and "Same as left" meaning "Same as left" are filled in a paper document Bk recorded in English, the same processing can be performed.

[0195] <Explanation of the program>

[0196] In addition, the processing performed by the terminal device 30 in the present embodiment is realized by the cooperation of software and hardware resources. That is, the CPU provided inside the terminal device 30 executes a program for realizing each function of the terminal device 30, and thus realizes the respective functions.

[0197] Therefore, the processing performed by the image forming apparatus 10 can also be understood as a program for causing a computer to realize the following functions: an acquisition function for acquiring the result of character recognition of a handwritten document; and a replacement function for replacing a character or symbol with the string represented by the character or symbol when the recognized string is a character or symbol that means including a string identical to the filled string.

[0198] As described above, the present embodiment has been described, but the technical scope of the present disclosure is not limited to the scope described in the above embodiment. It is clear from the description of the claims that those obtained by making various changes or improvements to the above embodiment are also included in the technical scope of the present disclosure.

Claims

1. An information processing apparatus, comprising a processor, The processor obtains a result of character recognition for a handwritten string filled in a box. When the string subjected to character recognition filled in the first box is a character or symbol meaning a string containing the same string as the filled string, when the second box is located on a side determined according to the character or symbol compared with the first box, and the attribute of the content of the second box is the same as the attribute of the content of the first box, the character or symbol in the first box is replaced with the string filled in the second box.

2. The information processing apparatus according to claim 1, wherein the processor displays a confirmation screen for the user to confirm the replaced string.

3. The information processing apparatus according to claim 2, wherein as the confirmation screen, the processor arranges and displays a handwritten string of the character or symbol and the string filled in the second box, and a result of character recognition of the string filled in the second box.

4. The information processing apparatus according to claim 2, wherein in addition to displaying the confirmation screen, the processor further displays an image of a document with a handwritten string. In the image of the document, the positions of the part of the character or symbol and the part of the string filled in the second box are displayed.

5. The information processing apparatus according to claim 4, wherein when the user selects any one in the confirmation screen, the processor displays the positions of the part of the character or symbol and the part of the string filled in the second box in the image of the document.

6. The information processing apparatus according to claim 1, wherein as the attribute, when the dictionaries used for character recognition are the same, the processor regards the attributes of the contents as the same.

7. The information processing apparatus according to claim 1, wherein when there are multiple boxes located on the side determined according to the character or symbol compared with the first box, and the attributes of the contents of the multiple boxes are the same as the attribute of the content of the first box, the processor replaces the character or symbol in the first box with the string in the box closest to the first box among the multiple boxes.

8. A character recognition system, comprising: A reading device that reads a document with a handwritten string in the form of an image; A character recognition device that performs character recognition on a handwritten string filled in a box; A display device that displays a confirmation screen for the user to confirm the result of character recognition performed by the character recognition device; And An image production device that produces an image of the confirmation screen; The image production device includes a processor, The processor obtains a result of character recognition from the character recognition device, When the character-recognized string filled in the first box is a character or symbol that means a string containing the same string as the filled string, when the second box is located on the side determined according to the character or symbol compared with the first box, and the attribute of the content of the second box is the same as the attribute of the content of the first box, replace the character or symbol in the first box with the string filled in the second box.

9. A computer-readable medium is a computer-readable medium storing a program for causing a computer to execute processing, and the processing includes the following steps: Obtain the result of character recognition of a handwritten string filled in a box; and When the character-recognized string filled in the first box is a character or symbol that means a string containing the same string as the filled string, when the second box is located on the side determined according to the character or symbol compared with the first box, and the attribute of the content of the second box is the same as the attribute of the content of the first box, replace the character or symbol in the first box with the string filled in the second box.

10. An information processing method includes the following steps: Obtain the result of character recognition of a handwritten string filled in a box; and When the character-recognized string filled in the first box is a character or symbol that means a string containing the same string as the filled string, when the second box is located on the side determined according to the character or symbol compared with the first box, and the attribute of the content of the second box is the same as the attribute of the content of the first box, replace the character or symbol in the first box with the string filled in the second box.

Citation Information

Patent Citations

  • Bill data input device and bill data input method

    CN103135894A

  • Character processor

    JP1997134357A