Information processing device, character recognition system and program

The information processing device addresses the user burden of recognizing repeated character strings by automatically replacing symbols with corresponding strings based on frame attributes, enhancing recognition efficiency and accuracy.

JP7581645B2Active Publication Date: 2024-11-13FUJIFILM BUSINESS INNOVATION CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2020069827
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-04-08
Publication Date
2024-11-13
Estimated Expiration
2040-04-08

AI Technical Summary

Technical Problem

Existing character recognition systems burden users with increased effort when they encounter characters or symbols indicating the same character string in multiple entry fields, requiring manual confirmation and correction.

Method used

An information processing device that uses a processor to recognize characters or symbols indicating the same string, searches for and compares the attributes of the frames in which they are written, and replaces them with the corresponding character strings, displaying the results alongside the original symbols for user confirmation.

Benefits of technology

Reduces user burden by accurately identifying and replacing characters or symbols indicating the same string, allowing for efficient and accurate character recognition without manual searching.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007581645000001
    Figure 0007581645000001
  • Figure 0007581645000002
    Figure 0007581645000002
  • Figure 0007581645000003
    Figure 0007581645000003
Patent Text Reader

Abstract

To provide an information processing device, etc., that is unlikely to increase a burden on a user to check a result of character recognition even if there are characters or symbols that are meant to contain the same character string as the one already entered.SOLUTION: The information processing device includes a processor. The processor obtains a result of character recognition for a handwritten character string. When the recognized character string is a character or symbol that is meant to contain the same character string as a character string that has already been filled in, the processor replaces the character or symbol with the character string that this character or symbol refers to.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device, a data structure, a character recognition system, and a program. [Background technology]

[0002] Conventionally, there has been a technology that uses OCR (Optical Character Recognition) to recognize handwritten characters written in a field in a document. Then, a process is sometimes performed to have a user confirm whether the result of the character recognition is correct or not.

[0003] Patent Document 1 discloses a character processing device. In this character processing device, document data input from an input means is stored in a data storage means, and when a match determination means determines that there is a character string that partially matches the beginning of the data stored in the data storage means, a character string replacement means reads out a specific character string from a specific character string storage means and replaces it, and further, a space insertion means supplements spaces by the replacement. Also, the specific character string in the data is restored to the original character string. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 9-134357 Summary of the Invention [Problem to be solved by the invention]

[0005] When the same character string is entered in multiple entry fields, characters or symbols such as "same as above," "same as above," and "〃" may be entered, which indicates that the same character string will be entered as the previous character string. However, if OCR were to directly recognize these characters or symbols, the user would have to search for and confirm the character strings they refer to in the document. This increases the burden on the user. The present invention aims to provide an information processing device, etc. that does not increase the burden on a user who checks the results of character recognition even if there is a character or symbol that means that the same character string as a character string already entered will be entered. [Means for solving the problem]

[0006] The invention described in claim 1 is The information processing device is equipped with a processor that obtains the results of character recognition of a handwritten character string, and when the recognized character string is a character or symbol that indicates that the same character string as an already written character string is to be inserted, searches for a character string in a direction determined by the character or symbol, compares information about a box in which the character or symbol is written with information about a box that exists in that direction, and, if the attributes of the content written within the box are the same, makes the character or symbol a candidate for the character string indicated by the character or symbol, and replaces the character or symbol with the character string obtained by the search. The invention described in claim 2 is The information processing device according to claim 1, characterized in that the processor determines that the content attributes are the same when the dictionaries used in character recognition are the same. The invention described in claim 3 is 2. The information processing apparatus according to claim 1, wherein when there are a plurality of candidates, the processor selects the one that is closest in position to the character or symbol. The invention described in claim 4 is 2. The information processing apparatus according to claim 1, wherein the processor displays a confirmation screen for prompting a user to confirm the replaced character string. The invention described in claim 5 is The information processing device according to claim 4, characterized in that the processor displays, as the confirmation screen, a handwritten version of the character or symbol and a string of characters indicated by the character or symbol, as well as the results of character recognition of the string of characters indicated by the character or symbol. The invention described in claim 6 is The information processing device according to claim 4, characterized in that, in addition to the confirmation screen, the processor further displays an image of a document containing a handwritten character string, and displays, in the image of the document, the location of the character or symbol and the location of the character string to which the character or symbol points. The invention described in claim 7 is The information processing device according to claim 6, characterized in that the processor indicates, when a user selects something on the confirmation screen, the location of the character or symbol and the location of the character string that the character or symbol points to in the image of the document. The invention described in claim 8 is The character recognition system includes a reading device that reads a document containing handwritten character strings as an image, a character recognition device that performs character recognition on the handwritten character strings, a display device that displays a confirmation screen that allows a user to confirm the results of character recognition by the character recognition device, and an image creation device that creates an image of the confirmation screen, wherein the image creation device has a processor that obtains the results of character recognition from the character recognition device, and, when the recognized character string is a character or symbol that indicates that the same character string as an already entered character string is to be inserted, searches for a character string in a direction determined according to the character or symbol, compares information about a box in which the character or symbol is written with information about a box that exists in that direction, and, if the attributes of the content written in the box are the same, sets the character or symbol as a candidate for the character string indicated by the character or symbol and replaces the character or symbol with the character string obtained by the search. The invention described in claim 9 is This program enables a computer to realize an acquisition function that acquires the results of character recognition of a handwritten document, and a replacement function that, when the recognized character string is a character or symbol that indicates that the same character string as an already written character string is to be entered, searches for character strings in a direction determined by the character or symbol, compares information about the box in which the character or symbol is written with information about the box that exists in that direction, and, if the attributes of the content written within the box are the same, makes the character or symbol a candidate for the character string indicated by the character or symbol, and replaces the character or symbol with the character string obtained by the search. Effect of the Invention

[0007] According to the invention of claim 1, To provide an information processing device that does not increase the burden on a user checking the result of character recognition even if there is a character or symbol that means that the same character string as an already entered character string is to be entered, and can more accurately search for a character string indicated by a character or symbol that means that the same character string as an already entered character string is to be entered, and can perform a search while taking into consideration the attribute of the character string to be entered in a box. According to the invention of claim 2, A search can be performed taking into account the contents of the character string entered in the box. According to the invention of claim 3, It is possible to extract the most accurate string of characters that is indicated by letters or symbols that mean that the same string as the string already entered will be entered. According to the invention of claim 4, The user can confirm the character string indicated by the letters or symbols which mean that the same character string as the one already entered will be entered. According to the invention of claim 5, It is possible to check by comparing the characters or symbols that indicate that the same character string as the already entered character string will be entered with the character strings that they indicate. According to the invention of claim 6, It is possible to confirm the positions in the original paper document of the characters and symbols which indicate that the same character string as that already entered will be entered, and the character strings which they indicate. According to the invention of claim 7, The user can select the character string that he or she wants to check. According to the invention of claim 8, To provide an information processing system that does not increase the burden on a user checking the result of character recognition even if there is a character or symbol that means that the same character string as an already entered character string is to be entered, and that can more accurately search for a character string indicated by a character or symbol that means that the same character string as an already entered character string is to be entered, and can perform a search while taking into consideration the attributes of the character string to be entered in a box. According to the invention of claim 9, To provide a function that does not increase the burden on a user checking the results of character recognition even when there is a character or symbol that means that the same character string as an already entered character string is to be entered, and to more accurately search for a character string indicated by a character or symbol that means that the same character string as an already entered character string is to be entered, and to perform a search while taking into consideration the attributes of the character string to be entered in the box. [Brief description of the drawings]

[0008] [Figure 1]1 is a diagram illustrating an example of the configuration of a character recognition system according to an embodiment of the present invention. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of an image forming apparatus. [Diagram 3] FIG. 2 is a diagram illustrating an outline of the operation of the character recognition system. [Figure 4] FIG. 1 is a diagram showing an example of a paper document according to an embodiment of the present invention. [Diagram 5] 1A to 1C are diagrams showing examples of conventional confirmation work performed on the OCR results in 1G of FIG. [Figure 6] 1 is a block diagram showing an example of a functional configuration of an information processing system according to an embodiment of the present invention. [Figure 7] 4 is a diagram showing a data structure of character recognition data stored in a storage unit. FIG. [Figure 8] 13(a) and 13(b) are diagrams showing a first example of a confirmation screen. [Figure 9] 13(a) and 13(b) are diagrams showing a second example of a confirmation screen. [Figure 10] 10 is a diagram showing a procedure for creating a display list for displaying a list of recognition results on the confirmation screen of FIG. 9. [Figure 11] 13 is a diagram showing a process in which a search unit searches a paper document for a character string indicated by "same as above." [Figure 12] 13 is a diagram showing a process in which a search unit searches a paper document for a character string indicated by "same as above." [Figure 13] 12(a) is a flowchart illustrating in more detail the processing of step 210 in Fig. 11. FIG. 12(b) is a flowchart illustrating in more detail the processing of step 310 in Fig. 12. [Figure 14] 13 is a diagram showing the operation of the terminal device when a user selects a portion of a recognition result in a recognition result list using an input device such as a mouse. [Figure 15] 6(a) to 6(c) are diagrams illustrating the operation of the replacement unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0010] <Explanation of the character recognition system> FIG. 1 is a diagram showing an example of the configuration of a character recognition system 1 according to the present embodiment. As shown in the figure, the character recognition system 1 of the present embodiment is configured by connecting an image forming device 10, a management server 20, and a terminal device 30 via a network 70 and a network 80. Although only one image forming apparatus 10 and one terminal apparatus 30 are shown in FIG. 1, a plurality of each may be provided.

[0011] The image forming device 10 is a device that forms an image on a recording medium such as paper and outputs it as a print medium. The image forming device 10 has a printer function, but in addition to this, it has other image processing functions such as a scanner function and a facsimile function. FIG. 2 is a diagram showing an example of the hardware configuration of the image forming apparatus 10. As shown in FIG. As shown in the figure, the image forming apparatus 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a HDD (Hard Disk Drive) 14, an operation panel 15, an image reading unit 16, an image forming unit 17, and a communication interface (hereinafter, referred to as "communication I / F") 18. These exchange necessary data via a bus B.

[0012] The CPU 11 loads various programs stored in the ROM 13 or the like into the RAM 12 and executes them to realize various functions, which will be described later.

[0013] The RAM 12 is a memory used as a working memory for the CPU 11, etc. The ROM 13 is a memory that stores various programs executed by the CPU 11. The HDD 14 is, for example, a magnetic disk device that stores image information read by the image reading unit 16, image information used in image formation by the image forming unit 17, and the like. The operation panel 15 is, for example, a touch panel that displays various information and accepts operation input from the user. For this reason, the display mechanism has a display unit such as a liquid crystal panel that displays content (information content) as an image in a predetermined area, and has a function of detecting the position of a contact object, such as a human finger or a stylus pen, when the contact object touches the liquid crystal panel. In this embodiment, the touch panel is not particularly limited, and various types such as a resistive film type and a capacitive type can be used.

[0014] The image reading unit 16 is an example of a reading device, and reads an image recorded on a document. In this embodiment, the document is a document containing handwritten character strings, and the image is read as the document. The image reading unit 16 may be, for example, a scanner, and may be of the CCD type, in which the reflected light from the light irradiated from a light source onto the document is reduced by a lens and received by a CCD (Charge Coupled Devices), or of the CIS type, in which the reflected light from the light irradiated sequentially onto the document from an LED light source is received by a CIS (Contact Image Sensor).

[0015] The image forming unit 17 is an example of a printing mechanism that forms an image on a recording medium. Here, the image forming unit 17 is, for example, a printer, and it is preferable to use an electrophotographic system that transfers toner attached to a photoconductor onto a recording medium such as paper to form an image, or an inkjet system that ejects ink onto a recording medium to form an image. The communication I / F 18 transmits and receives various types of information to and from other devices via the networks 70 and 80 .

[0016] The management server 20 is a server computer that manages the entire character recognition system 1. When the management server 20 is provided on a cloud, the management server 20 can also be said to be a cloud server. The management server 20, for example, authenticates a user who operates the image forming apparatus 10 or the terminal device 30, and transmits and receives electronic information between the image forming apparatus 10 and the terminal device 30. Here, "electronic information" refers to any information that has been digitized. The "electronic information" is not particularly limited in terms of the type, format, data structure, etc. of information as long as it can be handled by each of the image forming apparatus 10, the management server 20, and the terminal device 30. However, the electronic information handled by the character recognition system 1 of this embodiment is, for example, character information, image information, etc. Although details will be described later, the management server 20 is an example of a character recognition device, and performs character recognition on a handwritten character string read by the image reading unit 16 of the image forming apparatus 10.

[0017] The management server 20 includes a CPU as a computing means, a main memory as a storage means, and storage such as an HDD or SSD (Solid State Drive). The CPU executes various software such as an OS (operating system) and application software. The main memory is a storage area that stores various software and data used for executing the software, and the storage is a storage area that stores input data for the various software and output data from the various software. Furthermore, the management server 20 includes a communication I / F for communicating with the outside, and input devices such as a keyboard, a mouse, a touch panel, a stylus pen, etc. The management server 20 may also include a display device such as a liquid crystal display.

[0018] The terminal device 30 is a device where a user checks the results of character recognition performed by the management server 20. The terminal device 30 is, for example, a general-purpose personal computer (PC). The terminal device 30 runs various application software under the management of an OS (Operating System) to create electronic information, etc. The results of character recognition may hereinafter be referred to as "OCR results."

[0019] The terminal device 30, like the management server 20, includes a CPU, a main memory, a storage, a communication I / F, and an input device. These are examples of information processing devices or image creation devices, and create a confirmation screen that allows a user to confirm the OCR result generated by the management server 20. The CPU is an example of a processor. The terminal device 30 also includes a display device, which is an example of a display device that displays this confirmation screen.

[0020] The network 70 is a communication means used for information communication between the image forming apparatus 10, the management server 20, and the terminal apparatus 30, and is, for example, the Internet. Like the network 70, the network 80 is a communication means used for information communication among the image forming apparatus 10, the management server 20, and the terminal apparatus 30, and is, for example, a LAN (Local Area Network).

[0021] <Outline of the operation of the character recognition system 1> FIG. 3 is a diagram showing an outline of the operation of the character recognition system 1. As shown in FIG. In this example, the user who operates the image forming device 10 is User A, and the user who operates the terminal device 30 is User B. In addition, it is assumed that User A and User B have been authenticated by the management server 20 in advance.

[0022] First, user A scans paper document Bk (1A) using image forming device 10. The scanning is performed using image reading unit 16 of image forming device 10, and the scanned document is sent to management server 20 as electronic information consisting of image information (1B).

[0023] A paper document is an example of a document, and is a printed document Bp printed in a predetermined format on which a user A or the like has handwritten entries. The paper document Bk is not particularly limited, and may be, for example, a slip, a questionnaire, a voucher, an application form, a notification, or a request for approval.

[0024] FIG. 4 is a diagram showing an example of a paper document Bk according to the present embodiment. The illustrated paper document Bk is a slip, and shows a case where user A has written a handwritten character string Mt into a printed document Bp printed in a predetermined format. For example, entry fields Ra for name, address, etc. are printed in advance on the printed document Bp, and user A has written his / her name and address as a handwritten character string Mt into this entry field Ra. This entry field Ra can also be understood as a box into which user A writes, or a recognition box that is the target of character recognition.

[0025] Returning to Fig. 3, when the management server 20 receives the scanned electronic information, it performs character recognition using OCR (1C). For example, the management server 20 registers in advance the entry fields Ra in the print document Bp (1D). Then, based on the registered entry fields Ra, the handwritten characters Mt are extracted, and character recognition using OCR is performed on the extracted handwritten characters Mt. Then, the management server 20 transmits the OCR result to the terminal device 30 (1E). The terminal device 30 displays a confirmation screen Ga (1F) for prompting the user B to confirm whether the OCR result is correct or not, as will be described in detail later. That is, in this case, the user B is a person in charge or a manager who checks whether the OCR result is correct or not. User B checks the confirmation screen Ga, and if there are any corrections, the user corrects the OCR results (1G). After User B checks, the confirmation and correction results are output (1H). The confirmation and correction results are sent to the management server 20 and stored (1I).

[0026] 5(a) to (c) are diagrams showing an example of a conventional confirmation process performed in 1G of FIG. 3 on the OCR results. Of these, FIG. 5(a) shows the paper document Bk scanned in FIG. 3A 1A. This paper document Bk is a slip, and has address fields Ra1 and Ra2 for destination and sender information, respectively. The example shows a case where user A enters the destination address as a character string Mt1 in this entry field Ra1. The example also shows a case where the sender address is entered as a handwritten character string Mt2. However, in the example shown, since the destination address and the sender address are the same, the former has the actual address entered as the character string Mt1, while the latter has "same as above" entered as the character string Mt2, indicating that it is the same as the address already entered.

[0027] FIG. 5(b) shows the OCR result of the paper document Bk shown in FIG. 5(a). This is a part of the confirmation screen Ga displayed in 1F of Fig. 3. The OCR result shown in the figure indicates that for the destination address, handwritten character string Mt1 written by user A was character-recognized into a character string Mc1. Also, for the sender address, handwritten character string Mt2 written by user A was character-recognized into a character string Mc2.

[0028] FIG. 5(c) shows an example of the verification and correction results stored in 1H of FIG. In this example, the confirmation and correction results are output in CSV format. The output data shows that "Destination address, sender address, 1-2-34, ◇◇cho, △△city, XX prefecture, same as above."

[0029] However, as shown in FIG. 5B, if the character string Mc2, "same as above", is displayed as is on the confirmation screen Ga as the OCR result, user B who is checking the result needs to find the character string Mt1 corresponding to this "same as above" in the original paper document Bk and manually input the content. In this case, when user B finds the character string Mc2, "same as above", on the confirmation screen Ga, he / she finds the character string Mt2 in the image displaying the paper document Bk. Then, he / she further finds the character string Mt1 in this image and corrects the content of the character string Mc2 from "same as above" to "1-2-34, ◇◇-cho, △△-shi, XX-ken". In this case, the burden on user B who checks the OCR result increases.

[0030] Although FIGS. 5(a) to (c) show the case of "same as above," the same applies when characters or symbols such as "same as above" or "〃" are written on the paper document Bk. Therefore, in this embodiment, the character recognition system 1 is configured as follows to reduce the burden on user B.

[0031] <Explanation of the configuration of character recognition system 1> FIG. 6 is a block diagram showing an example of the functional configuration of the character recognition system 1 according to the present embodiment. In addition, here, among various functions possessed by the character recognition system 1, those related to this embodiment are selected and illustrated.

[0032] The image forming apparatus 10 of the character recognition system 1 includes a transmitting / receiving unit 110, a printing unit 120, a reading unit 130 that performs scanning, and a control unit 140. The transmitting / receiving unit 110 transmits and receives electronic information to and from the management server 20. The transmitting / receiving unit 110 transmits image information of a scanned paper document Bk to the management server 20. The transmitting / receiving unit 110 corresponds to the communication I / F 18, for example. The printing unit 120 is a printing mechanism that performs printing on a recording medium such as paper. The printing unit 120 corresponds to the image forming unit 17, for example. The reading unit 130 scans and reads the paper document Bk to obtain image information. The reading unit 130 corresponds to the image reading unit 16, for example. The control unit 140 performs overall control of the image forming apparatus 10. The control unit 140 corresponds to, for example, the CPU 11, the RAM 12, and the ROM 13.

[0033] The management server 20 includes a transmission / reception unit 210, a character recognition unit 220, and a storage unit 230. The transmitting / receiving unit 210 transmits and receives electronic information between the image forming apparatus 10 and the terminal device 30. The transmitting / receiving unit 210 receives image information of a paper document Bk scanned by the image forming apparatus 10, and also transmits the OCR results of the paper document Bk to the terminal device 30.

[0034] The character recognition unit 220 is an OCR unit that performs character recognition of a handwritten character string Mt in a paper document Bk. The storage unit 230 stores image information of the paper document Bk received by the transmission / reception unit 210, the OCR result which is the result of the character recognition, and definition information of the entry field Ra.

[0035] The definition information includes information on the attributes, position, and size of each entry field Ra in the paper document Bk. The attribute information includes a recognition frame name, which is the name of each entry field Ra and the name of the frame to be subjected to character recognition, and a dictionary name, which is the name of the dictionary used in each entry field Ra. This dictionary is used during character recognition, and the dictionary to be applied is determined in advance according to the contents of the character string Mt entered in each entry field Ra. For example, if the entry field Ra is a field in which a name is entered, a name dictionary is applied. That is, the name dictionary is, for example, a dictionary that collects people's names, and has contents suitable for performing character recognition of names. Also, if the entry field Ra is a field in which an address is entered, an address dictionary is applied. Other examples of dictionaries include a date dictionary used for the entry field Ra in which a date is entered. Also, the position and size information of each entry field Ra is expressed as coordinate information of each entry field Ra in the paper document Bk. Then, the character recognition unit 220 performs character recognition on the handwritten character string Mt based on the definition information.

[0036] The storage unit 230 associates the image information of the paper document Bk, the definition information of the entry fields Ra, and the OCR results obtained by the character recognition unit 220, and stores them as character recognition data. FIG. 7 is a diagram showing the data structure of the character recognition data stored in storage unit 230. As shown in FIG. The data structure shown in the figure can be expressed as a combination of image information of paper document Bk accepted by the transmitting / receiving unit 210 and a table T for storing the results of character recognition. This table T consists of records for each of the columns "recognition frame name", "OCR result", "dictionary name", "coordinate x", "coordinate y", "width w", and "height h". The "recognition frame name" is the name of each entry field Ra. The recognition frame name is also information that represents the content of the character string to be entered in this entry field Ra. In other words, names such as name, address 1, and date of birth are given, and it can be seen that the name, address, and date of birth are entered, respectively. The "OCR result" is the OCR result of the character string Mt entered in each entry field Ra. The "dictionary name" is the name of the dictionary used in each entry field Ra. In this case, the dictionary name is a dictionary prepared corresponding to the information that represents the content of the character string, and can also be said to be the name of the dictionary used during character recognition. The "coordinate x", "coordinate y", "width w", and "height h" are coordinate information of each entry field Ra in the paper document Bk, and correspond to the x coordinate, y coordinate, width, and height of the entry field Ra, respectively. In other words, the coordinate information represents the position of the entry field Ra in the paper document Bk and the size of the entry field Ra. The x and y coordinates indicate the position of the upper left corner of the entry field Ra, with the upper left corner of the paper document Bk being the origin (0, 0). The storage unit 230 also stores a dictionary used during character recognition.

[0037] The terminal device 30 includes a transmission / reception unit 310 for transmitting and receiving electronic information with an external device, a reception unit 320 for receiving instructions from a user, a determination unit 330 for determining whether or not a character string such as "same as above" is present, a search unit 340 for searching for a character string indicated by "same as above", a display control unit 350 for creating display data, a display unit 360 for displaying a confirmation screen Ga, and a replacement unit 370 for replacing character strings.

[0038] The transmitting / receiving unit 310 transmits and receives electronic information to and from the management server 20. The transmitting / receiving unit 310 acquires from the management server 20 the image information of the paper document Bk, the definition information of the entry field Ra, and the OCR result by the character recognition unit 220, which are data stored in the storage unit 230. The reception unit 320 receives instructions from the user. The user's instructions are given, for example, on the confirmation screen Ga, and include the selection of a paper document Bk and correction of the OCR results.

[0039] The determination unit 330 determines whether the character string that has been recognized is a character or symbol that indicates that the same character string as the character string already entered is to be entered. That is, the determination unit 330 determines whether the character string Mt entered in the entry field Ra is a character or symbol such as "same as above," "same as above," or "〃" based on the OCR result. The search unit 340 searches for character strings indicated by characters or symbols such as "same as above", "same as above", "same as above", "same as above", etc. In other words, it searches for character strings indicated by these characters in the paper document Bk. This can also be said to be searching for character strings cited by characters or symbols such as "same as above", "same as above", "same as above", etc.

[0040] The display control unit 350 creates display data for a screen to be displayed on the display unit 360. The display unit 360 displays an image based on the display data created by the display control unit 350. The image displayed here is the confirmation screen Ga described above. The replacement unit 370 replaces characters or symbols such as "same as above", "same as above", "〃" and the like with character strings that these characters or symbols indicate. In other words, characters or symbols such as "same as above", "same as above", "〃" and the like are modified to the character strings that they indicate.

[0041] <Details of confirmation screen Ga> Next, the confirmation screen Ga will be described in detail. 8(a) and (b) are diagrams showing a first example of a confirmation screen Ga. 8(a) is a diagram showing the entire confirmation screen Ga. In the illustrated confirmation screen Ga, an image Ga1 based on image information of a paper document Bk is displayed on the left side of the figure. In other words, this image Ga1 is an image of the paper document Bk scanned by the image forming device 10. Furthermore, in the illustrated confirmation screen Ga, the OCR results are displayed as a recognition result list Ga2 on the right side of the figure.

[0042] FIG. 8(b) is an enlarged view of a part of the recognition result list Ga2. In the illustrated example, the recognition results Nk1 to Nk3 for three entry fields Ra are arranged vertically in the figure. Among them, the first recognition result Nk1 and the second recognition result Nk2 from the top show the case where no characters or symbols such as "same as above", "same as above", or "〃" are written in the entry field Ra. In this case, a handwritten character string Mt11 is displayed in the upper row of the recognition result Nk1. Then, a character string Mc11 is displayed in the lower row as the OCR result of this character string Mt11. Similarly, a handwritten character string Mt21 is displayed in the upper row of the recognition result Nk2. Then, a character string Mc21 is displayed in the lower row as the OCR result of this character string Mt21.

[0043] On the other hand, the third recognition result Nk3 shows a case where the characters "same as above" have been written in the entry field Ra. In this case, a handwritten character string Mt31 is displayed in the middle of the recognition result Nk3. Here, it is shown that the character string "same as above" has been written in the entry field Ra. Then, in the upper row, a handwritten character string Mt32 is displayed as the character string pointed to by "same as above". Here, it is shown that the handwritten character string Mt32 pointed to by "same as above" is "Aichi City 〇-△-□". Furthermore, in the lower row, a character string Mc32 is displayed as the OCR result of this character string Mt32.

[0044] That is, here, the confirmation screen Ga displays handwritten characters or symbols and the string of characters that the characters or symbols point to (in this case, the strings Mt31 and Mt32, respectively), as well as the OCR results of the string of characters that the characters or symbols point to (in this case, the string Mc32). This makes it easier for User B, who checks the OCR results, to determine whether the OCR results are correct or not.

[0045] Furthermore, in this embodiment, user B can select a portion of the recognition result Nk3 using an input device such as a mouse. In this case, the position Pj31 of the entry field Ra in which the character string Mt31 "same as above" is entered and the position Pj32 of the entry field Ra in which the character string Mt32 pointed to by this "same as above" (written as "source" in the figure) is entered are highlighted in the image Ga1. In this case, the character string Mt32 pointed to by the character string Mt31 "same as above" is located above the character string Mt31. Note that even if the character string Mt31 is the symbol "〃", the positional relationship between the character strings Mt31 and Mt32 will be similar.

[0046] This can also be said to be displaying, in addition to the confirmation screen Ga, an image of a paper document Bk on which handwritten character strings Mt31 and Mt32 are written, and displaying the location of the character string Mt31 of "Same as above" and the location of the character string Mt32 pointed to by "Same as above" within the image of the paper document Bk. This allows user B, who checks the OCR results, to confirm the positions of character strings Mt31 and Mt32 in the paper document Bk, making it even easier to determine whether the OCR results are correct.

[0047] When user B selects the recognition result Nk1, the position of the entry field Ra in which the character string Mt11 is entered is highlighted in the image Ga1. Similarly, when user B selects the recognition result Nk2, the position of the entry field Ra in which the character string Mt21 is entered is highlighted in the image Ga1.

[0048] 9(a) and (b) are diagrams showing a second example of the confirmation screen Ga. Of these, the example shown in FIG. 9(b) is similar to the example shown in FIG. 8(b) except that the character string Mt31 has changed from "same as above" to "same as above." 9(a) shows a case where the position Pj31 of the entry field Ra in which the character string Mt31 "same as above" is entered and the position Pj32 of the entry field Ra in which the character string Mt32 pointed to by "same as above" (written as "source" in the figure) is entered are highlighted in the image Ga1. In this case, the character string Mt32 pointed to by the character string Mt31 "same as above" is located to the left of the character string Mt31.

[0049] <Detailed Description of the Operation of the Terminal Device 30> Next, the operation of the terminal device 30 will be described in detail. 10 to 14 are flowcharts showing the operation of the terminal device 30. In FIG. FIG. 10 is a diagram showing a procedure for creating a display list for displaying the recognition result list Ga2 on the confirmation screen of FIG. First, the transmitting / receiving unit 310 acquires the first record from the table T of character recognition data from the storage unit 230 of the management server 20 (step 101). In the example shown in Fig. 7, this is "Name, Test Taro, Name, 30, 50, 100, 30".

[0050] Next, the determination unit 330 determines whether the OCR result of the acquired record is a character or symbol such as "same as above," "same as above," or "〃" (step 102). Then, if the OCR result is a character or symbol such as "same as above," "same as above," or "〃" (Yes in step 102), the search unit 340 searches the paper document Bk for a character string indicated by the character or symbol, and the display control unit 350 updates the display list (step 103). The process of step 103 will be described in detail later. On the other hand, if the OCR result is not a character or symbol such as "same as above," "same as above," or "〃" (No in step 102), the display control unit 350 updates the display list based on this. Specifically, the handwritten character string Mk and the OCR result of this character string Mk are added to the display list (step 104). That is, a setting for displaying the recognition result Nk1 or the recognition result Nk2 in FIG. 8(b) is added.

[0051] After steps 103 and 104, the transmitting / receiving unit 310 determines whether or not there is a next record (step 105). As a result, if there is a next record (Yes in step 105), the transmitting / receiving unit 310 obtains the next record (step 106) and returns to step 102. On the other hand, if there is no next record (No in step 105), the series of processes ends.

[0052] 11 and 12 are flowcharts illustrating the process of step 103 in FIG. 10 in more detail. 11 is a diagram showing the process of searching for a character string indicated by "same as above" from within the paper document Bk by the search unit 340. The process of searching for a character string indicated by "" from within the paper document Bk is similar. First, the transmitting / receiving unit 310 acquires the first record from the table T of the character recognition data from the storage unit 230 of the management server 20 (step 201).

[0053] Then, the search unit 340 judges whether the coordinate x of the record acquired in step 201 is the same as the coordinate x of the record in which "same as above" is written (step 202). In other words, the search unit 340 judges whether the x coordinate of the entry field Ra acquired in step 201 is the same as the x coordinate of the entry field Ra in which "same as above" is written. As a result, if they are the same (Yes in step 202), the search unit 340 judges whether the y coordinate of the record acquired in step 201 is smaller than the y coordinate of the record in which "same as above" is written (step 203). In other words, the search unit 340 judges whether the entry field Ra acquired in step 201 is located above the entry field Ra in which "same as above" is written.

[0054] If it is smaller (Yes in step 202), that is, if it is located at the top, the search unit 340 judges whether the dictionary name of the record obtained in step 201 is the same as the dictionary name of the record in which "same as above" is entered (step 204). That is, here, it is judged whether the contents of the entry field Ra are the same depending on the dictionary used. Specifically, if the dictionary name is the same in both cases, for example, a name dictionary, it is considered that the same content, name, will be entered in both entry fields Ra. On the other hand, if the dictionary names are different, it is considered that different content, such as name and address, will be entered.

[0055] As a result, if they are the same (Yes in step 204), the search unit 340 judges whether the OCR result of the record acquired in step 201 is "same as above", "same as above", or "〃" (step 205). In other words, the search unit 340 judges whether the character string pointed to by "same as above" is "same as above", "same as above", or "〃". If the result is not "same as above," "same as above," or "same as above" (Yes in step 205), the record acquired in step 201 is added to the display candidate list (step 206).

[0056] Therefore, in steps 202 and 203, the search unit 340 first determines whether the entry field Ra of the record obtained in step 201 is directly above the entry field Ra of the record in which "same as above" is entered. Then, in step 204, if the same type of content is entered, the search unit 340 determines that the character string Mt in the entry field Ra obtained in step 201 is a candidate for the character string indicated by "same as above".

[0057] After step 206, the transmitting / receiving unit 310 judges whether or not there is a next record (step 207). Note that if the result in step 205 is "same as above," "same as above," or "〃" (No in step 205), and also if the results in steps 202, 203, and 204 are No, the transmitting / receiving unit 310 performs the process in step 207. As a result, if there is a next record (Yes in step 207 ), the transmitting / receiving unit 310 obtains the next record (step 208 ) and returns to step 202 . On the other hand, if there is no next record (No in step 207), the display control unit 350 determines whether or not there is one or more candidate records in the display candidate list (step 209).

[0058] As a result, if there is such a list (Yes in step 209), the display control unit 350 updates the display list in a predetermined manner (step 210), the details of which will be described later. On the other hand, if it does not exist (No in step 209), that is, if the entry field Ra in which the character string Mt pointed to by "same as above" is entered cannot be found, the display control unit 350 updates the settings of the display list based on this. Specifically, the handwritten character string Mk and the OCR result of this character string Mk are added to the display list (step 211). That is, a setting is added to display the recognition result Nk3 in Fig. 8(b) without the character string Mt32.

[0059] FIG. 12 is a diagram showing the process in which the search unit 340 searches the paper document Bk for the character string indicated by "same as above." In FIG. 12, step 301 and steps 304 to 311 are similar to step 201 and steps 204 to 211 in FIG. Therefore, steps 302 to 303 which are different from FIG. 11 will be mainly described here. In step 302, the search unit 340 determines whether the y coordinate of the record acquired in step 301 is the same as the y coordinate of the record in which "same as left" is entered (step 302). In other words, the search unit 340 determines whether the y coordinate of the entry field Ra acquired in step 301 is the same as the y coordinate of the entry field Ra in which "same as left" is entered. As a result, if they are the same (Yes in step 302), the search unit 340 judges whether the coordinate x of the record acquired in step 301 is smaller than the coordinate x of the record in which "Same as left" is written (step 303). In other words, the search unit 340 judges whether the entry field Ra acquired in step 301 is located to the left of the entry field Ra in which "Same as left" is written.

[0060] If it is smaller (Yes in step 302), that is, if it is located on the left side, the search unit 340 determines whether the dictionary name of the record obtained in step 301 is the same as the dictionary name of the record in which "same as left" is written (step 304).

[0061] Therefore, in steps 302 and 303, the search unit 340 first determines whether the entry field Ra of the record obtained in step 301 is to the left of the entry field Ra of the record in which "same as left" is entered. Then, in step 304, if the same type of content is entered, the search unit 340 determines that the character string Mt in the entry field Ra obtained in step 301 is a candidate for the character string indicated by "same as left".

[0062] As described in FIG. 11 and FIG. 12, the search unit 340 searches for a character string in a direction determined by a character or symbol such as "same as above," "same to the left," or "〃." That is, in the case of "same as above" or "〃," there is an entry field Ra in which a character string Mt is entered into the entry field Ra above the entry field Ra in which these characters are entered. Therefore, the search unit 340 performs a search in the upward direction. Also, in the case of "same to the left," there is an entry field Ra in which a character string Mt is entered into the entry field Ra above the entry field Ra in which these characters are entered. Therefore, the search unit 340 performs a search in the left direction. At this time, the search unit 340 uses the x coordinate and y coordinate of the entry field Ra in which the character string Mt is entered to determine the direction based on the position of the entry field Ra.

[0063] The search unit 340 also compares information on entry fields Ra in which characters or symbols such as "same as above," "same to the left," and "〃" are written with information on entry fields Ra that exist in these directions, and when the attributes of the contents written in the entry fields Ra are the same, it is determined to be a candidate for the character string indicated by the characters or symbols. In this case, the search unit 340 determines that the attributes of the contents are the same when the dictionaries used when performing character recognition are the same.

[0064] FIG. 13(a) is a flowchart explaining in more detail the process of step 210 in FIG. First, the display control unit 350 rearranges the records in the display candidate list in descending order of y coordinate (step 401). Next, the display control unit 350 sets the first record among the rearranged records as an item to be added to the display list (step 402). That is, if multiple records exist in the display candidate list, the one closest to the entry field Ra in which "Same as above" is entered is selected. Furthermore, the display control unit 350 updates the display list to reflect the values ​​in the record (step 403). Then, the display control unit 350 synthesizes images based on the display list in order to display the image showing the recognition result Nk3 shown in FIG. 8(b) (step 404). Moreover, the display control unit 350 adds the synthesized image to the recognition result list Ga2 (step 405).

[0065] FIG. 13(b) is a flowchart explaining in more detail the process of step 310 in FIG. Here, the display control unit 350 rearranges the records present in the display candidate list in descending order of x coordinate (step 501). Then, the subsequent steps 502 to 505 are the same as steps 402 to 405 in Fig. 13(a). Therefore, in this case as well, if multiple records exist in the display candidate list, the one closest to the entry field Ra in which "same as left" is entered is selected.

[0066] FIG. 14 is a diagram showing the operation of the terminal device 30 when a user selects a portion of a recognition result in the recognition result list Ga2 using an input device such as a mouse. First, the position information selected by the user in the recognition result list Ga2 is obtained (step 601). Then, the display control unit 350 determines whether or not the recognition result is displayed at this location (step 602). If the recognition result is displayed (Yes in step 602), the entry fields Ra in which characters or symbols such as "same as above," "same as above," and "" are entered in the image Ga1 are highlighted (step 603). Furthermore, the entry fields Ra they point to are highlighted (step 604). On the other hand, if the recognition result is not displayed (No in step 602), the series of processes ends.

[0067] 15(a) to (c) are diagrams showing the operation of the substitution section 370. In FIG. After user B checks the character recognition results on the confirmation screen Ga, the replacement unit 370 replaces characters and symbols such as "same as above," "same as above," and "〃" with character strings that these characters or symbols indicate. Of these, FIG. 15(a) shows the paper document Bk scanned in FIG. 3A 1A. This paper document Bk is similar to the paper document Bk shown in Fig. 5(a), and has address fields Ra1 and Ra2 for destination and sender information, respectively. In addition, the example shows a case where user A enters the destination address as a character string Mt1 in this entry field Ra1.

[0068] FIG. 15(b) shows the OCR result of the paper document Bk shown in FIG. 15(a). The OCR result shown in the figure indicates that for the destination address, a handwritten character string Mt1 written by user A has been character-recognized into a character string Mc1. Also, for the sender's address, a handwritten character string Mt2 written by user A has been character-recognized into a character string Mc3. However, the character string Mc3 has been replaced from the character string Mc2 by the replacement unit 370. That is, as shown in FIG. 5B, the character string Mt2 is first character-recognized as the character string Mc2, "same as above," but is replaced by the character string Mc3 pointed to by "same as above" by the replacement unit 370. In this case, the character string Mc2, "same as above," is replaced by the character string Mc3, "1-2-34, ◇◇-cho, △△-shi, XX-ken."

[0069] FIG. 15(c) shows an example of the checking and correction result. In this example, the confirmation and correction results are output in CSV format. The output data shows "Destination_Address, Sender_Address, 1-2-34 ◇◇cho, △△city, XX prefecture, 1-2-34 ◇◇cho, △△city, XX prefecture."

[0070] According to the example described above in detail, it is possible to provide an information processing device that does not increase the burden on the user who checks the results of character recognition, even if there are characters or symbols that mean that the same character string as a character string already entered will be entered.

[0071] In the above example, the case where characters and symbols such as "Same as above", "Same as left", and "〃" are written in Japanese has been described, but the same can be applied to other languages. For example, the same processing can be performed when character strings such as "Same as above" and "Same as left" are written in a paper document Bk written in English.

[0072] <Program Description> In addition, the processing performed by the terminal device 30 in this embodiment is realized by cooperation between software and hardware resources. That is, a CPU provided inside the terminal device 30 executes a program that realizes each function of the terminal device 30, and realizes each of these functions.

[0073] Therefore, the processing performed by image forming apparatus 10 can also be considered as a program for enabling a computer to realize an acquisition function for acquiring the results of character recognition of a handwritten document, and a replacement function for replacing a character or symbol with a character string indicated by the character or symbol when the recognized character string is a character or symbol indicating that the same character string as an already entered character string is to be inserted.

[0074] Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the claims that various modifications and improvements to the above embodiment are also included in the technical scope of the present invention. [Explanation of symbols]

[0075] 1...character recognition system, 10...image forming apparatus, 16...image reading unit, 20...management server, 30...terminal device, Bk...paper document, Ga...confirmation screen, Mt, Mc...character string, T...table

Claims

1. A processor is provided. The processor, Obtaining the results of character recognition for the handwritten character string; When the character string that has been recognized is a character or symbol that indicates that the same character string as the already entered character string is to be entered, the system searches for character strings in a direction determined by the character or symbol, compares the information on the box in which the character or symbol is written with the information on the box that exists in that direction, and if the attributes of the contents written within the box are the same, it is considered a candidate for the character string indicated by the character or symbol, and replaces the character or symbol with the character string obtained by the search. Information processing device.

2. 2. The information processing apparatus according to claim 1, wherein said processor determines that the content attributes are the same when the same dictionary is used for character recognition.

3. 2 . The information processing apparatus according to claim 1 , wherein, when there are a plurality of candidates, the processor selects the one that is closest in position to the character or symbol.

4. The information processing apparatus according to claim 1 , wherein the processor displays a confirmation screen for prompting a user to confirm the replaced character string.

5. The information processing device according to claim 4, characterized in that the processor displays, as the confirmation screen, the handwritten character or symbol and a character string indicated by the character or symbol, as well as the result of character recognition of the character string indicated by the character or symbol, side by side.

6. The processor further displays an image of a document having handwritten character strings written thereon in addition to the confirmation screen; 5. The information processing apparatus according to claim 4, further comprising: displaying, in the image of the document, the location of the character or symbol and the location of the character string that the character or symbol points to.

7. The information processing device according to claim 6, characterized in that, when a user selects something on the confirmation screen, the processor indicates the location of the character or symbol and the location of the character string pointed to by the character or symbol in the image of the document.

8. A reading device that reads a document containing handwritten character strings as an image; a character recognition device that recognizes characters in a handwritten character string; a display device that displays a confirmation screen for allowing a user to confirm a result of character recognition by the character recognition device; an image creation device for creating an image of the confirmation screen; Equipped with the image creation device comprises a processor; The processor, obtaining a character recognition result from the character recognition device; When the character string that has been recognized is a character or symbol that indicates that the same character string as the already entered character string is to be entered, the system searches for character strings in a direction determined by the character or symbol, compares the information on the box in which the character or symbol is written with the information on the box that exists in that direction, and if the attributes of the contents written within the box are the same, it is considered a candidate for the character string indicated by the character or symbol, and replaces the character or symbol with the character string obtained by the search. Character recognition system.

9. On the computer, an acquisition function for acquiring the results of character recognition for the handwritten document; a replacement function, when the recognized character string is a character or symbol that indicates that the same character string as the already entered character string is to be inserted, searching for character strings in a direction determined according to the character or symbol, comparing information on the box in which the character or symbol is written with information on the box that exists in the direction, and, if the attributes of the contents written within the box are the same, making it a candidate for the character string indicated by the character or symbol, and replacing the character or symbol with the character string obtained by the search; A program to achieve this.

Citation Information

Patent Citations

  • Character reader

    JP1987075885A

  • Character processor

    JP1997134357A

  • Retrieving device, recording medium and program

    JP2002063197A

  • Prescription reception device

    JP2007140632A

  • OCR system

    JP2008225895A