Information processing device and program

Abstract

To provide an information processing apparatus configured to efficiently check and correct results of character recognition.SOLUTION: A terminal device 20 includes: an acquisition unit 222 which acquires recognition information comprising character information indicating multiple characters CR read from a character image CI by character recognition processing for recognizing a character included in an image and accuracy information indicating accuracy of recognition results on each of the characters CR; a display control unit 224 which causes a display device 30 to display the characters CR indicated by the character information; and a restriction unit 226 which restricts operation on a protection character that satisfies conditions on the accuracy of the recognition results, out of the characters CR, on the basis of the accuracy indicated by the accuracy information.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] In information processing devices such as character recognition devices, optical character recognition (OCR) technology is generally adopted. OCR technology is a technology that captures characters written on a medium such as a form as an image by optical means such as a camera and an image scanner, and converts the characters in the captured image into character information (for example, character code) that can be used by a computer or the like. Since the accuracy of character recognition by OCR is not 100%, an operator visually checks the image on which character recognition has been performed and the result of character recognition, and corrects erroneous characters. Patent Document 1 discloses a character recognition method that classifies the accuracy of the result of character recognition into a number of categories, and one of the categories indicates the accuracy that the result of character recognition is absolutely correct. In this type of character recognition method, the operator's confirmation work can be omitted for characters that belong to a category that indicates the accuracy that the result of character recognition is absolutely correct. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-156938 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the confirmation process, an operator may make a mistake by mistakenly changing a character that is correct as a result of character recognition. For this reason, there is a demand for an information processing device that can efficiently confirm and correct the results of character recognition. [Means for solving the problem]

[0005] An information processing device according to a preferred embodiment of the present invention includes an acquisition unit that acquires recognition information including character information indicating a plurality of characters read from an image to be read by a character recognition process that recognizes characters contained in the image and accuracy information indicating the accuracy of the recognition result for each of the plurality of characters, a display control unit that causes a display device to display the plurality of characters indicated by the character information, and a restriction unit that restricts operations on protected characters among the plurality of characters for which a condition related to the accuracy of the recognition result is satisfied based on the accuracy indicated by the accuracy information.

[0006] An information processing device according to another preferred aspect of the present invention includes an image information acquisition unit that acquires image information indicating an image, a character recognition unit that recognizes a plurality of characters included in the image indicated by the image information and calculates an accuracy of a recognition result for each of the plurality of characters, a display control unit that causes the plurality of characters recognized by the character recognition unit to be displayed on a display device, and a restriction unit that restricts operations on protected characters among the plurality of characters for which a condition related to the accuracy of the recognition result is satisfied based on the accuracy calculated by the character recognition unit.

[0007] A program according to a preferred embodiment of the present invention causes a processor to function as: an acquisition unit that acquires recognition information including character information indicating a plurality of characters read from an image to be read by a character recognition process that recognizes characters contained in the image and accuracy information indicating the accuracy of the recognition result for each of the plurality of characters; a display control unit that displays the plurality of characters indicated by the character information on a display device; and a restriction unit that restricts operations on protected characters among the plurality of characters for which a condition related to the accuracy of the recognition result is satisfied, based on the accuracy indicated by the accuracy information.

[0008] A program according to another preferred aspect of the present invention causes a processor to function as an image information acquisition unit that acquires image information indicating an image, a character recognition unit that recognizes a plurality of characters contained in the image indicated by the image information and calculates an accuracy of a recognition result for each of the plurality of characters, a display control unit that displays the plurality of characters recognized by the character recognition unit on a display device, and a restriction unit that restricts operations on protected characters among the plurality of characters for which a condition related to the accuracy of the recognition result is satisfied based on the accuracy calculated by the character recognition unit. Effect of the Invention

[0009] According to the present invention, it is possible to provide an information processing device that is capable of efficiently checking and correcting the results of character recognition. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram for explaining an overview of a character recognition system according to an embodiment. [Diagram 2] FIG. 2 is an explanatory diagram for explaining an example of an input image. [Diagram 3] 13 is an explanatory diagram illustrating an example of an outline of a confirmation screen for confirming a result of the character recognition process. FIG. [Figure 4] 2 is a functional block diagram showing an example of the configuration of the character recognition device shown in FIG. 1. [Diagram 5] 5 is a diagram illustrating an example of a hardware configuration of the character recognition device illustrated in FIG. 4. [Figure 6] 2 is a functional block diagram showing an example of the configuration of a terminal device shown in FIG. 1. [Figure 7] FIG. 7 is a diagram illustrating an example of a hardware configuration of the terminal device illustrated in FIG. 6. [Figure 8] FIG. 7 is an explanatory diagram showing an example of an editing table shown in FIG. 6; [Figure 9] 7 is an explanatory diagram showing another example of the editing table shown in FIG. 6; [Figure 10] FIG. 2 is an explanatory diagram for explaining an overview of the operation of the terminal device shown in FIG. [Figure 11] 2 is a sequence chart showing an example of an operation of the character recognition system shown in FIG. 1. [Figure 12] FIG. 11 is an explanatory diagram for explaining an overview of the operation of a terminal device according to a first modified example. [Figure 13] FIG. 11 is an explanatory diagram for explaining an overview of the operation of a terminal device according to a second modified example. [Figure 14] FIG. 13 is a diagram showing an example of a confirmation screen according to a third modified example. [Figure 15] FIG. 13 is a diagram showing an example of a confirmation screen according to a fourth modified example. [Figure 16] 13 is a flowchart showing an example of the operation of a terminal device according to a fifth modified example. [Figure 17] FIG. 13 is an explanatory diagram for explaining an overview of a character recognition system according to a sixth modified example. [Figure 18] FIG. 13 is a functional block diagram showing an example of the configuration of a character recognition device according to a seventh modified example. [Figure 19] FIG. 19 is a diagram illustrating an example of a hardware configuration of the character recognition device illustrated in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. In each drawing, the dimensions and scale of each part are appropriately different from the actual ones. In addition, the embodiments described below are preferred examples of the present invention, and therefore various technically preferable limitations are attached, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0012] [1. Embodiment] First, an example of an overview of a character recognition system 1 according to an embodiment will be described with reference to FIG.

[0013] FIG. 1 is an explanatory diagram for explaining an overview of a character recognition system 1 according to an embodiment.

[0014] The character recognition system 1 shown in FIG. 1 includes a character recognition device 10, a terminal device 20, and a display device 30 provided in correspondence with the terminal device 20. The terminal device 20 is an example of an "information processing device." The character recognition device 10 and the terminal device 20 are connected to each other so as to be able to communicate with each other, for example, via a network NW. The network NW may include the Internet and a local area network. For example, the network NW includes one or both of a wired network and a wireless network. In addition, the connection between the network NW and the character recognition device 10 may be, for example, a connection that allows multiple elements to communicate with each other, and may be a connection using either a wired or wireless connection, or may be a connection using both a wired and wireless connection.

[0015] Any information processing device connectable to a network NW can be adopted as the character recognition device 10. The character recognition device 10 executes a character recognition process that recognizes characters included in an image, for example, by using an optical character recognition (OCR) technique. An example of an input image IIMG that is a target of the character recognition process is described later in FIG. 2. The configuration of the character recognition device 10 is described later in FIG. 4 and FIG. 5.

[0016] As the terminal device 20, any information processing device connectable to the network NW can be adopted. Specifically, the terminal device 20 may be, for example, a stationary information device such as a personal computer, or may be a portable information terminal such as a notebook personal computer or a tablet terminal. The terminal device 20 is, for example, communicably connected to the display device 30, and can display images and the like on a display unit 310 provided on the display device 30. The connection between the terminal device 20 and the display device 30 may be a wired or wireless connection, or may be both a wired and wireless connection. The display device 30 may be included in the terminal device 20. The terminal device 20 causes the display device 30 to display, for example, a confirmation screen CHS for a user (for example, an operator) to confirm the result of the character recognition process by the character recognition device 10. An example of the confirmation screen CHS will be described later in FIG. 3. The configuration of the terminal device 20 will be described later in FIG. 6 and FIG. 7.

[0017] 2 is an explanatory diagram for explaining an example of an input image IIMG. In this embodiment, it is assumed that the input image IIMG to be subjected to character recognition processing is an image of an application form, but the input image IIMG is not limited to an image of an application form. For example, the input image IIMG may be an image of a prescription, a slip, or the like.

[0018] The input image IIMG is, for example, an image of an application form provided with a plurality of entry fields EC corresponding to a plurality of items. In the example shown in Fig. 2, the plurality of items to be entered in the application form are application number, name (kana), name, postal code, address, e-mail, place of employment, place of employment phone number, and remarks. The input image IIMG is converted into input image information, which is a digital signal, by optical means such as a camera or an image scanner. Then, the character recognition device 10 recognizes the characters included in the input image IIMG using the input image information indicating the input image IIMG.

[0019] The character image portion (e.g., each of the multiple entry fields EC) of the input image IIMG that is the target of the character recognition process by the character recognition device 10 may be set manually by the user or automatically by the character recognition device 10. For example, the character recognition device 10 may use image information of an application form in which no handwritten characters or the like are written in the entry fields EC (a blank application form) as reference image information, specify a range to be the target of the character recognition process in advance on the reference image information, and set the specified range as the character image portion. Alternatively, the character image portion of the input image IIMG may be set by the user correcting the range that is automatically set by the character recognition device 10. In this way, when certain input image information is input, the character recognition device 10 acquires, from the form dictionary data stored in the storage unit, a form (an application form in this embodiment) that has a high degree of agreement with the application form of the input image information based on predetermined items and ruled line information. Then, the character recognition device 10 recognizes, as the character image portion, the range of XY coordinates of the input image information that is defined by the same XY coordinates as the XY coordinates of the character image portion that is preset in the acquired form.

[0020] In this embodiment, it is assumed that the multiple parts of the input image IIMG corresponding to the multiple entry fields EC are multiple character image parts that are the target of character recognition processing by the character recognition device 10. In the following, the character image parts of the input image IIMG that are the target of character recognition processing by the character recognition device 10 are also referred to as character images CI. In this embodiment, it is assumed that each of the multiple parts of the input image IIMG corresponding to the multiple entry fields EC is a character image CI. For example, the part surrounded by a dashed line in FIG. 2 indicates a character image CI that corresponds to the remarks entry field EC. Note that the input image IIMG and the character image CI are examples of the "image to be read". In the following, unless otherwise specified, it is assumed that the character image CI corresponds to the "image to be read".

[0021] Next, the confirmation screen CHS for confirming the result of the character recognition process will be described with reference to FIG.

[0022] Fig. 3 is an explanatory diagram for explaining an example of the outline of a confirmation screen CHS for confirming the result of the character recognition process. Fig. 3 shows an example of a confirmation screen CHS for confirming a plurality of characters CR read from a character image CI corresponding to a remarks entry field EC among a plurality of character images CI included in an input image IIMG. Note that the plurality of characters CR are an example of "a plurality of characters read from an image to be read". Hereinafter, the result of the character recognition process is also referred to as a recognition result.

[0023] The confirmation screen CHS is displayed, for example, on the display unit 310 of the display device 30. For example, the confirmation screen CHS displays a character string of "Notes" indicating the item to be confirmed, a character image area IW, an editing area EDW, and an end button BT. The end button BT is, for example, an image for a GUI (Graphical User Interface) for ending the display of the confirmation screen CHS.

[0024] A character image CI is displayed in the character image area IW. In the example shown in Fig. 3, a part of the character image CI (the character string "The mobile phone number is shown below" shown in Fig. 2) is not displayed in the character image area IW. Therefore, a horizontal scroll bar HS for moving the character image CI horizontally (horizontally in the figure) in the character image area IW and a vertical scroll bar VSi for moving the character image CI vertically (vertically in the figure) in the character image area IW are displayed on the confirmation screen CHS. Note that the character recognition device 10 or the terminal device 20 may reduce and display the character image CI so that the entire character image CI fits in the character image area IW.

[0025] Here, the character image CI is divided into, for example, a plurality of character parts (parts surrounded by dotted lines in the character image area IW shown in FIG. 3). For example, each of the plurality of character parts is a part of the character image CI that is separated as one character by character recognition processing. FIG. 3 illustrates a case in which each of the plurality of characters included in the character image CI is correctly separated as one character by character recognition processing. In addition, in FIG. 3, for ease of explanation, the character parts are shown by dotted lines in the character image area IW, but the dotted lines indicating the character parts may or may not be displayed in the character image area IW. In addition, for example, in a case in which the user compares an original such as an application form on which the character image CI is based with the plurality of characters CR, the character image area IW may not be displayed on the confirmation screen CHS.

[0026] In the editing area EDW, a plurality of characters CR read from the character image CI are displayed. In the example shown in FIG. 3, a portion of the plurality of characters CR (the character string "Your mobile phone number is shown below" shown in FIG. 2) is not displayed in the editing area EDW. For this reason, a vertical scroll bar VSe for moving the plurality of characters CR vertically (the vertical direction in the figure) in the editing area EDW is displayed on the confirmation screen CHS. In addition, in the editing area EDW, a cursor CS indicating the character input position is displayed. The display position of the cursor CS in the editing area EDW is moved, for example, by a user operation. Note that the character recognition device 10 or the terminal device 20 may reduce and display the plurality of characters CR read from the character image CI so that the entirety of the plurality of characters CR fits in the editing area EDW.

[0027] Here, in the editing area EDW shown in FIG. 3, the characters CR surrounded by a dashed line are characters CR whose accuracy of the result (recognition result) of the character recognition process by the character recognition device 10 is higher than a first accuracy determined in advance. The dashed line indicating the range of characters CR whose accuracy of the recognition result is higher than the first accuracy may or may not be displayed in the editing area EDW. The accuracy of the recognition result indicates the reliability of the recognition result. A character CR whose recognition result has a high accuracy has a higher probability of being correctly recognized than a character CR whose recognition result has a low accuracy. The first accuracy may be determined based on a trial result of the character recognition process by, for example, an administrator of the character recognition device 10 and a user of the terminal device 20. Alternatively, the first accuracy may be automatically set by the character recognition device 10 based on the past performance of the character recognition process.

[0028] The accuracy of the recognition result being higher than the first accuracy is an example of a "condition regarding the accuracy of the recognition result." For example, the terminal device 20 identifies a character CR having an accuracy of the recognition result higher than the first accuracy among a plurality of characters CR as a protected character for which the condition regarding the accuracy of the recognition result is satisfied, and restricts the user from performing operations such as correction, modification, and deletion on the identified protected character. Note that, in this embodiment, it is assumed that the condition regarding the accuracy of the recognition result is that the accuracy of the recognition result is higher than the first accuracy, but the condition regarding the accuracy of the recognition result is not limited to that the accuracy of the recognition result is higher than the first accuracy. Other examples of the condition regarding the accuracy of the recognition result will be described in the first modified example and the like described later.

[0029] In FIG. 3, it is assumed that the accuracy of the recognition result of a total of three characters, the one character "1" which is the fourth character counting from the end and two characters "7" in a character string ("090-0ab7-1171") made up of a plurality of characters included in the character image CI, is equal to or lower than the first accuracy. In the example shown in FIG. 3, the character recognition device 10 erroneously recognizes each of the two characters "7" in the character string made up of a plurality of characters included in the character image CI as the character "1" CR. Therefore, in the example shown in FIG. 3, the character string "090-0ab1-1111" is displayed in the editing area EDW as the plurality of characters CR. Of the plurality of characters CR ("090-0ab1-1111") displayed in the editing area EDW, the user's operation is restricted for the ten characters CR surrounded by the dashed line in FIG. 3, and the user's operation is not restricted for the three characters "1" CR not surrounded by the dashed line in FIG. 3.

[0030] For example, in the confirmation screen CHS shown at the top of FIG. 3, the cursor CS is displayed at the position of the character CR "1", which is the third character counting from the end, among a plurality of characters CR ("090-0ab1-1111"). In this case, the character CR selected by the cursor CS is a protected character for which the operation by the user is restricted, so that even if the user mistakenly performs a delete operation to delete the character CR, the character CR is not deleted. That is, when the character CR selected by the cursor CS is a protected character, the character CR selected by the cursor CS (protected character) does not reflect the operation of deleting, correcting, changing, etc., performed by the user. Note that the protected character may not be selectable by the cursor CS. In this case, when the user tries to select a protected character by operating the cursor CS, for example, the cursor CS skips the protected character and moves to the next character CR. This makes it easier for the user to recognize that the character CR is a protected character, and can reliably prevent erroneous operations on the protected character.

[0031] Also, for example, in the confirmation screen CHS shown at the bottom in Fig. 3, the display position of the cursor CS is moved by a user operation to the position of the character CR "1", which is the second character counting from the end, among the multiple characters CR. In this case, the character CR selected by the cursor CS is not a protected character whose correction by the user is restricted, so that when the user performs a correction operation to correct the erroneous character CR to the correct character, it is corrected to the correct character.

[0032] In this manner, in the present embodiment, correction of a character CR (protected character) having a recognition result accuracy higher than the first accuracy among a plurality of characters CR is restricted. This allows the user to efficiently check and correct the result of the character recognition process.

[0033] For example, in a form in which the user's operation on a character CR whose recognition result has a higher accuracy than the first accuracy is not restricted (hereinafter, also referred to as the first comparative example), an operation mistake may occur in which the user erroneously changes a character CR whose character recognition result is correct. Specifically, in a correction operation to correct the character CR "1" which is the second character counting from the end of a plurality of characters CR to the character "7", a case is assumed in which the user erroneously moves the cursor CS to the position of the character CR "1" which is the third character counting from the end. In this case, if the user performs a correction operation to correct the character CR "1" to the character "7" without noticing that the character CR selected by the cursor CS is not the character CR to be corrected, the character CR "1" whose character recognition result is correct is changed to the character "7" (incorrect character).

[0034] In contrast, in this embodiment, as described above, the character "1" CR (character CR selected by cursor CS), which is the third character counting from the end of the multiple characters CR, is a protected character. Therefore, even if a user performs a correction operation to correct the character "1" CR to the character "7" without noticing that the character CR selected by cursor CS is not the character CR to be corrected, the correction operation is not reflected in the character CR selected by cursor CS. Therefore, in this embodiment, it is possible to prevent a character CR that is correct as a result of character recognition from being changed to an incorrect character due to an erroneous operation by the user. As a result, in this embodiment, it is possible to improve the work efficiency of checking and correcting the result of character recognition processing compared to the first comparison example.

[0035] Next, functions of the character recognition device 10 and the terminal device 20 will be described with reference to FIGS.

[0036] FIG. 4 is a functional block diagram showing an example of the configuration of the character recognition device 10 shown in FIG.

[0037] The character recognition device 10 includes a control unit 120 that controls each unit of the character recognition device 10, a storage unit 140 that stores various information such as a control program PG1, and a communication unit 160 for executing communication with an external device such as a terminal device 20. In this embodiment, the control program PG1 includes, for example, an application program for the character recognition device 10 to execute a character recognition process. However, the control program PG1 may include, for example, an operating system program for the control unit 120 to control each unit of the character recognition device 10.

[0038] The control unit 120 has, for example, an image information acquisition unit 122 and a character recognition unit 124. The image information acquisition unit 122 acquires, from the terminal device 20 via the communication unit 160, input image information indicating an input image IIMG to be subjected to character recognition processing, for example.

[0039] The character recognition unit 124 recognizes characters included in the input image IIMG indicated by the input image information by executing character recognition processing using the input image information acquired by the image information acquisition unit 122. For example, the character recognition unit 124 recognizes characters included in a predetermined character image CI in the input image IIMG. In this way, a plurality of characters CR are read from each of the plurality of character images CI. That is, the character CR read from the character image CI is a character recognized by the character recognition processing as a character included in the character image CI. In addition, the character recognition unit 124 calculates an accuracy (i.e., an accuracy of the recognition result) indicating the likelihood of the plurality of characters CR read from each of the plurality of character images CI.

[0040] Furthermore, the character recognition unit 124 transmits recognition information on the result of the character recognition process for each of the multiple character images CI and character image information indicating each of the multiple character images CI to the terminal device 20 via the communication unit 160. For example, the recognition information includes character information indicating multiple characters CR read from each of the multiple character images CI and accuracy information indicating the accuracy of the recognition result for each of the multiple characters CR. Furthermore, the character image information may include, for example, character portion information indicating a character portion of the character image CI that is separated as one character by the character recognition process. The character portion information may indicate the range of the character portion in the character image CI using coordinates with a predetermined position in the character image CI as the origin.

[0041] The character recognition unit 124 may recognize characters included in the character image CI using a learning model that has learned the relationship between an image including characters and the characters included in the image. The learning model is, for example, a multi-layer neural network. That is, the character recognition unit 124 may use this learning model to perform character recognition processing by AI (Artificial Intelligence)-OCR.

[0042] Next, the hardware configuration of the character recognition device 10 will be described with reference to FIG.

[0043] FIG. 5 is a diagram showing an example of a hardware configuration of the character recognition device 10 shown in FIG.

[0044] The character recognition device 10 includes a processor 12 that controls each unit of the character recognition device 10 , a memory 14 that stores various information, and a communication device 16 .

[0045] The memory 14 includes, for example, one or both of a volatile memory such as a random access memory (RAM) that functions as a working area for the processor 12 and a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) that stores various information such as the control program PG1, and functions as a storage unit 140. The memory 14 may be detachable from the character recognition device 10. Specifically, the memory 14 may be a storage medium such as a memory card that is detachable from the character recognition device 10. The memory 14 may also be, for example, a storage device (e.g., online storage) communicatively connected to the character recognition device 10 via a network NW or the like.

[0046] The processor 12 includes, for example, one or more CPUs (Central Processing Units). The processor 12 executes, for example, a control program PG1 stored in the memory 14 and operates in accordance with the control program PG1, thereby functioning as the control unit 120. For example, the processor 12 that operates in accordance with an application program (program related to character recognition) included in the control program PG1 functions as the control unit 120 including an image information acquisition unit 122 and a character recognition unit 124. The control program PG1 may be transmitted from another device via the network NW.

[0047] Furthermore, for example, when the processor 12 is configured to include a plurality of CPUs, some or all of the functions of the control unit 120 may be realized by the plurality of CPUs operating in cooperation with each other in accordance with a program such as the control program PG1. Furthermore, the processor 12 may be configured to include hardware such as a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) in addition to one or more CPUs, or instead of some or all of the one or more CPUs. In this case, some or all of the control unit 120 realized by the processor 12 may be realized by hardware such as a DSP.

[0048] The communication device 16 is hardware for communicating with an external device that exists outside the character recognition device 10 via one or both of a wired network and a wireless network, and functions as a communication unit 160.

[0049] The configuration of character recognition device 10 is not limited to the examples shown in Fig. 4 and Fig. 5. For example, character recognition device 10 may have an operation device (similar to operation device 28 of terminal device 20 shown in Fig. 7 described later) that functions as an operation unit for accepting operations by an administrator of character recognition device 10 or the like.

[0050] Next, the configuration of the terminal device 20 will be described with reference to FIGS.

[0051] FIG. 6 is a functional block diagram showing an example of the configuration of the terminal device 20 shown in FIG.

[0052] The terminal device 20 includes a control unit 220 that controls each unit of the terminal device 20, a storage unit 240 that stores various information, a communication unit 260 for executing communication with an external device such as the character recognition device 10, and an operation unit 280 for accepting an operation by a user of the terminal device 20. The storage unit 240 stores, for example, a control program PG2 and an editing table TBL. In this embodiment, the control program PG2 includes, for example, an application program for the terminal device 20 to execute a correction support process that supports correction of the result of the character recognition process. However, the control program PG2 may include, for example, an operating system program for the control unit 120 to control each unit of the terminal device 20. The control program PG2 is an example of a "program." For example, the recognition information (character information and accuracy information) transmitted from the character recognition device 10 is stored in the editing table TBL. The editing table TBL will be described later with reference to FIG. 8 and FIG. 9.

[0053] The control unit 220 includes, for example, an acquisition unit 222, a display control unit 224, and a restriction unit 226.

[0054] The acquisition unit 222 acquires recognition information related to the result of the character recognition process on the character image CI and character image information indicating the character image CI from the character recognition device 10 via the communication unit 260. That is, the acquisition unit 222 acquires recognition information including character information indicating the multiple characters CR read from the character image CI by the character recognition process and accuracy information indicating the accuracy of the recognition result for each of the multiple characters CR. Then, the acquisition unit 222 stores the recognition information (character information and accuracy information) acquired from the character recognition device 10 in the editing table TBL.

[0055] The acquiring unit 222 may acquire input image information indicating the input image IIMG from an optical device such as a camera or an image scanner via the communication unit 260. Then, the acquiring unit 222 may transmit the input image information indicating the input image IIMG to the character recognition device 10 via the communication unit 260. One or both of the function of acquiring the input image information and the function of transmitting the input image information may be realized by a functional block separate from the acquiring unit 222.

[0056] The display control unit 224, for example, causes the display device 30 to display the confirmation screen CHS based on the character information and the character image information. That is, the display control unit 224 causes the display device 30 to display a plurality of characters CR read from the character image CI by character recognition processing based on the character information.

[0057] The restriction unit 226 restricts an operation on a protected character that satisfies a condition related to the accuracy of the recognition result among the multiple characters CR based on the accuracy indicated by the accuracy information stored in the editing table TBL. The accuracy information stored in the editing table TBL is, for example, the accuracy information included in the recognition information acquired by the acquisition unit 222.

[0058] Here, restricting an operation on a protected character may mean, for example, preventing all or some of a plurality of operations on the protected character from being reflected. Alternatively, restricting an operation on a protected character may mean disabling all or some of a plurality of operations on the protected character. The plurality of operations on the protected character may be, for example, an operation to delete the protected character, an operation to rewrite the protected character with another character, and the like. That is, restricting an operation on a protected character may mean, for example, prohibiting modification of the protected character. Alternatively, restricting an operation on a protected character may mean, for example, prohibiting modification of the protected character until a predetermined operation (for example, an operation to release the protection) is performed by the user. Furthermore, modification of the protected character may be, for example, deleting the protected character, rewriting the protected character with another character, and the like.

[0059] In this embodiment, as described above, it is assumed that the condition regarding the accuracy of the recognition result is that the accuracy of the recognition result is higher than the first accuracy, and therefore the restricting unit 226 specifies, among the multiple characters CR, a character CR whose accuracy of the recognition result is higher than the first accuracy as a protected character. That is, in this embodiment, the restricting unit 226 restricts the user's operation on a character CR whose accuracy of the recognition result is higher than the first accuracy (protected character) among the multiple characters CR.

[0060] Next, the hardware configuration of the terminal device 20 will be described with reference to FIG.

[0061] FIG. 7 is a diagram illustrating an example of a hardware configuration of the terminal device 20 illustrated in FIG.

[0062] The terminal device 20 includes a processor 22 that controls each unit of the terminal device 20, a memory 24 that stores various information, a communication device 26, and an operation device .

[0063] The memory 24 includes, for example, one or both of a volatile memory such as a RAM that functions as a working area for the processor 22 and a non-volatile memory such as an EEPROM that stores various information such as the control program PG2, and functions as a storage unit 240. Note that the memory 24 may be detachable from the terminal device 20, similar to the memory 14 of the character recognition device 10 described in Fig. 5, or may be a storage device (for example, online storage) communicatively connected to the terminal device 20 via a network NW or the like.

[0064] The processor 22 is configured similarly to the processor 12 of the character recognition device 10 described in FIG. 5. For example, the processor 22 is configured to include one or more CPUs. The processor 22 executes the control program PG2 stored in the memory 24 and operates in accordance with the control program PG2, thereby functioning as the control unit 220. For example, the processor 22 operating in accordance with an application program (a program relating to support for correcting the result of the character recognition process) included in the control program PG2 functions as the control unit 220 including the acquisition unit 222, the display control unit 224, and the restriction unit 226. The control program PG2 may be transmitted from another device via the network NW.

[0065] Furthermore, for example, when the processor 22 is configured to include multiple CPUs, some or all of the functions of the control unit 220 may be realized by these multiple CPUs operating in cooperation with each other in accordance with a program such as the control program PG2. Furthermore, the processor 22 may be configured to include hardware such as a GPU, a DSP, or an FPGA in addition to one or more CPUs, or in place of some or all of the one or more CPUs. In this case, some or all of the control unit 220 realized by the processor 22 may be realized by hardware such as a DSP.

[0066] The communication device 26 is hardware for communicating with an external device located outside the terminal device 20 via one or both of a wired network and a wireless network, and functions as a communication unit 260.

[0067] The operation device 28 is hardware for accepting operations by a user of the terminal device 20, and functions as an operation unit 280. For example, the operation device 28 may be configured from one or more devices including some or all of an operation button, a touch panel, a keyboard, and a mouse.

[0068] The configuration of the terminal device 20 is not limited to the examples shown in Fig. 6 and Fig. 7. For example, the terminal device 20 may have the display device 30 shown in Fig. 1. In this case, the terminal device 20 has a display unit 310 capable of displaying various images such as the confirmation screen CHS. The terminal device 20 may also have an optical device such as a camera and an image scanner for converting the input image IIMG into input image information that is a digital signal.

[0069] Next, the editing table TBL will be described with reference to FIG.

[0070] Fig. 8 is an explanatory diagram showing an example of the editing table TBL shown in Fig. 6. The editing table TBL stores, for example, recognition information (character information and accuracy information) transmitted from the character recognition device 10. In the example shown in Fig. 8, the editing table TBL stores recognition information and the like related to the result of the character recognition process for the character image CI corresponding to the remarks entry field EC shown in Fig. 2.

[0071] The editing table TBL stores, for example, text information including recognition information regarding the results of character recognition processing on the character image CI, and image information for identifying character image information indicating the character image CI (for example, information indicating the file name of the character image information).

[0072] The text information includes, for example, information about characters associated with each of the multiple characters CR, information about accuracy, and information about protection. For example, the number indicated by "No." in the editing table TBL corresponds to the row number in the character string made up of the multiple characters CR. Also, the number indicated by "character position" in the editing table TBL corresponds to the column number in the character string made up of the multiple characters CR. Note that the shaded areas in FIG. 8 indicate that there is no corresponding character in the character string made up of the multiple characters CR.

[0073] Also, for example, for one character CR among the multiple characters CR, the character-related information is character information indicating the one character CR, and the accuracy-related information is accuracy information indicating the accuracy of the recognition result of the one character CR. Also, the protection-related information for one character CR among the multiple characters CR is information indicating whether the one character CR is a protected character, and is set by the restriction unit 226. In the editing table TBL shown in Fig. 8, the code T indicates that the character CR is a protected character, and the code F indicates that the character CR is not a protected character.

[0074] Here, in FIG. 8, a case is assumed where the first accuracy is 90. In this case, for example, the limiting unit 226 sets, as a protected character, a character CR of which the accuracy of the recognition result is higher than 90 among the multiple characters CR. Specifically, the limiting unit 226 sets information on protection for a character CR of which the accuracy of the recognition result is higher than 90 among the multiple characters CR, as information indicating that the character is a protected character (symbol T in FIG. 8). In other words, among the multiple characters CR, a character CR of which the accuracy of the recognition result is 90 or less is not set as a protected character. In the example shown in FIG. 8, the character CR in the eighth column of the second row, the character CR in the tenth column of the second row, and the character CR in the twelfth column of the second row in the character string made up of the multiple characters CR are not protected characters.

[0075] Furthermore, in the editing table TBL, the information on the characters may reflect corrections made by the user. For example, the character CR in the eighth column of the second row and the character CR in the twelfth column of the second row of a character string made up of a plurality of characters CR are each correctly the character "7", but are not correctly recognized in the character recognition process and are recognized as the character CR of "1". Therefore, the user corrects the character CR in the eighth column of the second row and the character CR in the twelfth column of the second row of a character string made up of a plurality of characters CR to the correct character "7" in the confirmation screen CHS. In this case, in the editing table TBL, the information on the characters for the character CR in the eighth column of the second row and the character CR in the twelfth column of the second row of a character string made up of a plurality of characters CR may be updated to character information indicating the character "7".

[0076] The edit table TBL may store character information indicating characters corrected by the user (for example, information on edited characters shown in FIG. 9 described later) in an area separate from the area in which information on characters is stored. Hereinafter, characters corrected by the user may also be referred to as characters CR. The restriction unit 226 may set the corrected characters CR as protected characters.

[0077] Next, another example of the editing table TBL will be described with reference to FIG.

[0078] Fig. 9 is an explanatory diagram showing another example of the editing table TBL shown in Fig. 6. The editing table TBL shown in Fig. 9 is similar to the editing table TBL shown in Fig. 8, except that information about the recognition results other than the first candidate by the character recognition process is stored, and that characters corrected by the user are stored separately from the recognition results by the character recognition process.

[0079] For example, when a character recognition process identifies a plurality of candidate characters as candidates for a character included in a character image CI, the character with the highest accuracy among the plurality of candidate characters is the first candidate character. In the example shown in Fig. 9, the recognition results for each of the characters in the eighth column of the second line, the tenth column of the second line, and the twelfth column of the second line of a character string made up of a plurality of characters CR include a second candidate character that is a character other than the first candidate. The second candidate character may be used for predictive conversion or the like when a user corrects the character CR.

[0080] In the editing table TBL, the information on the edited character is, for example, character information indicating the character CR displayed in the editing area EDW of the confirmation screen CHS. If the character CR is not modified by the user, the information on the character and the information on the edited character are the same character information. Also, for example, among the information on the edited character, the information corresponding to the character CR modified by the user is character information reflecting the modification by the user. For example, in the editing table TBL shown in FIG. 9, the information on the edited character for each of the character CR in the eighth column of the second row and the character CR in the twelfth column of the second row of the character string made up of multiple characters CR has been updated to character information indicating the character "7".

[0081] The configuration of the editing table TBL is not limited to the examples shown in Figs. 8 and 9. For example, information related to protection may be omitted from the editing table TBL. Even in this case, the restriction unit 226 can recognize whether or not one character CR is a protected character by referring to the accuracy of the recognition result of one character CR among the multiple characters CR. That is, it is only necessary that the editing table TBL stores multiple characters CR and the accuracy of the recognition results of the multiple characters CR in association with each other.

[0082] Next, an overview of the operation of the terminal device 20 will be described with reference to FIG.

[0083] Fig. 10 is an explanatory diagram for explaining an overview of the operation of the terminal device 20 shown in Fig. 1. Note that, while Fig. 3 above illustrates a case where a part of the character image CI and a part of the multiple characters CR are not displayed on the confirmation screen CHS, Fig. 10 illustrates a case where the entire character image CI and all of the multiple characters CR are displayed on the confirmation screen CHS. For example, on the confirmation screen CHS shown in Fig. 10, the entire character image CI is displayed in the character image area IW, and all of the multiple characters CR are displayed in the editing area EDW.

[0084] 10, an overview of the operation of the terminal device 20 will be described taking as an example a case where there is an error in the separation of characters by the character recognition process. An example of a case where there is no error in the separation of characters by the character recognition process is shown in FIG. 3 described above. FIG. 10 assumes a case where the character recognition device 10 recognizes two characters, "shi" and "ma", of the multiple characters included in the character image CI, as one character, "ho". In this case, for example, in the editing table TBL shown in FIG. 8 and FIG. 9, the number of characters (the number of columns) in the first row is 16 characters, and the character information on the character CR in the 14th column of the first row is character information indicating the character CR of "ho". In addition, in the editing table TBL, the character information on the character CR in the 15th column of the first row is character information indicating the character CR of "su", and the character information on the character CR in the 16th column of the first row is character information indicating the character CR of ".".

[0085] 10, the characters CR surrounded by dashed lines are characters CR for which the accuracy of the result (recognition result) of the character recognition process by the character recognition device 10 is higher than the first accuracy. Therefore, in the editing area EDW shown in FIG. 10, operations by the user are restricted for the characters CR surrounded by dashed lines.

[0086] For example, in the confirmation screen CHS shown at the top of FIG. 10, the cursor CS is displayed at the position of the character CR "ho" among multiple characters CR ("The mobile phone number is shown below."). In this case, the character CR "ho" selected by the cursor CS is not a protected character whose manipulation by the user is restricted, so when the user performs a correction operation to correct the incorrect character CR to the correct character, it is corrected to the correct character. For example, the user performs an operation to correct the character "ho" selected by the cursor CS to the characters CR "shi" and "ma." As a result, the first line of the character string consisting of multiple characters CR is corrected to the correct character string ("The mobile phone number is shown below.").

[0087] In this case, for example, in the editing table TBL shown in Fig. 8 and Fig. 9, the text information in the 15th and 16th columns of the first row before the correction becomes the text information in the 16th and 17th columns of the first row after the correction. Also, for example, in the editing table TBL shown in Fig. 8, the character information about the character CR in the 14th column of the first row is updated to character information indicating the character CR of "shi", and the character information about the character CR in the 15th column of the first row is updated to character information indicating the character CR of "ma". Also, for example, in the editing table TBL shown in Fig. 9, the edited character information in the 14th column of the first row is updated to character information indicating the character CR of "shi", and the edited character information in the 15th column of the first row is updated to character information indicating the character CR of "ma". That is, the editing table TBL is updated so that the number indicated by the "character position" corresponds to the column number of the character string after the correction. Returning to Fig. 10 for explanation.

[0088] In the confirmation screen CHS shown at the bottom of Fig. 10, for example, the first line of the character string consisting of the multiple characters CR displayed in the editing area EDW is the correct character string. As for the second line of the character string consisting of the multiple characters CR, as described in Fig. 3, the incorrectly recognized character CR is not a protected character whose correction by the user is restricted, so the user corrects it to the correct character. Then, for example, when the user has finished correcting the multiple characters CR, the user selects (e.g., presses) the end button BT to end the process of checking and correcting the multiple characters CR.

[0089] The operation of the terminal device 20 is not limited to the example shown in Fig. 10. For example, a character CR that has been corrected to a correct character in the process of checking and correcting a plurality of characters CR may be newly set as a protected character whose correction is restricted.

[0090] Also, for example, when the character recognition device 10 recognizes one character among a plurality of characters included in the character image CI as a plurality of characters (for example, when recognizing one character of "記" as two characters of "言" and "己"), the terminal device 20 operates in the same manner as the operation shown in FIG. 10. Also in this case, the editing table TBL shown in FIGS. 8 and 9 is updated so that the number indicated by "character position" corresponds to the column number of the corrected character string. For example, the text information about the deleted character CR among the plurality of characters CR may be deleted from the editing table TBL, or may be stored in an area different from the area corresponding to the column number of the corrected character string as the text information about the deleted character CR.

[0091] Next, with reference to FIG. 11, an overview of the operation of the character recognition system 1 will be described.

[0092] FIG. 11 is a sequence chart showing an example of the operation of the character recognition system 1 shown in FIG. 1. The processes of steps S10 and S12 and the processes from step S100 to step S720 are executed by the terminal device 20. Also, the processes from step S20 to step S24 are executed by the character recognition device 10. Detailed descriptions of processes similar to those described in FIGS. 1 to 10 are omitted. In FIG. 11, for ease of explanation, the operation of the character recognition system 1 is described using as an example the case where there is one character image CI which is a predetermined character image portion of the input image IIMG.

[0093] First, in step S10, the terminal device 20 acquires input image information indicating the input image IIMG from an optical device such as a camera and an image scanner.

[0094] Next, in step S12, terminal device 20 transmits the input image information acquired in step S10 to character recognition device 10 via network NW. Then, terminal device 20 waits for the processes in steps S100 and after until the result of the character recognition process on input image IIMG is transmitted from character recognition device 10. Therefore, before describing the process in step S100, the processes in steps S20 and the like executed by character recognition device 10 will be described.

[0095] For example, in step S20, the character recognition device 10 acquires the input image information transmitted from the terminal device 20 in step S12. Then, in step S22, the character recognition device 10 executes character recognition processing using the input image information acquired in step S10. As a result, for example, the character recognition unit 124 recognizes a plurality of characters included in the character image CI, which is a predetermined character image portion of the input image IIMG indicated by the input image information. Next, in step S24, the character recognition device 10 transmits recognition information on the result of the character recognition processing on the character image CI and character image information indicating the character image CI to the terminal device 20 via the network NW. This ends the character recognition processing by the character recognition device 10. Also, since the result of the character recognition processing on the input image IIMG has been transmitted from the character recognition device 10, the terminal device 20 executes the processing of step S100.

[0096] For example, in step S100, the terminal device 20 acquires the recognition information and character image information transmitted from the character recognition device 10 in step S24.

[0097] Next, in step S200, the terminal device 20 sets a protected character based on the certainty information included in the recognition information acquired in step S100. For example, the terminal device 20 sets, as a protected character, a character CR whose certainty indicated by the certainty information is higher than a first certainty, among the multiple characters CR indicated by the character information included in the recognition information.

[0098] Next, in step S300, the terminal device 20 causes the display device 30 to display a confirmation screen CHS based on the recognition information and character image information acquired in step S100. For example, the terminal device 20 (more specifically, the display control unit 224) causes the display device 30 to display a character image CI and a plurality of characters CR read from the character image CI based on the character information and the character image information.

[0099] Next, in step S400, the terminal device 20 determines whether or not a user operation has been performed. If the result of the determination in step S400 is negative, the terminal device 20 returns the process to step S400. On the other hand, if the result of the determination in step S400 is positive, the terminal device 20 advances the process to step S500.

[0100] In step S500, the terminal device 20 determines whether the user operation is an end operation. The end operation is, for example, a user operation of selecting the end button BT. If the result of the determination in step S500 is positive, the terminal device 20 ends the display of the confirmation screen CHS and ends the process related to checking and correcting the result of the character recognition process. On the other hand, if the result of the determination in step S500 is negative, the terminal device 20 advances the process to step S600.

[0101] In step S600, the terminal device 20 judges whether the user operation is a protected character correction operation. The protected character correction operation is, for example, an operation for correcting a protected character among a plurality of characters CR. If the result of the judgment in step S600 is negative, the terminal device 20 executes a process corresponding to the user operation in step S700 and returns the process to step S400. On the other hand, if the result of the judgment in step S600 is positive, the terminal device 20 advances the process to step S720.

[0102] In step S720, the terminal device 20 notifies the user that the correction is not allowed. For example, the terminal device 20 displays information indicating that the correction is to a protected character on the display device 30. After executing the process of step S720, the terminal device 20 returns the process to step S400.

[0103] The operation of the character recognition system 1 is not limited to the example shown in Fig. 10. For example, the process of step S300 may be executed before the process of step S200, or may be executed in parallel with the process of step S200. For example, the process of step S720 may be omitted. For example, when the input image IIMG includes multiple character images CI, a series of processes from step S200 to step S720 is executed for each of the multiple character images CI.

[0104] As described above, in this embodiment, the terminal device 20 has an acquisition unit 222, a display control unit 224, and a restriction unit 226. The acquisition unit 222 acquires recognition information including character information indicating a plurality of characters CR read from a character image CI by a character recognition process for recognizing characters included in the image, and accuracy information indicating the accuracy of a recognition result for each of the plurality of characters CR. The display control unit 224 causes the display device 30 to display the plurality of characters CR indicated by the character information. The restriction unit 226 restricts an operation on a protected character for which a condition regarding the accuracy of the recognition result is satisfied, among the plurality of characters CR, based on the accuracy indicated by the accuracy information.

[0105] In this manner, in the present embodiment, the user's operation on the protected characters that satisfy the conditions regarding the accuracy of the recognition result among the multiple characters CR is restricted. As a result, in the present embodiment, it is possible to suppress the occurrence of an operational error in which the user erroneously changes the protected characters that have been correctly recognized as the result of the character recognition among the multiple characters CR. As a result, in the present embodiment, it is possible to provide the terminal device 20 that allows the user to accurately and efficiently confirm and correct the result of the character recognition.

[0106] Moreover, in this embodiment, the limiting unit 226 specifies, as a protected character, a character CR whose accuracy indicated by the accuracy information is higher than the first accuracy. That is, in this embodiment, the operation by the user on a character CR whose recognition result accuracy is higher than the first accuracy (character CR whose probability of being correctly recognized is high) among the multiple character CRs is restricted. As a result, in this embodiment, it is possible to suppress the occurrence of an operational error in which the user mistakenly changes a character CR whose recognition result accuracy is higher than the first accuracy among the multiple character CRs. In addition, there is a risk that the user will not notice the operational error, and if the operational error is left unnoticed, it may have a serious adverse effect on the subsequent process. The present invention can prevent an operational error made by a user without noticing, and as a result, it is possible to eliminate a serious adverse effect on the subsequent process.

[0107] [2. Modifications] The present invention is not limited to the above-described embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples may be combined.

[0108] [First Modification] In the above embodiment, a case where a user's operation on a character CR having a recognition result accuracy higher than a first accuracy among a plurality of characters CR is restricted is exemplified, but the present invention is not limited to such an aspect. For example, as shown in Fig. 12, when a plurality of characters CR are divided into a plurality of groups G (Gx and Gy), a user's operation on a group G including only characters CR having a recognition result accuracy higher than a first accuracy may be restricted or may not be restricted.

[0109] Fig. 12 is an explanatory diagram for explaining an overview of the operation of the terminal device 20 according to the first modified example. For ease of understanding, Fig. 12 illustrates an example in which the entire character image CI is displayed in the character image area IW, and all of the multiple characters CR are displayed in the editing area EDW.

[0110] As shown in Fig. 12, the character image CI is divided into a plurality of areas AR (ARx and ARy). Note that in Fig. 12, for ease of explanation, the area AR is indicated by a dashed line in the character image area IW, but the dashed line indicating the area AR may or may not be displayed in the character image area IW.

[0111] Furthermore, the multiple characters CR are characters CR read from any of the multiple areas AR (ARx and ARy), and belong to any of the multiple groups G (Gx and Gy) corresponding to the multiple areas AR (ARx and ARy). For example, among the multiple characters CR, each character CR in the character string "The mobile phone number is shown below" is a character CR read from the area ARx, and belongs to the group Gx corresponding to the area ARx. Furthermore, among the multiple characters CR, for example, the character string "090-0ab1-1111" is a character CR read from the area ARy, and belongs to the group Gy corresponding to the area ARy. The group Gx corresponds to the first line in the multiple characters CR, and the group Gy corresponds to the second line in the multiple characters CR.

[0112] Here, in the editing area EDW shown in FIG. 12, the characters CR surrounded by a dotted line and the shaded characters CR are characters CR whose accuracy of the result (recognition result) of the character recognition process by the character recognition device 10 is higher than the first accuracy. Furthermore, among the characters CR whose accuracy of the recognition result is higher than the first accuracy, the shaded characters CR are protected characters whose correction is restricted. In the confirmation screen CHS shown in FIG. 12, the protected characters among the multiple characters CR are displayed in a manner (shaded in FIG. 12) that allows them to be distinguished from characters other than the protected characters. That is, the display control unit 224 may cause the display device 30 to display the protected characters among the multiple characters CR in a manner that allows them to be distinguished from characters other than the protected characters. In this case, the user can easily visually recognize the characters CR whose correction is restricted.

[0113] In the example shown in Fig. 12, group Gx of the multiple groups Gx and Gy is a group G including only characters CR whose recognition result has a degree of accuracy higher than a first degree of accuracy. Also, group Gy of the multiple groups Gx and Gy is a group G including characters CR whose recognition result has a degree of accuracy higher than the first degree of accuracy and characters CR whose recognition result has a degree of accuracy equal to or lower than the first degree of accuracy. Note that group Gy is an example of a "first group" and group Gx is an example of a "second group".

[0114] For example, the group Gy is more likely to be selected by the user in the correction work because it contains a mixture of characters CR whose recognition result accuracy is higher than the first accuracy and characters CR whose recognition result accuracy is lower than the first accuracy. Therefore, among all the characters CR belonging to the group Gy, the operation by the user is restricted for the characters CR whose recognition result accuracy is higher than the first accuracy. Note that belonging to the group Gy containing a mixture of characters CR whose recognition result accuracy is higher than the first accuracy and characters CR whose recognition result accuracy is lower than the first accuracy, and having the recognition result accuracy higher than the first accuracy, is another example of the "condition related to the recognition result accuracy."

[0115] For example, the limiting unit 226 specifies a group Gy including characters CR whose recognition result accuracy is higher than a first accuracy and characters CR whose recognition result accuracy is equal to or lower than the first accuracy among the groups Gx and Gy. Then, the limiting unit 226 specifies characters CR whose recognition result accuracy is higher than the first accuracy as protected characters among all characters CR belonging to the group Gy. This makes it possible to prevent the occurrence of operational mistakes in which a user erroneously changes a character CR whose character recognition result is correct (protected character) in the correction operation of a character CR belonging to the group Gy.

[0116] Moreover, since the group Gx includes only characters CR whose recognition result has a certainty higher than the first certainty, the group Gx is less likely to be selected by the user in the correction operation than the group Gy in which characters CR whose recognition result has a certainty higher than the first certainty and characters CR whose recognition result has a certainty lower than the first certainty are mixed. That is, in the correction operation of a plurality of characters CR, the possibility that the character CR belonging to the group Gx is the correction target is lower than the possibility that the character CR belonging to the group Gy is the correction target. Therefore, even if the correction of the character CR belonging to the group Gx is not restricted, the possibility that it is erroneously corrected is lower than that of the character CR belonging to the group Gy. Therefore, in this modified example, the correction of all characters CR belonging to the group Gx may not be restricted.

[0117] The operation of the terminal device 20 according to this modification is not limited to the example shown in FIG. 12. For example, the protected characters among the multiple characters CR may not be displayed in a manner that makes them distinguishable from the characters CR other than the protected characters. Also, for example, among the multiple characters CR, the characters CR whose recognition result has a degree of accuracy higher than a first degree of accuracy may be displayed in a manner that makes them distinguishable from the characters CR whose recognition result has a degree of accuracy equal to or lower than the first degree of accuracy. In this case, the user can easily visually recognize the characters CR whose recognition result has a degree of accuracy higher than the first degree of accuracy.

[0118] As described above, the present modification can also provide the same effect as the above-described embodiment. In addition, in the present modification, the character image CI is divided into a plurality of regions AR, and each of the plurality of characters CR is a character CR read from any one of the plurality of regions AR, and belongs to any one of a plurality of groups G corresponding to the plurality of regions AR. In addition, the restriction unit 226 specifies a group G (first group) including a character CR whose certainty indicated by the certainty information acquired by the acquisition unit 222 is higher than the first certainty, and a character CR whose certainty indicated by the certainty information is equal to or lower than the first certainty, among the plurality of groups G. Then, the restriction unit 226 specifies a character CR whose certainty indicated by the certainty information is higher than the first certainty as a protected character, among all the characters CR belonging to the group G (first group) including a character CR whose certainty indicated by the certainty information is higher than the first certainty and a character CR whose certainty indicated by the certainty information is equal to or lower than the first certainty.

[0119] As a result, in this modified example, when correcting a character CR belonging to a group Gy that includes characters CR whose recognition result accuracy is higher than a first accuracy and characters CR whose recognition result accuracy is equal to or lower than the first accuracy, it is possible to prevent a correctly recognized character CR (protected character) from being erroneously changed.

[0120] In addition, in this modification, the restriction unit 226 identifies a group G (second group) that includes only characters CR whose certainty indicated by the certainty information is higher than the first certainty among the multiple groups G, and does not restrict user operations on all characters belonging to the identified group G. It is expected that the group G that includes only characters CR whose certainty is higher than the first certainty is highly unlikely to be operated by the user, and there is little need to set the entire group G as protected characters. As a result, in this modification, by not setting the group G that includes only characters CR whose certainty is higher than the first certainty as protected characters, it is possible to prevent the number of protected characters whose user operations are restricted from increasing more than necessary, and therefore to prevent the process of setting protected characters from becoming complicated.

[0121] In addition, in this modification, the display control unit 224 displays the protected characters among the multiple characters CR on the display device 30 in a manner that makes them distinguishable from the characters CR other than the protected characters. This allows the user to easily visually recognize the characters CR that are restricted from being modified in this modification.

[0122] [Second modified example] In the above-mentioned first modified example, the case where the operation for the character CR having the accuracy of the recognition result higher than the first accuracy among all the characters CR belonging to the group Gy (first group) is restricted is exemplified, but the present invention is not limited to such an embodiment. For example, as shown in Fig. 13, the operation by the user may not be restricted for all the characters CR belonging to the group Gy in which the character CR having the accuracy of the recognition result higher than the first accuracy and the character CR having the accuracy equal to or lower than the first accuracy are mixed.

[0123] Fig. 13 is an explanatory diagram for explaining an overview of the operation of the terminal device 20 according to the second modified example. The dashed and dotted lines and shading in Fig. 13 have the same meanings as those in Fig. 12. For example, in Fig. 13, among the multiple characters CR, the shaded characters CR are protected characters.

[0124] Among the multiple groups Gx and Gy, the group Gy is a group G including characters CR whose recognition result accuracy is equal to or lower than a first accuracy. In this modified example, the group G (e.g., group Gy) including characters CR whose recognition result accuracy is equal to or lower than the first accuracy is an example of the "first group."

[0125] In this modification, among the groups Gx and Gy, operations on all characters CR belonging to the group Gy including the character CR whose recognition result accuracy is equal to or lower than the first accuracy are not restricted. That is, in the group Gy including the character CR whose recognition result accuracy is equal to or lower than the first accuracy, operations on all characters CR belonging to the group Gy are not restricted regardless of the accuracy of the recognition result. This allows the user to collectively correct a character string (e.g., "1111" in the group Gy in FIG. 13) in which, for example, a character CR whose recognition result accuracy is higher than the first accuracy and a character CR whose recognition result accuracy is equal to or lower than the first accuracy are mixed, to a correct character string (e.g., "1171"). In some cases, the efficiency of the correction work is improved when the user collectively corrects the character string including the erroneous character CR than when the user corrects only the erroneous character CR.

[0126] Among the multiple groups Gx and Gy, operations by the user are restricted for all characters CR belonging to group Gx (second group) including only characters CR whose recognition result accuracy is higher than the first accuracy. Therefore, in this modified example, it is possible to suppress the occurrence of an operational error in which a user mistakenly changes a character CR belonging to group Gx including only characters CR whose recognition result accuracy is higher than the first accuracy. Belonging to group Gx including only characters CR whose recognition result accuracy is higher than the first accuracy is another example of a "condition related to the recognition result accuracy."

[0127] In this manner, in this modified example, in a plurality of groups G corresponding to a plurality of areas AR, the correction of the character CR is restricted for each group G.

[0128] As described above, the present modification can also provide the same effects as those of the above-described embodiment. In addition, in the present modification, the restriction unit 226 specifies, from among the multiple groups G, a group G (first group) including characters CR whose certainty indicated by the certainty information acquired by the acquisition unit 222 is equal to or lower than the first certainty, and does not restrict operations on all characters belonging to the specified group G. In addition, the restriction unit 226 specifies, from among the multiple groups G, a group G (second group) including only characters CR whose certainty indicated by the certainty information is higher than the first certainty, and specifies all characters CR belonging to the specified group G as protected characters.

[0129] As a result, in this modified example, it is possible to suppress a decrease in work efficiency when correcting a character string belonging to a group Gy (first group) including a character CR whose recognition result has a degree of accuracy equal to or lower than the first degree of accuracy. Also, in this modified example, it is possible to suppress the occurrence of an operational error in which a user mistakenly changes a character CR (protected character) belonging to a group Gx (second group) including only a character CR whose recognition result has a degree of accuracy higher than the first degree of accuracy. In other words, in the correction work of correcting a character string including a character CR that has been erroneously recognized by a character recognition process, it is possible to suppress the occurrence of a correction operation error while suppressing a decrease in work efficiency.

[0130] [Third Modification] In the above-described embodiment and modified example, a case where one of the multiple character images CI included in the input image IIMG is displayed on one confirmation screen CHS has been exemplified, but the present invention is not limited to such an embodiment. For example, all of the multiple character images CI included in the input image IIMG may be displayed on one confirmation screen CHS.

[0131] 14 is a diagram showing an example of a confirmation screen CHS according to Modification 3. The dashed and dotted lines, dotted lines, and shading in FIG. 14 have the same meanings as the dashed and dotted lines, dotted lines, and shading in FIG.

[0132] 14, the confirmation screen CHS displays, for example, a plurality of character image areas IW corresponding to a plurality of character images CI included in the input image IIMG, and one editing area EDW. In each of the plurality of character image areas IW, the entirety of the corresponding character image CI among the plurality of character images CI is displayed.

[0133] In the example shown in FIG. 14, an image including a plurality of character images CI (for example, input image IIMG) is another example of a "read target image". For example, in FIG. 14, the input image IIMG is divided into a plurality of character images CI as a plurality of areas AR (ARa, ARb, ARc, etc.) corresponding to a plurality of items on an application form. Therefore, each of the plurality of character images CI is an example of an "area". Also, each of the plurality of characters CR read from the input image IIMG is a character CR read from one of the plurality of character images CI, and belongs to one of a plurality of groups G (Ga, Gb, Gc, etc.) corresponding to the plurality of character images CI (a plurality of areas AR). The code of each of the plurality of groups G is suffixed with a lowercase alphabet that is the same as the lowercase alphabet suffixed to the code of the corresponding area AR. For example, the plurality of characters CR belonging to group Ga are characters CR read from area ARa (character image CI corresponding to the application number) of the input image IIMG.

[0134] In the editing area EDW, a plurality of characters CR read from each of the plurality of character images CI are displayed. In the editing area EDW, the plurality of characters CR are displayed in accordance with the display positions of the corresponding character images CI in the confirmation screen CHS. For example, in the confirmation screen CHS, the plurality of character images CI are displayed in the order of application number, name (kana), name, postal code, address, e-mail, place of employment, place of employment telephone number, and remarks. Therefore, in the editing area EDW, the plurality of characters CR are displayed in the order of application number, name (kana), name, postal code, address, e-mail, place of employment, place of employment telephone number, and remarks. Specifically, for example, all the characters CR ("180-3588") read from the character image CI (character image CI corresponding to the application number) displayed at the top in the confirmation screen CHS are displayed in the first line in the editing area EDW.

[0135] In the example shown in FIG. 14, operations on all characters belonging to group G (Gb, Gc, Gf, and Gi) including characters CR whose recognition result accuracy is equal to or lower than the first accuracy are not restricted. Also, operations on all characters CR belonging to group G (Ga, Gd, Ge, Gg, and Gh) including only characters CR whose recognition result accuracy is higher than the first accuracy are restricted. Thus, in the example shown in FIG. 14, in a plurality of groups G corresponding to a plurality of character images CI included in the input image IIMG, correction of characters CR is restricted for each group G. That is, in the example shown in FIG. 14, correction of characters CR is restricted for each entry field EC of an item in the application form shown in FIG. 2.

[0136] In the example shown in Fig. 14, the restriction on the user's operation on the character CR is not limited to the above example. For example, among the multiple characters CR belonging to each of the groups Gb, Gc, Gf, and Gi, the user's operation may be restricted for the character CR whose recognition result has a degree of accuracy higher than the first degree of accuracy. In this case, the user's operation may be restricted or not restricted for all the characters CR belonging to each of the groups Ga, Gd, Ge, Gg, and Gh. Also, in the example shown in Fig. 14, the character image CI may be divided into multiple areas AR.

[0137] 14 illustrates an example in which each of the multiple character images CI included in the input image IIMG is displayed in a corresponding one of the multiple character image areas IW, but this modification is not limited to this example. For example, all of the multiple character images CI included in the input image IIMG may be displayed in one character image area IW.

[0138] As described above, in this modified example, the same effects as those of the above-mentioned embodiment and modified example can be obtained. In addition, in this modified example, since a plurality of character images CI included in the input image IIMG are displayed on one confirmation screen CHS, it is possible to prevent the work of confirming and correcting a plurality of characters CR read from the plurality of character images CI from becoming complicated.

[0139] [Fourth Variation] In the third variant described above, a plurality of character image areas IW corresponding to a plurality of character images CI contained in the input image IIMG, and a plurality of editing areas EDW corresponding to the plurality of character image areas IW may be displayed on the confirmation screen CHS.

[0140] 15 is a diagram showing an example of a confirmation screen CHS according to Modification 4. The dashed and dotted lines, dotted lines, and shading in FIG. 15 have the same meanings as those in FIG.

[0141] As shown in Fig. 15, the confirmation screen CHS displays, for example, a plurality of character image areas IW corresponding to a plurality of character images CI included in the input image IIMG, and a plurality of editing areas EDW corresponding to the plurality of character image areas IW. The confirmation screen CHS shown in Fig. 15 is similar to the confirmation screen CHS shown in Fig. 14, except that a plurality of editing areas EDW are displayed.

[0142] For example, in each of the plurality of editing areas EDW, a plurality of characters CR are displayed which are read from a character image CI displayed in a corresponding one of the plurality of character image areas IW.

[0143] In addition, in this modification, the corresponding editing areas EDW and character image areas IW are arranged adjacent to each other on the confirmation screen CHS. This allows the user to easily compare the multiple characters CR with the multiple character portions for each item. The multiple items are, for example, application number, name (kana), name, postal code, address, email, place of employment, place of employment phone number, and remarks, as described in FIG. 2.

[0144] As described above, this modification can also provide the same effects as those of the above-described embodiment and modification. In the above-described first to fourth modifications, the multiple characters CR may be divided into multiple groups G, with each group being a character string or a sentence unit that is a unit of the meaning of a word. For example, the multiple characters CR may be divided into multiple groups G based on at least one of a word, a phrase, and a sentence by character recognition processing. Alternatively, the restriction unit 226 may analyze a character string made of the multiple characters CR, and divide the multiple characters CR into multiple groups G based on the analysis result by dividing the multiple characters CR into character strings or sentence units that are units of the meaning of a word. That is, the restriction unit 226 may divide the multiple characters CR into multiple groups G based on at least one of a word, a phrase, and a sentence.

[0145] [Fifth Modification] In the above-described embodiment and modified examples, the restriction unit 226 may change the degree of restriction on operations on protected characters depending on the accuracy of the recognition result.

[0146] Fig. 16 is a flowchart showing an example of the operation of the terminal device 20 according to the fifth modified example. Fig. 16 shows the operation of the terminal device 20 after the result of the character recognition process on the input image IIMG is transmitted from the character recognition device 10. The operation of the terminal device 20 and the operation of the character recognition device 10 until the result of the character recognition process on the input image IIMG is transmitted from the character recognition device 10 are similar to the operation (steps S10, S12, S20, S22 and S24) shown in Fig. 11 described above. Detailed explanations of the same processes as those described in Figs. 1 to 15 will be omitted.

[0147] A series of processes from step S100 to step S700 is similar to the series of processes from step S100 to step S700 shown in Fig. 11. However, in the operation shown in Fig. 16, if the result of the determination in step S600 is positive, the terminal device 20 advances the process to step S710.

[0148] In step S710, the terminal device 20 determines whether the accuracy of the protected character that is the target of the correction operation is higher than the second accuracy. The second accuracy is higher than the first accuracy. In other words, the first accuracy is lower than the second accuracy. The second accuracy may be determined by the administrator of the character recognition device 10 and the user of the terminal device 20, etc., like the first accuracy, or may be automatically set by the character recognition device 10.

[0149] If the result of the determination in step S710 is positive, the terminal device 20 notifies the user that modification is not possible in step S720, and the process returns to step S400. On the other hand, if the result of the determination in step S710 is negative, the terminal device 20 advances the process to step S712.

[0150] In step S712, the terminal device 20 determines whether or not to execute correction on the protected character that is the target of the correction operation. For example, the terminal device 20 causes the display device 30 to display a GUI image for selecting whether to continue correcting the protected character that is the target of the correction operation or to cancel the correction operation. If the user selects to continue correcting the protected character, the terminal device 20 determines to execute correction on the protected character. On the other hand, if the user selects to cancel the correction operation, the terminal device 20 determines not to execute correction on the protected character. Note that an operation for continuing correction on the protected character (for example, an operation for selecting to continue correction on the protected character in the GUI image) is an example of a "predetermined operation."

[0151] If the result of the determination in step S712 is negative, the terminal device 20 returns the process to step S400 without correcting the protected character that was the target of the correction operation. On the other hand, if the result of the determination in step S712 is positive, the terminal device 20 advances the process to step S714.

[0152] In step S714, the terminal device 20 corrects the protected character that is the target of the correction operation based on the correction operation, and then the terminal device 20 returns the process to step S400.

[0153] In this manner, in this modified example, the degree of restriction on the operation on the protected character is changed according to the accuracy of the recognition result. Specifically, for a protected character whose accuracy of the recognition result is higher than the second accuracy, the operation on the protected character is restricted so as not to be reflected. Also, for a protected character whose accuracy of the recognition result is equal to or lower than the second accuracy but higher than the first accuracy, if a predetermined operation is performed by the user, the operation on the protected character is not restricted. In other words, for a protected character whose accuracy of the recognition result is equal to or lower than the second accuracy but higher than the first accuracy, if a predetermined operation is not performed by the user, the operation on the protected character is restricted so as not to be reflected.

[0154] Note that the operation of the terminal device 20 according to this modification is not limited to the example shown in Fig. 16. For example, the terminal device 20 may execute a process related to removing the restriction instead of the process of step S712.

[0155] In the process of removing the restriction, the terminal device 20 first determines whether or not to remove the restriction on the operation on the protected character that is the target of the correction operation. For example, the terminal device 20 displays on the display device 30 a GUI image for selecting whether to remove the restriction on the operation on the protected character that is the target of the correction operation or to cancel the correction operation. Then, when the user selects to remove the restriction on the operation on the protected character, the terminal device 20 determines to remove the restriction on the operation on the protected character and removes the restriction on the operation on the protected character that is the target of the correction operation. After the restriction on the operation on the protected character is removed, the process of step S714 is executed. On the other hand, when the user selects to cancel the correction operation, the terminal device 20 determines not to remove the restriction on the operation on the protected character and returns the process to step S400 without correcting the protected character that is the target of the correction operation. Note that an operation for removing the restriction on the operation on the protected character (for example, an operation of selecting to remove the restriction on the operation on the protected character in the GUI image) is another example of the "predetermined operation".

[0156] As described above, in this modified example, the same effects as those of the above-mentioned embodiment and modified example can be obtained. Moreover, in this modified example, the restriction unit 226 changes the degree of restriction on operations on protected characters according to the certainty indicated by the certainty information acquired by the acquisition unit 222. As a result, in this modified example, it is possible to suppress a situation in which an erroneously recognized character CR cannot be corrected, while suppressing a situation in which an operation on a correctly recognized character CR becomes unrestricted.

[0157] In addition, in this modified example, the restriction unit 226 restricts the protected character CR whose certainty indicated by the certainty information is higher than the second certainty so that the content of the operation on the protected character is not reflected. In addition, in the protected character, the restriction unit 226 restricts the protected character CR whose certainty indicated by the certainty information is equal to or lower than the second certainty so that the operation on the protected character is not reflected when the user performs a predetermined operation, and restricts the protected character CR whose certainty is not performed by the user so that the content of the operation on the protected character is not reflected. In this modified example, when the certainty of the recognition result is equal to or lower than the second certainty and the recognition result of a protected character whose recognition result is higher than the first certainty is incorrect, it is possible to prevent the protected character whose recognition result is incorrect from being unable to be corrected.

[0158] [Sixth Modification] In the above-described embodiment and modified example, the case where the setting of the protected characters is executed by the terminal device 20 has been exemplified, but the present invention is not limited to such an aspect. For example, the restriction unit 226 shown in FIG. 6 may be included in the character recognition device 10.

[0159] Fig. 17 is an explanatory diagram for explaining an overview of character recognition system 1 according to a sixth modified example. Character recognition system 1 shown in Fig. 17 is similar to character recognition system 1 shown in Fig. 1, except that character recognition system 1 shown in Fig. 17 has character recognition device 10A and terminal device 20A instead of character recognition device 10 and terminal device 20 shown in Fig. 1.

[0160] Character recognition device 10A has a control unit 120 that controls each unit of character recognition device 10A, a storage unit 140 that stores various information, and a communication unit 160 for executing communication with an external device such as terminal device 20A. That is, character recognition device 10A is similar to character recognition device 10 shown in FIG. 4. However, character recognition device 10A differs from character recognition device 10 in that control unit 120 executes control program PG1a instead of control program PG1. For this reason, storage unit 140 stores control program PG1a instead of control program PG1. Character recognition device 10A is another example of an "information processing device", and control program PG1a is another example of a "program".

[0161] In this modification, the control program PG1a includes, for example, an application program for the character recognition device 10A to execute a character recognition process and a correction support process that supports the correction of the result of the character recognition process, etc. However, the control program PG1a may also include, for example, an operating system program for the control unit 120 to control each unit of the character recognition device 10A.

[0162] The control unit 120 has, for example, an image information acquisition unit 122, a character recognition unit 124, a display control unit 224a, and a restriction unit 226. In this way, the control unit 120 is similar to the control unit 120 shown in Fig. 4, except for having the display control unit 224a and the restriction unit 226. It should be noted that the restriction unit 226 is similar to the restriction unit 226 shown in Fig. 6. For this reason, a description of the restriction unit 226 will be omitted.

[0163] The display control unit 224a generates display information for displaying the confirmation screen CHS on the display unit 310 of the display device 30, and transmits the generated display information to the terminal device 20A via the communication unit 160. For example, the terminal device 20A displays the confirmation screen CHS indicated by the display information received from the character recognition device 10A on the display unit 310 of the display device 30. In this way, the display control unit 224a causes the display device 30 to display the multiple characters CR recognized by the character recognition unit 124, for example, by transmitting display information indicating the confirmation screen CHS to the terminal device 20A.

[0164] The terminal device 20A has a control unit 220 that controls each unit of the terminal device 20A, a storage unit 240 that stores various information, a communication unit 260 for executing communication with an external device such as the character recognition device 10A, and an operation unit 280 for accepting operations by a user of the terminal device 20A. That is, the terminal device 20A is similar to the terminal device 20 shown in Fig. 6. However, the terminal device 20A differs from the terminal device 20 in that the control unit 220 executes a control program PG2a instead of the control program PG2. For this reason, the storage unit 140 stores the control program PG2a instead of the control program PG2.

[0165] The control unit 220 includes, for example, an acquisition unit 222. In this manner, the control unit 220 is similar to the control unit 220 shown in FIG. 6, except that the display control unit 224 and the limiting unit 226 are omitted from the control unit 220 shown in FIG.

[0166] In this modification, for example, acquiring unit 222 may transmit operation information indicating the content of the operation accepted by operation unit 280 to character recognition device 10A via communication unit 260. The function of transmitting the operation information to character recognition device 10A may be realized, for example, by a functional block different from acquiring unit 222. Character recognition device 10A generates display information and the like for displaying a confirmation screen CHS on display unit 310 of display device 30, based on the operation information received from terminal device 20A.

[0167] In the operation of character recognition system 1 shown in Fig. 17, for example, character recognition device 10A executes the processes of steps S20 and S22 shown in Fig. 11, and then executes a series of processes from step S200 to step S720 shown in Fig. 11 or 16. However, in a process for displaying a confirmation screen CHS or the like on display device 30 (for example, the process of step S300), character recognition device 10A transmits display information indicating the confirmation screen CHS or the like to be displayed on display device 30 to terminal device 20A. In addition, a process based on the content of the operation accepted by operation unit 280 (for example, the process of step S400) is executed based on the operation information received from terminal device 20A.

[0168] 17, for example, after executing the processes of steps S10 and S12 shown in Fig. 11, terminal device 20A displays a confirmation screen CHS indicated by the display information received from character recognition device 10A on display unit 310 of display device 30. When operation unit 280 of terminal device 20A accepts an operation by a user, terminal device 20A transmits operation information indicating the content of the operation accepted by operation unit 280 to character recognition device 10A.

[0169] Here, the hardware configuration of the character recognition device 10A is the same as that of the character recognition device 10 shown in FIG. 5. For example, the processor 12 operating according to an application program included in the control program PG1a functions as a control unit 120 including an image information acquisition unit 122, a character recognition unit 124, a display control unit 224a, and a limiting unit 226. The control program PG1a may be transmitted from another device. Also, the hardware configuration of the terminal device 20A is the same as that of the terminal device 20 shown in FIG. 7. For example, the processor 22 operating according to an application program included in the control program PG2a functions as a control unit 220 including an acquisition unit 222. The control program PG2a may be transmitted from another device.

[0170] The configurations of character recognition device 10A and terminal device 20A are not limited to the example shown in FIG. 17. For example, character recognition device 10A may have an operation device (for example, operation device 18 shown in FIG. 19 described later) that functions as an operation unit for accepting operations by an administrator of character recognition device 10A. Terminal device 20A may also have an optical device such as a camera and an image scanner for converting input image IIMG into input image information that is a digital signal. Alternatively, terminal device 20A may have a display device 30. As described above, in this modification as well, the same effects as those of the above-mentioned embodiment and modification can be obtained.

[0171] [Seventh Variation] In the above-described embodiment and modified example, the character recognition device 10 and the terminal device 20 are separate from each other, but the present invention is not limited to such an embodiment. For example, the character recognition device 10 and the terminal device 20 may be integrated into one body.

[0172] FIG. 18 is a functional block diagram showing an example of the configuration of a character recognition device 10B according to the seventh modified example.

[0173] Character recognition device 10B has control unit 120 that controls each unit of character recognition device 10B, storage unit 140 that stores various information, communication unit 160 for executing communication with an external device that exists outside character recognition device 10B, operation unit 180, and imaging unit 190. Thus, character recognition device 10B is similar to character recognition device 10 shown in FIG. 4 except for having operation unit 180 and imaging unit 190. However, character recognition device 10B differs from character recognition device 10 in that control unit 120 executes control program PG1b instead of control program PG1. Therefore, storage unit 140 stores control program PG1b instead of control program PG1. Character recognition device 10B is another example of an "information processing device", and control program PG1b is another example of a "program".

[0174] In this modification, the control program PG1b includes, for example, an application program for the character recognition device 10B to execute a character recognition process and a correction support process that supports the correction of the result of the character recognition process, etc. However, the control program PG1b may also include, for example, an operating system program for the control unit 120 to control each unit of the character recognition device 10B.

[0175] The control unit 120 has, for example, an image information acquisition unit 122, a character recognition unit 124, a display control unit 224, and a restriction unit 226. The image information acquisition unit 122 and the character recognition unit 124 are similar to the image information acquisition unit 122 and the character recognition unit 124 shown in Fig. 4. Moreover, the display control unit 224 and the restriction unit 226 are similar to the display control unit 224 and the restriction unit 226 shown in Fig. 6.

[0176] The operation unit 180 accepts an operation by a user or the like. The imaging unit 190 images a subject and outputs image information indicating the image of the subject. For example, the imaging unit 190 may image an application form or the like and convert an input image IIMG, which is an image of the application form, into input image information. Then, the imaging unit 190 may store the input image information indicating the input image IIMG in the storage unit 140 or the like. In this case, the image information acquisition unit 122 may acquire input image information indicating the input image IIMG to be subjected to character recognition processing from, for example, the storage unit 140. Alternatively, the image information acquisition unit 122 may acquire the input image information from the imaging unit 190 without going through the storage unit 140. Alternatively, the image information acquisition unit 122 may acquire the input image information from an external device that exists outside the character recognition device 10B.

[0177] In the operation of character recognition device 10B, for example, after the processes of steps S20 and S22 shown in FIG. 11 are executed, a series of processes from step S200 to step S720 shown in FIG. 11 or FIG. 16 are executed. That is, the operation of character recognition device 10B is the same as the operation of character recognition system 1 shown in FIG. 11, except that a series of processes of steps S10, S12, S24, and S100 are omitted from the operation of character recognition system 1 shown in FIG. 11. For example, in step S20 shown in FIG. 11, image information acquisition unit 122 acquires input image information indicating input image IIMG to be a target of character recognition processing from imaging unit 190, storage unit 140, or an external device. Then, in step S22 shown in FIG. 11, character recognition unit 124 executes character recognition processing using the input image information (input image information acquired in step S20). 11, the restriction unit 226 sets protected characters based on the accuracy of the result of the character recognition process (the character recognition process of step S22) by the character recognition unit 124. Then, in step S300 shown in FIG 11, the display control unit 224 causes the display device 30 to display a confirmation screen CHS for confirming the result of the character recognition process (the character recognition process of step S22) by the character recognition unit 124.

[0178] Next, the hardware configuration of the character recognition device 10B will be described with reference to FIG.

[0179] FIG. 19 is a diagram illustrating an example of a hardware configuration of the character recognition device 10B illustrated in FIG.

[0180] Character recognition device 10B has a processor 12 that controls each unit of character recognition device 10B, a memory 14 that stores various information, a communication device 16, an operation device 18, and an imaging device 19. Thus, character recognition device 10B is similar to character recognition device 10 shown in FIG. 5 except for having operation device 18 and imaging device 19. Note that processor 12 that operates according to an application program included in control program PG1b functions as control unit 120 that includes, for example, image information acquisition unit 122, character recognition unit 124, display control unit 224, and limiting unit 226. Control program PG1b may be transmitted from another device.

[0181] The operation device 18 is hardware for accepting an operation by a user or the like, and functions as an operation unit 180. For example, the operation device 18 is configured similarly to the operation device 28 shown in FIG. 7. The imaging device 19 is an optical device such as a camera and an image scanner, and functions as an imaging unit 190. For example, the imaging device 19 captures an image of a subject and outputs image information indicating the captured image of the subject. The imaging device 19 has, for example, an imaging optical system and an imaging element. The imaging optical system is an optical system including at least one imaging lens. For example, the imaging optical system may have a zoom lens, a focus lens, or the like. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor, or the like.

[0182] The configuration of character recognition device 10B is not limited to the examples shown in Fig. 18 and Fig. 19. For example, character recognition device 10B may have a display device 30. Character recognition device 10B may not include imaging device 19. Display device 30 may be included in a terminal device such as a thin client terminal communicatively connected to character recognition device 10B via network NW, or may be communicatively connected to a terminal device such as a thin client terminal.

[0183] As described above, in this modified example as well, the same effects as those of the above-described embodiment and modified example can be obtained.

[0184] [Eighth Modification] In the above-described embodiment and modified examples, the character recognition device 10, 10A, or 10B may learn the content of corrections to the results of the character recognition process. As described above, the present modified example can also provide the same effects as the above-described embodiment and modified examples.

[0185] [Ninth Variation] In the above-described embodiment and modified example, when a character CR with high accuracy is set as a protected character, an operation on the protected character may be permitted by a stepwise operation by the user. For example, even if the restricting unit 226 sets a character CR with high accuracy as a protected character, the recognition result of the character CR set as the protected character may be incorrect because the recognition accuracy of the character recognition device 10, 10A, or 10B is not 100%. In addition, a case where the user intentionally wants to correct or change the protected character may be assumed. For this reason, for example, an operation to delete the protected character may be performed as a first operation by the user, and then an operation to change (correct) the protected character may be performed as a second operation by the user, so that the deletion of the protected character by the first operation and the change (correction) of the protected character by the second operation may be confirmed. In other words, when the user only performs the first operation, the deletion of the protected character to be operated is tentatively determined, and if the user does not perform the second operation thereafter, the first operation performed earlier is cancelled, and the operation to delete the protected character (first operation) is not confirmed.

[0186] In this manner, in this modified example, the character recognition device 10 can restrict careless manipulation of the protected characters while allowing the protected characters to be corrected or changed as required. [Explanation of symbols]

[0187] 1...character recognition system, 10, 10A, 10B...character recognition device, 12...processor, 14...memory, 16...communication device, 18...operation device, 19...imaging device, 20...terminal device, 22...processor, 24...memory, 26...communication device, 28...operation device, 30...display device, 120...control unit, 122...image information acquisition unit, 124...character recognition unit, 140...storage unit, 160...communication unit, 180...operation unit, 190...imaging unit, 220...control unit, 222...acquisition unit, 224, 224a...display control unit, 226...restriction unit, 240...storage unit, 260...communication unit, 280...operation unit, 310...display unit, AR...area, CI...character image, CR...character, G...group, IIMG...input image, NW...network.

Claims

1. an acquisition unit that acquires recognition information including character information indicating a plurality of characters read from a read target image by a character recognition process that recognizes characters included in the image and accuracy information indicating an accuracy of a recognition result for each of the plurality of characters; a display control unit that causes a display device to display the plurality of characters indicated by the character information; a restriction unit that restricts an operation on a character that satisfies a condition related to the accuracy of the recognition result among the plurality of characters based on the accuracy indicated by the accuracy information; Equipped with The plurality of characters are divided into a plurality of groups; the restriction unit is capable of identifying a first group including characters whose certainty indicated by the certainty information is higher than a first certainty and characters whose certainty indicated by the certainty information is equal to or lower than the first certainty, and determining whether or not to impose restrictions on operations on characters whose certainty indicated by the certainty information is higher than the first certainty so that the first group and groups other than the first group are different from each other.

23. An information processing apparatus comprising:

2. The limiting portion is restricting an operation on a character whose certainty indicated by the certainty information is higher than the first certainty among all characters belonging to the first group; 2. The information processing apparatus according to claim 1,

3. The limiting portion is identifying a second group including only characters whose certainty indicated by the certainty information is higher than the first certainty from among the plurality of groups, and not restricting operations on all characters belonging to the second group; 3. The information processing apparatus according to claim 2.

4. The limiting portion is identifying a second group among the plurality of groups, the second group including only characters whose certainty indicated by the certainty information is higher than the first certainty, and restricting operations on all characters belonging to the second group; 2. The information processing apparatus according to claim 1,

5. The limiting portion is varying a degree of restriction on an operation depending on the degree of certainty indicated by the degree of certainty information; 5. The information processing apparatus according to claim 1,

6. The display control unit is displaying, on the display device, a protected character, which is a character for which an operation is restricted, among the plurality of characters, in a manner that allows the protected character to be distinguished from characters other than the protected character; 6. The information processing apparatus according to claim 1,

7. The processor, an acquisition unit that acquires recognition information including character information indicating a plurality of characters read from a read target image by a character recognition process that recognizes characters included in the image and accuracy information indicating an accuracy of a recognition result for each of the plurality of characters; a display control unit that causes a display device to display the plurality of characters indicated by the character information; a restriction unit that restricts an operation on a character that satisfies a condition related to the accuracy of the recognition result among the plurality of characters based on the accuracy indicated by the accuracy information; Function as a The plurality of characters are divided into a plurality of groups; the restriction unit is capable of identifying a first group including characters whose certainty indicated by the certainty information is higher than a first certainty and characters whose certainty indicated by the certainty information is equal to or lower than the first certainty, and determining whether or not to impose restrictions on operations on characters whose certainty indicated by the certainty information is higher than the first certainty so that the first group and groups other than the first group are different from each other. A program characterized by:

Images