Information processing apparatus for obtaining a character string

Abstract

An information processing apparatus for obtaining a character string is disclosed. When attribute information such as a file name is set based on a character string obtained by performing character recognition processing on a scanned image, the correction content is made learnable based on a correction operation performed by a user on an attribute setting screen.

Description

TECHNICAL FIELD

[0001] The present application relates to an apparatus, a method, and a storage medium for obtaining a desired character string based on a character recognition result of a scanned image. BACKGROUND

[0002] Conventionally, there is a system that sets a file name of a form image obtained by scanning a paper form based on a recognition result obtained by performing a character recognition process on the form image. Japanese Patent Application Publication No. S62-051866 discusses performing a character recognition process on a predetermined region in a form image and using the result of the character recognition process as a file name of the form image. Japanese Patent Application Publication No. 2007-503032 discusses application of a file naming rule when performing optical character recognition (OCR) on an extraction region selected according to a user's specifications and using the OCR result as a file name. As the file naming rule, Japanese Patent Application Publication No. 2007-503032 discusses imposing a condition on the length of a file name (maximum length and minimum length), deleting forbidden characters, and preventing reuse of the same file name.

[0003] However, the technology discussed in Japanese Patent Application Publication No. 2007-503032 involves setting conditions in advance, such as characters that are prohibited from being used in a file name. Therefore, it is difficult for a user to flexibly set conditions by using a recognition result of a form image. SUMMARY

[0004] An information processing apparatus according to one aspect of the present disclosure includes a character recognition component configured to obtain a character recognition result by performing a character recognition process on a text region in a first scanned image, and a learning component configured to learn a correction content of a correction made to at least a portion of a character string of the character recognition result if the correction is made in setting attribute information on the first scanned image by using the character recognition result obtained by the character recognition component, wherein the character recognition component is configured to correct a character recognition result of a text region in a second scanned image based on the correction content learned by the learning component if the character recognition process is performed on the text region in the second scanned image, and output the corrected character recognition result.

[0005] Other features of the present disclosure will become apparent from the following description of example embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 is a diagram that illustrates the overall configuration of a system.

[0007] Figure 2 A hardware configuration example of a multifunction peripheral (MFP) is illustrated.

[0008] Figure 3 A hardware configuration example of the client personal computer (PC) and the MFP cooperation service is exemplified.

[0009] Figure 4 A software configuration example of the system is exemplified.

[0010] Figure 5 is a sequence diagram exemplifying the processing procedure between the devices.

[0011] Figure 6A and Figure 6B is a diagram exemplifying an example of a screen displayed on the MFP or the client PC.

[0012] Figure 7 is a flowchart exemplifying details of the processing for learning correction of a character recognition result.

[0013] Figure 8 is a diagram exemplifying an example of an attribute setting screen when a character recognition result is corrected.

[0014] Figure 9 is a flowchart exemplifying details of the correction processing when a character recognition processing is performed on a new image.

[0015] Figure 10 is a diagram exemplifying an example of an attribute setting screen.

[0016] Figure 11 is a diagram exemplifying details of data output as a character recognition result.

[0017] Figure 12 is a diagram exemplifying an example of an attribute setting screen.

[0018] Figure 13 is a diagram exemplifying details of data output as a character recognition result. DETAILED DESCRIPTION

[0019] <SYSTEM CONFIGURATION>

[0020] Figure 1 is a diagram exemplifying the overall configuration of a system for implementing an exemplary embodiment of the present application. The image processing system includes a multifunction peripheral (MFP) 110, a client personal computer (PC) 111, an MFP cooperation service 120, and a cloud storage 130. The MFP 110 and the client PC 111 are communicably connected to servers for providing various services on the Internet, such as the MFP cooperation service 120 and the cloud storage 130, via a local area network (LAN).

[0021] The MFP 110 has multiple functions such as a scanner function and a printer function, and is an example of an image processing apparatus. The client PC 111 is a computer that executes an application program for receiving provision of a service requested by the MFP cooperation service 120. The MFP cooperation service 120 is an example of a service having a function of storing an image file scanned by the MFP 110 in a server of the MFP cooperation service 120 or transferring the image file to a service capable of storing a file, such as another storage service. The cloud storage 130 is a service capable of storing a file via the Internet and obtaining the file via a web browser.

[0022] Although the image processing system according to the present exemplary embodiment includes the MFP 110, the client PC 111, the MFP cooperation service 120, and the cloud storage 130, the configuration of the image processing system is not limited thereto. For example, the MFP 110 can also function as the client PC 111 and / or the MFP cooperation service 120. Regarding the connection configuration, the MFP cooperation service 120 can be located on a server on a LAN instead of the Internet. The cloud storage 130 can be replaced with an e-mail server, and an e-mail attached with a scanned image can be sent.

[0023] <Hardware configuration of MFP>

[0024] Figure 2 is a hardware configuration diagram of the MFP 110. The MFP 110 includes a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223.

[0025] The control unit 210 includes units 211 to 219 to be described below, and controls the entire MFP 110. A central processing unit (CPU) 211 reads a control program stored in a read only memory (ROM) 212, and executes and controls various functions of the MFP 110, such as a reading, a printing, and a communication function. A random access memory (RAM) 213 functions as a temporary storage area, such as a work area of the CPU 211 or a main memory. In the present exemplary embodiment, a single CPU 211 is described to execute various processes shown in flowcharts to be described below by using a single memory (the RAM 213 or a hard disk drive (HDD) 214). However, this is not limiting. For example, the processes can be executed by cooperation of a plurality of CPUs and a plurality of RAMs or HDDs. The HDD 214 is a large capacity storage unit for storing image data and various programs. An operation unit interface (I / F) 215 is an I / F for connecting the operation unit 220 with the control unit 210.

[0026] The operation unit 220 includes a touch panel and a keyboard and accepts operations, inputs, and instructions provided by a user. The printer I / F 216 is an I / F for connecting the printer unit 221 with the control unit 210. Print image data is transferred from the control unit 210 to the printer unit 221 via the printer I / F 216 and is printed on a recording medium. The scanner I / F 217 is an I / F for connecting the scanner unit 222 with the control unit 210.

[0027] The scanner unit 222 reads a document placed on an unillustrated platen or an automatic document feeder (ADF) to generate image data, and inputs the image data into the control unit 210 via the scanner I / F 217. The MFP 110 has a function of printing and outputting (copying) the image data generated by the scanner unit 222 from the printer unit 221, a function of transferring the image data as a file through a network, and a function of transmitting an email attached with the image data. The modem I / F 218 is an I / F for connecting the modem 223 with the control unit 210.

[0028] The modem 223 realizes facsimile communication of image data with a facsimile device on a public switched telephone network (PSTN). The network I / F 219 is an I / F for connecting the control unit 210 (MFP 110) to a LAN. The MFP 110 transmits image data and information to various services on the Internet and receives various types of information by using the network I / F 219.

[0029] <Hardware configuration of client PC and MFP cooperative service>

[0030] Figure 3A hardware configuration example of the information processing apparatus that can implement each of the client PC 111 and the MFP cooperation service 120 is exemplified. The apparatus for implementing each of the client PC 111 and the MFP cooperation service 120 includes a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I / F 315. The CPU 311 controls the overall operation by reading a control program stored in the ROM 312 and executing various types of processing. The control program is installed via various networks such as the Internet and an intranet or a computer-readable storage medium such as a compact disc read-only memory (CD-ROM) and a universal serial bus (USB) memory. The RAM 313 is used as a temporary storage area such as a work area or a main memory of the CPU 311. The HDD 314 is a large-capacity storage unit for storing image data and various programs. The network I / F 315 is an I / F for connecting the client PC 111 or the MFP cooperation service 120 to the Internet. The MFP cooperation service 120 receives a processing request from another apparatus such as the MFP 110 via the network I / F 315 and transmits and receives various types of information.

[0031] <Software configuration of image processing system>

[0032] Figure 4 A software configuration example related to programs executed by the MFP 110 and the MFP cooperation service 120 in the image processing system according to the present exemplary embodiment is exemplified. The MFP 110 is broadly divided into two parts, a native function part 410 and an additional function part 420. The processing units included in the native function part 410 are processing units included in the MFP 110 by default, and the additional function part 420 is implemented by an application additionally installed on the MFP 110. For example, the additional function part 420 is a Java (registered trademark)-based application, and is capable of easily adding a function to the MFP 110. Although not shown, other additional applications can also be installed on the MFP 110.

[0033] The native function part 410 includes a scan execution unit 411 and an image data storage unit 412. The additional function part 420 includes a display control unit 421, a scan instruction unit 422, and a cooperation service request unit 423.

[0034] The display control unit 421 displays a user interface (UI) screen for accepting an operation of a user on a liquid crystal display unit of the operation unit 220 of the MFP 110 having a touch panel function. For example, the display control unit 421 displays a UI screen such as a screen for inputting authentication information for accessing the MFP cooperative service 120, a screen for making a scan setting, a screen for making a scan start operation, and a screen for displaying a preview image. The scan instruction unit 422 transmits a scan request including a scan setting based on a user instruction input via the UI screen to the scan execution unit 411.

[0035] The scan execution unit 411 receives a scan request including a scan setting from the scan instruction unit 422. Based on the scan request, the scan execution unit 411 generates scan image data by controlling the scanner unit 222 to read a document placed on a platen glass via the scanner I / F 217. The generated scan image data is transmitted to the image data storage unit 412. The scan execution unit 411 transmits a scan image identifier uniquely identifying the stored scan image data to the scan instruction unit 422. An example (not shown) of the scan image identifier includes a number, a symbol, and a letter to uniquely identify an image scanned by the MFP 110. The image data storage unit 412 stores the scan image data received from the scan execution unit 411 into the HDD 214.

[0036] The scan instruction unit 422 also obtains scan image data corresponding to the scan image identifier received from the scan execution unit 411 from the image data storage unit 412. The scan instruction unit 422 then requests the cooperative service request unit 423 to instruct the MFP cooperative service 120 to perform a desired processing on the obtained scan image data.

[0037] The cooperative service request unit 423 requests various types of processing from the MFP cooperative service 120. For example, the cooperative service request unit 423 requests a login, an analysis of a scan image, and a transmission of a scan image. The cooperative service request unit 423 communicates with the MFP cooperative service 120 by using a protocol such as a Representational State Transfer (REST) protocol or a Simple Object Access Protocol (SOAP), but other communication means can be used.

[0038] The MFP cooperative service 120 includes a request control unit 431, an image processing unit 432, a cloud storage access unit 433, a data management unit 434, and a display control unit 435.

[0039] The request control unit 431 is in a standby state in a state in which a request can be received from an external device. Upon receiving a processing request, the request control unit 431 appropriately instructs the image processing unit 432, the cloud storage access unit 433, and the data management unit 434 of the processing.

[0040] The image processing unit 432 performs recognition processing and editing processing on an image. Examples include text region analysis processing on an image, character recognition processing on an analyzed text region (hereinafter referred to as optical character recognition (OCR) processing), similar form determination processing (to be described below in the processing of steps S507, S508, and S509), and image rotation and skew correction processing. Figure 5

[0041] The cloud storage access unit 433 requests processing from the cloud storage 130. Cloud services generally publish various I / F for storing a file in a cloud storage or obtaining a stored file by using a protocol such as REST and SOAP. The cloud storage access unit 433 operates the cloud storage 130 by using the published I / F of the cloud storage 130.

[0042] The data management unit 434 stores user information and various types of setting data to be managed by the MFP cooperation service 120.

[0043] The display control unit 435 receives a request from a web browser running on another terminal (not shown) such as a PC and a mobile terminal connected via the Internet, and returns screen configuration information such as a Hyper Text Markup Language (HTML) and a Cascading Style Sheet (CSS) file to be used for screen display. The user checks user information registered in the MFP cooperation service 120 and changes scan settings via a screen displayed by the web browser.

[0044] Although Figure 4 An example of a configuration in which the additional function part 420 is installed on the MFP 110 is exemplified, but such a configuration is not restrictive. For example, the client PC 111 can include a program for realizing the functions of the additional function part 420.

[0045] <Overall processing procedure>

[0046] Figure 5 is a sequence chart exemplifying a processing procedure between devices in generating a file of an image scanned by the MFP 110 and transferring the file into the cloud storage 130. Here, mainly the transaction between devices will be described. In Figure 5 ​In the present embodiment, the MFP 110 is exemplified as performing the business with the MFP collaboration service 120. However, the client PC 111 can be configured to obtain the analysis result instead of the MFP 110 (step S513), provide the screen display (step S514), and give the learning instruction (step S515).

[0047] In a normal state, the MFP 110 displays a main screen including an array of buttons for executing various functions on the touch panel. If an additional application for transmitting a scanned form to the cloud storage 130 (hereinafter referred to as a scan application) is installed on the MFP 110, a button for using the function of the scan application is displayed on the main screen of the MFP 110. If the user presses this button, the CPU 311 of the MFP 110 executes the scan application, thereby displaying a screen for transmitting a scanned form to the cloud storage 130, and starts the process shown in the sequence diagram of FIG. 6. Figure 5

[0048] In step S501, the scan application (MFP 110) displays a login screen for letting the user input authentication information (a user name and a password) for accessing the MFP collaboration service 120. If the user inputs the authentication information and presses a login button, a login request is transmitted to the MFP collaboration service 120 in step S502. In step S503, the MFP collaboration service 120 verifies whether the authentication information (the user name and the password) included in the received login request is correct. If the authentication information is correct, the MFP collaboration service 120 returns an access token to the MFP 110. Various requests issued by the MFP 110 to the MFP collaboration service 120 are then transmitted with this access token, and the user making the request can be identified based on information about the access token. The user authentication is performed by using a conventional general technique such as basic access authentication, digest access authentication, and OAuth-based authentication.

[0049] ​After the login processing is completed, in step S504, the MFP 110 displays a scan setting screen. If the user makes various settings related to scanning, places a paper form to be scanned on a platen glass or an ADF, and presses a "start scan" button, then in step S505, the MFP 110 performs scanning to generate electronic image data on the paper form. In step S506, the MFP 110 transmits a scan image analysis request to the MFP federation service 120 together with the generated scan image data. In step S507, the MFP federation service 120 that received the scan image analysis request controls the image processing unit 432 of the MFP federation service 120 to start image analysis on the scan image data (scan image). Here, the MFP federation service 120 returns an identifier "processId" for uniquely identifying the analysis requested by the MFP federation service 120 to the MFP 110 without waiting for the end of the image analysis processing.

[0050] In the image analysis processing performed by the image processing unit 432, first, in step S508, the image processing unit 432 analyzes a text region included in the scan image. In step S509, using layout information on the analyzed text region in the scan image, the image processing unit 432 compares layout information on text regions in past scanned images with layout information on text regions in the current scan image, and determines whether there is an image in which text regions are in a similar layout (a similar form). This processing will be referred to as similar form determination. Layout information on text regions in past scanned images to be used in similar form determination is stored and accumulated by the processing of step S517 to be described below. In step S510, the image processing unit 432 performs OCR processing on the analyzed text region based on the determination result of step S509. The details of the OCR processing in step S510 will be described below with reference to FIG. 6. Figure 7 The details of the OCR processing in step S510 will be described below with reference to FIG. 6.

[0051] In step S511, using the identifier "processId" received as a response to the scanned image analysis request issued in step S506, MFP 110 periodically (e.g., every few hundred milliseconds to approximately every few milliseconds) transmits a request to MFP Collaboration Service 120 to check the processing status of the image analysis corresponding to the identifier "processId". Although not shown in the figure, the processing in step S511 continues to be executed periodically until a response instructing MFP Collaboration Service 120 to complete the image analysis is received (until the timing of step S512). MFP Collaboration Service 120 receives the request to check the processing status from MFP 110, checks the status of the requested processing corresponding to the identifier "processId", and returns a response. The response contains a string indicating the current processing status in its "status" field. For example, if the string in the "status" field is "processing", then the response indicates that MFP Collaboration Service 120 is processing. If the string in the "status" field is "completed", then the response indicates that MFP Collaboration Service 120's processing is complete. If processing fails, a "failed" status can be returned. A "completed" response upon completion includes the analysis results of the scanned image and information about the scan settings, along with a status indicating that processing is complete.

[0052] In step S512, if a response indicating processing completion is received as a result of checking the processing status, then in step S513, MFP 110 accesses the Uniform Resource Locator (URL) associated with the analysis result information included in the response and obtains the analysis result information about the scanned image. In step S514, MFP 110 displays an attribute setting screen for setting the filename and metadata of the scanned image using the analysis result information obtained in step S513. Figure 6B An example of a property settings screen is shown. The details of the property settings screen are described below. If the client PC 111 displays the property settings screen and sets the filename and metadata, the property settings screen can be displayed through the following procedure. First, the user logs into the MFP Collaboration Service 120 from a web browser on the client PC 111. A list of jobs related to the scanned forms stored in the MFP Collaboration Service 120 is displayed. Figure 6A Based on the user's instructions, the system selects a job related to the desired form. Then, the web browser on the client PC 111 accesses a URL related to the analysis results information of the scanned form image corresponding to the selected job, obtains the analysis results information, and displays an attribute settings screen for setting the file name and metadata of the form image.

[0053] If the user presses the "OK" button 620 after setting the file name and metadata of the scan form on the attribute setting screen, the MFP 110 transmits information on the text area for setting the file name and metadata to the MFP Collaboration Service 120 along with a learning request in step S515. If the user has corrected the character string extracted by the OCR processing on the attribute setting screen by using the keyboard, information on the corrected character string (information on the correction content) is also transmitted.

[0054] In step S516, the request control unit 431 of the MFP Collaboration Service 120 that received the learning request requests the image processing unit 432 to perform form learning. In step S517, the image processing unit 432 stores layout information on all the text areas included in the scanned image and information on the text area for the file name and metadata by the user. If the user has corrected the character string extracted on the attribute setting screen by the OCR processing, in step S518, the request control unit 431 requests the image processing unit 432 to learn the correction of the OCR processing result (OCR result). In step S519, the image processing unit 432 performs learning processing based on the correction content of the OCR processing result and stores information on the learning result. Details of step S519 will be described below.

[0055] Then, in step S520, the MFP 110 transmits the scanned image and information on the file name and metadata set on the attribute setting screen as a request to the MFP Collaboration Service 120. Upon receiving the request, the MFP Collaboration Service 120 starts file generation processing based on the received scanned image and returns a response indicating that the request was correctly received to the MFP 110. Upon receiving the response to the transmission, the MFP 110 ends the processing and returns to the display of the scan setting screen in step S504. In step S521, the MFP Collaboration Service 120 obtains information on the file format for transmitting the scanned image to the cloud storage 130 based on the scan setting registered in the MFP Collaboration Service 120 and generates a file of the scanned image based on the format information. Then, in step S522, the MFP Collaboration Service 120 sets the file name received in step S520 as the file name of the file generated in step S521 and transmits the file to the cloud storage 130.

[0056] Figure 6A and Figure 6B is a diagram illustrating an example of a screen displayed on the MFP 110 (or the client PC 111). Figure 6Ais a drawing illustrating an example of a scanned form list screen 600. The scanned form list screen 600 displays a list of jobs (a scanned form list) 601 related to scanned form images that have completed a scan and image analysis process (steps S505 to S510) and are temporarily stored in the MFP federation service 120 before being transferred to the cloud storage 130. In addition to the scanned form list 601, the scanned form list screen 600 also includes a send button 602, an edit button 603, and a delete button 604.

[0057] The scanned form list 601 includes a form name 605, a destination 606, a status 607, and a type 608. The form name 605 indicates an identifier of a name for uniquely identifying a form. The destination 606 indicates a name of the cloud storage 130 to which a file of a form image is transferred. The status 607 indicates a result of a similar form determination performed on the form image. The status 607 indicates "Not learned" or "Learned". "Not learned" means that a determination is made that there is no similar form. "Learned" means that a determination is made that there is a similar form. The type 608 indicates a type of a form determined automatically based on an OCR result of a string in a header in the form image, and indicates a type of a similar form if a similar form is found by the similar form determination. Examples include "Quotation" and "Invoice". If the status 607 is "Learned", a detailed type indicating an invoice format to which the form corresponds, such as "Invoice AAA" or "Invoice BBB", is also displayed. The detailed type is a type associated with a form determined as the most similar by the similar form determination process.

[0058] The send button 602 is a button for transferring a form image to the cloud storage 130. If a job related to a form image is selected from the scanned form list 601 by a user's operation and the send button 602 is pressed, the MFP federation service 120 transfers the selected form image to the cloud storage 130 set in the destination 606. If the transfer is normally completed, the job related to the form image is deleted from the scanned form list 601.

[0059] The edit button 603 is a button for transitioning to an attribute setting screen 610 to be described below. If a job related to a form image is selected from the scanned form list 601 by a user's operation and the edit button 603 is pressed, the screen transitions to the attribute setting screen 610 for setting attributes related to the selected form image.

[0060] The delete button 604 is a button for deleting a form. If a job related to a form image is selected from the scanned form list 601 by a user's operation and the delete button 604 is pressed, the job related to the selected form image is deleted.

[0061] Figure 6B is a diagram illustrating an example of the attribute setting screen 610 for setting attribute information such as a file name and metadata for a form image. An attribute area 611 is an area for displaying attribute information set for a form image. The attribute area 611 displays item names of attributes such as "file name" and "company name", and includes text fields and a clip image display area under the respective item names. For example, as will be described below, a text field 618 for inputting attribute information and a clip image 619 related to the text area for inputting attribute information are displayed under the item name "company name". If it is determined in step S509 that there is no similar form (i.e., in the case of a form that has not been learned), the attribute setting screen 610 is displayed without selecting any text area (in a state where the text field is empty and no clip image is set). On the other hand, if it is determined in step S509 that there is a similar form, the character recognition result of the text area at the corresponding position of the form image to be processed and the clip image of the text area are automatically displayed in the text field and the clip image display area based on the position information about the text area selected in the past in the similar form.

[0062] A preview area 612 displays a preview image of the scanned form.

[0063] If attribute information about the respective items in the attribute area 611 is set by using the client PC 111, the setting can be made as follows. For example, if the user selects the item "company name" and then places the mouse over a desired text area in the preview image, the mouse-over text area is highlighted (e.g., the text area is displayed with a colored frame) based on the analysis result information obtained in step S513. Here, the string of the character recognition result in the mouse-over text area is displayed in the text field 618, and the clip image of the text area is displayed in the clip image display area 619. The display contents of the text field 618 and the clip image display area 619 are switched accordingly each time the mouse is hovered over a different text area in the preview image. If the user clicks the mouse on the desired text area, the character recognition result and the clip image of the mouse-clicked text area are set as determined input to the text field 618 and the clip image display area 619 under the selected item name. The string input to the text field based on the character recognition result can be subjected to correction processing such as correcting the character recognized erroneously and deleting unnecessary characters by using the keyboard (not shown) of the client PC 111.

[0064] If the attribute information about the item in the attribute area 611 is set through the touch operation on the MFP 110, the user selects a desired attribute item and then touches a desired text area. Then, the character recognition result and the cut image corresponding to the touched text area are set. On the preview image, the text area for setting the attribute information is highlighted.

[0065] The text area deletion button 613 is a button for clearing the text area for setting the attribute information, and is displayed at the upper right of the highlighted text area. If the text area deletion button 613 is pressed, the attribute information set in the attribute area 611 based on the text area is cleared, and the text area stops being highlighted on the preview image.

[0066] The preview enlargement button 614 is an operation button for increasing the magnification of the preview image displayed in the preview area 612. The preview reduction button 615 is an operation button for reducing the magnification of the preview image displayed in the preview area 612. The preview initial display button 616 is an operation button for restoring the initial magnification and the initial display position in a case where the magnification is changed by pressing the preview enlargement button 614 or the preview reduction button 615 or in a case where the display position of the preview image is moved by an operation such as a swipe.

[0067] The OK button 620 is a button for giving an instruction to transmit the attribute information (the file name and the metadata such as the company name) set on the attribute setting screen 610 to the MFP collaboration service 120 and the position information about the text area for setting the attribute information. If the correction processing has been performed on the string of the character recognition result in the text field, the information about the correction content is also transmitted in response to the pressing of the OK button 620. If the OK button 620 is pressed and the transmission of the information is completed, the screen returns to the scanned form list screen 600. The MFP collaboration service 120 receives the information, and performs the learning processing based on the result of the text area analysis processing and the correction of the OCR result (steps S516 to S519).

[0068] Figure 7 is a flowchart illustrating the details of the learning processing based on the correction content of the OCR result if the OCR result is corrected when the attribute information is set on the attribute setting screen 610 (step S519). The details of the processing of a new form image by using the learning result of the learning processing in Figure 9 will be described below. Figure 7

[0069] ​If the user corrects the character recognition result string displayed in the text field on the attribute setting screen 610, information about the correction content is transmitted from the client PC 111 (or the MFP 110) in step S515. In this case, in step S701, the image processing unit 432 of the MFP collaboration service 120 obtains the information about the correction content. For example, assume that, as shown in Figure 6B the text field 618 displays "Shimomaruko Inc" as the OCR processing result of the text region 617 selected on the preview image. Also assume that the user determines that the part "Inc" is not needed, and deletes the part and leaves the string "Shimomaruko", as shown in the text field 813 of Figure 8 If such a deletion operation is performed on the string, the attribute setting screen 810 can be controlled to highlight the text region 811 corresponding to the string of the OCR result remaining in the text field 813 and display the cutout image 812 corresponding to the text region 811. If the deletion operation is performed, the client PC 111 (or the MFP 110) determines that the string of the OCR processing result is corrected, and transmits information about the deleted string as the information about the correction content. Although the MFP collaboration service 120 is described here to obtain the information about the correction content transmitted from the client PC 111 (or the MFP 110), this is not limiting. For example, the client PC 111 (or the MFP 110) can be configured to transmit the uncorrected string and the corrected string of the OCR result, so that the MFP collaboration service 120 obtains the correction content by extracting the difference between the uncorrected string and the corrected string to recognize the deleted string.

[0070] In step S702, the image processing unit 432 determines whether the correction content is the deletion of a part of the string of the OCR result and the deleted character constitutes a string registered in a general term dictionary. The general term dictionary refers to a dictionary in which strings other than proper nouns, such as "Inc" and "invoice", are registered. For example, if "Inc" is deleted from "Shimomaruko Inc" as shown in Figure 8 the correction content is determined to be the deletion of the string "Inc", and "Inc" is determined to be a string registered in the general term dictionary. In this case (step S702 is "Yes"), the process proceeds to step S703. In step S703, the image processing unit 432 generates a regular expression based on the original string and the deleted string, and learns the correction content of the OCR processing result. For example, if "Inc" is deleted from "Shimomaruko Inc" as shown in Figure 8"Shimomaruko Inc" is deleted, then the image processing unit 432 learns the correction content of the OCR processing result by registering the regular expression ".*Inc" and the target form number (a number assigned to a form in the same format) as shown in Table 1 below. The form number determined during the similar form determination in step S509 is used as the target form number. Since the same form number is assigned to forms in which the text region is in the same layout (forms determined to be in the same format), it is determined whether to reflect the learning result based on the form number when performing the OCR processing on a new form image, as described below.

[0071] Table 1

[0072] Number Regular expression Target form number 1 .*Inc 1

[0073] In Figure 8 In the example described above, the regular expression (such as shown in Table 1) is generated when the latter half of the string of the OCR result is deleted. However, this is not limiting. For example, if a character in the first part or in the middle of the string of the OCR result is deleted and the string to be deleted is registered in the general term dictionary, then a regular expression expressing the correction content is similarly generated.

[0074] In step S704, the image processing unit 432 analyzes the uncorrected string and the corrected string of the OCR processing result and learns the correction of the OCR processing result based on the analysis result. For example, if the OCR processing result of the selected text region is the string "AL" and the user corrects the string to "AI" (the string includes a character "I" that was misrecognized as "L" and the user corrected the misrecognition), then the image processing unit 432 registers "AL" as the target string (replaces the previous string) and registers "AI" as the replaced string as shown in Table 2 below. Similar to Table 1, the form number determined during the similar form determination in step S509 is used as the target form number.

[0075] Table 2

[0076] Number Target string Replaced string Target form number 1 AL AI 2

[0077] As described above, by the learning processing of steps S703 and S704, the regular expression information shown in Table 1 and the replaced string information shown in Table 2 are stored. If a new form image in the same format is subsequently scanned, then when performing the character recognition processing of step S510 on the new form image, the OCR result can be automatically corrected by using the information (learning data) shown in Tables 1 and 2.

[0078] Figure 9is a detailed flowchart of the process of correcting the OCR result by using the learning data shown in Tables 1 and 2 above when performing the character recognition process of the new form image in step S510.

[0079] In step S901, the image processing unit 432 of the MFP cooperation service 120 obtains a character string as a character recognition result by performing a character recognition (OCR) process on a partial image corresponding to the respective text region analyzed in step S508.

[0080] In step S902, the image processing unit 432 determines whether there is a character string that can be corrected by comparing the character string of the OCR process result obtained in step S901 with the regular expression in Table 1 and the target character string in Table 2. In other words, if the character string of the OCR process result matches the regular expression in Table 1 or the target character string in Table 2, the image processing unit 432 determines that the character string can be corrected (YES in step S902), and then the process proceeds to step S903.

[0081] In step S903, the image processing unit 432 determines whether the new form image to be processed is determined to have a similar form in step S509, and whether the form number of the determined similar form is among the form numbers in at least one of Table 1 and Table 2. In other words, the image processing unit 432 determines whether the form scanned this time to generate the new form image has the same format as the format of the previously OCR-corrected form. If it is determined that the form has the same format as the format of the previously corrected form (YES in step S903), the process proceeds to step S904. In step S904, the image processing unit 432 automatically applies the same correction as the correction made before to the current OCR result. On the other hand, if it is determined that the form is of a different format (NO in step S903), the process proceeds to step S905. In step S905, the image processing unit 432 includes the corrected character string into the OCR result, so that the OCR result can be presented to the user. In this way, it is possible to determine whether to apply the same correction as the correction made before to the current OCR result based on the user's instruction.

[0082] In other words, in step S904, the image processing unit 432 corrects the character string of the character recognition result obtained in step S901. Specifically, if the character string of the character recognition result matches the regular expression in Table 1, the image processing unit 432 deletes the character string portion expressed by the regular expression. If the character string of the character recognition result coincides with the target character string in Table 2, the image processing unit 432 replaces the character string of the character recognition result with the corresponding replacement character string in Table 2. For example, assume that the form scanned in step S508 is a form of Figure 8Similar to form images Figure 10 The form image, and in relation to Figure 8 The text region corresponding to text region 811 is used to obtain "Kosugi Inc" as the initial result of character recognition. In this case, since the string of the character recognition result matches the regular expression ".*Inc" registered in Table 1, the image processing unit 432 outputs the string obtained by deleting "Inc" as the OCR processing result. Specifically, the string "Kosugi" and its corresponding coordinates are output as follows: Figure 11 The OCR processing result data. When displaying the attribute setting screen in step S514 based on this output result, the string "Kosugi" is displayed in text field 1013, such as... Figure 10 As shown in the image, a cut-out portion of the "Kosugi" text is displayed in the cut-out image display area 1012. The corresponding text area 1011 is highlighted on the preview image.

[0083] In step S905, the image processing unit 432 outputs the string of character recognition results obtained in step S901 and the string corrected based on the information in Table 1 or Table 2 as the OCR processing result. For example, suppose that in step S903 it is determined that there are no similar forms in the form image to be processed, and the string obtained by character recognition of the selected text region matches the regular expression ".*Inc" in Table 1. Figure 12 In this case, two strings (i.e., the string "Kawasaki Inc" as the character recognition result of the selected text region 1211, and the string "Kawasaki" obtained by removing the "Inc" part based on the regular expression in Table 1) are output as OCR processing result data. Specifically, as Figure 13 As shown, the string "Kawasaki Inc" and its replaced string candidate "Kawasaki" as the character recognition result of the target text region 1211 are output as OCR processing result data. Then, as... Figure 12 As shown, based on the OCR processing results, the string is displayed as a drop-down list in text field 1212. If the user selects "Kawasaki" in the drop-down list (i.e., selects the replaced string candidate), then the form number associated with the new form image can be additionally registered in the target form number in Table 1 or Table 2, and that form can subsequently be used as a similar form.

[0084] In this exemplary embodiment, in Figure 9In step S903, it is determined whether to automatically replace the character string with the replaced character string or to display the character string as a candidate based on the form number of the similar form. However, other conditions can be used. For example, it can be determined whether to apply the correction contents of Table 1 and Table 2 for each user or for each group to which the user belongs.

[0085] As described above, if the character string of the character recognition result of the text region in the scanned image is corrected when the attribute information such as the file name and the metadata is set by using the character string of the character recognition result, the correction content is registered. At the time of setting the attribute information on the new scanned image, the registered correction content is reflected on the display of the character string of the character recognition result. This can reduce the time and effort of the user for correcting the character string of the character recognition result.

[0086] <Other Embodiments>

[0087] The embodiment(s) of the present disclosure can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on storage media (which can also be referred to more fully as a 'non-transitory computer-readable storage medium') to perform the functions of one or more of the above-described embodiment(s) and / or that includes one or more circuits (e.g., application specific integrated circuits (ASICs)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage media to perform the functions of one or more of the above-described embodiment(s) and / or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer can comprise one or more processors (e.g., central processing units (CPUs), micro processing units (MPUs)) and can include a network of the separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions can be provided to the computer, for example, from a network or the storage media. The storage media can include, for example, one or both of a hard disk, a random access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM), a flash memory device, a memory card, and the like.

[0088] Embodiments of the present invention can also be realized by a method that provides software (a program) to a system or apparatus for performing the functions of the above-described embodiments by a network or various storage media, and a method that reads out and executes the program by a computer of the system or apparatus or a central processing unit (CPU), a micro processing unit (MPU).

[0089] While the present disclosure has been described with reference to example embodiments, it is to be understood that the disclosure is not limited to the disclosed example embodiments. The scope of the claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An information processing apparatus comprising: a character recognition component configured to obtain a character recognition result by performing a character recognition process on a text region in a first scan image; a learning component configured to learn a correction content of a correction made on at least a portion of a character string of the character recognition result if the correction is made in setting attribute information on the first scan image by using the character recognition result obtained by the character recognition component, and a determination component configured to determine whether a second scan image is similar to the first scan image, wherein the character recognition component is configured to, if performing a character recognition process on a text region in the second scan image, a) correct a character recognition result of the text region in the second scan image based on the correction content learned by the learning component and output the corrected character recognition result if it is determined that the second scan image is similar to the first scan image, and b) output both a character recognition result of the text region of the second scan image and a corrected character recognition result corrected based on the correction content learned by the learning component as candidate results if the character recognition result of the text region in the second scan image matches the correction content learned by the learning component. 2.The information processing apparatus according to claim 1, wherein the learning component is configured to learn a regular expression based on a deleted character string as the correction content if the correction made in setting the attribute information on the first scan image is a deletion of the portion of the character string of the character recognition result, and wherein the character recognition component is configured to correct the character recognition result of the text region in the second scan image by deleting a portion of the character string of the character recognition result of the text region in the second scan image that matches the regular expression learned by the learning component and output the corrected character recognition result if performing the character recognition process on the text region in the second scan image and the character string of the character recognition result of the text region in the second scan image matches the regular expression learned by the learning component. 3.The information processing apparatus according to claim 1, wherein the learning component is configured to learn a regular expression based on a deleted character string as the correction content if the correction made in setting the attribute information on the first scan image is a deletion of the portion of the character string of the character recognition result and the deleted character string is a character string registered in a predetermined term dictionary, and wherein the learning component is configured to correct the character recognition result of the text region in the second scan image by deleting a portion of the character string of the character recognition result of the text region in the second scan image that matches the regular expression learned if performing the character recognition process on the text region in the second scan image and the character string of the character recognition result of the text region in the second scan image matches the regular expression learned. 4.The information processing apparatus according to any one of claims 1 to 3, wherein the learning means is configured to learn, as a correction content, a string before the replacement and a replaced string if the correction made in setting the attribute information on the first scanned image is a replacement of the part of the string of the character recognition result, and wherein the character recognition means is configured to correct the character recognition result by replacing the string of the character recognition result of the text region in the second scanned image with the learned replaced string if the character recognition processing is performed on the text region in the second scanned image and the string of the character recognition result of the text region in the second scanned image matches the string before the replacement learned by the learning means, and output the corrected character recognition result.

5. A computer-readable storage medium storing instructions to cause a computer to perform: obtaining a character recognition result by performing character recognition processing on a text region in a first scanned image; learning a correction content of the correction if at least a part of a string of the obtained character recognition result is corrected in setting attribute information on the first scanned image by using the character recognition result; and determining whether a second scanned image is similar to the first scanned image, wherein if character recognition processing is performed on a text region in the second scanned image, a) in a case where it is determined that the second scanned image is similar to the first scanned image, correcting a character recognition result of the text region in the second scanned image based on the learned correction content, and b) in a case where it is determined that the second scanned image is not similar to the first scanned image, if the character recognition result of the text region in the second scanned image matches the learned correction content, outputting both the character recognition result of the text region of the second scanned image and a corrected character recognition result corrected based on the learned correction content as a candidate result.

6. The computer-readable storage medium according to claim 5, wherein learning a regular expression based on a deleted string as a correction content if the correction made in setting the attribute information on the first scanned image is a deletion of the part of the string of the character recognition result, and wherein if the character recognition processing is performed on the text region in the second scanned image and the string of the character recognition result of the text region in the second scanned image matches the learned regular expression, the character recognition result is corrected by deleting the part of the string of the character recognition result of the text region in the second scanned image that matches the regular expression.

7. The computer-readable storage medium according to claim 5, wherein learning a regular expression based on a deleted string as a correction content if the correction made in setting the attribute information on the first scanned image is a deletion of the part of the string of the character recognition result and the deleted string is a string registered in a predetermined term dictionary, and wherein, if the character recognition processing is performed on the text region in the second scan image and the string of the character recognition result of the text region in the second scan image matches the learned regular expression, the character recognition result is corrected by deleting the portion of the string of the character recognition result of the text region in the second scan image that matches the regular expression.

8. The computer-readable storage medium according to any one of claims 5 to 7, wherein, if the correction made in setting the attribute information on the first scan image is replacement of the portion of the string of the character recognition result, the string before replacement and the replaced string are learned as the correction content, and wherein, if the character recognition processing is performed on the text region in the second scan image and the string of the character recognition result of the text region in the second scan image matches the learned string before replacement, the character recognition result is corrected by replacing the string of the character recognition result of the text region in the second scan image with the learned replaced string.

9. An information processing method to be executed by an information processing apparatus, the information processing method comprising: obtaining a character recognition result by performing character recognition processing on a text region in a first scan image; if a correction is made on at least a portion of a string of the obtained character recognition result in setting attribute information on the first scan image by using the character recognition result, learning a correction content of the correction; and determining whether a second scan image is similar to the first scan image, wherein, if the character recognition processing is performed on a text region in the second scan image, a) in a case where it is determined that the second scan image is similar to the first scan image, the character recognition result of the text region in the second scan image is corrected based on the learned correction content; and b) in a case where it is determined that the second scan image is not similar to the first scan image, if the character recognition result of the text region in the second scan image matches the learned correction content, both the character recognition result of the text region of the second scan image and the corrected character recognition result corrected based on the learned correction content are output as candidate results.

10. The information processing method according to claim 9, wherein if the correction made in setting the attribute information on the first scan image is deletion of the portion of the string of the character recognition result, a regular expression based on the deleted string is learned as the correction content, and wherein, if the character recognition processing is performed on the text region in the second scan image and the string of the character recognition result of the text region in the second scan image matches the learned regular expression, the character recognition result is corrected by deleting the portion of the string of the character recognition result of the text region in the second scan image that matches the regular expression.

11. The information processing method according to claim 9, wherein If the correction made in setting attribute information about the first scan image is deletion of the part of the string of the character recognition result and the deleted string is a string registered in a predetermined term dictionary, a regular expression based on the deleted string is learned as the correction content, and wherein if the character recognition process is performed on the text region in the second scan image and the string of the character recognition result of the text region in the second scan image matches the learned regular expression, the character recognition result is corrected by deleting the part of the string of the character recognition result of the text region in the second scan image that matches the regular expression.

12. The information processing method according to any one of claims 9 to 11, wherein If the correction made in setting attribute information about the first scan image is replacement of the part of the string of the character recognition result, the string before replacement and the replaced string are learned as the correction content, and wherein if the character recognition process is performed on the text region in the second scan image and the string of the character recognition result of the text region in the second scan image matches the learned string before replacement, the character recognition result is corrected by replacing the string of the character recognition result of the text region in the second scan image with the learned replaced string.

Images