Information processing device, control method for information processing device, processing program, and system

Abstract

To solve such a problem that the stand-by time of a user is long if a series of processing to reception of a confirmation result of the user through a UI from recommendation is simply repeated when recommending a file name or the like by scanning a plurality of documents at once and dividing the documents to different files.SOLUTION: An image processing device acquires image data of a plurality of documents, divides the image data to create a plurality of files, executes OCR-associated processing, performs matching between the document of the file and a registered document and determines the type of the document. The image processing device generates a recommendation content about an index from the document of the file and presents it to a user in accordance with an index extraction rule linked to the registered document, executes correction processing of the index and the index extraction rule on the basis of a correction instruction of the user, and gives the index to the file. The image processing device performs parallel processing of the first processing including the OCR-associated processing and the second processing including the correction processing.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image processing device having a function of dividing multiple documents into files when scanned and recommending file names for each document, a control method for the image processing device, and a program therefor. [Background technology]

[0002] There are workflows in which paper documents such as forms are scanned and digitized using an image scanning device. In this process, the digitized document images are typically given file names and meta information according to the document type and content, and then stored in a designated folder. Manually assigning file names, adding meta information, and specifying destination folders can become a significant burden as the number of paper documents to be digitized increases. In particular, with multifunction peripherals (MFPs), the burden on users increases when the user interface (UI) requires inputting file names and meta information using a software keyboard. Specifying destinations and creating new folders can also be a significant burden on users.

[0003] A method for solving this problem is disclosed in Patent Document 1. In this method, first, document types are associated with index extraction rules such as file naming rules, meta information rules, and destination rules and are registered in advance. Next, when documents of the same type are scanned, the document type is recognized, and the associated index extraction rules are used to automatically determine the file name, meta information, and destination, and recommend them to the user. This method significantly reduces the user's workload. Furthermore, the technique of Patent Document 2 not only recommends registered documents but also provides a confirmation UI that allows users to modify the recommendation results and to name unregistered documents. This makes it possible for the user to automatically learn index extraction rules simply by repeatedly performing naming and modification operations, without the need for initial setting of index extraction rules. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-15348 [Patent Document 2] Japanese Patent Application Publication No. 2018-124656 Summary of the Invention [Problem to be solved by the invention]

[0005] On the other hand, there is a demand for a system that can scan multiple documents at once, split them into separate files, and recommend file names, meta information, and delivery destinations for each file. However, there are challenges in automatically learning index extraction rules while recommending multiple files, as in the technology of Patent Document 2. For example, if the series of processes described in Patent Document 2, from recommendation to receiving user confirmation results through a UI, is repeated for the number of files, a waiting time equivalent to the processing time for the recommendation occurs each time before confirming each file. Furthermore, if image processing related to the recommendation is performed in the backend of the user's confirmation operation to reduce waiting time, the newly registered index extraction rule cannot be applied to other documents in the same scan. In this case, even if the same scan contains unregistered documents of the same type or documents of the same type that require changes to the index extraction rule, the same operations must be repeated, which is time-consuming for the user. The present invention has been made in consideration of the above circumstances, and aims to reduce the user's effort when recommending multiple files while automatically learning index extraction rules. [Means for solving the problem]

[0006] An image processing device comprising: an acquisition means for acquiring image data of a plurality of documents; a creation means for dividing the image data to create a plurality of files; an OCR-related processing means for performing OCR-related processing on the files; a determination means for matching the document of the file with a registered document and determining the document type; a generation processing means for generating recommended content related to an index from the document of the file in accordance with an index extraction rule linked to the registered document; a correction processing means for presenting the recommended content to a user and performing correction processing of the index and index extraction rule based on a correction instruction from the user; and an assignment means for assigning an index to the file, wherein a first processing means including the OCR-related processing means and a second processing means including the correction processing means are started in parallel. [Effects of the Invention]

[0007] This invention reduces the user's workload by dividing multiple documents into files when scanning them and recommending index items such as file names for each document. In particular, when multiple documents of the same type are found in the same scan, the rules registered or modified by the user can be immediately applied, reducing the user's workload. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a system according to an embodiment. [Figure 2] 1 is a diagram showing a hardware configuration of an image processing apparatus according to an embodiment; [Figure 3] Flowchart of image processing procedure according to the embodiment [Figure 4] FIG. 10 is a diagram illustrating an example of a process for determining a file division position according to an embodiment. [Figure 5] 1 is a flowchart of a processing procedure for OCR-related processing according to an embodiment. [Figure 6] FIG. 10 is a diagram showing an example of block selection processing according to an embodiment; [Figure 7] Flowchart of confirmation and correction processing according to the embodiment [Figure 8] 10 is an example of an image of the first page after file division according to an embodiment of the present invention. [Figure 9A] Example of index extraction rules according to the embodiment [Figure 9B] Example of index extraction rules according to the embodiment [Figure 10] 1 is a flowchart of a process for generating recommendation content according to an embodiment; [Figure 11A] Example of file name specification using the confirmation / edit screen related to the embodiment [Figure 11B] Example of file name specification using the confirmation / edit screen related to the embodiment [Figure 12] Example of file name recommendation using the confirmation / edit screen according to the embodiment [Figure 13] 10 is a flowchart of a processing procedure according to another embodiment. [Figure 14] Flowchart of confirmation and correction processing according to another embodiment [Figure 15A] Example of a second index extraction rule according to another embodiment [Figure 15B] Example of a second index extraction rule according to another embodiment [Figure 16] 10 is a flowchart of a process for generating recommendation content according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. Note that the following examples do not limit the scope of the present invention, and not all of the combinations of features described in the examples are necessarily essential to the solution of the present invention. [Example]

[0010] <System configuration> Fig. 1 is a diagram showing the overall configuration of a system to which this embodiment can be applied. As shown in Fig. 1, an image processing device 100 is connected to a LAN 102 and is capable of communicating with a terminal 101 such as a PC via the Internet 103 or the like. Note that, in realizing this embodiment, the terminal 101 such as a PC is not necessarily required, and the configuration may be such that only the image processing device 100 is included. The image processing device 100 is, for example, a multifunction peripheral (MFP) having a display / operation unit, a scanner unit, and a printer unit, and can be used as a scanning terminal that scans document manuscripts using the scanner unit. It also has a display / operation unit such as a touch panel and hard buttons, and displays a user interface for displaying recommended file names and storage destinations and accepting instructions from the user. In this embodiment, an example will be described in which the image processing device 100 alone performs the scanning process, file division process, OCR-related process, recommended content generation process, and file transmission, but a system configured to perform some of these processes on another terminal 101 may also be used. For example, first, the scanning process and file division process are performed on the image processing device 100, and the scanned image is transmitted to the other terminal 101 via a network. Then, the OCR-related process and recommended content generation process are performed on the other terminal 101, and the recommendation results are returned to the image processing device 100, and the image processing device 100 generates a file and transmits the file.

[0011] <Hardware configuration of image processing device> 2 is a block diagram showing the configuration of the image processing device 100. In the control unit 110, a CPU 111, a storage device 112, a printer I / F unit 113, a network I / F unit 114, a scanner I / F unit 115, and a display / operation I / F unit 116 are connected to each other via a system bus 117 so that they can communicate with each other. The control unit 110 controls the operation of the entire image processing device 100. The CPU 111 reads and executes a control program stored in the storage device 112, thereby functioning as a unit for executing each process (such as reading control, image processing, and display control) in the flowcharts described below. The storage device 112 stores and holds the above programs, image data, metadata, setting data, processing result data, etc. The storage device 112 includes a ROM 118 which is a non-volatile memory, a RAM 119 which is a volatile memory, and an HDD 120 which is a large-capacity storage area. The ROM 118 is a non-volatile memory that stores control programs and the like, and the CPU 111 reads out the control programs and performs control. The RAM 119 is a volatile memory used as a temporary storage area such as a main memory or work area for the CPU 111 . The network I / F unit 114 connects the control unit 110 to the LAN 102 via a system bus 117. The network I / F unit 114 transmits image data to external devices on the LAN 102 and receives various types of information from external devices on the LAN 102.

[0012] Scanner I / F unit 115 connects scanner unit 122 and control unit 110 via system bus 117. Scanner unit 122 reads a document original to generate scanned image data, and inputs the scanned image data to control unit 110 via scanner I / F unit 115. Scanner unit 122 is equipped with a document feeder, and is capable of feeding multiple documents placed on a tray one by one, thereby enabling continuous reading. The display / operation unit I / F unit 116 connects the display / operation unit 123 and the control unit 110 via a system bus 117. The display / operation unit 123 is provided with a liquid crystal display unit having a touch panel function, hard buttons, and the like. The printer I / F unit 113 connects the printer unit 121 and the control unit 110 via a system bus 117. The printer unit 121 receives image data generated by the CPU 111 via the printer I / F unit 113, and performs a print process on recording paper using the received image data. As described above, the image processing device 100 according to this embodiment can provide image processing functions with the above hardware configuration.

[0013] <Processing procedure executed by the image processing device> In this embodiment, the image processing device 100 scans multiple document manuscripts and divides the image data into multiple files according to the file division settings. Then, file names to be assigned to each file are generated using the character strings of the OCR results on the first page of each file based on the index extraction rules, and the recommended content is presented to the user via the display and operation unit 123. The user checks whether the file names and the recommended content are correct on the display and operation unit 123, and if they are incorrect, instructs the user to correct them. If any corrections are made based on the user's operation results, the index extraction rules are updated. After the user has completed the confirmation and correction process for all the divided files, file names are assigned to each file and the files are sent. Next, the processing procedure executed by the image processing device 100 in this embodiment will be described in detail with reference to the flowchart in Fig. 3. The CPU 111 of the image processing device 100 loads a processing program stored in the ROM 118 into the RAM 119 and executes it, thereby functioning as a processing unit that executes the processing of each step in Fig. 3. In step S301, when the image processing apparatus 100 receives a user instruction from the display / operation unit 123, the scanner unit 122 reads a plurality of document originals one by one from the tray of the original feeder, and acquires image data. In step S302, the CPU 111 acquires file division settings from the storage device 112. The file division settings may be acquired in advance as a user instruction received from the display / operation unit 123, or may be predetermined fixed settings.

[0014] In step S303, CPU 111 determines the division positions of the multiple pieces of image data acquired in step S301 based on the file division setting acquired in step S302. Then, the image data acquired in S301 is grouped according to the division positions. This group is image data that will be combined into one file when creating a file, which will be described later, and one group contains at least one piece of image data. Therefore, the image data is divided to generate multiple files. Examples of file division settings and division position determination will be described later using FIG. 4. In step S304, the CPU 111 starts the confirmation and correction process shown in step S309 in parallel, and the process proceeds to step S305. After the OCR-related processing in the following steps S305 to S307 is completed, the processing in step S309 is executed in parallel for the files to which the unconfirmed attribute has been assigned. In step S305, CPU 111 selects one of the files for which OCR-related processing has not been performed from the image group determined in step S303, in order of earliest read time. In step S306, CPU 111 executes OCR-related processing on the image of the first page of the file selected in step S305. Details of the OCR-related processing will be described later. In step S307, CPU 111 assigns an unconfirmed attribute to the file selected in step S305 and subjected to OCR-related processing in step S306. The unconfirmed attribute indicates that the file is awaiting processing for the confirmation / correction processing started in parallel in step S304.

[0015] In step S308, CPU 111 determines whether the OCR-related processing in step S306 has been completed for all image groups determined in step S303. If the OCR-related processing has not been completed for all image groups, the process returns to step S305. If the OCR-related processing has been completed for all image groups, the process proceeds to step S310. Meanwhile, in step S309, which is processed in parallel, the CPU 111 presents the file names, which are the recommended contents, to the user for all the image groups determined in step S303. Thereafter, a confirmation / correction process is executed to confirm the file names based on the confirmation / correction instructions of the user. The confirmation / correction process will be described in detail later. In step S310, the CPU 111 determines whether the checking and correcting process has been completed for all image groups determined in step S303. The CPU 111 waits until the checking and correcting process has been completed for all image groups, and if so, the process proceeds to step S311. In step S311, CPU 111 displays on display / operation unit 123 a list of the file names that have been confirmed, corrected, and determined in step S309. In step S312, CPU 111 creates a file from each of the image groups determined in step S303. Each file is given a file name determined after confirmation and correction by the user in step S309. In this embodiment, an example is shown in which the files are converted to PDF (Portable Document Format) and saved. In the case of PDF, it is possible to save images divided into pages, and each piece of image data in the image group determined in step S303 is saved as a separate page in a single file. In step S313, the CPU 111 transmits the file created in step S312 to the destination via the LAN 102.

[0016] <Division position determination process> Next, the division position determination process in step S303 will be described. Figure 4 shows an image of the page splitting process for each file split setting. 4A shows an example of the division position when page number division is set as the file division setting. Page number division is a method of determining the division position for each preset number of pages. The number of pages is specified by the user via the display / operation unit 123, or a fixed number of pages is specified in advance according to the scan settings. For example, if image data for six pages is acquired in step S301 and division into two pages is set, the division positions are set between the second and third pages and between the fourth and fifth pages. Figure 4(B) shows an example of a file split setting that uses divider pages. Divider page splitting is a method that analyzes image data, and if it recognizes a divider page, removes the divider page and sets the split position at the location where the divider page was located. The user must insert a divider page between the documents they want to split, and then place the documents in the document feeder. For example, if image data for six pages is acquired in step S301 and pages three and five are divider pages, the image data for pages three and five are removed first, and the division positions are set between the images for pages two and four, and between the images for pages four and six. Divider pages can be blank paper with nothing printed on it, or paper with a barcode or two-dimensional code printed on it. When blank paper is used as a divider page, the divider page is recognized by determining whether the paper is blank based on the number of pixels below a certain pixel value contained in the image data or whether a colored pixel block with a certain area or more is included. When using paper with a barcode or two-dimensional code printed on it, the barcode or two-dimensional code is first detected from the image data. The detected barcode or two-dimensional code is then decoded to determine whether the result contains information identifying the divider page. Divider page paper with a barcode or two-dimensional code may be printed using a specific PC application. Alternatively, an application within the image processing device 100 may receive information such as the number of sheets and paper size from the display / operation unit 123 and print the pages using the printer unit 121.

[0017] Figure 4(C) shows an example of a file split setting that includes cover page splitting. Cover page splitting is a method of analyzing image data, recognizing a cover page, and then setting the split position between the page immediately before that page. As an example, in step S301, when image data for six pages is acquired and the first page, the third page, and the fifth page are recognized as cover pages, the split positions are set between the second page and the third page and between the fourth page and the fifth page. As a method for recognizing the display page, for example, the image data of the cover page is registered in advance and recognized by comparing the images. The image data of the cover page may be registered by using an application in the image processing apparatus 100 to read and register a document manuscript that will be the cover page with the scanner unit 122, or may be registered from the outside through the LAN 102 and the network I / F unit 114. In the image comparison, it is recognized whether it is the same as the registered cover page based on the degree of coincidence of the local feature amounts calculated from the image. Note that the recognition of the cover page is not limited to image comparison. As another method, for example, layout features such as the arrangement of characters and ruled lines included in the document in the image may be extracted and recognized by comparing the layout features. Also, instead of registering the cover page in advance, it may be recognized based on whether a specific character string, barcode, or two-dimensional code is included in the cover page.

[0018] <OCR-related processing> Next, the details of the OCR-related processing in step S306 will be described with reference to FIG. 5. FIG. 5 is a flowchart showing a processing procedure for executing OCR-related processing including OCR and its preprocessing for one sheet of image data. In step S501, the CPU 111 detects the angle of inclination from the image data, and obtains image data with the inclination corrected by rotating the image in the reverse direction by the detected inclination. The inclination to be corrected is an inclination such that when reading by the scanner unit 122, the original manuscript cannot be read straight due to wear of the rollers in the original feeder, or cannot be printed straight during printing of the original manuscript. Tilt detection first detects objects contained in the image data and connects groups of adjacent objects horizontally or vertically. The tilt is then calculated by determining the angle formed by connecting the centers of the connected objects relative to the horizontal or vertical direction. Note that tilt detection is not limited to the above method. For example, the center coordinates of objects contained in the image data can be obtained, and the center coordinates can be rotated in 0.1-degree increments to determine the angle at which the center coordinates are most closely aligned horizontally or vertically. Correcting the tilt of an image through this tilt correction can improve the accuracy of the rotation correction, block selection processing, and OCR processes described below.

[0019] In step S502, CPU 111 obtains an image by rotating the tilt-corrected image obtained in step S501 in 90-degree increments so that the characters in the document are oriented upright. Using the tilt-corrected image obtained in step S501 as a reference image, four images are prepared: the reference image, an image rotated 90 degrees, an image rotated 180 degrees, and an image rotated 270 degrees. Then, a simple OCR process that allows high-speed processing is performed on each image, and the image with the largest number of characters recognized with a certain degree of confidence above a certain value is obtained as the rotation-corrected image. Note that the method of rotation correction is not limited to the above. In step S503, the CPU 111 performs block selection processing on the rotation-corrected image acquired in step S502. The block selection processing is a determination process in which the image is classified into a foreground region and a background region, and the foreground region is divided into text blocks and other blocks.

[0020] Specifically, contour tracing is performed on the query image, which has been binarized to black and white, to extract clusters of pixels surrounded by black pixel contours. For clusters of black pixels larger than a certain area, contour tracing is also performed on the white pixels within the cluster to extract clusters of white pixels. Furthermore, clusters of black pixels are recursively extracted from within clusters of white pixels larger than a certain area. The clusters of black pixels thus obtained are designated as foreground regions, and are then classified by size and shape into regions with different attributes. For example, pixel clusters with an aspect ratio close to 1 and within a certain range of size are designated as character clusters, and areas where adjacent characters can be grouped in a well-aligned order are designated as character regions (TEXT). Flat pixel clusters are designated as line regions (LINE). The area occupied by black pixel clusters larger than a certain size and containing well-aligned rectangular white pixel clusters is designated as table regions (TABLE). Regions with scattered clusters of irregularly shaped pixels are designated as photo regions (PHOTO). Finally, pixel clusters of any other shape are designated as picture regions (PICTURE). From among the regions thus divided for each attribute of the object, blocks determined to have character attributes are identified as text blocks.

[0021] FIG. 6 shows an example of the results of the block selection process. FIG. 6(A) shows the image after rotation correction, and FIG. 6(B) shows the results of the block selection process for the query image. The information about each text block obtained in this step (information indicating the attributes and position of each block) is used in the OCR process and similarity calculations described below. Note that the reason why only text blocks are extracted in this step is because the position of character strings effectively represents the structure of a document image and is closely related to scan assist information. Therefore, this does not preclude the use of information about blocks determined to have other attributes, such as photo areas or table areas, in subsequent processing. Returning to the explanation of the flowchart in FIG. 5, in step S504, CPU 111 executes OCR on the text blocks acquired in step S503, and acquires character strings corresponding to each text block as a result.

[0022] <Confirmation and correction process> The confirmation and correction process (S309 in FIG. 3) will be described in detail. In this embodiment, the confirmation and correction process (S309) is started in step S304 in FIG. 3 and operates in parallel with the process flow (S305 to S308) in FIG. 3. The confirmation and correction process is a process in which a recommended file name to be assigned to each divided file is presented to the user and confirmed based on the user's instructions. FIG. 7 is a flowchart showing the processing steps of the confirmation and correction process. FIG. 8 shows image data of the first page of each image group, and the order of reading in FIGS. 8(A), 8(B), and 8(C) is the order in which they were read. FIGS. 8(A) and 8(B) are documents of the same type but with some differences in content, while FIG. 8(C) is a document of a different type. Below, the behavior of the confirmation and correction process when the image data in FIG. 8 is read will be described using the flowchart in FIG. 7.

[0023] In step S701, CPU 111 checks whether any of the image groups scanned in step S301 of Fig. 3 and determined in step S303 have been assigned an unconfirmed attribute. An unconfirmed attribute refers to an attribute assigned in step S307 of Fig. 3 before the execution of this confirmation and correction process. The process waits until an image group with an unconfirmed attribute appears, and if there is at least one, the process proceeds to step S702. In step S702, CPU 111 selects one of the image groups to which the unconfirmed attribute has been assigned in order of earliest read. In step S703, the CPU 111 acquires the index extraction rule from the HDD 120 and expands it in the RAM 119 before acquiring it. Examples of index extraction rules are shown in FIGS. 9A and 9B. FIG. 9A shows a state in which one document with document ID "0001" is registered, while FIG. 9B shows a state in which a document with document ID "0002" is also registered. The index extraction rule is a combination of a document ID, document identification information, and recommendation information for each registered document, and these combinations are stored for the number of registered documents. The document ID is a unique ID that indicates the type of document. The document identification information is information required to identify the type of document used in document matching, which will be described later, and uses text blocks obtained by executing OCR-related processing. The recommendation information is a rule for extracting recommended content from an input document, and includes coordinates of specific text blocks within a registered document and information on the order in which the OCR results of those text blocks should be concatenated to form a file name. In this embodiment, details of an example of recommending a file name will be described later, but the recommendation information may also include information on the folder to which the file is to be sent, or information for adding the OCR results of the text block group as metadata. Also, as shown in Figures 9A and 9B, thumbnails of the scanned image data may also be stored. In step S704, the CPU 111 executes a recommendation content generation process to generate a file name to be recommended for the image group selected in step S702 based on the index extraction rule acquired in step S703. Fig. 10 is a flowchart showing the processing procedure of the recommendation content generation process.

[0024] <Recommendation content generation process> In step S1000, the CPU 111 executes document matching. Document matching is a process that determines whether a document of the same type as the input document is included in the registered documents registered in the image processing device's database along with the index extraction rules, and, if a document of the same type is found, identifies the type of the document. In this embodiment, a similarity is first calculated between the input document and each registered document, representing the degree of similarity in the shape and layout of text blocks obtained by OCR-related processing. The similarity calculation first aligns the entire text blocks of the input document with the entire text blocks of the registered document. Next, the similarity is calculated by dividing the square of the sum of the overlapping areas of each text block in the input document and the text blocks of the registered document by the product of the sum of the text block areas of the input document and the registered document. The above-described similarity calculation is performed between the input document and all registered documents. If the highest similarity is equal to or greater than a certain value, the input document is determined to be of the same type as the registered document. If the highest similarity is less than the certain value, the input document is determined to be of a type not found in the registered documents. Note that document matching is not limited to the above method. For example, a group of character strings contained in a document obtained as a result of OCR-related processing may be stored as document identification information, and matching may be performed based on their similarity. Alternatively, image data or image features obtained from the image data may be stored as document identification information, and similarity between the image features may be used. In step S1001, if the CPU 111 determines that the input document is a registered document as a result of the document matching performed in step S1000, the process proceeds to step S1002, and if the input document is not a registered document, the process ends. In step S1002, CPU 111 assigns the same document ID to the input document as that of the document determined to be of the same type in step S1000.

[0025] In step S1003, CPU 111 obtains recommendation information associated with the document ID of the document determined in step S1000 from the index extraction rules, and generates a file name representing the recommendation content based on the rule. Specifically, first, a group of recommendation text blocks in the input document corresponding to a group of recommendation text blocks at a position set for each document type is obtained. In this case, the position of the entire image between the input document and the registered document may be misaligned depending on the scan timing, or the size of the text blocks may change depending on the content. Therefore, the CPU 111 aligns the entire text blocks in the input document with the entire text blocks in the registered document, and then obtains a group of recommendation text blocks in the input document that overlap with the recommendation text blocks in the registered document. Then, the CPU 111 concatenates character strings obtained as the OCR results of the group of recommendation text blocks in the obtained input document, and recommends the result as a file name.

[0026] <Confirmation / Editing screen> Returning to the explanation of the flowchart in FIG. 7, CPU 111 displays a confirmation and correction screen on operation and display unit 123. This confirmation and correction screen will be described with reference to Figures 11A and 11B. Note that, hereinafter, Figures 11A and 11B will be collectively referred to as Figure 11. In FIG. 11, the display and operation unit 123 includes a touch panel display area 1101 and hard keys 1102 (such as numeric keys and start keys for various processes). A preview image of scanned image data of an input document is displayed in the touch panel display area 1101. The user can scroll the preview image in any direction by swiping on the preview image. Also displayed are a text field 1103 for displaying and editing the file name, a delete button 1104 for deleting the file name, and buttons 1105 and 1106 for enlarging and reducing the display magnification of the preview image. Also displayed are a button 1107 for canceling the file name decision and returning to the previous file, and a button 1108 for deciding the file name. Here, if the document shown in FIG. 8A is selected in step S702 and the index extraction rule shown in FIG. 9A is acquired in step S703, the selected document is determined to be an unregistered document in step S704. Therefore, the recommended file name is not generated, and the file name text field 1103 is displayed blank, as shown in FIG. 11A. On this confirmation / editing screen, the user can specify the file name to be assigned to the file by touching a text block on the previewed image data. For example, as shown in FIG. 11B, if the user touches the text block 1109 representing the sending company, the text block 1109 is highlighted in the preview. Then, the file name text field 1103 displays "ABC Co.", the text resulting from the OCR of the text block 1109. Similarly, the results of touching the text block 1110 representing the document number and the text block 1111 representing the date are shown in FIGS. 11C and 11D, respectively. In this way, when multiple text blocks are touched, the file name is created by connecting them with an underscore, which is a predetermined separator. On the other hand, if a file name that is a recommendation is generated in step S704, the text block to be used for the file name is highlighted from the beginning, as shown in FIG. 11(D), and a confirmation / edit screen is displayed showing the file name. Note that it is not necessary to use the text of the OCR result as is for the file name. For example, as shown in FIG. 11(D), assume that the text obtained as the OCR result of text block 1111 is "2019 / 04 / 01." This text data may be identified as being in the YYYY / MM / DD format, which is a date, and converted to "20190401" in the YYYYMMDD format. If the user presses the OK button 1108 to confirm the file name, the process proceeds to step S706.

[0027] In step S706, the CPU 111 creates an index extraction rule corresponding to the selected document based on the content of the instruction given by the user via the confirmation / correction screen in step S705. In step S707, CPU 111 determines whether the user modified the file name generated in step S704 in step S705. If the recommendation content has been modified, the process proceeds to step S708; if not, the process proceeds to step S711. Note that if it is determined in step S704 that the selected document is not a registered document, the process proceeds to step S708, just as if the document had been modified. In step S708, CPU 111 determines whether a document ID has been assigned to the selected image group. If a document ID has been assigned, the process proceeds to step S709; if a document ID has not been assigned, the process proceeds to step S710. In step S709, the CPU 111 updates the document identification information and recommendation information of the corresponding document ID based on the index extraction rule created in step S706, and the process proceeds to step S711. In step S710, the CPU 111 issues a new document ID, combines it with the document identification information and recommendation information created in step S706, and registers a new index extraction rule, and the process proceeds to step S711. According to the above steps S706 to S708 and S710, when the user specifies a file name on the confirmation / edit screen (FIGS. 11(B) to (D)), a new index extraction rule corresponding to document ID "0002" in FIG. 9B is registered.

[0028] In step S711, the CPU 111 changes the attribute of the selected image group from the unconfirmed attribute to the confirmed attribute. In step S712, CPU 111 determines whether or not the confirmed attribute has been assigned to all of the image groups determined in step S303. If the confirmed attribute has been assigned to all of the image groups, the process ends; if not, the process returns to step S701. A case will be described in which the images shown in FIG. 8(B) and FIG. 8(C) are processed successively by the above-mentioned confirmation and correction processing. First, when the document shown in FIG. 8(B) is selected in step S702, the updated index extraction rule shown in FIG. 9B is obtained in step S703. In step S704, it is determined to be a registered document (document ID "0002"), and the file name of the recommendation result is obtained. In this case, the confirmation / edit screen displayed in step S705 is shown in FIG. 12(A). In this way, even if documents of the same type that were not registered before scanning are scanned at the same time, the user can specify the file name of one document on the confirmation / edit screen, and the file name of the other document can be recommended.

[0029] Next, a case will be described in which the document shown in FIG. 8(C) is selected in step S702, and the updated index extraction rule shown in FIG. 9B is acquired in step S703. In step S704, the selected document is determined to be a registered document (document ID "0001"), and a file name is recommended based on the index extraction rule previously registered. The confirmation and correction screen displayed in step S705 in this case is shown in FIG. 12(B). In this way, even if different types of files are scanned at the same time, it is possible to recommend file names according to the type of each document based on the index extraction rule previously registered. As described above, according to the first embodiment, when multiple documents are scanned, file division is performed and index items such as file names are recommended for each document, reducing the user's workload. Even if multiple documents of the same type are included in the same scan, registered and modified rules can be applied on the spot. Furthermore, by first performing OCR-related processing on all documents in the backend where the user is checking and modifying them, the user's waiting time can be significantly reduced. [Example]

[0030] In the first embodiment, an example was described in which the OCR-related processing of step S306 in FIG. 3 is executed for all image groups in the backend of the confirmation and correction processing, and the recommendation content generation processing shown in S704 in FIG. 7 is executed before the confirmation and correction screen for each image group is displayed. In the second embodiment, not only the OCR-related processing but also the generation of recommendation content is performed in the backend of the confirmation and correction processing. Then, an example is described in which it is determined whether the recommendation content needs to be regenerated before the confirmation and correction screen for each image group is displayed, and the index extraction rule regeneration processing is performed only if regeneration is necessary. Note that only the differences from the first embodiment will be described below. 13 is a flowchart of the processing procedure according to this embodiment. First, the differences between FIG. 13 and the flowchart of the first embodiment shown in FIG. 3 will be described. After the confirmation / correction processes are started in parallel in step S304, in step S1301, the CPU 111 acquires the first index extraction rule from the HDD 120, expands it in the RAM 119, and then acquires it. In this embodiment, the first index extraction rule is read only once and applied once to each of the image groups divided in step S303. This is the index extraction rule used in the recommendation content generation process in step S1302, which will be described later. After the OCR-related processes are performed in steps S305 and S306, the CPU 111 executes a recommendation content generation process in step S1302. Here, the CPU 111 generates a file name to be recommended for the image group selected in step S305 based on the first index extraction rule acquired in step S1301. Details of the recommendation content generation process are the same as those described in the first embodiment with reference to FIG. 10.

[0031] In step S1303 of the parallel processing, the CPU 111 recommends file names to the user for all image groups divided in step S303, and executes confirmation and correction processing to confirm the file names based on confirmation and correction instructions from the user. Details of the confirmation and correction processing will be described later. <Detailed explanation of confirmation and correction process> Details of the confirmation and correction process shown in S1303 of Fig. 13 will be explained using the flowchart of Fig. 14. In this embodiment, the confirmation and correction process is started in step S304 of Fig. 13 and operates in parallel with the processing flow of Fig. 13. Only the differences between Fig. 14 and the flowchart of the first embodiment shown in Fig. 7 will be explained. In step S1401, the CPU 111 creates a second index extraction rule and loads it in the RAM 119. The second index extraction rule is an index extraction rule that temporarily stores the updated state when the first index extraction rule acquired in step S1301 of FIG. 13 is updated or newly registered through the confirmation and correction process. The second index extraction rule has, in addition to the first index extraction rule, update information such as whether the rule has been updated or newly registered as an attribute. The initial state of the second index extraction rule is the same as the document ID, document identification information, and recommendation information, which are the contents of the first index extraction rule acquired in step S1301, and the initial update information is a state where no update has occurred. Examples of the second index extraction rule are shown in FIGS. 15A and 15B. FIG. 15A shows a state generated based on the first index extraction rule in which one document type, document ID "0001," is registered. FIG. 15B shows a state in which an index extraction rule corresponding to document ID "0002" has been newly registered.

[0032] In step S 1402 , the CPU 111 acquires the second index extraction rule developed in the RAM 119 . In step S1403, the CPU 111 executes a second recommendation content generation process based on the second index extraction rule acquired in step S1402. Details of the second recommendation content generation process will be described later. After steps S705 and S706, if the second index extraction rule is modified (S707) and there is a document ID (S708), the process proceeds to step S1404, and if there is no document ID, the process proceeds to S1405. In step S1404, the CPU 111 updates the document identification information and recommendation information of the corresponding document ID based on the second index extraction rule created in step S706, and stores the update information as updated. In step S1405, the CPU 111 issues a new document ID, combines it with the document identification information and recommendation information created in step S706, registers it as a new document in the second index extraction rule, and holds the updated status as a new registration. If the confirmation / correction process for all groups has been completed, then in step S1406, the CPU 111 reflects the second index extraction rule in the first index extraction rule and stores it in the HDD 120.

[0033] <Detailed explanation of the second recommendation content generation process> FIG. 16 shows a flowchart of the second recommendation content generation process. In step S1601, the CPU 111 determines whether the image group selected in step S702 in Fig. 14 has been determined to be a registered document in the recommendation content generation process in step S1302 in Fig. 13 and has been assigned a document ID. If a document ID has not been assigned, that is, if the document image is not registered in the second index extraction rule, the process proceeds to step S1602, and if a document ID has been assigned, the process proceeds to step S1607. In step S1602, the CPU 111 determines whether or not there is a new rule for the attribute of the update information among the second index extraction rules acquired in step S1402 of Fig. 14. If there is a new rule, the process proceeds to step S1603, and if there is no new rule, the process ends. As a result, if there is no new rule, the result of the recommendation content generation process in step S1302 of Fig. 13 is recommended in step S705 of Fig. 14, and the time required to display the confirmation / edit screen can be reduced. In step S1603, the CPU 111 performs document matching between the selected document and only newly registered documents in the second index extraction rule. Generally, the processing time for document matching increases according to the number of registered documents to be matched, so limiting the matching targets to only newly registered documents in this way can reduce the processing time. In step S1604, if the CPU 111 determines that the input document is a newly registered document as a result of the document matching executed in step S1603, the process proceeds to step S1605; otherwise, the process ends. If the input document is not a newly registered document, the file name that is the recommendation content is not generated, and the confirmation / display screen accepts the user's instruction for the file name.

[0034] In step S1605, CPU 111 assigns the same document ID to the input document as that of the document determined to be of the same type as the document matching result in step S1603. In the next step S1606, the CPU 111 acquires recommendation information linked to the document type determined in step S1603 from the second index extraction rule, and generates a file name as the recommendation content based on the rule. On the other hand, if the image group contains a document ID, in step S1607, the CPU 111 references the update information associated with the document ID assigned to the input document in the second index extraction rule and determines whether an update has occurred. If an update has occurred, the process proceeds to step S1608; if an update has not occurred, the process ends. As a result, if there has been no update, in step S705 of FIG. 14, the result of the recommendation content generation process in step S1302 of FIG. 13 is recommended, thereby reducing the time required to display the confirmation / edit screen. Furthermore, even if there has been an update, document matching is not required, and only recommendation content needs to be generated in step S1608 (described later), thereby reducing the time required to display the confirmation / edit screen. In step S1608, the CPU 111 acquires recommendation information linked to the document ID of the input document from the second index extraction rule, and generates a file name as the recommendation content based on the recommendation information. As described above, according to the second embodiment, the backend where the user confirms and modifies the documents first executes the process of generating the recommendation contents for all documents once, and then regenerates the recommendation contents as necessary. This reduces the time it takes to display the confirmation and modification screen for each document, thereby significantly reducing the user's waiting time compared to the first embodiment.

[0035] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

Claims

1. A means for causing a scanner to scan; a control means for, when a series of multiple originals set by a user is scanned continuously by the scanner, starting character recognition processing on image data consisting of multiple pages obtained by the scanning, in order from the first page of the multiple pages, without waiting for an operation by the user on the image data; a providing means for providing a screen for displaying a character string corresponding to the result of the character recognition processing on the image data of the first page of the image data, the screen being capable of receiving a correction instruction from the user for the character string; A processing program that causes a computer to function as while the correction screen for the image data of the first page provided by the providing means is displayed, the character recognition process for the image data of the subsequent page of the first page is executed by the control means. A processing program characterized by:

2. When the computer receives an instruction from the user, it instructs the scanner to perform a scan.

2. The processing program according to claim 1.

3. A processing program as described in claim 1 or 2, characterized in that the character recognition processing for the image data of the subsequent pages is executed by the control means regardless of the acceptance of operations from the user on the correction screen for the image data of the first page.

4. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page.

4. The processing program according to claim 1, wherein the processing program is a program for executing ...

5. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page while there is a page among the plurality of pages for which the character recognition process has not been performed at all.

4. The processing program according to claim 1, wherein the processing program is a program for executing ...

6. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page before completion of the character recognition process for the remaining pages of the plurality of pages.

4. The processing program according to claim 1, wherein the processing program is a program for executing ...

7. A processing program described in any one of claims 1 to 6, characterized in that the input of the image data of the first page and the image data of the subsequent pages is performed by a single action by the user.

8. A processing program described in any one of claims 1 to 7, characterized in that the providing means provides the correction screen on the display of the computer.

9. A processing program described in any one of claims 1 to 7, characterized in that the correction screen is displayed on a display of a device separate from the computer.

10. A processing program described in any one of claims 1 to 9, characterized in that the character string corresponding to the result of the character recognition processing displayed on the correction screen is at least one of a character string indicating the file name of the file related to the image data of the first page, a character string indicating the folder in which the file is saved, and a character string indicating the result of the character recognition processing.

11. A processing program described in any one of claims 1 to 10, characterized in that the target of the correction instruction is displayed on the correction screen alongside a preview image of the image data of the first page.

12. A management means for managing position information of a character area corresponding to the result of the character recognition process used for the character string displayed on the correction screen in association with a type of document; an identification means for identifying the type of document for each page based on the result of image processing on the image data consisting of the plurality of pages; and The processing program according to any one of claims 1 to 11, characterized in that the character string is displayed on the correction screen based on the results of character recognition processing corresponding to the character area at the position indicated by the position information managed in association with the type of manuscript of the first page.

13. An information processing device that executes a processing program, means for causing the scanner to scan; a control means for, when a series of multiple originals set by a user is successively scanned by the scanner, starting character recognition processing for image data consisting of multiple pages obtained by the scanning in order from the first page of the multiple pages in accordance with the processing program without waiting for an operation by the user on the image data; a providing means for providing, by the processing program, a screen for displaying a character string corresponding to the result of the character recognition processing on the image data of the first page of the image data, the correction screen being capable of receiving a correction instruction from the user for the character string; and while the correction screen for the image data of the first page provided by the providing means is displayed, the character recognition process for the image data of the subsequent page of the first page is executed by the control means.

1. An information processing device comprising:

14. When the information processing device receives an instruction from the user, it instructs the scanner to perform a scan.

14. The information processing apparatus according to claim 13,

15. An information processing device as described in Claim 13 or 14, characterized in that the character recognition processing for the image data of the subsequent pages is executed by the control means regardless of the acceptance of operations from the user on the correction screen for the image data of the first page.

16. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page.

16. The information processing device according to claim 13, wherein:

17. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page while there is a page among the plurality of pages for which the character recognition process has not been performed at all.

16. The information processing device according to claim 13, wherein:

18. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page before completion of the character recognition process for the remaining pages of the plurality of pages.

16. The information processing device according to claim 13, wherein:

19. An information processing device according to claim 13, wherein the image data of the first page and the image data of the subsequent pages are input by a single action by the user.

20. An information processing device according to claim 13, wherein the providing means provides the correction screen on a display of the information processing device.

21. An information processing device according to claim 13, wherein the correction screen is displayed on a display of a device other than the information processing device.

22. An information processing device described in any one of claims 13 to 21, characterized in that the character string corresponding to the result of the character recognition processing displayed on the correction screen is at least one of a character string indicating the file name of the file related to the image data of the first page, a character string indicating the folder in which the file is saved, and a character string indicating the result of the character recognition processing.

23. An information processing device described in any one of claims 13 to 22, characterized in that the target of the correction instruction is displayed on the correction screen alongside a preview image of the image data of the first page.

24. A management means for managing position information of a character area corresponding to the result of the character recognition process used for the character string displayed on the correction screen in association with a type of document; an identification means for identifying the type of document for each page based on the result of image processing on the image data consisting of the plurality of pages; and An information processing device according to any one of claims 13 to 23, characterized in that the character string is displayed on the correction screen based on the results of a character recognition process corresponding to a character area at a position indicated by the position information managed in association with the type of manuscript of the first page.

25. A method for controlling an information processing device that executes a processing program, comprising: scanning the image with a scanner; a control step in which, when a series of multiple originals set by a user are successively scanned by the scanner, the processing program starts character recognition processing on image data consisting of multiple pages obtained by the scanning, starting from the first page of the multiple pages, without waiting for an operation by the user on the image data; a providing step of providing, by the processing program, a screen that displays a character string corresponding to the result of the character recognition processing on image data of a first page of the image data, the correction screen being capable of receiving a correction instruction from the user for the character string; and while the correction screen for the image data of the first page provided in the providing step is displayed, the character recognition process for the image data of the subsequent page of the first page is executed by the control step.

2. A method for controlling an information processing apparatus comprising:

26. A system including a scanner, the scanner for scanning an original; a control means for, when a series of multiple originals set by a user is scanned continuously by the scanner, starting character recognition processing on image data consisting of multiple pages obtained by the scanning, in order from the first page of the multiple pages, without waiting for an operation by the user on the image data; a providing means for providing a correction screen that displays a character string corresponding to the result of the character recognition process on the image data of the first page of the image data, and that can receive a correction instruction from the user for the character string; and while the correction screen for the image data of the first page provided by the providing means is displayed, the character recognition process for the image data of the subsequent page of the first page is executed by the control means. A system characterized by:

27. ​​The providing means is provided in a computer rather than in the scanner, When the computer receives an instruction from the user, it instructs the scanner to perform a scan; The series of multiple document sheets is scanned based on instructions from the computer.

27. The system of claim 26.

28. A system as described in Claim 26 or 27, characterized in that the character recognition processing for the image data of the subsequent pages is executed by the control means regardless of the acceptance of operations from the user on the correction screen for the image data of the first page.

29. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page. The system according to any one of claims 26 to 28.

30. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page while there is a page among the plurality of pages for which the character recognition process has not been performed at all. The system according to any one of claims 26 to 28.

31. The providing means provides the correction screen for the first page in response to completion of the character recognition process for the first page before completion of the character recognition process for the remaining pages of the plurality of pages. The system according to any one of claims 26 to 28.

32. A system described in any one of claims 26 to 31, characterized in that the image data of the first page and the image data of the subsequent pages are input by a single action by the user.

33. A system described in any one of claims 26 to 32, characterized in that the providing means provides the modified screen on a display of a device having the control means.

34. A system described in any one of claims 26 to 32, characterized in that the correction screen is displayed on a display of a device other than the device having the control means.

35. A system described in any one of claims 26 to 34, characterized in that the character string corresponding to the result of the character recognition processing displayed on the correction screen is at least one of a character string indicating the file name of the file related to the image data of the first page, a character string indicating the folder in which the file is saved, and a character string indicating the result of the character recognition processing.

36. A system described in any one of claims 26 to 35, characterized in that the target of the correction instruction is displayed on the correction screen alongside a preview image of the image data of the first page.

37. A management means for managing position information of a character area corresponding to the result of the character recognition process used for the character string displayed on the correction screen in association with a type of document; an identification means for identifying the type of document for each page based on the result of image processing on the image data consisting of the plurality of pages; and The system described in any one of claims 26 to 36, characterized in that the character string is displayed on the correction screen based on the results of character recognition processing corresponding to the character area at the position indicated by the position information managed in association with the type of manuscript of the first page.

Images