Information processing device, information processing method, and program
The system enhances user understanding of correction rules by generating explanatory texts for modification rules, addressing the complexity of interpreting abstract notations in character string processing.
Patent Information
- Application Number
- JP2025077322
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2026-11-19
AI Technical Summary
Users find it difficult to understand highly abstract correction rules constructed for character strings, necessitating interpretation of regular expressions.
An information processing system that includes acquisition, generation, and output means for generating modification rules and explanatory texts to facilitate user understanding of correction rules.
Enables users to comprehend and interact with correction rules more intuitively, improving the usability of character string correction systems.
Smart Images

Figure 2026181734000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an information processing technique for correcting an input character string.
Background Art
[0002] Conventionally, there is a system that sets the file name of a document image based on a character recognition result obtained by performing character recognition processing on the document image obtained by scanning a paper document. Patent Document 1 describes that when assigning attribute information to a document image based on a character string obtained by performing character recognition processing on the document image, the content of corrections made by the user to the obtained character string is learned, and a correction rule corresponding to the learned correction content is constructed. According to Patent Document 1, it is possible to automatically correct character strings obtained from similar document images based on the constructed correction rules.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, in Patent Document 1, in order to confirm whether the constructed correction rule is the correction rule intended by the user, the user needs to interpret highly abstract notation such as regular expressions, and it is difficult for the user to properly understand the content of the correction rule from the highly abstract notation.
[0005] The present disclosure aims to make it easier for users to understand the constructed correction rules.
Means for Solving the Problems
[0006] This disclosure is characterized by comprising: an acquisition means for acquiring a string contained in an input document image and a modified string obtained by the user modifying the string; a generation means for generating a modification rule for obtaining the modified string from the string based on the string and the modified string; and an output means for outputting explanatory information including an explanatory text for the modification rule generated by the generation means. [Effects of the Invention]
[0007] According to this disclosure, the established modification rules can be made easier for users to understand. [Brief explanation of the drawing]
[0008] [Figure 1] This figure shows an example of the system configuration according to one embodiment. [Figure 2] This figure shows an example of the hardware configuration of an image forming apparatus according to one embodiment. [Figure 3] This figure shows the functional configuration of an image forming apparatus according to one embodiment. [Figure 4] This is a flowchart illustrating the file generation process for scanned images according to Embodiment 1. [Figure 5] This figure shows an example of the string area detection process according to Embodiment 1. [Figure 6] This figure shows an example of a registered document record according to Embodiment 1. [Figure 7] This figure shows an example of a confirmation and correction process using the confirmation and correction screen according to Embodiment 1. [Figure 8] This is a flowchart illustrating the process of constructing modification rules according to Embodiment 1. [Figure 9] This figure shows the screen for displaying the explanation of the revised rules and receiving instructions for the revised rules according to Embodiment 1. [Figure 10] This figure shows the screen during the process of updating the explanation of the correction rule according to Embodiment 2. [Figure 11] This figure shows the screens for suggesting candidate modification rules and receiving multiple instructions according to Embodiment 3. [Modes for carrying out the invention]
[0009] The embodiments for carrying out the present invention will be described below with reference to the figures. Note that the following embodiments are not intended to limit the technology of the present invention as defined in the claims, and not all combinations of features described in these embodiments are necessarily essential to the solutions of the technology of this disclosure.
[0010] <Embodiment 1> A first embodiment of the present invention will be described below. First, the system configuration and overall processing flow outline in this embodiment will be described. Next, the features and effects of this embodiment with respect to the configuration according to the present invention will be described in detail.
[0011] [System Configuration] (Overall system configuration) Figure 1 shows the overall configuration of a system to which this embodiment can be applied. The system 105 of this embodiment includes an image forming apparatus 100 and a terminal 101. As shown in Figure 1, the image forming apparatus 100 is connected to a LAN 102 and can communicate with a terminal 101 such as a PC via the internet 103, etc. In this embodiment, the terminal 101 is not required, and the system may consist only of the image forming apparatus 100.
[0012] (Overview of the image forming apparatus 100) As shown in Figure 2, the image forming apparatus 100 is a multifunction printer (MFP) having a printer unit 121, a scanner unit 122, and a display / operation unit 123. The image forming apparatus 100 can be used as a scanning terminal to scan document originals using the scanner unit 122. It also has a display / operation unit 123 equipped with a touch panel and hard buttons, and displays a user interface for displaying file names and storage location suggestions, and for receiving instructions from the user.
[0013] (Hardware configuration of the image forming apparatus 100) Figure 2 shows an example of the hardware configuration of the image forming apparatus 100. The image forming apparatus 100 of the present embodiment includes a printer unit 121, a scanner unit 122, a display and operation unit 123, and a control unit 110.
[0014] The control unit 110 includes a CPU 111, a storage device 112 (ROM 118, RAM 119, HDD 120), a printer I / F unit 113, a network I / F unit 114, a scanner I / F unit 115, and a display and operation I / F unit 116. Further, the control unit 110 is an information processing apparatus in which the respective units are communicably connected to each other via a system bus 117, and controls the overall operation of the image forming apparatus 100.
[0015] The CPU 111 functions as a unit that executes respective processes such as reading control, image processing, and display control in flowcharts described later by reading and executing a control program stored in the storage device 112.
[0016] The storage device 112 stores and holds a control program, image data, metadata, setting data, processing result data, and the like. The storage device 112 includes the ROM 118 which is a non-volatile memory, the RAM 119 which is a volatile memory, and the HDD 120 which is a large-capacity storage area. The ROM 118 is a non-volatile memory that holds the control program and the like, and the CPU 111 reads out the control program to perform control. The RAM 119 is a volatile memory used as a temporary storage area such as a main memory and a work area for the CPU 111.
[0017] The network I / F unit 114 connects the control unit 110 (image forming apparatus 100) to a LAN 102 via the system bus 117. The network I / F unit 114 transmits image data to an external apparatus on the LAN 102, and receives various types of information from the external apparatus on the LAN 102.
[0018] The scanner interface unit 115 connects the scanner unit 122 and the control unit 110 via the system bus 117. The scanner unit 122 reads the original document and generates scanned image data, and inputs the scanned image data to the control unit 110 via the scanner interface unit 115. The scanner unit 122 is also equipped with a document feeder, which feeds multiple documents placed in a tray one by one, enabling continuous scanning.
[0019] The display and operation interface unit 116 connects the display and operation unit 123 and the control unit 110 via the system bus 117. The display and operation unit 123 is equipped with a liquid crystal display unit with touch panel functionality, hard buttons, and the like.
[0020] The printer interface unit 113 connects the printer unit 121 and the control unit 110 via the system bus 117. The printer unit 121 receives image data generated by the CPU 111 via the printer interface unit 113, and uses the received image data to perform printing on recording paper. As described above, the image forming apparatus 100 according to this embodiment can provide the functionality of an image processing apparatus with the above hardware configuration.
[0021] (Functional configuration of the image forming apparatus 100) Figure 3 shows an example of the functional configuration of the image forming apparatus 100 according to this embodiment. Note that Figure 3 shows only the functions related to the processing of scanning a document original, digitizing it (creating a file), and saving it, among the various functions of the image forming apparatus 100.
[0022] The display control unit 301 displays a user interface screen (UI screen) on the touch panel of the display and operation unit 123 to accept various user operations. These various user operations include, for example, scan settings, instructions to start scanning, instructions to correct entered strings, instructions for registration methods, file name settings, and instructions to save files.
[0023] The scan control unit 302, in response to user operations performed on the UI screen (for example, pressing the "Start Scan" button), instructs the scan execution unit 303 to perform the scan process along with the scan setting information. The scan execution unit 303, following the scan execution instruction from the scan control unit 302, causes the scanner unit 122 to perform a scan operation of the document via the scanner I / F unit 115 and generates scanned image data. The generated scanned image data is saved to the HDD 120 by the scanned image management unit 304.
[0024] The image processing unit 305 performs image analysis processing on scanned image data, including text region detection, OCR (optical character recognition), and similar document determination, as well as image processing such as rotation and tilt correction, and automatic text correction processing according to the correction rules described later. The image processing unit 305 also allows the image forming apparatus 100 to function as an image processing device. The text regions detected from the scanned image are also called "text blocks." Details of the image processing will be described later.
[0025] The functions of each part shown in Figure 3 are realized by the CPU 111 of the image forming apparatus 100 loading the program code stored in the ROM 118 into the RAM 119 and executing it. Alternatively, some or all of the functions of each part in Figure 3 may be realized by hardware such as an ASIC or electronic circuit.
[0026] [Overall processing flow] Figure 4 shows a flowchart illustrating the processes performed by the image forming apparatus 100 in this embodiment. The image forming apparatus 100 scans a document, performs image processing on the image obtained as a result of scanning the first page of the document (hereinafter referred to as the scanned image), and extracts a string from the scanned image. The overall processing flow will be described, which involves generating a file name using the obtained string, and then sending the created file after confirming and correcting the string.
[0027] The series of processes shown in the flowchart of Figure 4 are performed by the CPU 111 of the image forming apparatus 100 loading the program code stored in the ROM 118 into the RAM 119 and executing it. Alternatively, some or all of the functions of the steps in Figure 4 may be implemented by hardware such as an ASIC or electronic circuit. The symbol "S" in the description of each process indicates a step in that flowchart, and the same applies to subsequent flowcharts.
[0028] In S400, when the scan control unit 302 receives a scan command from the user via the display and operation unit 123, it instructs the scan execution unit 303 to scan multiple document originals one by one from the document feeder tray of the scanner unit 122. The scan control unit 302 then acquires the scanned image data obtained as a result of the scan.
[0029] In S401, the image processing unit 305 analyzes the scanned image data acquired in S400 and performs string extraction processing. The string extraction processing includes image processing, string region detection processing, similar document determination processing, character recognition processing (OCR processing), and automatic string correction processing. The similar document determination processing is performed using document identification information contained in the registered document record described later. The character recognition processing performs character recognition processing on the detected string region in accordance with the string extraction rules contained in the registered document record described later. The automatic string correction processing performs correction processing on the input string in accordance with the correction rules described later. Details of the character recognition processing, automatic string correction processing, document record, and correction rules will be described later as features of this embodiment.
[0030] A string extraction rule is one of the pieces of information included in a document record. It is a rule for extracting strings of predetermined items used as filenames or metadata, such as the document title, management number, and company name, from scanned image data. In this embodiment, a string extraction rule is constructed for each group of documents (hereinafter referred to as the "similar document group") that the image processing unit 305 determines to be similar documents in the aforementioned similar document determination process. The string extraction rule is used to generate the filename when saving the scanned image data, and filename rules are set that define modification rules for each item to be extracted, as well as the order in which the items to be extracted are arranged. Note that the string extraction rule is not limited to generating filenames; it may also be used to set other property information, such as extracting metadata or generating folder paths. In short, a string extraction rule is a rule for extracting information that is set as a property (attribute) related to scanned image data from scanned image data, and one piece of information set using a string extraction rule is the filename.
[0031] A modification rule is a rule that defines how to modify a string when a modification is set for a string recognized in a string area that has been targeted for extraction based on a string extraction rule. This rule is constructed by the modification rule construction process described later. In this embodiment, string modification is a process that performs one or more modification operations on a substring that matches the regular expression or string representing the target of modification set in the modification rule, so that it matches the set modified regular expression or string. Individual modifications that make up a modification rule include "modification of similar character shapes," "change of the number of digits in numeric strings," "modification of common nouns (deletion, conversion, addition)," and "modification of delimiters (deletion, conversion, addition)." In this embodiment, the scope of application of the modification rule is set for each extraction target item to which the string extraction rule constructed for each group of similar documents is applied, among documents used by all users of the same image forming apparatus 100. The scope of application of the modification rule may be, for example, all extraction target items in a group of similar documents, a specific extraction target item in all documents used by the same user, or a specific group of documents used by image forming apparatuses connected to the same LAN 102.
[0032] In S402, the display control unit 301 generates a file name for the scanned image data using the string extracted in S401, and displays a confirmation / correction screen containing the string and the generated file name on the display / operation unit 123 for the user. The display control unit 301 also accepts confirmation or correction of the presented string from the user. When the display control unit 301 receives confirmation or correction from the user via the display / operation unit 123, it sets the presented file name, or the corrected file name if corrected, as the file name for the scanned image data. Details of this confirmation / correction process will be described later with reference to Figure 7, which is a feature of this embodiment.
[0033] In S403, the image processing unit 305 determines whether the user modified the string in S402. If the user has modified the string, it updates the string extraction rules for the document or performs a process to register a new document (document registration process). Details of the string extraction rule update and document registration process will be described later in the explanation of the confirmation and modification process as a feature of this embodiment.
[0034] In S404, the image processing unit 305 determines whether the user modified the string via the display / operation unit 123 in S402. If the user has modified the string, it performs a process to construct a modification rule (modification rule construction process). Details of the modification rule construction process will be described later with reference to Figure 8, which is a feature of this embodiment.
[0035] In S405, the image processing unit 305 determines whether or not it constructed a correction rule in S404. If the image processing unit 305 did not construct a correction rule, the process proceeds to S410. If the image processing unit 305 constructed a correction rule, the process proceeds to S406.
[0036] In S406, the image processing unit 305 performs a process to generate an explanatory text for the correction rule constructed in S404 (correction rule explanatory text generation process). Details of the correction rule explanatory text generation process will be described later as a feature of this embodiment.
[0037] In S407, the display control unit 301 displays the explanatory text of the correction rule generated in S406 on the display / operation unit 123 and presents it to the user.
[0038] In S408, the display control unit 301 receives user instructions regarding the modification rules. Here, user instructions regarding modification rules in this embodiment refer to instructions to "automatically apply the modification rules" and instructions to "not automatically apply the modification rules." The presentation of the explanation of the modification rules and the reception of user instructions regarding the modification rules will be described later as features of this embodiment. If the display control unit 301 receives the instruction to "not automatically apply the modification rules," the process proceeds to S410. If the display control unit 301 receives the instruction to "automatically apply the modification rules" (S408 automatically applies them), the process proceeds to S409.
[0039] In S409, the image processing unit 305 sets the automatic application of correction rules. As mentioned above, the scope of automatic application of correction rules in this embodiment is each item to be extracted by the string extraction rules constructed for each group of similar documents among the documents used by all users of the same image forming apparatus 100. Therefore, each item to be extracted is set as the target of automatic string correction for each correction rule, based on the string extraction rules held by the document records of each group of similar documents. Details will be explained later as part of the correction rule construction process.
[0040] In S410, the image processing unit 305 creates a file from the scanned image data acquired in S400 and sets the file name determined in S402. In this embodiment, as an example, the scanned image data is saved as a PDF (Portable Document Format). In the case of PDF, if multiple paper documents are scanned in S400, the scanned image data corresponding to each paper document is saved as separate pages in one file.
[0041] In S411, the scanned image management unit 304 sends the file created in S410 to a designated destination via LAN 102.
[0042] [Features of Embodiment 1] This embodiment's features, including character recognition processing, automatic string correction processing, confirmation and correction processing, correction rule construction processing, correction rule explanation text generation processing, presentation of correction rule explanation text, and acceptance of instructions for correction rules, will be explained in detail using diagrams.
[0043] (Character recognition processing) The details of the character recognition process, which is an internal process of the string extraction process (S401), will be explained using Figures 5 and 6, and Table 1.
[0044] Figure 5 shows an example of the result of the string region detection process, which is an internal process of the string extraction process (S401) in this embodiment. Figure 5(a) is a scanned image after image processing, which is an internal process of the string extraction process (S401). Figure 5(b) is the result of the string region detection process on the scanned image of Figure 5(a), and the rectangle indicated by the dashed line represents the foreground region detected as a string region. The string region in Figure 5(b) is used in the similar document determination process and character recognition process, which are internal processes of the string extraction process (S401).
[0045] Figure 6 shows an example of a registered document record in this embodiment. This is a registered document record used when performing similar document determination processing using the string area in Figure 5(b). Each document record includes information such as "Document ID," "Registered Name," "Scanned Image," "Document Identification Information," and "String Extraction Rule."
[0046] The "Document ID" is a unique identification number assigned to each group of similar documents when registering a document record.
[0047] The "Registered Name" is the name set when the document was registered, and the "Scanned Image" is the scanned image data used when the document record was registered. This information is used by the user to identify the registered document. The "Scanned Image" only needs to contain enough information for the user to identify the document's features; for example, the resolution may be reduced to around 150 dpi.
[0048] "Document identification information" is information used by the image processing unit 305 to identify document records that contain scanned image data similar to the scanned image data acquired by S400. Specifically, it is information on the position and size of the string area obtained as a result of performing a string area detection process on the scanned image data. However, "document identification information" does not need to be limited to information on the position and size of the string area; it does not need to be any information necessary to identify document records that contain scanned image data similar to the scanned image data acquired by S400. For example, it may retain the position and shape of ruled lines contained in the document, or it may use combinations of strings that appear in the document.
[0049] The "string extraction rule" is information used to generate filenames from scanned image data, and includes the extraction item ID, which is an identification number for each extraction item, the order in which to assign filenames for each extraction item ID, and the coordinates and size of the string area for each extraction item ID. Figure 600 shows the coordinates and size information of the string area for each extraction item ID placed at coordinates corresponding to the scanned image, along with the extraction item ID. The filename rule 601 included in the "string extraction rule" in Figure 6 shows the rule for generating filenames using strings recognized in the string area corresponding to the extraction item ID. As shown in Figure 6, the filename rule 601 is a rule that concatenates the strings recognized in the string areas corresponding to extraction item IDs "01", "02", and "03" with underscores in that order.
[0050] As mentioned above, for the scanned image data shown in Figure 5(a), a similar document determination process is performed for each registered document record using the position and size information of the text area shown in Figure 5(b). In the similar document determination process, the overlap between the text area of the scanned image data acquired by S400 and the text area registered as document identification information contained in each registered document record is calculated as the similarity. Then, if the similarity is greater than or equal to a predetermined value, the registered document record with the highest similarity is determined to be the similar document. If there are no registered document records with a similarity greater than or equal to the predetermined value, it is determined that there are no similar documents. One example of a method for calculating the similarity is as follows: First, the entire text area of the scanned image data acquired by S400 is aligned with the entire text area of each registered document record, and the square of the sum of the overlapping areas of the text area of the scanned image data and the text area of each registered document record (value A) is calculated. Furthermore, the product of the sum of the areas of the text areas of the scanned image data acquired by S400 and the sum of the areas of the text areas of each registered document record (value B) is calculated. Then, the similarity score is calculated by dividing value A by value B.
[0051] In this embodiment, if the scanned image data is determined to be a similar document to the document registered in the document record with document ID "0001", the scanned image data is assigned the document ID "0001". Then, according to the string extraction rule shown in Figure 6, the items to be extracted are detected from the scanned image data and character recognition processing is performed. If, for example, the scanned image data in Figure 5(a) is determined not to be a similar document to the document registered in the registered document record, a new document record is generated and registered based on the scanned image data. The new document record is assigned a unique identification number different from the existing document ID as the document ID, the scanned image in Figure 5(a) is registered as the scanned image, and the string area in Figure 5(b) is registered as document identification information. The display control unit 301 also accepts user input regarding the string area where the string to be used for the file name is located, or the corresponding extraction item ID, the file name rule that defines their order, etc., and the registration name. In this way, the new document record is registered with the registration name and the string extraction rule.
[0052] To estimate the string regions to be processed for character recognition in the scanned image data of Figure 5(a), a target string region estimation process is performed as a preprocessing step for character recognition. The target string region estimation process estimates the string regions in the scanned image data that have a large overlap with the string regions shown in image 600 of the "string extraction rule" in Figure 6. This target string region estimation process also includes a process to perform overall alignment between the scanned image of the registered document record and the scanned image of Figure 5(a). Note that the method for estimating the string regions to be processed for character recognition is not limited to the above. For example, for string regions registered as string extraction rules, surrounding string regions can be extracted as partial patterns from the group of string regions of the document registered in the document record. Then, using the extracted partial patterns, pattern matching can be performed within a pre-set search range between them and the string regions of the input document after overall alignment to perform local alignment and estimate the target string regions.
[0053] Table 1 shows the results obtained by the image processing unit 305 executing partial OCR (Optical Character Recognition) on the character string region group that is the character recognition processing target of the scanned image in Fig. 5(a) estimated in the aforementioned target character string region estimation processing, and recognizing the character string corresponding to each character string region. As shown in Table 1, for the scanned image in Fig. 5(a), "見積書" (with spaces in between) is recognized as the character string for the extraction item ID "01", "ABC Co., Ltd." is recognized as the character string for the extraction item ID "02", and "2019 / 4 / 3" is recognized as the character string for the extraction item ID "03".
[0054]
Table 1
[0055] (Automatic Character String Correction Processing) The details of the automatic character string correction processing, which is an internal process of the character string extraction processing (S401), will be described using Table 2.
[0056] Table 2 is a table showing an example of correction rules, and shows an example of performing automatic character string correction processing on the character string recognition result of the document with the document ID "0001" shown in Table 1. In the correction rules, "correction rule ID" and "correction target ID" are associated with each other on a correction rule unit basis. In addition, a correction rule may be dividable into a plurality of correction units, and correction is performed on the partial character string to be corrected by the correction operation for each correction unit so as to match the regular expression or character string of the corrected partial character string. When one character string includes a plurality of partial character strings to be corrected, the correction operation is executed on the character string a plurality of times.
[0057] The correction rule ID is a unique identification number for each correction rule. The correction target ID is an identification number associated with the target character string region for which automatic application of the correction rule is set. The correction target ID in the present embodiment is represented in a form in which the target document ID and the extraction item ID are connected by an underscore. Note that since one correction rule may be applied to a plurality of character string regions, a plurality of correction target IDs may be associated with one correction rule ID.
[0058] [Table 2]
[0059] In Table 2, as correction rules targeting the document with document ID "0001", the correction rule with correction rule ID "1" targeting extraction item IDs "01" and "03", and the correction rule with correction rule ID "2" targeting extraction item ID "02" are registered.
[0060] The correction rule with correction rule ID "1" is a rule that, with correction operation number "1", corrects the partial string to be corrected that matches the regular expression "\s" (blank character) to the post-correction partial string "" (no characters).
[0061] The correction rule with correction rule ID "2" corrects, with correction operation number "1", the partial string to be corrected that matches the special character "(Kabushiki)" to the post-correction partial string "Kabushiki-gaisha". Subsequently, this is a rule that, with correction operation number "2", corrects the partial string to be corrected that matches the regular expression "\s" (blank character) to the post-correction partial string "" (no characters).
[0062] The character string of extraction item ID "01" is subject to the automatic correction rule application setting for correction rule ID "1". Since the extracted character string is "Estimate Document" (with spaces) from Table 1, the two " (space)" among "Estimate", " (space)", "Document", " (space)", "Form" match the partial string to be corrected for correction operation number "1". Therefore, the two " (space)" are automatically corrected to "" (no characters). As a result, the character string of extraction item ID "01" becomes "EstimateDocument" with no spaces between characters.
[0063] The character string of extraction item ID "02" is subject to the automatic correction rule application setting for correction rule ID "2", but since the extracted character string is "ABC Co., Ltd." from Table 1, it does not match the partial string to be corrected, and the character string remains as "ABC Co., Ltd.".
[0064] The string with extraction item ID "03" is subject to the automatic application setting of correction rule ID "1," but since the extracted string is "2019 / 4 / 3" from Table 1, it does not match the substring to be corrected, and the string remains "2019 / 4 / 3".
[0065] (Confirmation and correction process) The details of the verification and correction process (S402) will be explained using Figure 7.
[0066] Figure 7 shows the confirmation and correction screen displayed on the display and operation unit 123 when, in the string extraction process (S401), a document record containing scanned image data with a similarity to the scanned image data acquired in S400 that is equal to or greater than a predetermined value is identified, and a string is extracted.
[0067] The preview area 700 displays the scanned image data acquired by S400 as a preview image. A rectangle indicating the estimated location of the text area is also overlaid and highlighted on the scanned image. Buttons 701 and 702 allow zooming in and out of the preview image's display magnification. Button 703 allows zooming in or out of the preview image to fit the width or height of the preview area. The file name text field 704 displays the file name generated by combining the strings extracted by S401 according to a pre-set string extraction rule. The extracted item information area 705 displays the extracted item number area 706, the partial preview area 707, and the extracted item text field 708 for each extracted item. The extracted item number area 706 displays a number representing the extracted item ID. The partial preview area 707 displays an image partially cut from the scanned image corresponding to the estimated text area. The extracted item text field 708 displays the string extraction result of the text area, and the user can edit the string extraction result if it is incorrect.
[0068] For strings where string estimation fails, the extracted item information area 705 displays a number representing the extracted item ID in the extracted item number area 706, but the partial preview area 707 and the extracted item text field 708 are displayed empty.
[0069] This section describes the procedure for correcting a string area if the estimated location of the string area in S401 is incorrect or if the estimation of the string area fails. To correct an existing extracted item, the user first selects one of the extracted items corresponding to the string to be corrected, displayed in the extracted item information area 705. Next, the user specifies the correct location of the string area on the preview area 700. The partial preview area 707 is then updated to match the string area specified by the user, the newly processed string with character recognition and automatic string correction is displayed in the extracted item text field 708, and the file name text field 704 is updated. To add an extracted item, the user can specify the location of the string area on the preview area 700 while the extracted item information area 705 is not selected. In this case, a new extracted item is added to the extracted item information area 705, and an image corresponding to the string area specified by the user is displayed in the partial preview area 707. The processed string with character recognition and automatic string correction is then displayed in the extracted item text field 708, and the file name text field 704 is updated.
[0070] Next, we will explain the procedure for correcting strings in cases where the estimated position of the string area in S401 is correct, but the extracted string is incorrect, or when the user wishes to correct the notation. First, the user selects the extraction item text field 708 where the string to be corrected is displayed. Then, the user uses the software keyboard or other overlay displayed on the display / operation unit 123 to perform cursor operations, input, and deletion operations on the extraction item text field 708. When the string on the extraction item text field 708 is corrected by the operation received by the display / operation unit 123, the file name text field 704 is updated. If S401 determines that there is no corresponding registered document record, the extraction item will be displayed in the extraction item information area 705 for all strings. In this case, each extraction item will be displayed with the partial preview area 707 and the extraction item text field 708 empty, similar to the case where the estimation of the string area fails as described above. It is also possible to specify a new position for each string area by following the string area correction procedure described above.
[0071] When the user completes confirmation and correction and presses the OK button 709, the file name, string for each extraction item ID, and string area are confirmed. If the string area is modified, an item is added, or an item is deleted, the string extraction rule in the document record is updated accordingly. Also, if S401 determines that there is no corresponding registered document record, a new document record is generated with a new document ID assigned, and the registration name of the generated new document record is accepted from the user via the display / operation unit 123 and confirmed. Then, the scanned image data and document identification information, such as the scanned image shown in Figure 5(a) and the string area shown in Figure 5(b), are registered, and the newly added extraction item ID and string area are registered as string extraction rules.
[0072] (Revision rule construction process) The details of the modification rule construction process (S404) will be explained using Figure 8 and Table 3.
[0073] Figure 8 shows a flowchart illustrating the process of constructing correction rules performed for each extracted string in this embodiment. Note that the correction rules in this embodiment target the correction of the string itself, and do not target the position correction of the string area.
[0074] In S800, the image processing unit 305 acquires the results obtained in the string extraction process (S401) and the results confirmed in the confirmation and correction process (S402). In this embodiment, the information of the results to be acquired is the string for each extracted item ID obtained as a result of the string extraction process and the string for each confirmed extracted item ID, but input operations such as key operations when correcting by the user may also be acquired as they are. If the string area correction described above is performed in the confirmation and correction process (S402), the strings obtained by the character recognition process and automatic string correction process performed after the string area correction are acquired as the strings for each extracted item ID obtained as a result of the string extraction process.
[0075] In S801, the image processing unit 305 selects one of the extracted items obtained as a result of the string extraction process acquired in S800.
[0076] In S802, the image processing unit 305 determines whether or not the user has modified the string of the extracted item selected in S801. If the image processing unit 305 determines that the user has not modified the string, it terminates the process of constructing modification rules for the string of that extracted item. If the image processing unit 305 determines that the user has modified the string, it proceeds to S803. The determination method is as follows: for the same extracted item ID, if the string obtained as a result of the string extraction process does not match the confirmed string, it is determined that the string has been modified; otherwise, it is determined that the string has not been modified.
[0077] In S803, the image processing unit 305 determines the correction content for each extracted item that was determined to have undergone character string correction in S801. Here, in the present embodiment, the individual correction operations that are the determination targets and included in the constructed correction rules are "similar character shape correction", "digit count change of numeric character string", "common noun change (deletion, conversion, addition)", and "delimiter change (deletion, conversion, addition)". Note that the correction operations included in the correction rules are not limited to these, and corrections for various purposes such as "half-width and full-width conversion", "calendar conversion", and "date display order change" may be included in the correction rules. In addition, since the correction content for each extracted item may include a plurality of corrections, the process is repeated for each extracted item until all the above corrections have been determined. In the present embodiment, the priority order of corrections to be determined when judging the correction content is "similar character shape correction", "digit count change of numeric character string", "common noun change (deletion, conversion, addition)", and "delimiter change (deletion, conversion, addition)" from top to bottom.
[0078] A method for determining whether or not the above corrections are included in the correction content in the present embodiment will be described below.
[0079] The determination of whether or not the correction is "similar character shape correction" is performed based on whether or not the corrected partial character string of the corrected portion is a similar character shape of the pre-correction partial character string. Here, the similar character shape refers to a character string that has a high frequency of being erroneous relative to the correct character string during character recognition due to similar character shapes, and is managed in a similar character shape dictionary. The similar character shape dictionary is a dictionary in which a group of similar character shape character strings for a prescribed character string is registered. As an example, "shellfish" is registered as a similar character shape character string of "see", and "see" is registered as a similar character shape character string of "shellfish". In this way, they are often registered as similar character shape character strings in both directions. Further, as "word blue" is registered as a similar character shape character string of "request", not only characters but also character strings may be similar character shapes. Further, as "3" and "B" are registered as similar character shapes of "8", a plurality of similar character shape character strings may be registered for one character string.
[0080] The determination of whether a change constitutes a "modification (deletion, conversion, or addition) of a common noun" is based on whether the deleted, converted, or added substring matches a common noun before and after the modification. Here, a common noun is a noun other than a proper noun, and is managed in a common noun dictionary. Examples of common nouns managed in the common noun dictionary include "corporation," "stock," "limited company," "company," "quote," "estimate," "invoice," "invoice," "issue date," "issue," "date," and "Western calendar year." Furthermore, there are also "telephone number," "address," "postal code," "name," and "full name."
[0081] Whether or not a delimiter has been changed (deleted, converted, or added) is determined based on whether the deleted, converted, or added character matches the delimiter before and after the modification. Here, the delimiter is a character that separates strings and is managed in the delimiter dictionary. Examples of delimiters managed in the delimiter dictionary include "_", " / ", ",", ".", "、", ".", "-", "(space)", "year", "month", "day", etc.
[0082] The determination of whether or not a change is a "change in the number of digits in a numeric string" is based on whether or not the number of digits displayed in a numeric-only string separated by a string composed of non-numeric characters has been changed. For example, if "January 14, 2023" is changed to "January 14, 2023", the number of digits in the numeric-only substring "1" has been changed to "01", which represents the same number. Therefore, this case is determined to be a change in the number of digits in a numeric string. However, if the "1" is changed to a different number "10", as in "October 14, 2023", it is not determined to be a change in the number of digits in a numeric string.
[0083] In S804, the image processing unit 305 proceeds to S805 if there is a predefined correction operation that matches the correction content, and proceeds to S813 if there is no matching correction operation.
[0084] In S805, when the image processing unit 305 determines that the correction operation matching the correction content is "similar character shape correction", the process proceeds to S806, and when it is determined that the correction operation is not "similar character shape correction", the process proceeds to S807.
[0085] In S806, the image processing unit 305 constructs a correction operation for similar character shape correction. For the correction operation of similar character shape correction, for example, when character recognition incorrectly recognizes "見積書" as "貝積書", the user corrects "貝積書" to "見積書". In this case, in order to automatically correct "貝", which is registered in the dictionary as a similar character shape of "見", to "見", "貝積書" is set as the partial character string to be corrected, and "見積書" is set as the corrected partial character string. Since character errors of similar character shapes may occur bidirectionally, by defining the correction operation to be limited to correction of character strings having a fixed number of characters instead of correction of single characters, the corrected character string is regarded as a correct word, thereby suppressing the occurrence of bidirectional similar character shape errors. In addition, since digits have a high possibility of bidirectional errors even as character strings, they are not targeted for automatic similar character shape correction.
[0086] In S807, when the image processing unit 305 determines that the correction operation matching the correction content is "digit count change of numeric character string", the process proceeds to S808, and when it is determined that the correction operation is not "digit count change of numeric character string", the process proceeds to S809.
[0087] In S808, the image processing unit 305 constructs a correction operation for digit count change of a numeric character string. In the following, the correction operation constructed when the user corrects "April 3, 2019" to "April 03, 2019" will be described as an example. First, a regular expression is used to set "\d{1,4}\D+\d{1,2}\D+\d{1,2}" (1 to 4 digits, 1 to 2 digits, and 1 to 2 digits, separated by a character string of 1 or more characters composed of non-digit characters) as the partial character string to be corrected. Subsequently, a regular expression is used to set "\d{04}\D+\d{02}\D+\d{02}" (4 zero-padded digits, 2 digits, and 2 digits, separated by a character string of 1 or more characters composed of non-digit characters) as the corrected partial character string.
[0088] However, in practice, a plurality of processes as described below are performed on the three numerical parts "2019", "4", and "3" mentioned above. First, replacement is performed to add leading zeros for the specified number of digits, as "0000\d", "00\d", and "00\d" respectively. Subsequently, by replacing "0*\d" with "\d{4}", "0*\d" with "\d{2}", and "0*\d" with "\d{2}" respectively again, a process is performed to obtain zero-padded numbers with the specified number of digits. For convenience, in the present embodiment, the plurality of correction operations described above are represented as a single correction operation using the regular expressions described above.
[0089] Incidentally, in the example described above, the number of digits of the part "2019" is not changed, but since correction for changing the number of digits (fixing the number of digits) has been performed on other numerical values in the item, a correction operation is constructed to fix the number of digits for all numerical values in the item.
[0090] Further, for example, there is a possibility that the user corrects "April 03, 2019" to "April 3, 2019". In this case, contrary to the example described above, first, a regular expression is used to set "\d{4}\D+\d{2}\D+\d{2}" (4 digits, 2 digits, and 2 digits separated by one or more characters composed of non-numeric characters) as the partial string to be corrected. Subsequently, a regular expression is used to set "\d{1,4}\D+\d{1,2}\D+\d{1,2}" (1 to 4 digits, 1 to 2 digits, and 1 to 2 digits separated by one or more characters composed of non-numeric characters) as the corrected partial string.
[0091] In S809, if the image processing unit 305 determines that the correction operation matching the correction content is "change of common noun (deletion, conversion, addition)", the process proceeds to S810, and if it is determined that the correction operation is not "change of common noun (deletion, conversion, addition)", the process proceeds to S811.
[0092] In S810, the image processing unit 305 constructs a correction operation for common noun modification. For the correction operation of common noun modification, for example, when the user corrects "ABC Co., Ltd." to "ABC Kabushiki Kaisha", using "Co., Ltd." and "Kabushiki Kaisha" registered in the common noun dictionary, "(Co., Ltd.)" is set as the partial character string to be corrected, and "Kabushiki Kaisha" is set as the corrected partial character string. In addition, a correction operation for newly adding a character string registered in the common noun dictionary to the beginning or end of a character string, or before or after a specific character string, or a correction operation for deleting a common noun may be constructed. Although not implemented in the present embodiment, words in the common noun dictionary may be associated with each other as synonyms, and conversion of common nouns may be performed only between synonyms.
[0093] In S811, when the image processing unit 305 determines that the correction operation matching the correction content is "delimiter change (deletion, conversion, addition)", the process proceeds to S812, and when it is determined that the correction operation is not "delimiter change (deletion, conversion, addition)", the process proceeds to S813.
[0094] In S812, the image processing unit 305 constructs a correction operation for delimiter change. For the constructed correction operation, for example, when the user corrects "03-0000-0000" to "0300000000", using "-" registered in the delimiter dictionary, "-" is set as the partial character string to be corrected. In addition, "" (no character) is set as the corrected partial character string. Further, the correction operation constructed when the user corrects "April 03, 2019" to "2019.04.03" will be described below as an example. First, using "year", "month", and "." registered in the delimiter dictionary, "[yearmonth]" (either year or month) is set as the partial character string to be corrected using a regular expression, and "\." (dot) is set as the corrected partial character string using a regular expression. Further, in order to delete the trailing "day", using "day" registered in the delimiter dictionary, "day" is set as the partial character string to be corrected, and "" (no character) is set as the corrected partial character string.
[0095] After executing S806, S808, S810, and S812, the process proceeds to S813.
[0096] In S813, the image processing unit 305 determines whether or not the correction operation construction processing has been performed for all the correction contents determined in S801 to S812. If the correction operation construction processing has not been performed for all the correction contents, the process returns to S804. If the correction operation construction processing has been performed for all the correction contents, the process proceeds to S814.
[0097] In S814, the image processing unit 305 determines whether or not one or more correction operations have been constructed. If one or more correction operations have not been constructed, the correction rule construction processing for the character string of the extracted item is ended. If one or more correction operations have been constructed, the process proceeds to S815.
[0098] In S815, for each extracted item for which one or more correction operations have been constructed, the image processing unit 305 determines whether or not the automatic correction rule application has already been performed on the character string of that extracted item. If the correction rule has not been automatically applied, the process proceeds to S817, and if the correction rule has already been automatically applied, the process proceeds to S816.
[0099] In S816, the image processing unit 305 inherits inheritable correction operations from among the correction operations included in the automatically applied correction rule. Here, the inheritable correction operation refers to a correction operation that, among the user's character string correction results acquired in S800, the corrected partial string is not the partial string to be corrected among the correction operations included in the automatically applied correction rule. As an example, description will be given of a case where "April 3, 2019" is extracted under a correction rule that only includes a correction operation for correcting the partial string to be corrected "\s" (blank character) to the corrected partial string "" (no character). First, the character string is automatically corrected to "April 3, 2019" by the above-described correction rule. Subsequently, for example, if the user corrects the string to "2019 4 3" (spaces as delimiters), the space newly inserted by the user matches the partial string to be deleted, so this correction operation is a non-inheritable correction operation.
[0100] In S817, the image processing unit 305 constructs a new correction rule using all the constructed correction operations and the correction operations inherited in S816. At this time, correction operations that result in identical substrings after correction are merged. Specifically, in correction operations that change delimiters, the delimiters set in the substring to be corrected are merged, and if there are more than a specified number of delimiters, all delimiters are treated as the substring to be corrected. In addition, all the constructed correction operations and all the correction operations inherited in S816 are rearranged based on the priority of the correction content as described above. Finally, a new correction rule ID is assigned to the newly constructed correction rule.
[0101] Table 3 shows an example of how a correction rule is added to Table 2 when the user changes the extracted string "2019 / 4 / 3" for the extracted item ID "03" in the scanned image in Figure 5(a) to "20190403".
[0102] Furthermore, although the aforementioned modification rule construction process (S404) has created and registered a new modification rule with modification rule ID "3", whether or not to set it as an automatic application target will be determined by the instructions received in S408. Therefore, in Table 3, no modification target IDs are set.
[0103] [Table 3] (Process for generating explanations of correction rules) The details of the correction rule explanation generation process (S406) are explained using Table 4.
[0104] Table 4 is a table showing standard phrases for explanatory texts. In this embodiment, standard phrases corresponding to each modification operation constructed in the modification rule construction process (S404) are stored in a storage device 112 such as the ROM 118 of the image forming apparatus 100.
[0105] [Table 4]
[0106] In the correction rule description text generation process (S406), the description text of the correction rule is generated using the fixed phrases shown in Table 4. Here, the character strings enclosed in curly brackets {} are treated as variables, and are replaced with the number of digits, common nouns, and the like set as the partial character string to be corrected by the correction rule. The character strings enclosed in square brackets [] are repeated zero or more times according to the number of variables enclosed in curly brackets {}. For the character strings enclosed in parentheses () and separated by a vertical line "|", one of them is selected according to the condition. Regarding the selection conditions, in the case of "(in the character string |)", the condition for selecting "in the character string" is that the correction operation has a correction operation number of "1". In addition, in the case of "(do.|and,)", the condition for selecting "do." is that the correction operation has the largest correction operation number. Further, in the case of "(beginning | between "{character}" and "{character}" | end)", the selection is made according to the insertion position of the character string to be added.
[0107] Here, as an example of description text generation using fixed phrases, description texts generated by fixed phrases for correction rule IDs "1", "2", and "3" shown in Table 3 are presented.
[0108] The description text of the correction rule with correction rule ID "1" will be explained. This correction rule corrects the space (blank character) represented by the regular expression "\s" to "" (no character) as delimiter deletion with correction operation number "1". Therefore, the description text generated using the fixed phrase is "Delete the "space" in the character string."
[0109] The description text of the correction rule with correction rule ID "2" will be explained. For this correction rule, first, as common noun conversion with correction operation number "1", the special character "(Co., Ltd.)" is corrected to "Kabushiki Gaisha". Subsequently, as delimiter deletion with correction operation number "2", the space (blank character) represented by the regular expression "\s" is corrected to "" (no character). Therefore, the description text generated using the fixed phrase is "Convert "(Co., Ltd.)" in the character string to "Kabushiki Gaisha", and delete the "space"."
[0110] This explains the description of the correction rule with correction rule ID "3". This correction rule first changes the number of digits in the numeric string with correction operation number "1". The substring to be corrected is a substring that conforms to the regular expression "\d{1,4}\D+\d{1,2}\D+\d{1,2}" (where 1 to 4 digits, 1 to 2 digits, and 1 to 2 digits are separated by a string of one or more non-numeric characters). Then, the substring to be corrected is modified to conform to the regular expression "\d{04}\D+\d{02}\D+\d{02}" (where 4 zero-padded digits, 2 digits, and 2 digits are separated by a string of one or more non-numeric characters). Subsequently, with correction operation number "2", the delimiter is removed by modifying a substring that conforms to the regular expression "[\s / ]" (either a whitespace character or / ) to "" (no character). Therefore, the generated pre-defined explanation will read: "Converts the numbers in the string to '4' digits, '2' digits, and '2' digits, and removes 'spaces' and ' / '."
[0111] In this embodiment, in addition to the explanatory text for the modification rule using a standard phrase, explanatory information is generated that includes the modification result when the modification rule is constructed, and as an example of modification, the string obtained when the modification rule is applied to a string of system information such as the date held by the image forming apparatus 100.
[0112] For example, based on the date string (year=2023, month=11, day=14) contained in the system information, strings are generated by applying multiple modification rules such as "2023 / 11 / 14", "20231114", "2023.11.14", and "2023-11-14". Also, based on the company name string (name=DEF Corporation, legal entity=corporation) contained in the system information, strings are generated by applying multiple modification rules such as "DEF Corporation", "DEF(Co., Ltd.)", and "DEF". Among the strings generated based on the system information, the string with the highest similarity to the substring targeted for modification by the presented modification rule is used as the pre-modification string for the modification example. The selection criteria may be the string with the highest similarity, or multiple strings whose similarity exceeds a threshold.
[0113] (Presentation of explanatory text for the revised rules and acceptance of instructions regarding the revised rules) The presentation of explanatory texts for the revised rules (S407) and the acceptance of instructions regarding the revised rules (S408, S409) will be explained using Figure 9 and Table 5.
[0114] Figure 9 shows the screen for displaying the explanatory text of the modification rule and receiving instructions for the modification rule according to this embodiment. It is an example showing explanatory information including the explanatory text for the modification rule with modification rule ID "3" shown in Table 3. When the confirmation process (S402) is completed and a new modification rule is constructed, the modification rule confirmation screen 900 is overlaid on the confirmation / modification screen. The upper part of the modification rule confirmation screen 900 displays the modification result 901 at the time of modification rule construction. The middle part of the modification rule confirmation screen 900 has a modification rule explanation area 902, where the explanatory text 903 of the modification rule and an example of modification rule application 904 are displayed. Buttons 905 and 906 are located at the bottom of the modification rule confirmation screen 900. Button 905 is a button that instructs the automatic application setting of the modification rule corresponding to the presented explanatory information, and button 906 is a button that instructs not to automatically apply the rule.
[0115] The image processing unit 305 displays the correction result 901 when the correction rule is constructed, the correction rule explanation text 903, and the correction rule application example 904 in the correction rule explanation area 902 via the display and operation unit 123. It also detects when button 905 or button 906 is pressed and receives instructions on whether or not to apply the displayed correction rule.
[0116] Table 5 shows an example of a correction rule after receiving an instruction to automatically apply a correction rule. It shows the correction rule when button 905 is pressed and an instruction to set automatic application of correction rule ID "3" is received. Upon receiving the instruction to set automatic application of correction rule ID "3", "0001_03" is newly set as the target ID for correction rule ID "3". In this embodiment, a maximum of one correction rule is applied to each target ID, so "0001_03" is deleted from the target ID for correction rule ID "1". As a result, the correction rule confirmed by the user will be applied the next time a similar document is scanned. If button 906 is pressed and an instruction not to automatically apply correction rule ID "3" is received, the newly constructed correction rule for correction rule ID "3" will be deleted.
[0117] [Table 5]
[0118] Furthermore, while an example of a correctly constructed correction rule was given, for instance, if a user corrects "ABC" to "ABC Corporation," the correction rule will be constructed to add a common noun, inserting "Corporation" at the end. However, if the above correction rule is set to be automatically applied, then in the future, if "ABC Corporation" or "ABC(Co.)" are recognized as strings with the same extracted item ID in similar documents, they will be corrected to strings such as "ABC Corporation Corporation" or "ABC(Co.) Corporation." Also, if "Corporation" precedes the company name, or if the legal entity is not "Corporation," there is a possibility of unintended string corrections. Therefore, this correction rule is likely not the rule the user intended. However, by checking the explanatory text "Insert "Corporation" at the end," which is generated in this embodiment, the user can confirm whether it is the intended correction rule or not, thus suppressing the automatic application of unintended correction rules.
[0119] [Effects of Embodiment 1] As explained above, according to this embodiment, it is possible to determine whether the constructed modification rule is the modification rule intended by the user by checking the explanatory text and application examples written in simple language, thereby suppressing the automatic correction of incorrect modification rules.
[0120] Furthermore, in this embodiment, by displaying not only explanatory text about the modification operations that constitute the modification rules as explanatory information, but also the results of the modifications performed by the user and examples of their application, the user's understanding of the modification rules can be improved. still,
[0121] <Embodiment 2> In Embodiment 1, the system is configured to present the user with an explanatory text for the modification rule generated based on a standard phrase and to accept instructions for applying the modification rule. However, the method for generating the explanatory text for the modification rule is not limited to the method described above; for example, a trained natural language generation AI may be used to generate a text explaining the modification operation of the modification rule. On the other hand, in any explanatory text generation method, if the modification rule contains a large number of modification details, the explanatory text may become complex and difficult for the user to understand. Therefore, in this embodiment, as a means to help the user understand the explanatory text, if the user finds it difficult to understand, a more detailed explanatory text is presented. In this embodiment, the processing prior to the modification rule explanatory text generation process (S406) is the same as in Embodiment 1, so only the distinctive parts will be described.
[0122] [Features of Embodiment 2] The presentation of the explanatory text for the modification rules, the acceptance of instructions for the modification rules, and the processing of updating the explanatory text for the modification rules according to this embodiment will be explained in detail with reference to Figure 10.
[0123] Figure 10(a) shows a screen for displaying the explanation of a correction rule, receiving instructions regarding the correction rule, and receiving instructions for displaying a more detailed explanation of the correction rule. It is an example showing the explanation for correction rule ID "3" as shown in Table 3.
[0124] Similar to Embodiment 1, when a confirmation process is performed in the confirmation / correction process (S402) and a new correction rule is constructed, the correction rule confirmation screen 1000a is overlaid on the confirmation / correction screen. The upper part of the correction rule confirmation screen 1000a displays the correction result 1001 at the time the correction rule was constructed. The middle part of the correction rule confirmation screen 1000a contains the correction rule explanation area 1002, where the correction rule explanation text 1003 and the correction rule application example 1004 are displayed. Buttons 1005 and 1006 are located at the bottom of the correction rule confirmation screen 1000a. Button 1005 is a button that instructs the automatic application setting of the correction rule corresponding to the presented explanation information, and button 1006 is a button that instructs not to apply it automatically. In this embodiment, unlike Embodiment 1, button 1007 is located within the correction rule explanation area 1002. Button 1007 is a button used to instruct the display of the detailed screen of the correction rule explanation. When the image processing unit 305 detects that button 1007 has been pressed via the display / operation unit 123, it displays the detailed screen of the correction rule explanation.
[0125] Figure 10(b) shows the screen after button 1007 is pressed and the modification rule details screen 1000b is displayed. The modification operation number area 1008 on the modification rule details screen 1000b displays the modification operation number assigned to each unit modification operation that constitutes the modification rule. The individual modification operation explanation area 1009 displays an explanation for each unit modification operation. The modification content area 1010 displays the modification content for each unit modification operation as shown in the "Modification Content" column of Table 3. Button 1011 is located at the bottom of the modification rule details screen 1000b and is a button that instructs the user to return to the modification rule confirmation screen 1000a.
[0126] When a user requests a more detailed explanation and instructs the system to select either the modification operation number in the modification operation number area 1008 or one of the explanatory texts in the individual modification operation explanation area 1009, the image processing unit 305 detects the selection via the display / operation unit 123. The system then displays the detailed explanatory text of the selected modification operation on the display / operation unit 123. The detailed explanatory text could include, for example, examples of the text before and after the modification operation, or highlighting parts of the string used in the examples that match the regular expression to be modified by changing the text color, font size, or font style (bold, italics, etc.). This makes it easier to determine which parts of the string are modified and how in each modification.
[0127] In this description of the embodiment, we have described a configuration in which user instructions are received through button presses or item selections. However, for example, the system may also receive instructions regarding a string entered by the user in a text box or a string entered by the user via speech recognition, and a trained natural language generation AI may perform rephrasing of the explanatory text or add more detailed explanations in a dialogue format. Furthermore, the corrected results of the user-entered string may be output as an example of the corrected text.
[0128] [Effects of Embodiment 2] As explained above, according to this embodiment, it is possible to determine whether the constructed modification rule is the modification rule intended by the user by checking the explanatory text written in simple language, and then judge the appropriateness of the modification rule, thereby suppressing erroneous automatic modifications.
[0129] Furthermore, in this embodiment, the user's understanding of the modification rules can be improved by updating the user-specified portion of the explanatory text into a detailed explanatory text.
[0130] <Embodiment 3> In Embodiment 1, when a confirmation process (S402) is performed and a new correction rule is constructed, a description of the correction rule is presented to the user on the confirmation / correction screen. The user then instructs whether or not to automatically apply the constructed correction rule to the extracted item. However, there are cases where the user wants to apply the constructed correction rule to other extracted items within the same document, or to all extracted items in a document used by the same user. Also, if there are multiple conflicting correction rules for the same extracted item, the user may want to be able to check the application column for those multiple correction rules before selecting which one to use. Therefore, in this embodiment, multiple candidate correction rules are presented for the extracted item, and instructions are accepted for outputting the application column for those multiple correction rules, setting the automatic application range of the correction rule, and deleting the correction rule. Below, only the confirmation / correction process (S402), which is a feature of this embodiment and differs from Embodiment 1, will be described, including the form in which candidate correction strings and candidate correction rules for the extracted item are presented to the user within the confirmation / correction process (S402) and various instructions are accepted.
[0131] [Features of Embodiment 3] The verification and correction process (S402) according to this embodiment will be explained in detail with reference to Figure 11.
[0132] Figure 11(a) shows the confirmation / correction screen, which is displayed on the display / operation unit 123 when a registered document of the same type as the input document is identified and a string is extracted during the string extraction process (S401). Similar to Embodiment 1, the file name text field 1100, the extracted item information area 1101, and the extracted item text field 1102 are displayed. However, the difference from the confirmation / correction screen of Embodiment 1 is that a correction rule candidate display button 1103 is additionally placed to the right of the extracted item text field 1102 in the extracted item information area 1101. Note that if there are no correction rule candidates for the item to be extracted, the correction rule candidate display button 1103 will not be displayed.
[0133] When the correction rule suggestion display button 1103 is detected to have been pressed, the correction rule suggestion display screen for the pressed extraction target item is displayed as an overlay.
[0134] Figure 11(b) shows a confirmation / correction screen displaying multiple correction rule candidates. It shows the screen where the correction rule candidate display button 1103 for extraction item ID "03" has been pressed, and the correction rule candidate display screen 1104 is displayed as an overlay. In this embodiment, two correction rule candidates are registered for extraction item ID "03," and this shows the display when neither is set to be automatically applied.
[0135] The correction rule candidate display screen 1104 displays a correction rule candidate area 1105 for each correction rule candidate. The correction rule candidate area 1105 displays and arranges the correction rule application column 1106, the correction rule description text 1107, the automatic correction rule application setting checkbox 1108, the correction rule application range setting button 1109, and the correction rule delete button 1110.
[0136] The column 1106, which displays the corrected string obtained by applying the correction rule to the string that is the character recognition result of the extracted item, shows the corrected string.
[0137] The explanation text generated during the revision rule explanation text generation process (S406) is displayed in revision rule explanation text 1107.
[0138] The automatic correction rule application setting checkbox 1108 is checked when the automatic correction rule application setting is enabled. When it detects that the checkbox has been clicked, it accepts the instruction to apply the automatic correction rule to the selected item, applies the automatic correction rule, and checks the checkbox. When the automatic correction rule is applied, the string displayed in the extracted item text field 1102 is corrected according to the correction rule, and the string displayed in the file name text field 1100 is also corrected in the same way.
[0139] When the correction rule application range setting button 1109 is detected to have been pressed, the application range setting screen 1111 is overlaid on the confirmation / correction screen as shown in Figure 11(c). The application range setting screen 1111 displays and arranges the application range display name area 1112 and the application range setting slider bar 1113. When an application range setting instruction is received by changing the position of the application range setting slider bar 1113, the application range is set according to the application range display name displayed at the horizontal position of the application range setting slider bar 1113.
[0140] The delete modification rule button 1110, upon detecting that it has been pressed, accepts the delete instruction and deletes the target modification rule.
[0141] When a selection is detected within the correction rule candidate area 1105, excluding the correction rule automatic application setting checkbox 1108, the correction rule application range setting button 1109, and the correction rule delete button 1110, the correction rule application instruction is accepted. Then, the string displayed in the extracted item text field 1102 is corrected according to the correction rule, and the string displayed in the file name text field 1100 is corrected in the same way.
[0142] [Effects of Embodiment 3] As explained above, according to this embodiment, it is possible to determine whether the constructed modification rule is the modification rule intended by the user by checking the explanatory text written in simple language, and then judge the appropriateness of the modification rule, thereby suppressing erroneous automatic modifications.
[0143] Furthermore, in this embodiment, multiple correction rule candidates are held for a single extracted item, and the system is configured to accept the user's selection of a correction rule to be automatically applied after presenting the correction results when multiple correction rule candidates are applied to the string. This allows the user to use different correction rules depending on the situation, making it easy to set the correction rules as intended even in documents that require various corrections, thereby reducing the user's burden of string correction work. In addition, since the scope of application of correction rules can be set, the time spent checking explanations of unnecessary correction rules or explanations of the same correction rule can be reduced.
[0144] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.
[0145] This disclosure includes the following configurations and methods:
[0146] (Composition 1) A means for obtaining a string of characters contained in an input document image and a modified string of characters after the user has modified that string, A generation means for generating a modification rule for obtaining the modified string from the string, based on the string and the modified string, An output means that outputs explanatory information including the explanatory text of the correction rule generated by the generation means, An information processing device characterized by having the following features.
[0147] (Configuration 2) A receiving means that displays the aforementioned explanatory information and receives instructions from the user on whether or not to use the aforementioned modification rules. The information processing device according to configuration 1, further comprising the features described above.
[0148] (Composition 3) The instruction to use the aforementioned correction rule is an instruction to apply the aforementioned correction rule to strings contained in other document images. The information processing apparatus according to configuration 2, characterized in that...
[0149] (Composition 4) The generation means is capable of generating one or more of the correction rules for each predetermined extraction item in the string, The receiving means displays explanatory information for one or more of the correction rules generated for one of the extracted items. An information processing apparatus according to configuration 2 or 3, characterized by the above.
[0150] (Composition 5) Multiple correction rules generated for the same extracted item are conflicting correction rules. The information processing apparatus according to configuration 4, characterized by the features described above.
[0151] (Composition 6) The aforementioned receiving means receives instructions from the user to delete the aforementioned modification rule. An information processing device according to any one of configurations 2 to 5, characterized in that it is an information processing device.
[0152] (Composition 7) The receiving means accepts user input specifying a part of the explanatory information, The output means outputs additional information relating to the part of the explanatory information. An information processing device according to any one of configurations 2 to 6, characterized by the above.
[0153] (Composition 8) The aforementioned receiving means receives the scope of application of the aforementioned modification rule from the user. An information processing device according to any one of configurations 2 to 7, characterized by the above.
[0154] (Composition 9) The generation means generates the correction rules based on the unit correction operations that constitute the corrections made by the user, The output means outputs a descriptive text for each of the predetermined unit modification operations included in the modification rule as descriptive information. An information processing device according to any one of configurations 1 to 8.
[0155] (Composition 10) The generation means identifies the predetermined unit modification operation based on the regular expressions of the string and the modified string, which are registered for each predetermined unit modification operation. The information processing apparatus according to configuration 9, characterized by the features described therein.
[0156] (Composition 11) The output means outputs the result of applying the correction rule to the string as the explanatory information. An information processing apparatus according to any one of configurations 1 to 10, characterized by the above.
[0157] (Composition 12) The output means outputs the result of applying the modification rule to the system information held by the information processing device as explanatory information. An information processing device according to any one of configurations 1 to 11, characterized by the above.
[0158] (Composition 13) The system further includes a receiving means for receiving user input that specifies a portion of the aforementioned explanatory information, The output means outputs a portion of the explanatory information specified by the user in a different form. An information processing apparatus according to any one of configurations 1 to 12, characterized by the above.
[0159] (Composition 14) The aforementioned string is obtained by performing character recognition processing on the input document image. An information processing device according to any one of configurations 1 to 13, characterized by the above.
[0160] (Composition 15) An output means that outputs explanatory information about correction rules for correcting strings, which includes an explanatory text for the correction rules, A display control means that causes the outputted explanatory information to be displayed on a display means, An information processing device characterized by having the following features.
[0161] (Composition 16) The output means outputs an explanatory text of the modification rule, which includes an explanatory text for each of the predetermined unit modification operations that constitute the modification rule. An information processing apparatus according to any one of configurations 1 to 15, characterized by the above.
[0162] (Composition 17) The steps include obtaining the string contained in the input document image and the modified string after the user has modified that string, A step of generating a modification rule for obtaining the modified string from the string based on the string and the modified string, A step of outputting explanatory information including an explanatory text for the correction rule generated by the above generation step, An information processing method characterized by having the following features.
[0163] (Composition 18) A program for causing a computer to function as an information processing device as described in any one of items 1 to 16.
Claims
1. A means for obtaining a string of characters contained in an input document image and a modified string of characters after the user has modified that string, A generation means for generating a modification rule for obtaining the modified string from the string, based on the acquired string and the modified string, An output means that outputs explanatory information including an explanatory text for the generated correction rule, An information processing device characterized by having the following features.
2. A receiving means that displays the aforementioned explanatory information and receives instructions from the user on whether or not to use the aforementioned modification rules. The information processing apparatus according to claim 1, further comprising the features described above.
3. The instruction to use the aforementioned correction rule is an instruction to apply the aforementioned correction rule to strings contained in other document images. The information processing apparatus according to feature 2.
4. The generation means is capable of generating one or more of the correction rules for each predetermined extraction item in the string, The receiving means displays explanatory information for one or more of the correction rules generated for one of the extracted items. The information processing apparatus according to feature 2.
5. Multiple correction rules generated for the same extraction item are conflicting correction rules. The information processing apparatus according to feature 4.
6. The aforementioned receiving means receives instructions from the user to delete the aforementioned modification rule. The information processing apparatus according to feature 2.
7. The receiving means accepts user input specifying a part of the explanatory information, The output means outputs additional information relating to the part of the explanatory information. The information processing apparatus according to feature 2.
8. The aforementioned receiving means receives the scope of application of the aforementioned modification rule from the user. The information processing apparatus according to feature 2.
9. The generation means generates the modification rule based on a predetermined unit modification operation that constitutes the modification performed by the user, The output means outputs a descriptive text for each of the predetermined unit modification operations included in the modification rule as descriptive information. The information processing apparatus according to feature 1.
10. The generation means identifies the predetermined unit modification operation based on the regular expressions of the string and the modified string, which are registered for each predetermined unit modification operation. The information processing apparatus according to feature 9.
11. The output means outputs the result of applying the correction rule to the string as the explanatory information. The information processing apparatus according to feature 1.
12. The output means outputs the result of applying the modification rule to the system information held by the information processing device as explanatory information. The information processing apparatus according to feature 1.
13. The system further includes a receiving means for receiving user input that specifies a portion of the aforementioned explanatory information, The output means outputs a portion of the explanatory information specified by the user in a different form. The information processing apparatus according to feature 1.
14. The aforementioned string is obtained by performing character recognition processing on the input document image. The information processing apparatus according to feature 1.
15. An output means that outputs explanatory information about correction rules for correcting strings, which includes an explanatory text for the correction rules, A display control means for displaying the generated explanatory information on a display means, An information processing device characterized by having the following features.
16. The output means outputs an explanatory text of the modification rule, which includes an explanatory text for each of the predetermined unit modification operations that constitute the modification rule. The information processing apparatus according to feature 15.
17. The steps include obtaining the string contained in the input document image and the modified string after the user has modified that string, A step of generating a modification rule for obtaining the modified string from the string based on the string and the modified string, A step of outputting explanatory information including an explanatory text for the correction rule generated by the above generation step, An information processing method characterized by having the following features.
18. A program for causing a computer to function as an information processing device according to any one of claims 1 to 16.
Citation Information
Patent Citations
Information processing apparatus
JP2021163178A
