Information processing device, control method for information processing device, and program

Abstract

To provide a mechanism for displaying a character string corresponding to an attribute in image data to be edited and receiving a specification from a user as to whether to select the displayed character string as a target for masking.SOLUTION: A screen including image data and a plurality of items indicating types of character strings is provided, and instructions regarding masking based on a user's specification via the screen are accepted, and on the screen, one or more character strings of the types indicated by each item contained in the image data are displayed in correspondence with each other, and the user's specification is a specification to select or deselect any one of the plurality of items and the plurality of character strings displayed in correspondence with each of the items, and when a specification to select is made, the selected character string is masked, and when a specification to deselect is made, the deselected character string is displayed without masking.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a technique for masking a scanned image. [Background technology]

[0002] When documents that include personal information or confidential information, such as identity documents, application documents, or drawings, are shared with others, parts of the document may be masked by filling them in (also known as "redaction" or "masking"). Patent Document 1 discloses a technology that masks confidential information corresponding to pre-registered attributes such as "name," "date of birth," and "address." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-73172 Summary of the Invention [Problem to be solved by the invention]

[0004] When masking confidential information corresponding to an attribute as in Patent Document 1, it is considered that different character strings are masked for different image data. Therefore, there may be cases where a user wants to check a character string corresponding to an attribute in image data to be edited and specify whether or not to select the character string as a target for masking.

[0005] Therefore, an object of the present invention is to provide a mechanism for displaying character strings corresponding to attributes in image data to be edited and receiving a specification from the user as to whether or not to select the displayed character string as a target for masking. [Means for solving the problem]

[0006] In order to achieve the above object, the information processing device of the present invention has a providing means for providing a screen including image data to be edited and a plurality of items indicating types of character strings, and a receiving means for receiving an instruction regarding masking of the image data based on a user's specification via the screen, wherein on the screen, for each of the plurality of items, one or more character strings of the type indicated by each item contained in the image data are displayed in association with each of the plurality of items, the user's specification is a specification for selecting or deselecting any of the plurality of items and a plurality of character strings displayed in association with each of the items, and when an item is selected by the user's specification, all of the plurality of character strings displayed in association with the item are masked on the image data of the screen, and when a character string is selected, the character string is displayed without masking on the image data of the screen, and when an item is deselected by the user's specification, all of the plurality of character strings displayed in association with the item are displayed without masking on the image data of the screen, and when a character string is deselected, the character string is displayed without masking on the image data of the screen. Effect of the Invention

[0007] According to the information processing device of the present invention, a mechanism can be provided in which a character string corresponding to an attribute in image data to be edited is displayed and a user's specification of whether or not to select the displayed character string as a target for masking is received. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the overall configuration of the present system. [Diagram 2] FIG. 2 is a diagram illustrating the hardware configuration of an MFP. [Diagram 3] FIG. 2 is a hardware configuration diagram of a storage device. [Figure 4] FIG. 2 is a diagram illustrating the software configuration of the present system. [Diagram 5] A flowchart showing the flow of processing performed by the MFP in this system. [Figure 6]1 is a flowchart of an area designation process performed by the MFP in this system. [Figure 7] 13 is a diagram showing an example of a mask designation method selection screen displayed by the MFP. [Figure 8] FIG. 13 is a diagram illustrating an example of a preset selection screen displayed by the MFP. [Figure 9] FIG. 13 is a diagram illustrating an example of a scan screen displayed by the MFP. [Figure 10] FIG. 13 is a diagram illustrating an example of a preview screen displayed by the MFP. [Figure 11] FIG. 13 is a diagram illustrating an example of a preview screen displayed by the MFP. [Figure 12] 13 is a diagram showing an example of an execution confirmation screen displayed by the MFP. [Figure 13] 11 is a diagram showing an example of area information stored in the MFP. [Figure 14] 11 is a diagram illustrating an example of preset information stored in the MFP. [Figure 15] FIG. 13 is a diagram showing an example of a preset name setting screen displayed by the MFP. [Figure 16] 11 is a diagram showing an example of an attribute-classified OCR character string stored in the MFP. FIG. [Figure 17] FIG. 13 is a diagram showing an example of a screen for selecting an attribute displayed by the MFP. [Figure 18] 11A and 11B are diagrams illustrating an example of attributes, OCR character strings, and item names stored in the MFP. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims, and all of the combinations of features described in the embodiments are not necessarily essential to the solution of the invention.

[0010] <First embodiment> <Overall composition> FIG. 1 is a diagram showing the overall configuration of the present system. The image processing system of FIG. 1 includes an MFP (Multifunction Peripheral) 110 and an external storage 120. The MFP 110 is communicably connected to a server that provides various services on the Internet via a LAN (Local Area Network).

[0011] The MFP 110 is a multifunction device having a plurality of functions such as a scanner and a printer, and is an example of the information processing apparatus of the present invention. The MFP 110 also has a function of transferring a scanned image file to a service capable of file storage such as an external storage service. Note that the information processing apparatus of the present invention is not limited to a multifunction device having a scanner and a printer, and may be a PC.

[0012] The external storage (service) 120 is a service that can store files received via the Internet or acquire files from an external device via a web browser. The external storage 120 is, for example, a cloud service. There may be a plurality of external storages, not limited to just 120.

[0013] The image processing system of the present embodiment is configured to include the MFP 110, the external storage 120, and the external storage 130, but is not limited thereto. For example, a part of the functions and processes of the MFP 110 may be implemented on another server arranged on the Internet or the LAN. Also, the external storage 120 may be arranged on the LAN instead of on the Internet. Further, the external storage 120 may be replaced with a mail server or the like to attach and transmit a scanned image to an email. The MFP 110 may also have the storage function of the external storage 120.

[0014] <Hardware Configuration of MFP> FIG. 2 is a hardware configuration diagram of the MFP 110. The MFP 110 is composed of a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223. The control unit 210 is composed of the following units 211 to 219, and controls the operation of the entire MFP 110. The CPU 211 reads out a control program stored in the ROM 212 or the HDD 214, and executes and controls various functions of the MFP 110, such as reading, printing, and communication. The RAM 213 is used as a temporary storage area such as the main memory and work area of ​​the CPU 211. Note that in this embodiment, one CPU 211 executes each process shown in the flowchart described later using one memory (RAM 213 or HDD 214), but is not limited to this. For example, each process may be executed by cooperating multiple CPUs and multiple RAMs or HDDs. The HDD 214 is a large-capacity storage unit that stores image data and various programs.

[0015] The operation unit I / F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 is provided with a touch panel, a keyboard, and the like, and accepts operations / inputs / instructions by a user. The printer I / F 216 is an interface that connects the printer unit 221 and the control unit 210. Image data for printing is transferred from the control unit 210 to the printer unit 221 via the printer I / F 216 and printed on a recording medium. The scanner I / F 217 is an interface that connects the scanner unit 222 and the control unit 210. The scanner unit 222 reads an original placed on a document tray or an ADF (Auto Document Feeder), not shown, generates image data, and inputs the image data to the control unit 210 via the scanner I / F 217. The MFP 110 can print (copy) the image data generated by the scanner unit 222 from the printer unit 221, as well as transmit the image data as a file or by e-mail. The modem I / F 218 is an interface that connects the modem 223 and the control unit 210. The modem 223 communicates image data by facsimile with a facsimile device on the PSTN. The network I / F 219 is an interface that connects the control unit 210 (MFP 110) to a LAN. The MFP 110 uses the network I / F 219 to transmit image data and information to various services on the Internet and to receive various information.

[0016] <Hardware configuration of external storage> FIG. 3 is a hardware configuration diagram of the external storage 120. The external storage 120 is composed of a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I / F 315. The CPU 311 controls the overall operation by reading out the control program stored in the ROM 312 and executing various processes. The RAM 313 is used as a temporary storage area such as the main memory and work area of the CPU 311. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I / F 315 is an interface that connects the external storage 120 to the Internet. The external storage 120 receives a process request from another device (such as the MFP 110) via the network I / F 315 and transmits and receives various information.

[0017] <Software Configuration of MFP> FIG. 4 is a software configuration diagram of the image processing system according to this embodiment. The MFP 110 is roughly divided into two major parts: a native function unit 410 and an additional function unit 420. Each function unit is realized by the CPU 211 reading out and executing the program stored in the ROM 212 or HDD 214 of the MFP 110 into the RAM 213. Each part included in the native function unit 410 is standardly provided in the MFP 110, while the additional function unit 420 is an application additionally installed in the MFP 110. The additional function unit 420 is an application based on Java (registered trademark), and can easily realize the addition of functions to the MFP 110. Note that other additional applications (not shown) may be installed in the MFP 110.

[0018] The native function unit 410 has a scan execution unit 411, an image data storage unit 412, and a print execution unit 413. The additional function unit 420 has a display control unit 421, a scan instruction unit 422, an image processing unit 423, a data management unit 424, a print instruction unit 425, a data conversion unit 426, and an external storage access unit 427.

[0019] The display control unit 421 displays a UI (user interface) screen for accepting operations by a user on a liquid crystal display unit having a touch panel function of the operation unit 220 of the MFP 110. For example, it displays an operation screen for accepting operations for scan setting and scan start, a preview of a scanned image and a designation operation of a mask area described later, a preview of a mask composite image, and an operation for output setting and output start.

[0020] The scan instruction unit 422 requests the scan execution unit 411 to perform a scan process according to the scan settings input via the UI screen. The scan execution unit 411 receives a scan request including the scan settings from the scan instruction unit 422. In accordance with the scan request, the scan execution unit 411 generates scan image data by reading a document placed on a platen glass with the scanner 222 via the scanner I / F 217. The generated scan image data is sent to the image data storage unit 412. The scan execution unit 411 sends an image identifier that uniquely identifies the stored scan image data to the scan instruction unit 422. The image identifier is a number, symbol, alphabet, or the like (not shown) for uniquely identifying an image scanned in the MFP 110. The image data storage unit 412 stores the scan image data received from the scan execution unit 411 in the HDD 214.

[0021] The image processing unit 423 performs analysis and processing on the scanned image. The image processing unit 423 receives an image identifier from the scan instruction unit 422, and acquires scanned image data corresponding to the image identifier from the image data storage unit 412. The image processing unit 423 performs image recognition processing such as character area analysis in the acquired image data, OCR (Optical Character Recognition), barcode area analysis, and image rotation and tilt correction. Furthermore, the image processing unit 423 has in advance learning data for determining whether the scanned document is a fixed document or an unfixed document, and if the scanned image is similar to the learned fixed document based on the analysis result of the scanned image, it is determined to be a fixed document. Furthermore, the image processing unit 423 synthesizes a mask image according to an area designation process (described later in FIG. 6) according to a mask area input via a UI screen, and instructs the image data storage unit 412 to store the generated masked synthetic image data. The image processing unit 423 sends the image identifier to a print instruction unit 425, a data conversion unit 426, and an external storage access unit 427 in accordance with the output settings set for the image after the mask synthesis.

[0022] The data management unit 424 holds information such as coordinates, character strings, attributes, file names, and save destinations designated in the area designation process in association with the image identifier. In addition, the coordinates, character strings, attributes, file names, and save destinations designated in the area designation process are saved as presets in the HDD 214. In this embodiment, a preset is a template in which information indicating the coordinates, character strings, attributes, and file names and save destinations of the area to be masked in the scanned image are saved in association with the type of document. By selecting an item indicating the type of document displayed as a preset button on a screen 800 described later, it is possible to perform processing using the settings saved in the template without making individual settings again. The attributes will be described later.

[0023] The print instruction unit 425 sends to the print execution unit 413 a print processing request according to the print settings input via the UI screen and the image identifier received from the image processing unit 423. The print execution unit 413 receives the print request including the print settings and the image identifier from the print instruction unit 425. The print execution unit 413 obtains from the image data storage unit 412 corresponding to the image identifier and generates image data for printing according to the print request. The print execution unit 413 prints the mask composite image on a recording medium by the printer 221 via the printer I / F 216 according to the generated image data for printing.

[0024] The data conversion unit 426 obtains an image corresponding to the image identifier received from the image processing unit 423 from the image data storage unit 412, and converts it into a file format according to the output settings input via the UI screen. The data conversion unit 426 obtains a file name corresponding to the image identifier received from the image processing unit 423 from the data management unit, and sets it as the file name of the converted file. The data conversion unit 426 sends the converted file and the corresponding image identifier to the data management unit 424, and the data management unit 424 holds the received file in association with the image identifier.

[0025] The external storage access unit 427 transmits a processing request to a cloud service or the like that provides a storage function (storage service). Cloud services generally publish various interfaces using protocols such as REST and SOAP for saving files in cloud storage and retrieving saved files from external devices. The external storage access unit 426 operates the cloud service using the published interface of the cloud service. The external storage access unit 427 acquires a file and transmission information corresponding to the image identifier received from the image processing unit 423 from the data management unit 424. The external storage access unit 426 transmits the file acquired from the data management unit 424 to the external storage 120 via the network I / F using the transmission information acquired from the data management unit 424.

[0026] The attribute classification unit 428 classifies the character string extracted from the result of the OCR processing in the image processing unit 423 into attributes by natural language processing capable of understanding the meaning of the OCR character string, character recognition processing such as key-value extraction, etc. An attribute is an item indicating the type of character string. Examples of attributes include "date", "personal name", "email address", "telephone number", "company name", "product name", etc. The attribute classification unit 428 stores teacher data in which character strings and attributes are associated with each other, and determines which attribute the character string of the OCR result corresponds to by learning using the teacher data, and classifies it. Alternatively, the attribute classification unit 428 cooperates with the image processing unit 423 to extract a character string ("key") that becomes an item name from the character string obtained by OCR of the image data. For example, if the character string "date: xx month xx day" is included in the image data, "date" is determined to be the key. Then, the attribute classification unit 428 detects the character string ("value") corresponding to the key from the positional relationship with the character string of the key, etc., and classifies the value by associating it with the attribute of the item name of the key. In addition, a regular expression for a character string corresponding to an attribute may be preset, and a character string that matches the preset regular expression may be classified into that attribute. The attribute-classified OCR character string is stored in the RAM 213 with the attribute name and the OCR character string corresponding to each other, as shown in FIG.

[0027] <Overall process flow> In the processing described below, the CPU 211 of the MFP 110 reads out a control program stored in the ROM 212 or HDD 214, and executes and controls various functions of the MFP 110 and functions of additional applications.

[0028] 5 is a flowchart showing a process flow when a mask image is synthesized with an image scanned by the MFP 110, the file is converted, and the file is transmitted to cloud storage (external storage 120). Note that, in this embodiment, an example is described in which the display control unit 421 displays a screen on the touch panel of the operation unit 220, but the display control unit 421 may provide each screen in this embodiment to another device, and the operation unit of the other device may display each screen.

[0029] A display control unit 421 displays, on the touch panel of the operation unit 220, a main screen (not shown) on which buttons for executing applications provided by the MFP 110 are arranged.

[0030] An additional application of the present invention that masks a part of a scanned image and transmits it to a cloud service (hereinafter, referred to as a masking application) can be used by installing it on the MFP 110. When the masking application is installed on the MFP 110, a button for using the functions of the application is displayed on the main screen of the MFP 110.

[0031] When the pressing of the masking application button is detected, the flow of FIG. 5 starts, and in S501, the MFP 110 starts the masking application (reads and executes the program). When the masking application starts, in S502, the display control unit 421 of the MFP 110 displays the mask setting method selection screen 700 of FIG. 7 as the initial screen of the masking application. 701 is a button for instructing the MFP 110 to display a screen 800, which will be described later. The preset is a function for saving the setting values, such as the scan setting, mask area, and file saving information, once set, in the HDD 214 of the MFP 110 and reusing them. 702 is a button for instructing a process of manually setting various initial values ​​of the masking application. When it is detected that the user has selected any button on the mask setting method selection screen 700, in S503, the display control unit 421 determines whether the "preset" of 701 or the "manual" of 702 has been selected.

[0032] When the selection of "Manual" in 702 is detected, the determination in S503 becomes "No", and in S505 the display control unit 421 displays the scan screen 900 in FIG. 9. 901 is an item for specifying the output method of the masked scanned image. The output method can be selected from print only, save only, or print and save. 902 is an item for specifying the file format at the time of saving that is valid when the setting of 901 is set to "print and save" or "save only". 903 is an item for specifying the scan settings, and accepts setting changes of "original size", "color", "double-sided", and "detailed settings". In "detailed settings", settings such as resolution, image quality, mixed original sizes, and density are accepted. 904 is a button for executing a scan using the scan settings set in 903. 905 is an area for displaying the number of pages scanned. 906 is a button for returning to the previous screen. If the determination in S503 is "No", the scan screen 900 is displayed with the output settings and scan settings held by the MFP 110 as the initial values ​​of the masking application set to 901 to 903.

[0033] On the other hand, when the selection of "preset" 701 is detected, S503 becomes "Yes", and in S504, the display control unit 421 displays the preset list screen 800 of FIG. 8. 801 is an example of a registered preset button. The preset button is a button corresponding to a template stored in the data management unit 424. When it is detected that any preset button is selected, the setting value of the template stored in association with the button ID of the preset button is used in the subsequent processing. The button ID is an ID generated by the data management unit 424 when the preset button is registered, and is a number, symbol, alphabet, etc. for uniquely identifying a group of registered setting values ​​(not shown). 802 is a button for returning to the mask setting method selection screen 700 of FIG. 7. For example, when it is detected that "investigation report" 801 is selected, the display control unit 421 reads out the setting related to the button ID of "investigation report" and displays the scan screen 900. At this time, the setting related to the button ID is reflected in the output setting and scan setting of 901 to 903.

[0034] When the operation unit 220 detects that the user has placed a document to be scanned on the document glass or ADF and pressed the "scan" button 904, the MFP 110 executes scanning in S506 and generates an image identifier that uniquely indicates image data obtained by digitizing the document. The generated scanned image and image identifier are associated with each other and stored in the HDD 214. Each time the MFP 110 completes the generation of a scanned image for one page, the number of scanned pages 905 increases.

[0035] When scanning of the document to be scanned is completed and pressing of "Next" 907 is detected, the image processing unit 423 executes an analysis process of the scanned image in S507. In the scanned image analysis process, the image processing unit 423 detects a character area existing in the image and acquires the coordinates of the detected character area. In addition, the image processing unit 423 executes a character recognition process on the scanned image and extracts a character string in the detected character area. Note that the image processing unit 423 may execute the character recognition process only on the image data included in the detected character area. Then, the coordinates of the acquired character area and information on the extracted character string are stored in the HDD 214 as the analysis result in association with the image identifier. Furthermore, the attribute classification unit 428 classifies the character string obtained by the character recognition process by the image processing unit 423 into attributes by the method described above.

[0036] When the process of the scan image analysis in S507 is completed, the MFP 110 executes an area designation process in S508. The area designation process is a process of accepting a designation of an area to be masked on the scan image and generating a composite image in which the designated area is masked. In this embodiment, the designation of the area to be masked is accepted by selecting a character string to be masked from among character strings extracted in the scan image, or by selecting an attribute of the character string to be masked. FIG. 6 is a flowchart of the process executed in the area designation process in S508. In the area designation process in S508, the process changes depending on whether or not "preset" is selected on the mask setting method selection screen 700. First, in S601, the display control unit 421 determines whether or not "preset" is selected on the mask setting method selection screen 700. If "preset" is selected in S601, the display control unit 421 acquires preset information from the data management unit 424 in S602. If "manual" is selected in S601, the process proceeds to S604 without acquiring preset information.

[0037] If it is determined in S601 that "preset" is not selected, the process proceeds to S603, where the display control unit 421 displays an attribute selection screen 1700 in FIG. 17. The attribute selection screen 1700 displays the attribute names when the character string extracted in S507 is classified into attributes. 1701 is an area that displays a list of attribute names, and the area in which each attribute name is displayed is a button that accepts input from the user. When each attribute name button is pressed, the background color of the button changes, so that it is possible to know that the button is selected. 1702 is a button that returns to the mask setting method selection screen 700. 1703 is a button for transitioning to a preview screen 1000 (described later) for specifying a mask area in FIG. 10. The attribute name selected by pressing the button in 1701 is held in the RAM 213.

[0038] In S604, the display control unit 421 displays the preview screen 1000 of FIG. 10. FIG. 10(a) is an example of the preview screen 1000 that is first displayed in S604. If "preset" is not selected in S601, the display control unit 421 acquires information on the attribute selected via the attribute selection screen 1700 in S603, and reflects the information on the preview screen 1000. Specifically, based on the acquired information on the attribute, the OCR character string belonging to the attribute is masked and displayed on the preview screen 1000. If "preset" is selected in S601, the display control unit 421 acquires information on the attribute registered in the template corresponding to the selected preset button in S602. Then, based on the acquired information on the attribute, the OCR character string belonging to the attribute is masked and displayed. The masking is to fill in a rectangular area surrounding the target character string and display it. In addition, the display control unit 421 displays the masked character string on the preview screen 1000 in a selected state in an attribute list display area 1012 described later.

[0039] Reference numeral 1012 denotes an attribute list display area that displays the OCR character string in association with each attribute.

[0040] The display control unit 421 displays attributes and OCR character strings in a tree structure in the attribute list display area 1012. The parent is the attribute name, and the child is the OCR character string. That is, for each of a plurality of attributes, one or more OCR character strings classified into each attribute are displayed in association with each other. The display control unit 421 displays check boxes in the attribute list display area 1012 that accept a tap operation to select each attribute and each OCR character string as a mask target or not. The display control unit 421 accepts a specification from the user to select or deselect each OCR character string via the check boxes.

[0041] Specifically, when a user checks a checkbox for an OCR character string, the operation is accepted as a designation for selecting the OCR character string. Then, based on the designation, an instruction to mask the OCR character string is accepted. Then, the OCR character string is masked on the preview screen 1000.

[0042] When a user unchecks a checkbox for an OCR character string, the operation is accepted as a designation for deselecting the OCR character string. Then, based on the designation, an instruction to unmask the OCR character string is accepted. Then, the OCR character string is displayed on the preview screen 1000 without being masked.

[0043] Furthermore, when a user checks a checkbox of an attribute, the operation is accepted as a designation for selecting all OCR character strings associated with the attribute. Then, based on the designation, an instruction to mask all OCR character strings associated with the attribute is accepted. Then, all the OCR character strings are masked on the preview screen 1000.

[0044] When a user unchecks a checkbox of an attribute, the operation is accepted as a designation for deselecting all OCR character strings associated with the attribute. Then, based on the designation, an instruction to unmask all OCR character strings associated with the attribute is accepted. Then, all the OCR character strings are displayed on the preview screen 1000 without being masked.

[0045] Furthermore, the display control unit 421 displays a scroll bar if the OCR character strings to be displayed do not fit in the attribute list display area. In this embodiment, the association between the attributes and the OCR character strings is expressed in a tree structure, but other expression methods such as a list structure may be used as long as the attributes and the OCR character strings are associated with each other. This attribute list display area 1012 makes it easy to add or delete attributes and character strings to be masked.

[0046] Furthermore, when an attribute registered in a preset is subject to an attribute lock (described later), the display control unit 421 controls the attribute list display area 1012 so that the attribute subject to the attribute lock cannot be released. For example, the attribute subject to the attribute lock and its check box, and all OCR character strings associated with the attribute and its check box are displayed in gray so that they cannot be deselected. The attribute lock has the effect of preventing the operator of this application from coming into contact with confidential information such as personal information. For this reason, the attribute lock function may be controlled so as to be used in the case of a specific user authority, and may be applied, for example, in the case where a user who calls and operates a preset has a specific authority. Specifically, when the user information management unit (not shown) of the MFP 110 determines that the attribute lock setting (described later) is enabled for at least one attribute in the called preset, the user information management unit checks the information of the user who has logged in to the MFP 110. When the information is information indicating a general user, the display control unit 421 notifies the display control unit 421 of the fact. When the display control unit 421 receives information indicating a general user, the display control unit 421 controls display so that the selection of the OCR character string cannot be deselected for the attribute for which the attribute lock is set to be enabled in the preset.

[0047] Reference numeral 1001 denotes an area in which a preview of the image scanned in S506 is displayed. Reference numeral 1002 denotes a preview image of the image scanned in S506. If the preview image does not fit on the screen, a scroll bar is automatically displayed.

[0048] A button 1003 is used to cancel the mask area selected on the preview image 1002. When the display control unit 421 detects that the OCR character string selected as the mask target is touched with a finger on the image data of the preview image, the display control unit 421 recognizes that the touched OCR character string is selected for the next operation. Then, when the display control unit 421 detects the pressing of the deselect button 1003 following the selection, the display control unit 421 cancels the OCR character string from the mask target. Specifically, first, the display of the masking on the preview image 1002 is cancelled. Furthermore, the selection is also cancelled from the attribute list display area 1012 described later. That is, the selection of the check box corresponding to the OCR character string is cancelled. Furthermore, when all the character strings included in the attribute to which the deselected character string belongs are in a state of being unselected due to the deselection, the selection of the attribute is also cancelled.

[0049] Button 1004 reduces the display magnification of preview image 1002 by a fixed amount. Button 1005 changes the magnification to the maximum magnification at which the entire preview image 1002 fits within preview area 1001. Button 1006 increases the display magnification of preview image 1002 by a fixed amount to enlarge it. Button 1007 displays the previous page of scanned image when there are multiple pages of scanned images. Button 1008 displays the page of the currently displayed scanned image and the total number of pages. Button 1009 displays the next page of scanned image when there are multiple pages of scanned images.

[0050] Reference numeral 1010 denotes a mask color designation button for designating the color of a mask area, which will be described later. The color for filling the mask area may be not only a single color but also a fill effect such as a mosaic or stamp. That is, in this embodiment, the button 1010 is described as a mask color designation button, but is not limited to this. It may also be a button for selecting the type of mask image. For example, it may be configured so that the color and pattern of the mask image can be selected.

[0051] Reference numeral 1011 denotes a toggle button for semi-transparently displaying the mask region on the preview. When a selection for enabling the setting for transparent display is accepted via 1011, the display control unit 421 displays the character string selected as the mask target in a transparent masked manner so that the character string can be seen. When a selection for disabling the setting for transparent display is accepted via 1011, the display control unit 421 displays the character string selected as the mask target in a non-transparent masked manner so that the character string can be seen.

[0052] Reference numeral 1018 denotes a button for returning to the previous process, and if "preset" was selected in S601, the display control unit 421 displays a scan screen 900 upon detecting the pressing of the button 1018. If "preset" was not selected in S601, the display control unit 421 displays an attribute selection screen 1700 upon detecting the pressing of the button 1018. Reference numeral 1019 denotes a button for proceeding to the next process, and the display control unit 421 displays an execution confirmation screen 1200 (described later) of FIG. 12 upon detecting the pressing of the button 1019.

[0053] In this embodiment, a method of specifying a mask area by attributes is described, but the mask area may be specified by other methods such as character string specification or coordinate specification. Character string specification is a method in which a user specifies a character string to be masked, and the display control unit 421 searches for the specified character string from among the character strings extracted in the scanned image and selects it as a mask target. Coordinate specification is a method in which a user specifies an area to be masked on a scanned image, and the display control unit 421 detects a touch operation by the user and sets the corresponding area as the area to be masked. In addition, it is also possible to use multiple area specification methods on the same screen.

[0054] Reference numerals 1020 to 1030 show how the display control unit 421 masks and displays OCR character strings that are included in the presets or the attributes selected on the attribute selection screen 1700. Before receiving a user's instruction to select or deselect a character string, the display control unit 421 displays the screen 1000 with 1020 to 1030 masked. The display control unit 421 also displays the masked OCR character strings in a state where they are selected by a check mark in the attribute list display area 1012. Furthermore, among the attributes displayed in the attribute list display area 1012, attributes for which all of the associated OCR character strings are selected are displayed in a state where they are selected by a check mark.

[0055] Fig. 10B is an example of the preview screen 1000 after the user edits the mask area, and shows how the user adds or deletes a mask target after the preview screen 1000 of Fig. 10A is displayed. When the display control unit 421 receives an instruction to deselect the attribute name "date" in the attribute list display area 1012, it cancels the mask image displayed in 1022 and displays the character string of the "date" attribute without masking. When the display control unit 421 receives an instruction to deselect the OCR character strings "XXX" and "AAA" in the attribute list display area 1012, it cancels the mask image that has been hiding "XXX" and "AAA" among the mask images displayed in 1027 to 1029. Then, it displays the character strings "XXX" and "AAA" without masking. Furthermore, when the display control unit 421 detects that the attribute name “personal name” in the attribute list display area 1012 has been newly selected, it displays a mask image 1030 and masks the character string “Koyanno” associated with the “personal name”.

[0056] FIG. 13(a) shows area information 1300 such as coordinates stored in the RAM 213 in relation to the area specified on the preview screen of FIG. 10(a). When the display control unit 421 detects that a mask area has been specified on the preview screen, the area information 1300 is stored in the data management unit 424. 1301 is an ID that uniquely indicates each area information. 1302 is the type of area specification means. In addition to "attribute", the types of 1302 include "coordinates" and "text". "Attribute" indicates the type of mask area specified by specifying an attribute on the preview screen of FIG. 10(a). "Coordinates" indicates the type of mask area specified by specifying coordinates. "Text" indicates the type of mask area specified by specifying a character string. 1303 is the start point of the area. The coordinates of the horizontal direction and the vertical direction are shown in the (X, Y) format with the upper left corner of the scanned image as the origin. 1304 is the end point of the area. That is, the target area is a rectangle with the coordinates of 1303 as the upper left point and the coordinates of 1304 as the lower right point. Coordinate information of the character string is obtained by image analysis processing such as OCR by the image processing unit 423. 1305 is the RGB value of the color. 1306 is the OCR character string to be masked. It can be seen that the area information 1300 stores 10 mask areas (MSK1 to 10) corresponding to the area information of the preview area 1001 in FIG. 10(a).

[0057] Fig. 13(b) shows region information 1301 such as coordinates stored in RAM 213 in association with the region specified on the preview screen in Fig. 10(b). In Fig. 10(b), 1022 (2022 / 12 / 22), 1027 (XXX), 1028 (AAA), and 1029 (AAA) have been deleted from the region information as targets to be masked, and 1030 (Koyano) has been newly added. Therefore, it can be seen that eight mask regions (MSK1 to 7) corresponding to the region information of preview area 1001 in Fig. 10(b) are stored in region information 1301.

[0058] In S607, the image processing unit 423 composites an image of the specified color with respect to the area specified as the mask area in the processing of S604 with respect to the scanned image, and saves the composite mask image in the HDD 214. That is, the mask image is composited with the area corresponding to the character string selected in the attribute list display area 1012, and a composite mask image is generated. In S608, the mask area information used in the mask composition in S607 is associated with the image identifier of the scanned image data and saved in the HDD 214. The saved mask area information is the information shown in FIG. 13. The above is the detailed processing executed in the area designation processing of S508.

[0059] When the area designation process in S508 is completed, in S509 the display control unit 421 displays the execution confirmation screen 1200 in FIG. 12. Reference numeral 1201 denotes a print setting display area that is displayed when an item including print is selected in the output method 901 in FIG. 9. In the print setting display area 1201, the setting of the number of copies and a button for "other print settings" are arranged. In addition, when the "other print settings" button is pressed, a screen (not shown) is displayed in which settings such as double-sided printing, paper size, and imposition can be changed. Reference numeral 1202 denotes a save setting confirmation display area that is displayed when an item including save is selected in the output method 901 in FIG. 9. The save setting confirmation display area 1202 displays the setting values ​​of the save destination and file name set in FIG. 12(b) or FIG. 12(c).

[0060] A button 1206 is for displaying an edit screen for setting the external storage and folder path of the save destination. When the save destination edit button 1206 is detected to be pressed, the display control unit 421 displays a save destination setting screen 1210 of FIG.

[0061] An item 1211 is for setting the external storage to which the scanned image is to be sent, and in this example, "Cloud A" is selected. When it is detected that a storage has been selected by the operation in 1211, the MFP 110 starts connecting to the external storage set in 1211. If authentication is required to access the external storage set in 1211, a screen required for authentication (not shown) is displayed. When authentication is complete, a list 1213 (described below) of folders that can be used under the external storage in 1211 is displayed, and the user can select a desired folder and change the folder path.

[0062] An area 1212 displays the currently selected folder path under the external storage set in the area 1211. In the example of FIG. 12(b), it can be seen that "Investigation Report" is set as the folder path.

[0063] Reference numeral 1213 denotes a list of folders that can be used under the external storage. The user can select a desired folder and change the folder path. When a folder is selected, the folder path is displayed in a folder path display area 1212.

[0064] A button 1214 is used to discard the changes made on the save destination setting screen 1210 and return to the execution confirmation screen 1200.

[0065] A button 1215 is used to confirm and save the changes made on the save destination setting screen 1210 and return to the execution confirmation screen 1200. When it detects that the decision button 1215 is pressed, the MFP 110 saves the settings and displays the set information in the save setting confirmation display area 1202. At this time, the storage name and folder path are concatenated into one and displayed so that the save destination path is clear to the user. In the save setting confirmation display area 1202 in FIG. 12(b), it can be seen that "Cloud A" is set as the external storage and "Investigation Report" is set as the save destination folder path.

[0066] A button 1207 is for displaying an edit screen for changing the file name settings. When the file name edit button 1207 is pressed, the MFP 110 displays a name input screen 1220 shown in FIG.

[0067] 1221 is an area that displays the set file name. 1222 to 1223 are areas that display buttons used to input the file name. 1222 is an area that displays buttons for inputting fixed text. Any text or a pre-prepared delimiter (such as a hyphen, underscore, or blank) can be input into the file name. When "Text" is selected, a keyboard (not shown) for inputting text is displayed on the touch panel, and the user can input any text. 1223 is an area that displays buttons for inputting the date and time information at the time of scanning into the file name. By inputting the date and time information into the file name using the button in this date and time area 1223, it is possible to easily input the date and time information into the file name. 1224 is a name rule display area that displays which button in the areas 1222 to 1223 was used for input. When input is made from the button in the fixed value area 1222, the corresponding text is displayed as is in the name rule display area 1224, but when input is made from any other area, it is displayed in the format of [variable name]. [Variable name] is filled with information related to the name of the button in each area. In FIG. 12C, it can be seen from name rule display area 1224 that the "year", "month", and "day" buttons in date and time area 1223 have been used as information other than fixed values.

[0068] 1225 is a button for deleting a character string entered in a file name. The delete button 1225 deletes from the last character string of a file name, but if the deletion target is not a fixed value, the character string corresponding to the [variable name] in the name rule display area is deleted in one deletion. That is, when it is detected that the user has pressed the delete button 1225 on the screen of FIG. 10(c), the character string "05" is deleted in one go. At that time, the [day] corresponding to the deleted character string is also deleted from the name rule display area 1224. 1226 is a button for canceling the editing on the name input screen 1220 and returning to the execution confirmation screen 1200. 1227 is a button for confirming and saving the editing on the name input screen 1220 and returning to the execution confirmation screen 1200. The confirmed file name is displayed in the save setting confirmation display area 1202. The setting contents of the name rule display area 1224 are stored as the setting values ​​of the preset button when the preset button is registered as described later. Therefore, values ​​input using variable values ​​other than fixed values ​​can change depending on the date and time of the actual scan and information obtained from the scanned image.

[0069] 1203 is an option for registering various areas and various setting information as a preset. In S503 and S504, the process of accepting the selection of a template that stores information on scan settings and masking settings has been described. In this embodiment, the settings newly set in S505, S508, and S509 or changed from the template can be saved in the data management unit 424. A new template can be registered with a preset name, which will be described later. It is also possible to update an existing template by giving it an already existing preset name. For example, if a preset has been selected in S503 and S504, the template can be updated by giving the same preset name.

[0070] 1204 is a button for returning to the preview screen 1100 of FIG. 10 or FIG. 11. 1205 is a button for executing printing, saving, and registering a preset. When detecting that the execute button 1205 is pressed, the display control unit 421 judges whether the preset registration option 1203 is set in S510. If the preset registration option 1203 is ON, the judgment in S510 becomes "Yes", and the display control unit 421 displays the preset name setting screen 1500 in S511. The user can register an arbitrary preset name from the preset name input area 1501 of the preset name setting screen 1500. In the attribute lock designation area 1502, it is possible to set to designate that a specific attribute cannot be released from the mask target when information is called up in the preset. The attribute lock setting does not necessarily have to be configured in this embodiment. The attribute lock designation area 1502 shown in FIG. 15 is displayed based on the attribute list display area 1012 in FIG. 10(b). 15 shows a state in which three items of personal information, namely, telephone number, email address, and individual name, are selected as items to be locked, and the company name and product name, which are not personal information, are excluded from the items to be locked. Information specified in the attribute lock specification area 1502 is saved in attribute lock 1415 of area information 1405, which will be described later. When pressing of the back button 1503 is detected, the screen transitions to the execution confirmation screen 1200.

[0071] When it is detected that the enter button 1504 has been pressed, in step S512 the data management unit 424 of the MFP 110 stores in the HDD 214 the various areas and setting information currently set, and the preset name, in association with the button ID.

[0072] FIG. 14 shows an example of preset information 1400 saved by the data management unit 424. 1401 is a preset button ID. 1402 is a preset name input by the user in S511. 1403 and 1404 are the output method and scan settings set on the scan screen 900 of FIG. 9. 1405 is area information set on the preview screen 1000 of FIG. 10(a). As can be seen from the area information 1405, an attribute name 1415 used as a method for specifying a mask area is stored. When specifying a mask target by attribute, the coordinate position of the OCR character string to be masked may change depending on the input scanned image, so the coordinate information does not necessarily need to be saved as preset information. Similarly, the OCR character string included in each attribute may also change depending on the scanned image, so it is not included in 1405, but both the coordinate information and the OCR character string may be included in the area information 1405. Also, information such as specifying or excluding a specific OCR character string may be included.

[0073] Numeral 1411 is an ID that uniquely identifies each piece of region information, similar to 1301. Numeral 1412 is the type of region designation means, similar to 1302, and in addition to "attribute", there are types such as "coordinates" and "text". Numeral 1413 is the RGB value of the color, similar to 1305. Numeral 1414 is the attribute name set via the attribute list display area 1012.

[0074] 1415 is an attribute lock setting for specifying whether or not the attribute saved in the preset can be released from the mask target. If YES, when the preset is called and the mask target is specified, the attribute set in the preset as YES cannot be released from the mask target. If NO, it can be released. The attribute lock setting 1515 receives the attribute to be locked from the user in S517.

[0075] 1406 is the save setting set on the execution confirmation screen 1200 in FIG. 12. The save setting 1406 stores the name rule for the save destination and the folder path rule. The print setting 1407 is the print setting set on the execution confirmation screen 1200 in FIG. 12. As described above, by registering various areas and various settings that have been set once as presets, the same settings can be easily reused the next time. The registered preset information can be called up in S504 via the preset list screen 800 in FIG. 8.

[0076] When the saving in S512 is completed, in S513 the print instruction unit 425 determines whether printing has been set. If printing has been set, the result in S513 becomes "Yes," and in S514 the print instruction unit 425 instructs the print execution unit 413 to execute a print process for the image combined with the mask, and the print execution unit 413 executes the printing.

[0077] Next, in S515, the external storage access unit 427 determines whether or not a save setting has been made. If a save setting has been made, S515 becomes "Yes", and in S516 the external storage access unit 427 executes a process of saving the image combined with the mask. At this time, the data conversion unit 426 converts the image combined with the mask into a file format specified on the scan screen 900 in FIG. 9. Then, the external storage access unit 427 transmits the image to the destination specified on the preview screen 1000 in FIG. 10 under the specified file name. Then, the transmitted file is saved in the external storage 120.

[0078] When the save execution in S516, i.e., the transmission execution, is completed, all the processing of the masking application is completed. In other words, a screen transition is executed, such as returning to the main screen of the MFP 110, or returning to the mask setting method selection screen in FIG. 7 for the next scan.

[0079] As described above, convenience is improved by displaying character strings corresponding to attributes in the image data to be edited and accepting a specification from the user as to whether or not to select the displayed character string as a target for masking.

[0080] <Second embodiment> In this embodiment, a method for further improving the convenience of mask area setting by switching the display order depending on the type of attribute classification means will be described with reference to Fig. 11 and Fig. 18. Descriptions of processes similar to those in the first embodiment will be omitted.

[0081] The attribute classification unit 428 classifies character strings extracted from the results of the OCR processing in the image processing unit 423 into attributes by natural language processing capable of semantic understanding of the OCR character strings and character recognition processing such as key-value extraction. The characteristics of attribute names differ between classification by semantic understanding and attribute classification by key-value extraction; in classification by semantic understanding, the attribute names are predefined, whereas in classification by key-value extraction, the attribute names are character strings that actually exist in the input image data. Therefore, by using character strings that actually exist in the image data as attribute names, it becomes easier to find by associating character strings on the preview screen with attribute names.

[0082] FIG. 11 is a diagram showing attribute names classified by key-value extraction displayed at the top of the list. In the attribute list display area 1012 of the preview screen 1000 in FIG. 11, an area labeled "List by Item" is displayed, and character strings associated with keys and values ​​are displayed from the contents of the document displayed on the preview screen. Here, four attribute names, "Date of Creation", "Person in Charge", "TEL", and "Email", are displayed as parts corresponding to keys. These four key character strings are character strings that actually exist in the preview screen. As parts corresponding to values, each OCR character string, "2022 / 12 / 22", "Kiyano", "090-0000-2111", and "kiyano@example.com", is displayed in association with each other. The OCR character string "Kiyano" of the attribute name "Person in Charge" is not selected as a mask target here, so it is hidden here.

[0083] Fig. 18 shows information in which item names have been added to the information in which attribute names and OCR character strings are associated with each other as shown in Fig. 16. The item names are attribute names classified by key-value extraction. Both attribute names classified by semantic understanding and item names classified by key-value extraction can be associated with each OCR character string, and OCR character strings to which item names have been added are displayed at the top of the list in the attribute list display area 1012.

[0084] As described above, by switching the display order depending on the type of attribute classification means, the convenience of setting a mask area by specifying an attribute is further improved.

[0085] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of each of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC or FPGA) that realizes one or more functions. [Explanation of symbols]

[0086] 110 MFP 421 Display control unit 423 Image Processing Unit

Claims

1. A receiving means for receiving the selection of one or more attributes of a string to be masked, A display control means that displays one or more strings corresponding to the one or more selected attributes identified by the analysis process of the data, and a preview of what would happen if masking was performed on the one or more strings contained in the data. It has, The one or more strings are displayed in a second area different from the first area where the preview is displayed, along with information indicating that each of the one or more strings has been selected as the target of the masking process. In response to a user operation received in the second area to deselect one of the one or more strings as the target of the masking process, the preview in the first area is updated to show the preview as it would be if the masking process for that one string were deselected, and the display in the second area is updated so that one of the one or more strings is displayed along with information indicating that it has not been selected as the target of the masking process. An information processing device characterized by the following:

2. The receiving means receives the selection of multiple attributes of the string to be subjected to the masking process, The aforementioned analysis process identifies multiple strings corresponding to the multiple attributes, In the second area, the multiple strings are displayed separately for each of the multiple attributes. The information processing apparatus according to feature 1.

3. The display control means further displays in the second area a display item for collectively deselecting one or more strings as masking processes corresponding to one of the plurality of attributes, In response to user interaction with the display item, the preview in the first area is updated to show the preview as if the masking process for one or more strings corresponding to the attribute had been removed, and the display in the second area is updated so that the one or more strings corresponding to the attribute are displayed along with information indicating that they have not been selected as targets for masking. The information processing apparatus according to feature 2.

4. For any of the selected one or more attributes for which no corresponding string was identified by the analysis process, an indication that no string was identified is displayed in the second area. The information processing apparatus according to feature 2.

5. After updating the display of the preview and the second area, in response to receiving an instruction to perform the masking process, the masking process is performed on the data, with the string displayed in the updated second area along with information indicating that it has been selected as the target of the masking process being treated as the target of the masking process. The information processing apparatus according to feature 1.

6. Further comprising a scanner, The aforementioned data is image data obtained by scanning with the scanner. The information processing apparatus according to feature 1.

7. The preview is displayed so that the string to be masked is visible by making the image superimposed on the string to be masked transparent. The information processing apparatus according to feature 1.

8. The one or more attributes of the string to be subjected to the masking process are managed by associating them with a single item, Based on the selection of one of the aforementioned items, the selection of one or more of the aforementioned attributes is accepted. The information processing apparatus according to feature 1.

9. A receiving step of receiving the selection of one or more attributes of a string to be subjected to masking, A display control step that displays one or more strings corresponding to the one or more selected attributes identified by the analysis process of the data, and a preview of what would happen if masking was performed on the one or more strings contained in the data. It has, The one or more strings are displayed in a second area different from the first area where the preview is displayed, along with information indicating that each of the one or more strings has been selected as the target of the masking process. In response to a user operation received in the second area to deselect one of the one or more strings as the target of the masking process, the preview in the first area is updated to show the preview as it would be if the masking process for that one string were deselected, and the display in the second area is updated so that one of the one or more strings is displayed along with information indicating that it has not been selected as the target of the masking process. A control method characterized by the following:

10. A receiving means for receiving the selection of one or more attributes of a string to be subjected to masking, Display control means for displaying one or more strings corresponding to the one or more selected attributes identified by the analysis process of the data, and a preview of what would happen if masking was performed on the one or more strings contained in the data. It is a program that makes a computer function as such. The one or more strings are displayed in a second area different from the first area where the preview is displayed, along with information indicating that each of the one or more strings has been selected as the target of the masking process. In response to a user operation received in the second area to deselect one of the one or more strings as the target of the masking process, the preview in the first area is updated to show the preview as it would be if the masking process for that one string were deselected, and the display in the second area is updated so that one of the one or more strings is displayed along with information indicating that it has not been selected as the target of the masking process. A program characterized by the following features.

Images