Image processing device, image processing method, and program

Abstract

To properly check a masked image.SOLUTION: An image processing device has acquisition means for acquiring a document image, image processing means for generating a masked image where a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized, and control means for controlling the presentation of the predetermined character string so that it is recognizable by a user on the basis of the user's instruction.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure relates to processing of document images. [Background technology]

[0002] Document data, such as scanned images obtained by scanning documents such as invoices, may be sent to related departments or other companies.

[0003] Patent Document 1 discloses a method for masking information such as personal information and confidential information in document data by blacking out character strings indicating the information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-68023 Summary of the Invention [Problem to be solved by the invention]

[0005] Before outputting a masked image obtained by masking document data using the method of Patent Document 1, it may be necessary to check the masked image. When a preview of the masked image is displayed to check the masked image, the masked character string in the masked image is displayed so that the user cannot recognize it. For this reason, the user cannot check whether the masked character string has been erroneously masked. [Means for solving the problem]

[0006] The image processing device disclosed herein is characterized by having an acquisition means for acquiring a document image, an image processing means for generating a masking image in which a target area in the document image containing a specified character string is masked so that the user cannot recognize the specified character string, and a control means for controlling the presentation of the specified character string so that the user can recognize it, based on a user's instruction. Effect of the Invention

[0007] According to the present disclosure, it is possible to appropriately check a masking image. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an image processing system. [Diagram 2] FIG. 1 is a block diagram showing the hardware configuration of an MFP. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of an external storage device. [Figure 4] FIG. 1 is a block diagram showing the functional configuration of an MFP. [Diagram 5] 11 is a flowchart for explaining a flow from generating a masking image to transmitting the same. [Figure 6] FIG. 13 is a diagram showing an example of a setting screen. [Figure 7] 10 is a flowchart for explaining details of a masking region specifying process. [Figure 8] 10 is a flowchart for explaining details of acquisition of an information type to be masked. [Figure 9] 11 is a flowchart for explaining a masking image generating process. [Figure 10] FIG. 4 is a diagram for explaining a preview display screen. [Figure 11] 10 is a flowchart illustrating details of a printing process for confirmation. [Figure 12] FIG. 11 is a diagram showing an example of a printed matter for confirming a masking image. [Figure 13] FIG. 13 is a diagram showing an example of a list print screen. [Figure 14] 10 is a flowchart for explaining details of a preset registration process. [Figure 15] FIG. 13 is a diagram showing an example of a preset registration screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments for carrying out the technology of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the technology according to the claims, and not all of the combinations of features described in the following embodiments are necessarily essential to the solution of the technology of the present disclosure.

[0010] <First embodiment> [System Configuration] 1 is a diagram showing an example of the configuration of an image processing system according to this embodiment. The image processing system 1 includes an MFP (Multifunction Peripheral) 110 and an external storage 120. The MFP 110 is communicably connected to a server that provides various services on the Internet via a LAN (Local Area Network).

[0011] The MFP 110 is an image forming device having multiple functions such as a scanner function and a printer function, and also has the functions of an information processing device and an image processing device. The MFP 110 also has a function of transferring an image file of a scanned image obtained by scanning a paper document to an external storage 120 capable of storing the file.

[0012] The external storage 120 is a device that provides a service that enables a user to store files received via the Internet and to obtain files from an external device via a web browser. The service provided by the external storage 120 is, for example, a cloud service. The device that provides the storage service is not limited to the external storage 120, and multiple devices may exist in the image processing system 1.

[0013] Although the image processing system 1 of this embodiment has a configuration including the MFP 110 and the external storage 120, for example, part of the functions of the MFP 110 may be realized by another server device or the like arranged on the Internet or on a LAN. Also, the external storage 120 may be arranged on the LAN rather than on the Internet. In addition to the external storage 120, a mail server may be included in the image processing system 1. The mail server may transmit the scanned image generated by the MFP 110 by attaching the scanned image to an email. The MFP 110 may include a storage function that the external storage 120 has.

[0014] [MFP hardware configuration] 2 is a block diagram showing the hardware configuration of the MFP 110. The MFP 110 includes a control unit 210, an operation unit 220, a printer 221, a scanner 222, and a modem 223.

[0015] The control unit 210 is composed of each unit described below, and controls the operation of the entire MFP 110. The CPU 211 reads various control programs stored in the ROM 212 or the HDD 214, and executes processes for implementing various functions of the MFP 110, such as scanning, printing, and communication. The RAM 213 is used as a temporary storage area such as a main memory and a work area of ​​the CPU 211. Note that, in this embodiment, one CPU 211 executes each process shown in a flowchart described later using one memory (RAM 213 or HDD 214), but is not limited to this. For example, each process may be executed by cooperating with multiple CPUs and multiple RAMs or HDDs. The HDD 214 is a large-capacity storage unit that stores image data and various programs.

[0016] The operation unit I / F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 is provided with a touch panel that also functions as a display unit, a keyboard, and the like, and receives operations, inputs, and instructions from a user. Note that touch operations on the touch panel include operations with a person's finger and operations with a touch pen.

[0017] The printer I / F 216 is an interface that connects the printer 221 and the control unit 210. Image data to be printed is transferred from the control unit 210 to the printer 221 via the printer I / F 216, and the image is printed on a recording medium such as paper.

[0018] The scanner I / F 217 is an interface that connects the scanner 222 and the control unit 210. The scanner 222 optically reads a paper document (original) set on a platen or an ADF (Auto Document Feeder), not shown, to generate a scanned image, and inputs the scanned image to the control unit 210 via the scanner I / F 217. The MFP 110 can print (copy output) the scanned image read and generated by the scanner 222 on the printer 221, store it in the HDD 214, or send the image as a file or by email to an external device such as the external storage 120 via a LAN.

[0019] The modem I / F 218 is an interface that connects the modem 223 to the control unit 210. The modem 223 transmits image data by facsimile communication with a facsimile device (not shown) on the PSTN.

[0020] The network I / F 219 is an interface that connects the control unit 210 (MFP 110) to a LAN. The MFP 110 uses the network I / F 219 to transmit scanned images to devices that provide various services on the Internet and to receive various types of information.

[0021] [External storage hardware configuration] 3 is a block diagram showing a hardware configuration of the external storage 120. The external storage 120 has a CPU 311, a ROM 312, a RAM 313, a HDD 314, and a network I / F 315. The CPU 311 controls the overall operation by reading out a control program stored in the ROM 312 and executing various processes. The RAM 313 is used as a temporary storage area such as a main memory and a work area for the CPU 311. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I / F 315 is an interface that connects the control unit 310 to the Internet. The external storage 120 receives various processing requests from other devices (such as the MFP 110) via the network I / F 315 and transmits and receives various information.

[0022] [MFP functional configuration] 4 is a block diagram showing the functional configuration of the MFP 110 of this embodiment. Of the various functions of the MFP 110, the following description focuses on functions related to processing from scanning a document to digitizing (filing) it and storing it in the external storage 120.

[0023] The functions of the MFP 110 include two functional modules: a native function unit 410 and an additional function unit 420. The native function unit 410 is an application that is provided as standard in the MFP 110, whereas the additional function unit 420 is an application that is additionally installed in the MFP 110. The additional function unit 420 is an application based on Java (registered trademark), and functions can be easily added to the MFP 110. Note that other additional applications (not shown) may be installed in the MFP 110. The functional units of the MFP 110 are realized by the CPU 211 reading a program stored in the ROM 212 or HDD 214 of the MFP 110 into the RAM 213 and executing the program.

[0024] The native function unit 410 includes a scan execution unit 411 , an internal data storage unit 412 , a print execution unit 413 , and a UI display control unit 414 .

[0025] The additional function unit 420 has a main processing unit 421, a scan instruction unit 422, an image processing unit 423, a data management unit 424, a print instruction unit 425, an Internet access unit 426, a display control unit 427, an information detection unit 428, and a document type determination unit 429. It also has a masking target identification unit 430.

[0026] The main processing unit 421 performs processing related to all of the functional units included in the additional functional unit 420. Specifically, the main processing unit 421 controls all of the processing executed by the additional functional unit 420, and requests each functional unit included in the additional functional unit 420 to perform processing.

[0027] The scan instruction unit 422 requests the scan execution unit 411 to perform scan processing according to the scan settings input via the setting screen 600 (see FIG. 6). The scan instruction unit 422 requests scan processing in response to pressing of the scan execution button 601 (see FIG. 6) on the setting screen 600.

[0028] The scan execution unit 411 acquires a scan request including scan settings from the scan instruction unit 422. The scan execution unit 411 causes the scanner 222 to execute a reading operation of a paper document placed on a platen via the scanner I / F 217 in accordance with the acquired scan request, and generates a scanned image. An image representing the contents of a document, such as the scanned image generated in this way, is called a document image.

[0029] The internal data storage unit 412 stores the data of the document image in the HDD 214. Information on an image identifier that uniquely identifies the stored document image is notified to the scan instruction unit 422. The image identifier is a number, a symbol, an alphabet, or the like for uniquely identifying an image stored in the MFP 110.

[0030] The image processing unit 423 is a functional unit that performs analysis and processing on the document image. The image processing unit 423 acquires an image identifier from the scan instruction unit 422, and acquires document image data corresponding to the acquired image identifier from the internal data storage unit 412. The image processing unit 423 performs image rotation and tilt correction processing, character string area detection processing, and character recognition processing (Optical Character Recognition processing or OCR processing) on ​​the acquired document image.

[0031] The image processing unit 423 of this embodiment synthesizes a mask so that the mask is superimposed on a target area in a document image, and instructs the internal data storage unit 412 to store data of a masked image, which is an image obtained as a result. The process of synthesizing a mask on a document image is called masking (processing) or blackout processing. Alternatively, the masking process may be called masking. The target area that is the subject of the masking process is called a masking area. In other words, the masking image is an image obtained by masking the masking area in the document image. The masked area in the masking image that has been subjected to the masking process may be called a mask. Details of the process of the image processing unit 423 and other functions will be described later.

[0032] In the description of this embodiment, the masking area is assumed to be a rectangular area, but the shape of the masking area may be any shape, such as an ellipse or a triangle. The position and size of a rectangular masking area are represented by the coordinates of the starting point, which is the upper left position, and the coordinates of the end point, which is the lower right position. For example, if the coordinates of the starting point are (441,957) and the coordinates of the end point are (1369,1 7) The position and size of the masking region are expressed as (441,957), (1369,1057).

[0033] The image processing unit 423 sends the image identifier to a print instruction unit 425 and an Internet access unit 426 in accordance with the output settings set for the masking image.

[0034] The data management unit 424 stores information obtained by processing the document image, such as the document type of the document image, a character string corresponding to the information type detected from the document image, masking information, and masking display information, in the storage unit in association with the image identifier. The data management unit 424 also stores the information type set when generating the masking image in the HDD 214 as a preset in association with the document type. In this embodiment, the preset is a template in which the information type to be masked is stored in association with the document type. The preset may further be associated with a file name, a save destination, and the like.

[0035] The print instruction unit 425 acquires an image identifier, which is an identifier of the image to be printed, from the image processing unit 423. The print instruction unit 425 sends to the print execution unit 413 a print processing request including the print settings input via the UI screen, and the acquired image identifier.

[0036] The print execution unit 413 acquires a print request and an image identifier from the print instruction unit 425. The print execution unit 413 acquires an image corresponding to the image identifier from the internal data storage unit 412. The print execution unit 413 performs processing to convert the acquired image data into image data for printing according to the print settings included in the print request. The print execution unit 413 causes the printer 221 to execute printing based on the image data for printing via the printer I / F 216, and generates a printed matter. If the image to be printed is a masking image, the masking image is printed on the recording medium.

[0037] The internet access unit 426 transmits a processing request to the external storage 120 that provides a cloud service such as a storage function (storage service). In the cloud service, a process of saving a file in the cloud storage and retrieving the saved file from an external device is generally performed using a protocol such as REST or SOAP. The cloud service publishes various interfaces for executing such a process. The internet access unit 426 performs an operation on the cloud service using the published interface of the cloud service. The internet access unit 426 acquires an image file and transmission information corresponding to the image identifier acquired from the image processing unit 423 from the data management unit 424. The internet access unit 426 transmits the image file acquired from the data management unit 424 to the external storage 120 via the network I / F using the transmission information acquired from the data management unit 424.

[0038] The display control unit 427 controls where necessary components such as buttons and images included in a UI (User Interface) screen for accepting user operations are displayed. The display control unit 427 controls displaying a UI screen on a liquid crystal display unit of a touch panel included in the operation unit 220 of the MFP 110 via the UI display control unit 414. The UI display control unit 414 displays a UI screen for accepting various user operations on the touch panel of the operation unit 220. For example, the UI display control unit 414 controls displaying a setting screen 600 (see FIG. 6) on which scan settings and scan start operations can be performed. In addition, the UI display control unit 414 displays a preview display screen 1000 (see FIG. 10) on which a preview of a masking image is displayed and on which the user can correct the masking area and instruct to start output.

[0039] The information detection unit 428 performs information detection processing to obtain a character string recognized from a document image from the image processing unit 423 and detect a character string corresponding to a predetermined item (called an information type) such as personal information or confidential information from the obtained character string. The information detection unit 428 requests the data management unit 424 to store the detected character string and the information type corresponding to the detected character string in the HDD 214 in association with each other.

[0040] The information type is the type of concept expressed by a character string, and is used to specify the character string to be masked. The information types that the information detection unit 428 can detect include, but are not limited to, types of personal information or confidential information, such as "company name," "name," "telephone number," and "address."

[0041] As a result of the information detection process by the information detection unit 428, for example, a character string "Yamada Taro" is detected as a character string corresponding to the information type "name". In this embodiment, the information detection unit 428 is assumed to hold items of predetermined detectable information types, but the place where the detectable information types are held is not limited to the information detection unit 428.

[0042] The document type determination unit 429 determines the document type of the document image based on the character string recognized from the document image. The document type is, for example, a type that indicates the purpose of the document, such as "contract," "delivery note," and "invoice." In this embodiment, a trained model generated by machine learning the features of the document type using samples of multiple document types as training data is used to determine the document type. In this embodiment, a trained model obtained by machine learning based on TranSformer is used. Other models that may be used include a bidirectional Long Short-Term Memory (LSTM) neural network, a Sequence to Sequence model, and a recurrent neural network (RNN).

[0043] The masking target specification unit 430 determines the information type to be masked based on the preset, the processing result of the information detection unit 428, and the processing result of the document type determination unit 429. Details will be described later.

[0044] [Overall processing flow] Fig. 5 is a flowchart for explaining a flow of generating a masking image from a document image obtained by scanning a document and transmitting the masking image. The series of processes shown in the flowchart of Fig. 5 are performed by the CPU 211 of the MFP 110 expanding program codes stored in the ROM 212 or HDD 214 into the RAM 213 and executing them, thereby implementing each functional unit to execute the processing of each step. Alternatively, some or all of the functions of the steps in Fig. 5 may be implemented by hardware such as an ASIC or an electronic circuit.

[0045] In S501, the display control unit generates a setting screen for the user to select settings regarding the output method, scanning, and masking, and causes the operation unit 220 to display the generated setting screen via the UI display control unit 414.

[0046] 6 is a diagram showing an example of the setting screen displayed in S501. The setting screen 600 includes an output method setting area 610, a scan setting area 620, a masking setting area 630, a scan execution button 601, and a back button 602.

[0047] The output method setting area 610 includes an output method selection button 611, an output format selection pull-down 612, and an output destination designation button 613. The output method selection button 611 is a button for the user to designate the output method of the masking image. The user can select the output method from "print only," "save only," or "print and save." In the following explanation, it is assumed that "save only" has been selected.

[0048] The output format selection pull-down 612 is a pull-down for selecting the file format of the masking image when "Print and save" or "Save only" is selected from the output method selection buttons 611.

[0049] The output destination designation button 613 is a button for selecting a destination where the masking image is saved when "Print and save" or "Save only" is selected from the output method selection buttons 611. From the output destination designation button 613, the user can select a destination from among "New input", "Address book", and "Send to self".

[0050] The "new input" included in the output destination designation button 613 is a button for the user to arbitrarily designate a destination of the masking image. In addition to the HDD 214 or the external storage 120, the user can designate a destination by an IP address via the network I / F 219 as the destination. The "address book" included in the output destination designation button 613 is a button for sending the masking image by email to an email address entered by the user. The "send to me" included in the output destination designation button 613 is a button for sending the image to an authentication server (not shown) that identifies the user based on the user information entered by the user. The authentication server sets a registered email address based on the authenticated user information. The user can input an arbitrary destination, email address, and user information via the operation unit 220. The output destination designation button 613 may be configured to further include other save destination buttons. In this embodiment, the description will be given assuming that the destination of the file of the masking image is the external storage 120.

[0051] The scan setting area 620 includes items for the user to select scan settings. For example, it includes items such as "original size," "gray / color," "single-sided / double-sided," and "advanced settings," which are changeable settings. When "advanced settings" is pressed, a screen (not shown) is displayed, and changes to the detailed settings such as resolution, image quality, mixed original sizes, and density are accepted on that screen. The scan execution button 601 is a button for executing a scan using the scan settings selected in the scan setting area 620.

[0052] The masking setting area 630 includes a masking selection button 631 , a masking method selection button 632 , and a masking mode selection pull-down 633 .

[0053] The masking selection button 631 is a button that accepts the selection of whether to mask or not mask the document image generated by pressing the scan execution button 601. The flowchart in Fig. 5 is a flowchart for explaining the process when "mask" is selected in the masking selection button 631.

[0054] The masking method selection button 632 is a button that allows the user to select whether to automatically or manually set a masking area for the scanned image when "masking" is selected with the masking selection button 631.

[0055] The masking mode selection pull-down 633 is a pull-down for manually selecting a mode for setting a masking area when "Manual masking setting selection" is selected with the masking method selection button 632. When "Automatic masking setting" is pressed with the masking method selection button 632 to select automatically setting a masking area, "Automatic" is set in the masking mode selection pull-down 633. In the following explanation, it is assumed that "Automatic masking setting" is pressed with the masking method selection button 632 to select automatically setting a masking area.

[0056] An application (called a masking application) that performs a process of masking a partial area of ​​a document image and transmitting the image to a cloud service can be used by installing it on the MFP 110. When the masking application is installed on the MFP 110, a masking setting area 630 including a button that enables the use of a masking function is displayed on the setting screen 600. When a scan execution button 601 is pressed, job information including the contents of the settings selected on the setting screen 600 is generated. Then, the process proceeds to S502.

[0057] In S502, the main processing unit 421 requests the scan execution unit 411 to scan a paper document placed on a platen via the scan instruction unit 422. Data of a scanned image obtained as a result of the scan is stored in the RAM 213.

[0058] In the following description, the document image is assumed to be a scanned image obtained as a result of S502, and the masking process is performed on the scanned image, but the document image to be masked is not limited to a scanned image. The main processing unit 421 may obtain data of the document image stored in the HDD 214 via the internal data storage unit 412, and subject the document image to the masking process. Alternatively, the main processing unit 421 may obtain data of the document image to be masked from the external storage 120 via the Internet access unit 426. Data of the document image to be masked may be obtained using any other method.

[0059] In S503, the main processing unit 421 requests the image processing unit 423 to perform processing for identifying a masking region based on the job information generated in S501 and the document image data generated in S502, and the image processing unit 423 performs processing based on the request. Details of the processing of the image processing unit 423 in S503 will be described later.

[0060] In S504, the main processing unit 421 requests the image processing unit 423 to create a masking image, and the image processing unit 423 performs processing based on the request. Details of the processing of the image processing unit 423 in S504 will be described later.

[0061] In S505, the main processing unit 421 outputs the masking image created in S504 to the external storage 120 via the Internet access unit 426.

[0062] As described above, the generated masking image will be described as being output to the external storage 120, but the output method of the masking image is not limited to transmission to the outside. The main processing unit 421 may perform output processing of the masking image so that it is sent by email to an arbitrary destination. Alternatively, the main processing unit 421 may store the masking image in the HDD 214 via the internal data storage unit 412. Alternatively, the main processing unit 421 may perform processing for executing printing via the print instruction unit 425 so that the masking image is printed and output. Alternatively, the main processing unit 421 may store the masking image using other methods.

[0063] In S506, the main processing unit 421 generates a preset registration screen 1500 (see FIG. 15) for the display control unit 427. Then, the main processing unit 421 requests the UI display control unit 414 to display the preset registration screen. The UI display control unit 414 displays the preset registration screen 1500 on the touch panel of the operation unit 220, and registers a preset based on a selection received from the user. Details will be described later.

[0064] In this embodiment, an example will be described in which the display control unit 427 displays a UI screen on the touch panel of the operation unit 220. However, the display control unit 427 may provide each screen in this embodiment to another device, and a display unit of the other device may display each screen.

[0065] [Masking area identification process] FIG. 7 is a flowchart for explaining details of the process of S503 in this embodiment.

[0066] In S701, the image processing unit 423 acquires a document image from the HDD 214. The image processing unit 423 performs corrections such as skew correction and rotation correction on the acquired document image. The document image in the following steps refers to the document image corrected in S701. Data of the document image after the correction is stored in the RAM 213.

[0067] In S702, the image processing unit 423 converts the document image into a one-channel image so as to improve the accuracy of the OCR process. Then, a character string area included in the converted document image is extracted. Then, OCR process (character recognition process) is performed on the extracted character string area to obtain data of the character string included in the document image (character string data). The recognized character string and the character string area corresponding to the recognized character string are stored in the RAM 213.

[0068] Character string regions are extracted by performing block selection (BS) processing. In block selection (BS) processing, a document image is divided into object units and block regions within the document image are extracted. Then, attributes of each extracted block region are determined. Specifically, for example, attributes such as text, photographs, and diagrams are determined, and the document image is divided into block regions with different attributes. Block selection processing can be realized using known region determination techniques.

[0069] In S703, the information detection unit 428 performs information detection processing on each character string recognized in S702. As a result, character strings corresponding to predetermined information types are detected from the character strings recognized in S702. The information types in this embodiment are described as "company name", "name", "telephone number", and "address", but the information types to be detected are not limited.

[0070] Table 1 is an example of an "information detection processing result list" showing an example of the results of the information detection processing executed in S703. In the information detection processing result list of this embodiment, information types and character strings corresponding to the information types detected from the document image are stored in association with each other. The information detection processing result list is stored in the RAM 213.

[0071] When the information detection unit 428 detects "Iroha Co., Ltd." as a character string corresponding to the information type "company name," as shown in Table 1, "Iroha Co., Ltd." is stored in the same row as "company name" as a character string corresponding to the information type of "company name." Similarly, "Koyano" and "Tanaka" are detected as character strings corresponding to the information type of "name," and "090-0000-2111" is detected as a character string corresponding to the information type of "telephone number," so the character strings corresponding to the information types are stored in the corresponding rows as shown in Table 1. It is assumed that a character string corresponding to the information type of "address" was not detected. The information detection processing result list of this embodiment is generated so as not to include information types for which no character strings were detected.

[0072] [Table 1] In S704, the document type determination unit 429 determines the document type of the document image from the character string extracted in S702. In this embodiment, the document type determination unit 429 is described as determining the document type of the document image using a trained model that has been trained in advance to output a document type when a character string is input, but the document type may be determined by other methods. For example, the document type may be determined by setting a keyword corresponding to the document type in advance.

[0073] In S705, the masking target identification unit 430 determines whether or not the document type has been determined in S704. If the masking target identification unit 430 determines that the document type has been determined (YES in S705), the process proceeds to S706, and if the masking target identification unit 430 determines that the document type has not been determined (NO in S705), the process ends this flowchart.

[0074] In S706, the masking target identification unit 430 acquires the information type of the masking target corresponding to the document type.

[0075] FIG. 8 is a flowchart for explaining the details of the process of S706.

[0076] In S801, the masking target identification unit 430 requests the data management unit 424 to acquire a preset corresponding to the document type. The data management unit 424 acquires a preset corresponding to the document type determined in S704 from the external storage 120 via the Internet access unit 426. The acquired preset is stored in the RAM 213. The preset may be held in the HDD 214, in which case the preset may be acquired from the HDD 214.

[0077] The preset is information generated in advance for each document type, and includes an information type to be masked for each document type. The preset is stored in the external storage 120 or the HDD 214 by, for example, a user registering an information type and a document type in advance. Alternatively, the preset may be registered from a preset registration screen 1500 (see FIG. 15).

[0078] For example, if the document type is determined to be an invoice in S704, a preset associated with the invoice is acquired in S801. Table 2 is a table showing an example of information about a preset associated with an invoice.

[0079] [Table 2] In S802, the masking target identification unit 430 determines whether or not a preset associated with the document type of the document image in S801 has been acquired. If a preset has been acquired (YES in S802), the process proceeds to S803, and if a preset has not been acquired (NO in S802), the process ends this flowchart as there is no information type to be masked, and proceeds to S707.

[0080] In S803, the masking target identification unit 430 stores the information type of the masking target included in the preset acquired in S801 in the RAM 213. Then, this flow ends and the process proceeds to S707 in Fig. 7. If the document type determined in S704 is an invoice and a preset associated with an invoice is acquired, as shown in Table 2, "name", "telephone number", and "address" are stored in the RAM 213 as the information types of the masking target.

[0081] In S707, the image processing unit 423 determines an information type that matches the information type to be masked acquired in S706, among the information types corresponding to the character string detected in S703. The image processing unit 423 determines an area in the document image that includes a character string corresponding to the information type determined to match as an area where masking is required. A character string area that includes a character string indicating that masking is required becomes a masking area. The image processing unit 423 determines a character string area that includes a character string corresponding to a non-matching information type as an area where masking is not required.

[0082] For example, suppose that character strings corresponding to the information types of "company name", "name", and "phone number" are detected in S703 as shown in Table 1. Also suppose that "name", "phone number", and "address" are acquired as the information types to be masked in S706. In this case, the matching information types are "name" and "phone number". For this reason, it is determined that the masking of the character strings corresponding to the information types of "name" and "phone number" among the character strings detected in S703 is "necessary". On the other hand, it is determined that the masking of the character string corresponding to the information type of "company name" is "not required".

[0083] The image processing unit 423 acquires the character string area of ​​the character string in which the masking necessity is "Needed" from the character string area extracted in S702. The image processing unit 423 generates masking information as a processing result of S707. Table 3 is a masking information list showing an example of the masking information.

[0084] [Table 3] The masking information is information in which the information type to be detected in the information detection process in S703, a character string corresponding to the information type detected as a result of the information detection process in S703, whether masking is required, and a character string area are associated with each other. The masking information list in Table 3 shows an example of masking information generated as a result of the image processing unit 423 performing the process in S707 based on the information detection process result information list in Table 1 and the preset information in Table 2.

[0085] The character string area of ​​the masking information holds coordinate information in the document image of the character string area containing a character string corresponding to the information type. For "company name", "name", and "phone number", character strings corresponding to the information type were detected, so character string area information is held. On the other hand, since a character string corresponding to the information type of "address" was not detected, "- (none)" is held in the character string area corresponding to the information type of "address", indicating that no character string was detected. The image processing unit 423 saves the masking information, document type, and information detection results in the RAM 213.

[0086] It should be noted that if it is determined that the document type cannot be determined (NO in S705) or if a preset corresponding to the document type is not found, the masking information is not generated in the process of S503.

[0087] [Masking image generation] FIG. 9 is a flowchart for explaining the masking image generation process in S504 of FIG.

[0088] In S901, the image processing unit 423 acquires the document image corrected in S701, the masking information generated in S707, and the masking display information stored in advance from the RAM 213. If the masking information is changed in S905 (described later), the changed masking information is acquired. If the masking display information is changed in S908 or S909, the changed masking display information is acquired from the RAM 213. The image processing unit 423 performs masking processing on the document image based on the acquired masking information and masking display information.

[0089] Table 4 is a table showing an example of the masking display information. The masking display information includes information on the display type, display color, and transparency. In this embodiment, the display type is a rectangle, and the display color is black. The image processing unit 423 performs masking on the document image in accordance with the masking display information. The transparency is a value in the range of 0.0 to 1.0, and the lower the value, the lower the transparency. A transparency of 0.0 indicates a state in which the masked character string cannot be seen by the user.

[0090] The method of masking with a transparency of 0.0 may be any method that makes it impossible for the user to recognize the character string contained in the masking area. For example, if the display color is black, masking may be performed by filling the masking area with black. If the display color is the background color, masking may be performed by hiding the masking area with the background color. Masking may be performed in any other manner.

[0091] [Table 4] The image processing unit 423 acquires coordinate information of a character string area where the masking necessity in the acquired masking information is "Needed". The image processing unit 423 performs masking on the masking area, which is a rectangular area on the document image indicated by the coordinates, so that a mask based on the masking display information is synthesized, and generates a masked image. The image processing unit 423 stores the generated masked image in the RAM 213. If masking information has not been generated, no masking process is performed on the document image.

[0092] In S902, the main processing unit 421 requests the display control unit 427 to display a preview of the masked image generated in S901. Specifically, the display control unit 427 acquires the masked image generated in S901 from the RAM 213, generates a preview display screen 1000 (see FIG. 10), and displays it on the touch panel of the operation unit 220 via the UI display control unit 414. Note that if masking processing has not been performed on the document image as a result of S901, a preview of the document image that has not been masked is displayed. Even in this case, for convenience of explanation, the image previewed on the preview display screen 1000 will be referred to as a masked image.

[0093] Fig. 10 is a diagram for explaining a preview display screen. Fig. 10(a) is a diagram showing an example of the preview display screen generated in S902. The preview display screen 1000 includes an image display area 1010, a document type display area 1020, a masking information display area 1030, a send button 1041, a save settings button 1042, a confirmation print button 1043, and a list display button 1044.

[0094] The components included in the image display area 1010 will be described. The preview display area 1001 is an area where a preview of the masking image generated in S901 is displayed. If the generated masking image does not fit within the preview display area 1001, a scroll bar is automatically displayed.

[0095] The enlarge display area button 1002 is a button for increasing the display magnification of the image displayed in the preview display area 1001 by a fixed amount, and enlarging and previewing the generated masking image. The fit display area button 1003 is a button for changing the magnification to the maximum magnification so that the entire generated masking image fits within the preview display area 1001. The reduce display area button 1004 is a button for decreasing the display magnification of the image displayed in the preview display area 1001 by a fixed amount, and reducing and previewing the generated masking image.

[0096] A previous page button 1007 is a button for displaying the masking image of the previous page when there are multiple pages of masking images. A next page button 1009 is a button for displaying the masking image of the next page when there are multiple pages of masking images. A page number display area 1008 is an area for displaying the page of the masking image currently being displayed and the total number of pages.

[0097] The masking area selection button 1013 is a button for selecting a new masking area from the preview display area 1001. The mask deletion button 1011 is a button for setting the masking necessity of the masking area selected in the preview display area 1001 to "unnecessary" and canceling the mask. The masking image previewed in the preview display area 1001 includes a mask 1005 superimposed on the character string to be masked. When the masking area selection button 1013 is pressed, a mask such as the mask 1005 becomes selectable.

[0098] The masking instruction button 1012 is a button for instructing that a masking process be performed on an unmasked area in the current masking image.

[0099] The mask display check box 1014 is a check box for generating and previewing a confirmation masking image that makes the mask on the masking image transparent so that the user can view the masked character string. Each time the mask display check box 1014 is pressed, the state transitions between a checked state and an unchecked state. Details will be described later.

[0100] The document type display area 1020 includes a document type selection dropdown 1021. The document type selection dropdown 1021 is displayed with the document type determined in S704 selected as the default. If the document type cannot be determined, a blank not shown, indicating that the document type is "not set", is displayed as the default in the document type display area 1020. The user can select a document type from the document type selection dropdown 1021.

[0101] The masking information display area 1030 is an area that displays the contents of the masking information corresponding to the document type selected in the document type selection dropdown 1021. If masking information is generated in S503, the contents of the generated masking information are displayed by default.

[0102] When blank, indicating that the document type is not set, is selected in the document type selection dropdown 1021, no masking information exists and therefore no information is displayed in the masking information display area 1030.

[0103] When a document type is selected from the document type selection dropdown 1021, the masking target identification unit 430 performs the same processes as in S706 and S707 to obtain the information type to be masked that corresponds to the selected document type, and generates masking information. The display control unit 427 displays the contents of the newly generated masking information in the masking information display area 1030.

[0104] The masking information display area 1030 displays information types and character strings corresponding to the information types in a tree structure, with the parent being an information type and the child being a character string corresponding to the information type. That is, one or more character strings corresponding to each of a plurality of information types are displayed in association with each other.

[0105] The masking information display area 1030 has a check box 1036 corresponding to each information type. It also has a check box 1037 corresponding to the character string corresponding to each information type. When the check box 1036, 1037 is checked, the information type or character string corresponding to the check box indicates that masking is "necessary." When the check box 1036, 1037 is not checked, the information type or character string corresponding to the check box indicates that masking is "not required." To improve visibility, information types for which masking is "necessary" are displayed at the top, but the display order is not important.

[0106] Check boxes 1037 for character strings corresponding to information types for which the masking information indicates whether masking is required and whose coordinates are held in the masking area are displayed in a pre-checked state. Check boxes 1036, 1037 for information types or character strings for which the masking information indicates whether masking is not required are displayed in a pre-unchecked state.

[0107] Information types that are not subject to masking and for which the masking necessity is "unnecessary" are not included in the presets. However, the information types that are subject to detection by the information detection process are items that indicate personal information or confidential information, and are items for which the user is likely to instruct masking. For this reason, information types that are not subject to masking and for which the masking necessity is "unnecessary" are also displayed for the purpose of presenting them to the user.

[0108] The warning mark 1031 is a mark for warning the user about an information type for which masking is "necessary" but no character string is detected. The masking area edit buttons 1032 to 1035 are buttons for editing or adding a masking area. Details will be described later.

[0109] In S903, the display control unit 427 determines whether or not the send button 1041 on the preview display screen 1000 has been pressed by the user via the UI display control unit 414. If the display control unit 427 determines that the send button 1041 has been pressed (YES in S903), the process proceeds to S911. If the display control unit 427 determines that the send button 1041 has not been pressed (NO in S903), the process proceeds to S904.

[0110] In S904, the display control unit 427 determines via the UI display control unit 414 whether or not an operation that requires a change in masking information has been performed on the preview display screen 1000. An operation that requires a change in masking information is an operation of pressing the Delete Mask button 1011, the Masking Instruction button 1012, or the Edit Masking Area buttons 1032 to 1035, pressing the check boxes 1036 and 1037, or an operation of selecting a different document type from the document type selection dropdown 1021.

[0111] If the display control unit 427 determines that an operation requiring a change in masking information has been performed (YES in S904), the process proceeds to S905. If the display control unit 427 determines that an operation requiring a change in masking information has not been performed (NO in S904), the process proceeds to S906.

[0112] In S905, the image processing unit 423 acquires the current masking information, and changes the masking information based on the user's operation on the preview display screen 1000. The masking information generated in S707, or, if S905 has been executed previously, the updated masking information is acquired, and a process of changing the acquired masking information is performed.

[0113] For example, assume that the user presses any one of the masking area edit buttons 1032 to 1035, and then the user drags on the touch panel on which the preview display area 1001 is displayed to input a start point and an end point. The display control unit 427 acquires the positions of the start point and the end point selected on the touch panel of the operation unit 220 via the UI display control unit 414, and converts them into a rectangular area on the masking image. Specifically, when it detects that the masking image on the preview display area 1001 is touched with a finger, it detects the point touched with the finger as the start point of the rectangular area. The display control unit 427 detects the point where the finger is released after dragging from the start point as the end point of the rectangular area. This operation corresponds to an operation that requires a change of the masking information. In this case, in S905, the image processing unit 423 updates the masking information so that the detected rectangular area is included as information on the masking area of ​​the information type corresponding to the masking area edit buttons 1032 to 1035.

[0114] In the masking information list of Table 3, the information type "address" has the masking necessity set to "necessary", but the character string area (masking area) is not specified. When adding masking area information for the information type "address", the user presses the masking area edit button 1034 associated with "address" in the masking information display area 1030. Then, as shown in the dotted rectangular area 1006 in FIG. 10(b), the user drags his / her finger on the preview display area 1001 to specify a rectangular area. The image processing unit 423 causes the character string area corresponding to the information type "address" of the masking information to hold the coordinate information of the rectangular area specified by the user, and updates the masking information as shown in Table 5.

[0115] [Table 5] For example, when the masking instruction button 1012 is selected, the display control unit 427 detects that the masking image in the preview display area 1001 is touched with a finger and detects the point touched with the finger as the start point. The display control unit 427 detects the point where the finger is released after dragging from the start point as the end point. This operation corresponds to an operation that requires a change of the masking information. In this case, in S905, the image processing unit 423 updates the masking information so that a rectangular area specified by the detected start point and end point is held as information of a new masking area. Specifically, the image processing unit 423 adds the information type "rectangular area" to the acquired masking information. The character string corresponding to the information type "rectangular area" holds "- (none)" and the masking necessity "necessary". Then, the character string area corresponding to the information type "rectangular area" holds the coordinates of the start point and the coordinates of the end point of the rectangular area designated by the user. The updated masking information is stored in the RAM 213.

[0116] Table 6 shows the masking information after a character string area in which the masking necessity of the new masking area "rectangle area" is "necessary" is added. The masking area may be added by other methods.

[0117] [Table 6] As another example, when the masking area selection button 1013 is selected, the display control unit 427 recognizes that the touched mask has been selected when it detects that the mask on the masking image in the preview display area 1001 has been touched with a finger. Then, the operation of pressing the mask deletion button 1011 following the selection corresponds to an operation that requires a change of the masking information. In this case, in S905, the image processing unit 423 changes the masking necessity associated with the masking area corresponding to the selected mask in the masking information to "not required."

[0118] Alternatively, it is assumed that the display control unit 427 receives a press on the check box 1036, 1037, and the checked state of the checked check box is unchecked. In this case, the image processing unit 423 changes the corresponding masking necessity in the masking information from "necessary" to "unnecessary." Alternatively, it is assumed that the display control unit 427 receives a press on the check box 1036, 1037, and the unchecked check box is changed to a checked state. In this case, the image processing unit 423 changes the corresponding masking necessity in the masking information from "unnecessary" to "necessary."

[0119] When the masking information is updated in S905, the process returns to S901. In S901, the image processing unit 423 creates a masked image based on the changed masking information. Then, in S902, the created masked image is displayed in the preview display area 1001. For example, as shown in FIG. 10(c), assume that a masking area for the information type "Address" has been newly set by the user. In this case, a masked image in which the address character string "Ota-ku, Tokyo" is masked so that it cannot be recognized by the user is displayed in the preview display area 1001.

[0120] On the other hand, in S906, the display control unit 427 determines whether or not the mask display check box 1014 has been pressed via the UI display control unit 414. If the display control unit 427 determines that the mask display check box 1014 has been pressed (YES in S906), the process proceeds to S907. If the display control unit 427 determines that the mask display check box 1014 has not been pressed (NO in S906), the process proceeds to S910.

[0121] In S907, the display control unit 427 determines whether the mask display check box 1014 is checked. If the display control unit 427 determines that the mask display check box 1014 is checked (YES in S907), the process proceeds to S908. If the display control unit 427 determines that the mask display check box 1014 is not checked (NO in S907), the process proceeds to S909.

[0122] In S909, the display control unit 427 changes the transparency of the masking display information to 0.5. Then, the process returns to S901. In the next step S901, a masked image is generated based on the masking display information with a transparency of 0.5, and in S902, a preview of the masked image is displayed in which the masking area, which is the character string area in which the masking necessity is "Needed", is masked with a transparency of 0.5. The masked image masked with a transparency of 0.5 is an image for user confirmation, and is controlled so as not to be transmitted to the outside.

[0123] FIG. 10(d) is a diagram showing an example of an image display area 1010 of a preview display screen 1000 in which a masked image generated based on masking display information with a transparency of 0.5 is previewed. A transparency of 0.5 means that the transparency of the mask is increased, so the masking image is generated so that a transparent mask is synthesized. Therefore, the character string included in the masking area in the masking image is displayed in a state in which it can be seen through and recognized. In this way, in this embodiment, by unchecking the mask display check box 1014, the masked image can be previewed in a mask-free state so that the masked character string can be seen. Therefore, the user can check the location of the masking area and which character string is masked, and can check whether the masking area is set erroneously.

[0124] In this embodiment, the transparency of the mask when the mask display checkbox 1014 is not checked is set to 0.5, but the transparency may be a value other than 0.5 as long as it is visible to the user.

[0125] On the other hand, in S908, the display control unit 427 sets the transparency of the masking display information to 0.0. Then, the process returns to S901. By setting the transparency to 0.0, the character string included in the masked area of ​​the masking image becomes unrecognizable on the preview display area 1001.

[0126] In this manner, in this embodiment, when the mask display check box 1014 is checked, the mask 1005 is displayed in the masked image displayed as a preview. When the mask display check box 1014 is not checked, the mask 1005 is displayed in an unmasked state, allowing the display to be switched.

[0127] On the other hand, in S910, the display control unit 427 performs processing for checking the masking processing before the output to the external storage 120 in S505.

[0128] FIG. 11 is a flowchart for explaining the details of the process of S910.

[0129] In S1101, the display control unit 427 determines whether or not the confirmation print button 1043 on the preview display screen 1000 has been pressed. If the display control unit 427 determines that the confirmation print button 1043 has been pressed (YES in S1101), the process proceeds to S1102. If the display control unit 427 determines that the confirmation print button 1043 has not been pressed (NO in S1101), the process proceeds to S1103.

[0130] In S1102, the display control unit 427 performs confirmation printing of the masking image generated in S901. First, the display control unit 427 acquires the masking image generated in S901 from the RAM 213. The display control unit 427 requests the print execution unit 413 to print the masking image via the print instruction unit 425. The print execution unit 413 executes printing (confirmation printing) of the acquired masking image.

[0131] Fig. 12 is a diagram showing an example of a printed matter obtained by checking and printing the masked image generated in S901. A printed matter 1201 in Fig. 12(a) is a printed matter obtained by checking and printing the masked image generated in S901 based on the masking display information with a transparency of 0.0. Since the transparency is set to 0.0, the information types "Name", "Telephone number", and "Address" to be masked are "Koyano", "090-0000-2111", and "Ota-ku, Tokyo" that correspond to the information types "Name", "Telephone number", and "Address" are not visible to the user in the printed matter of the masked image.

[0132] 12B is a printout obtained by checking and printing the masked image generated in S901 based on the masking display information with a transparency of 0.5. Because the transparency is set to 0.5, unlike the printout 1201, the masked image is printed in a state in which the character strings "Koyanno," "090-0000-2111," and "Ota-ku, Tokyo," which are the targets of masking, can be seen.

[0133] In this embodiment, a method has been described for generating a masking image in which the character string to be masked is visible when the mask display check box 1014 is not checked (mask display release state). The method for generating a masking image in which the character string to be masked is visible when the mask display release state is reached is not limited. Alternatively, when the mask display release state is reached, the display type of the masking display information in Table 4 may be changed to "underline". In this case, in S901, the image processing unit 423 generates a masking image by compositing an underline into the masking area in the document image.

[0134] A printout 1203 in Fig. 12(c) is an example of a printout obtained by checking and printing a masked image generated based on masking display information with a display type of "underline". By underlining the masked target in this way, the masked area and the contents of the masked character string can be visually confirmed. In addition, when the mask display is released, a confirmation masking image may be generated in which the area other than the masked area is filled with a background color and only the masked area is displayed.

[0135] In addition, in order to ensure that a printout of the masking image for confirming the mask display release state is promptly discarded, the masking image may be controlled so that a character string or image urging the user to discard it is superimposed on the masking image and then the masking image is printed for confirmation.

[0136] In S1103, the display control unit 427 determines whether or not the list display button 1044 of the preview display screen 1000 in Fig. 10(a) has been pressed. If the display control unit 427 determines that the list display button 1044 has been pressed (YES in S1103), the process proceeds to S1104. If the display control unit 427 determines that the list display button 1044 has not been pressed (NO in S1103), the process ends this flowchart and returns to S901 in Fig. 9.

[0137] In S1104, the display control unit 427 acquires the data of the document image and the masking information from the RAM 213 via the image processing unit 423. The display control unit 427 cuts out a masking area, which is a character string area associated with the masking necessity of the masking information being "necessary", from the document image to generate a partial image. The partial image is stored in the RAM 213.

[0138] In S1105, the display control unit 427 acquires the partial image and masking information generated in S1104 from the RAM 213, and generates a list print screen for the user to check the masking area. The display control unit 427 displays the generated list print screen on the touch panel of the operation unit 220 via the UI display control unit 414.

[0139] 13 is a diagram showing an example of a list print screen. The list print screen 1300 includes a masking information list 1301, a back button 1302, and a list print button 1303. The back button 1302 is a button for closing the list print screen 1300. The list print button 1303 is a button for instructing printing of the masking information list 1301.

[0140] The masking information list 1301 is a list display image for confirmation in which the partial images generated in S1104, the information types corresponding to the partial images, and character strings corresponding to the information types are associated with each other on a line basis. The information types are information types associated with masking areas referenced when the partial images are cut out in the masking information. The character strings corresponding to the information types are character strings corresponding to the information types associated with masking areas referenced when the partial images are cut out in the masking information.

[0141] In S1106, the display control unit 427 determines whether or not the back button 1302 has been pressed. If the display control unit 427 determines that the back button 1302 has been pressed (YES in S1106), this flowchart ends and the process returns to S901 in Fig. 9. If the display control unit 427 determines that the back button 1302 has not been pressed (NO in S1106), the process proceeds to S1107.

[0142] In S1107, the display control unit 427 determines whether or not the list print button 1303 has been pressed. If the display control unit 427 determines that the list print button 1303 has been pressed (YES in S1107), the process proceeds to S1108. If the display control unit 427 determines that the list print button 1303 has not been pressed (NO in S1107), the process returns to S1105.

[0143] In S1108, the image processing unit 423 generates a list display image representing the masking information list 1301 and saves it in the RAM 213. The display control unit 427 acquires the generated list display image from the RAM 213. The display control unit 427 requests the print execution unit 413 to print the acquired list display image via the print instruction unit 425. The print execution unit 413 prints the list display image and returns to S1105.

[0144] In addition, in order to ensure that the printed matter obtained by printing the list display image is promptly discarded, the list display image may be controlled so that a character string or image urging the user to discard the printed matter is superimposed on the list display image before the list display image is printed.

[0145] In S1108, in addition to requesting printing of the list display image, the display control unit 427 may obtain the masking image generated in S901 from the RAM 213 and request the print execution unit 413 to print the obtained masking image via the print instruction unit 425.

[0146] In this way, a confirmation print or a list display image can be printed before outputting a masked image. For example, when the touch panel of the MFP 110 is small and the screen size is small, the display of the masked part of the previewed image may be small, resulting in low visibility. In such a case, there is a risk of missing the confirmation of the masked part. In this embodiment, when the confirmation print or the printing of the list display image is selected, the user can check the printed matter. Therefore, the user can perform the confirmation work in a state where the visibility of the masked part is improved. Therefore, the user can accurately check the excess, deficiency, and error of the masked part, and it is possible to appropriately mask and output the character string related to the confidential information or the range of disclosure. Returning to FIG. 9, the rest of the flowchart will be described.

[0147] In S911, the display control unit 427 acquires the document image corrected in S701, the masking information generated in S707, and the masking display information from the RAM 213 via the image processing unit 423. If the masking information is changed in S905, the changed masking information is acquired. The image processing unit 423 acquires the start and end coordinates of the character string area in which the masking necessity is "necessary" in the acquired masking information, and performs masking on the masking area, which is an area corresponding to the coordinate position on the document image, to generate a masked image. The image processing unit 423 performs masking using the display type and display color among the information included in the acquired masking display information. In S911, the image processing unit 423 performs masking with the transparency set to 0.0 without using the transparency included in the acquired masking display information. That is, in S911, the masking image is generated based on the masking display information in Table 4. The image processing unit 423 holds the generated masking image for transmission in the RAM 213. In this way, the masking image for output is masked with a transparency of 0.0 so that the character string in the masked area is not visible. After S911, the masking display information is changed to the state shown in Table 2.

[0148] [About preset registration] FIG. 14 is a flowchart for explaining the details of the preset registration process in S506 in this embodiment.

[0149] In S1401, the display control unit 427 displays, on the touch panel of the operation unit 220 via the UI display control unit 414, a screen (not shown) for confirming with the user whether or not to perform preset registration.

[0150] In S1402, the display control unit 427 determines whether or not a user operation has been performed to register a preset. If the display control unit 427 determines that a user operation has been performed to register a preset (YES in S1402), the process proceeds to S1403. If the display control unit 427 determines that a user operation has not been performed to register a preset (NO in S1402), the process ends.

[0151] In S1403, the display control unit 427 requests the UI display control unit 414 to display a preset registration screen on the touch panel of the operation unit 220. If the information type of the generated masking information holds items other than "rectangular area", the display control unit 427 generates a preset registration screen 1500 (see FIG. 15) for selecting an information type. If the information type of the masking information holds only "rectangular area", the display control unit 427 generates a preset registration screen (not shown) for area selection.

[0152] Fig. 15 is a diagram showing an example of a preset registration screen that is displayed when an item other than "rectangular area" is held as the information type of the masking information. The preset registration screen 1500 in Fig. 15 has a preset name display area 1510, a document type selection area 1520, a registration information display area 1530, a save button 1503, and a cancel button 1504. The cancel button 1504 is a button for ending this flowchart without performing the preset registration process.

[0153] The preset name display area 1510 is an area for displaying the preset name, which is a name for identifying the preset to be registered. Pressing the preset name display area 1510 transitions to a screen for correcting the preset name, and the correction of the preset name is accepted. It is assumed that the preset name is a character string representing the document type that is set in advance.

[0154] The document type selection area 1520 includes a pull-down menu for the user to select the document type to be registered as a preset. The drop-down menu included in the document type selection area 1520 displays the document type selected in the document type selection drop-down 1021 of the document type display area 1020 as the default selection.

[0155] The registration information display area 1530 includes display areas 1531-1533 generated for each information type. The display areas 1531-1533 include toggle buttons 1535-1537 for selecting whether or not to register the corresponding information type in the preset as an information type to be masked. When the toggle buttons 1535-1537 are selected to be ON, the corresponding information type is registered in the preset as an information type to be masked. When the toggle buttons 1535-1537 are selected to be OFF, the corresponding information type is not registered in the preset. The initial display of the toggle buttons 1535-1537 on the preset registration screen 1500 is set according to whether or not masking of the masking information is required.

[0156] A save button 1503 is a button for registering the registration type selected in the registration information display area 1530 as a masking target as a preset for the document type selected in the document type selection area 1520. As a result, preset information such as that shown in Table 2 is generated.

[0157] In S1404, the display control unit 427 requests the data management unit 424 to store the preset information generated in S1403. The data management unit 424 stores the preset information in the external storage 120 via the Internet access unit 426. Alternatively, the preset information may be stored in the HDD 214 via the internal data storage unit 412.

[0158] When a preset of the same document type is saved in the HDD 214 or the external storage 120, the display control unit 427 may request the touch panel of the operation unit 220 to display a warning display inquiring the user whether or not to overwrite and save the preset. Furthermore, when the preset registration is normally completed, the display control unit 427 may request the operation unit 220 to display a message indicating that the preset registration is completed.

[0159] As described above, according to this embodiment, since the masked image can be displayed in the mask display release state, it is possible to check the contents of the masked parts of the generated masked image before outputting the masked image. Therefore, it is possible to prevent excesses, shortages, or errors in the masked parts. In addition, it is possible to mask and transmit character strings related to confidential information or disclosure range so that they cannot be viewed.

[0160] <Second embodiment> In this embodiment, a method for more appropriately checking the masking process on the list print screen 1300 will be described. The present embodiment will be described focusing on the differences from the first embodiment. Parts that are not specifically mentioned have the same configuration and process as the first embodiment.

[0161] 9, the image processing unit 423 acquires a document image, masking information, and masking display information from the RAM 213, similarly to the first embodiment. Furthermore, in the present embodiment, the image processing unit 423 acquires masking display correction information from the RAM 213 in S901.

[0162] Table 7 is a table showing an example of the masking display correction information. The masking display correction information includes an area correction width and an area correction height. For example, the display correction width is 5, and the area correction height is 2.

[0163] [Table 7] In S901, the image processing unit 423 changes the coordinates of the start point and the end point of the character string area included in the acquired masking information according to the numerical value of the masking display correction information, and holds the changed coordinate information as information of the changed character string area. The x-coordinate value of the start point coordinate is subtracted by the area correction width, and the y-coordinate value is subtracted by the area correction height. In addition, the x-coordinate value of the end point coordinate is added by the area correction width, and the y-coordinate value is subtracted by the area correction height. For example, if the character string area is "(341,461), (997,545)", the changed character string area becomes "(336,459), (1002,547)". In S901, a character string for which masking is "necessary" may be masked based on the changed coordinates, or may be masked based on the coordinates before the change.

[0164] 11, the display control unit 427 cuts out a masking area, which is a character string area associated with the masking information's masking necessity field set to "necessary", from the document image to generate a partial image. In this embodiment, the masking area is cut out based on the coordinates of the character string area (masking area) changed in S901 to generate a partial image. Therefore, a partial image is generated in which a larger area is cut out than in the first embodiment.

[0165] In S1105, the display control unit 427 generates a list print screen based on the partial image and masking information generated in S1104, and displays the list print screen on the touch panel of the operation unit 220 via the UI display control unit 414.

[0166] In this manner, in this embodiment, a partial image of an area obtained by expanding the area to be masked can be displayed as a list print image. Therefore, the user can confirm whether a character string detected as a character string corresponding to a certain information type is actually a character string corresponding to that information type. For example, as shown in Table 1, assume that the character string "Kiyano" is detected as a character string corresponding to the information type "name". However, there is a possibility that "Kiyano" included in the character string "Kiyano Co., Ltd." of the company name is detected as a character string corresponding to the information type "name". Therefore, in this embodiment, a partial image including a surrounding area of ​​the character string area where the masking necessity is "necessary" is displayed as the list print screen of FIG. 13. Therefore, the user can confirm as an image whether "Kiyano" detected as a character string corresponding to a certain information type is a part of a certain character string, such as "Kiyano Co., Ltd.". Therefore, it is possible to confirm whether an appropriate character string has been detected as a masking target.

[0167] <Other embodiments> The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0168] The disclosure of the above-described embodiment includes the following configurations.

[0169] (Configuration 1) An acquisition means for acquiring a document image; an image processing means for generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user; a control means for controlling the presentation of the predetermined character string in a manner recognizable by the user based on an instruction from the user; 13. An image processing device comprising:

[0170] (Configuration 2) The control means presents a confirmation image that enables the user to recognize the position of the target area in the document image and the predetermined character string including the target area. 2. The image processing device according to claim 1,

[0171] (Configuration 3) the masking image is an image in which a mask is combined with the target area of ​​the document image, The control means The confirmation image in which a mask having a higher transparency than the transparency of the mask in the output masking image is combined with the target region of the document image is presented to the user. 3. The image processing device according to configuration 2.

[0172] (Configuration 4) The control means Presenting the confirmation image in which an underline is synthesized in the target area of ​​the document image. 3. The image processing device according to configuration 2.

[0173] (Configuration 5) The control means The confirmation image is displayed on a display unit to present the confirmation image. 5. The image processing device according to any one of configurations 2 to 4.

[0174] (Configuration 6) The control means The confirmation image is presented by processing the image to be printed. 6. The image processing device according to any one of configurations 2 to 5,

[0175] (Configuration 7) The control means the masking image in which the target area is masked so that the user cannot recognize the predetermined character string included in the document image; and the confirmation image in which the user can recognize the predetermined character string included in the document image. Switch and display based on the user's operation 6. The image processing device according to configuration 5.

[0176] (Configuration 8) The image processing means includes: further generating a screen showing a list of the predetermined character strings included in the target area of ​​the document image; The control means The screen is presented to the user, so that the predetermined character string included in the target area is presented so as to be recognizable by the user. 2. The image processing device according to claim 1,

[0177] (Configuration 9) The image processing means includes: A partial image corresponding to the target region in the document image is generated, and the screen including the partial image is generated. 9. The image processing device according to configuration 8.

[0178] (Configuration 10) The image processing means includes: Generate a sub-image of an area of ​​the document image that includes the target area and is larger than the target area. 10. The image processing device according to configuration 9.

[0179] (Configuration 11) On the screen, the partial image, the character string represented by the partial image, and the information type of the character string are associated with each other. 11. The image processing device according to configuration 9 or 10.

[0180] (Configuration 12) The control means The screen is displayed on a display unit. 12. The image processing device according to any one of configurations 8 to 11,

[0181] (Configuration 13) The control means Process the screen so that it is printed 13. The image processing device according to any one of configurations 8 to 12.

[0182] (Configuration 14) The method further includes receiving means for receiving an instruction to change the target area from the user. The image processing means generates the masking image in which the target area changed by the user is masked. 14. The image processing device according to any one of configurations 1 to 13,

[0183] (Configuration 15) The masking image is an image in which the target area is filled with a predetermined color. 15. The image processing device according to any one of configurations 1 to 14.

[0184] (Configuration 16) The document image further includes a detection unit for detecting the predetermined character string from the document image. 16. The image processing device according to any one of configurations 1 to 15.

[0185] (Configuration 17) The method further includes outputting the generated masking image based on an instruction from the user. 17. The image processing device according to any one of configurations 1 to 16,

[0186] (Configuration 18) An acquisition step of acquiring a document image; an image processing step of generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user; a control step of performing control for presenting the predetermined character string in a manner recognizable by the user based on an instruction from the user; 13. An image processing method comprising:

[0187] (Configuration 19) A program for causing a computer to execute each of the means of the image processing device according to any one of configurations 1 to 17. [Explanation of symbols]

[0188] 110 MFP 427 Display control section 423 Image Processing Unit

Claims

1. An acquisition means for acquiring a document image; an image processing means for generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user; a control means for controlling the presentation of the predetermined character string in a manner recognizable by the user based on an instruction from the user; 13. An image processing device comprising:

2. The control means presents a confirmation image that enables the user to recognize the position of the target area in the document image and the predetermined character string including the target area.

2. The image processing device according to claim 1,

3. the masking image is an image in which a mask is combined with the target area of ​​the document image, The control means The confirmation image in which a mask having a higher transparency than the transparency of the mask in the output masking image is combined with the target region of the document image is presented to the user.

3. The image processing device according to claim 2.

4. The control means Presenting the confirmation image in which an underline is synthesized in the target area of ​​the document image.

3. The image processing device according to claim 2.

5. The control means The confirmation image is displayed on a display unit to present the confirmation image.

3. The image processing device according to claim 2.

6. The control means The confirmation image is presented by processing the image to be printed.

3. The image processing device according to claim 2.

7. The control means The masking image, in which the target area is masked so that the user cannot recognize the predetermined character string included in the document image, and the confirmation image, in which the user can recognize the predetermined character string included in the document image, are switched and displayed based on an operation by the user.

6. The image processing device according to claim 5,

8. The image processing means includes: further generating a screen showing a list of the predetermined character strings included in the target area of ​​the document image; The control means The screen is presented to the user, so that the predetermined character string included in the target area is presented so as to be recognizable by the user.

2. The image processing device according to claim 1,

9. The image processing means includes: A partial image corresponding to the target region in the document image is generated, and the screen including the partial image is generated.

9. The image processing device according to claim 8,

10. The image processing means includes: Generate a sub-image of an area of ​​the document image that includes the target area and is larger than the target area.

10. The image processing device according to claim 9,

11. On the screen, the partial image, the character string represented by the partial image, and the information type of the character string are associated with each other.

10. The image processing device according to claim 9,

12. The control means The screen is displayed on a display unit.

9. The image processing device according to claim 8,

13. The control means Process the screen so that it is printed 9. The image processing device according to claim 8,

14. The method further includes receiving means for receiving an instruction to change the target area from the user. The image processing means generates the masking image in which the target area changed by the user is masked.

2. The image processing device according to claim 1,

15. The masking image is an image in which the target area is filled with a predetermined color.

2. The image processing device according to claim 1,

16. The document image further includes a detection unit for detecting the predetermined character string from the document image.

2. The image processing device according to claim 1,

17. The method further includes outputting the generated masking image based on an instruction from the user.

2. The image processing device according to claim 1,

18. An acquisition step of acquiring a document image; an image processing step of generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user; a control step of performing control for presenting the predetermined character string in a manner recognizable by the user based on an instruction from the user; 13. An image processing method comprising:

19. A program for causing a computer to execute each of the means of the image processing apparatus according to any one of claims 1 to 17.

Images