Image processing apparatus

Abstract

To easily change personal information to be anonymized.SOLUTION: An image processing apparatus comprises an image reading unit, a control unit, and an operation display unit. After generating image data for output, the control unit displays a preview image corresponding to the image data for output. The operation display unit receives a first editing operation and a second editing operation. The first editing operation is an operation to designate a non-anonymized region. The second editing operation is an operation to designate an anonymized region. When the first editing operation is performed, the control unit performs anonymization processing on the region designated by the first editing operation. When the second editing operation is performed, the control unit returns the region designated by the second editing operation to a state before the anonymization processing.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an image processing device. [Background technology]

[0002] Conventionally, there has been known an image processing device that reads a document containing personal information and performs anonymization processing (in other words, confidentiality processing) on ​​the personal information area of ​​the image data obtained by reading the document. Such an image processing device is disclosed, for example, in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-113606 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, multiple pieces of personal information may be written on a single document. Some of the personal information may be anonymized while others may not. In this case, the personal information you want to anonymize may not be anonymized, or the personal information you do not want to anonymize may be anonymized. If this happens, you will have to review the settings related to the anonymization of the personal information and re-scan the document. This is inconvenient for users.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide an image processing device that can easily change the personal information to be anonymized in a configuration that anonymizes personal information in image data obtained by reading a document. [Means for solving the problem]

[0006] To achieve the above object, an image processing device according to one aspect of the present invention includes an image reading unit that reads a document containing personal information; a control unit that generates output image data in which the personal information has been anonymized by performing an anonymization process on original image data obtained by reading the document with the image reading unit; and an operation display unit that displays information and accepts operations. When generating the output image data, the control unit extracts text data by performing an OCR process on the original image data, extracts personal information from the text data, recognizes an area of ​​the original image data containing the personal information as a target area, and performs an anonymization process to anonymize the information in the target area. After generating the output image data, the control unit displays a preview image corresponding to the output image data on the operation display unit and causes the operation display unit to accept an editing operation for editing the output image data. When the editing operation is performed, the control unit edits the output image data based on the content of the editing operation. The operation display unit accepts a first editing operation and a second editing operation as the editing operation. The first editing operation is an operation for designating an area of ​​the preview image that is not anonymized as an area to be edited. The second editing operation is an operation of designating an anonymized region of the preview image as an editing target. When the first editing operation is performed, the control unit performs an anonymization process on the region designated by the first editing operation. When the second editing operation is performed, the control unit returns the region designated by the second editing operation to the state before the anonymization process. [Effects of the Invention]

[0007] In the present invention, in a configuration in which personal information in image data obtained by scanning a document is anonymized, the personal information to be anonymized can be easily changed. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram of a multifunction peripheral according to an embodiment. [Figure 2] FIG. 1 is a block diagram of a multifunction peripheral according to an embodiment. [Figure 3] 1 is a diagram schematically illustrating an example of a document (personal information) that can be read by the multifunction peripheral of an embodiment. [Figure 4] 10 is a flowchart illustrating a flow of an anonymization job executed by the multifunction peripheral of the embodiment. [Figure 5] FIG. 10 illustrates a setting screen displayed when an information selection operation is accepted by the multifunction peripheral of the embodiment. [Figure 6] FIG. 10 illustrates a setting screen displayed when a method selection operation is accepted by the multifunction peripheral of the embodiment. [Figure 7] 10 is a flowchart illustrating a flow of a personal information extraction process executed by the multifunction peripheral of the embodiment. [Figure 8] 10A and 10B are diagrams for explaining anonymization processing (filling in) executed by the multifunction peripheral of an embodiment; [Figure 9] 10A and 10B are diagrams for explaining anonymization processing (initialization and labeling) executed by the multifunction peripheral of an embodiment. [Figure 10] 10A and 10B are diagrams for explaining a first editing operation accepted by the multifunction peripheral of the embodiment. [Figure 11] 10A and 10B are diagrams for explaining a second editing operation accepted by the multifunction peripheral of the embodiment. [Figure 12] 10A and 10B are diagrams for explaining a display switching operation accepted by the multifunction peripheral of the embodiment; [Figure 13] 10A and 10B are diagrams for explaining a candidate display operation accepted by the multifunction peripheral of an embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0009] <Configuration of the multifunction device> An image processing apparatus according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 13, taking as an example a multifunction peripheral 100 having multiple functions such as a scanning function, a printing function, and a transmission function.

[0010] As shown in FIG. 1, the multifunction device 100 (corresponding to an "image processing device") includes a printing unit 1. The printing unit 1 constitutes the main body of the multifunction device 100. The printing unit 1 prints an image on a sheet S. The printing method of the printing unit 1 is an electrophotographic method. However, this is not limited to this. The printing method of the printing unit 1 may also be an inkjet method.

[0011] The printing unit 1 forms an image based on image data input to the multifunction device 100. The printing unit 1 also transports a sheet S along a sheet transport path. The printing unit 1 prints an image on the sheet S while it is being transported. In FIG. 1, the sheet transport path is indicated by a dashed line.

[0012] The printing unit 1 includes a paper feed roller 11. The paper feed roller 11 comes into contact with a sheet S stored in a sheet cassette CA and rotates in this state, thereby feeding the sheet S from the sheet cassette CA to a sheet transport path.

[0013] The printing unit 1 includes an image forming unit 12. The image forming unit 12 includes a photosensitive drum 12a and a transfer roller 12b. The photosensitive drum 12a carries a toner image on its peripheral surface. The transfer roller 12b is in pressure contact with the photosensitive drum 12a, forming a transfer nip between the photosensitive drum 12a and the transfer roller 12b. The transfer roller 12b rotates together with the photosensitive drum 12a. The image forming unit 12 transfers the toner image onto the sheet S while transporting the sheet S that has entered the transfer nip.

[0014] Although not shown, the image forming unit 12 further includes a charging device, an exposure device, and a developing device. The charging device charges the circumferential surface of the photosensitive drum 12a. The exposure device forms an electrostatic latent image on the circumferential surface of the photosensitive drum 12a. The developing device develops the electrostatic latent image on the circumferential surface of the photosensitive drum 12a into a toner image.

[0015] The printing unit 1 includes a fixing unit 13. The fixing unit 13 includes a heating roller 13a and a pressure roller 13b. The heating roller 13a has a built-in heater (not shown). The pressure roller 13b is pressed against the heating roller 13a, forming a fixing nip between the heating roller 13a and the pressure roller 13b. The pressure roller 13b rotates together with the heating roller 13a. The fixing unit 13 fixes the toner image transferred onto the sheet S onto the sheet S while transporting the sheet S that has entered the fixing nip. The sheet S that has exited the fixing nip is discharged onto an ejection tray ET.

[0016] The multifunction device 100 also includes an image reading unit 2. The image reading unit 2 is disposed on top of the main body of the multifunction device 100. In a job involving reading an original D, the original D is set in the image reading unit 2. The image reading unit 2 reads the original D set in the image reading unit 2 and generates image data of the read original D.

[0017] The image reading unit 2 includes contact glasses G1 and G2. The contact glasses G1 and G2 are installed in a housing RH of the image reading unit 2. The housing RH has an opening on the top surface. The contact glasses G1 and G2 are attached to the opening on the top surface of the housing RH.

[0018] The image reading unit 2 includes a document transport device DP. The document transport device DP is attached to the housing RH. When viewed from the front of the multifunction device 100, the document transport device DP pivots around its rear portion as if swinging its front portion up and down. The document transport device DP opens and closes relative to the top surface of the housing RH.

[0019] The document transport device DP has a set tray ST on which a document D is set. The document transport device DP transports the document D set on the set tray ST onto the contact glass G1.

[0020] In the conveying reading mode, the user sets the document D on the set tray ST. Then, the document D is automatically conveyed onto the contact glass G1 by the document conveying device DP (in other words, the document D passing over the contact glass G1) and is read. On the other hand, in the placed reading mode, the user sets the document D on the contact glass G2, and the document D on the contact glass G2 is read.

[0021] The image reading unit 2 includes a light source 21, an image sensor 22, a mirror 23, and a lens 24. The light source 21, the image sensor 22, the mirror 23, and the lens 24 are installed inside the housing RH. The image reading unit 2 performs a scanning operation in which light is irradiated from the light source 21 toward the contact glass G1 or G2 and photoelectrically converted by the image sensor 22.

[0022] The light source 21 has a plurality of LED elements. The plurality of LED elements are arranged in a line in the main scanning direction (a direction perpendicular to the paper surface of FIG. 1). The image sensor 22 has a plurality of photoelectric conversion elements arranged in the main scanning direction. The mirror 23 reflects light toward the lens 24. The lens 24 collects the light reflected by the mirror 23 and guides it to the image sensor 22.

[0023] The light source 21 and the mirror 23 are mounted on a carriage 25 that is movable in a sub-scanning direction (left and right direction in FIG. 1) that is perpendicular to the main scanning direction. As the carriage 25 moves in the sub-scanning direction, the reading line of the image reading unit 2 moves in the sub-scanning direction.

[0024] As shown in FIG. 2, the multifunction device 100 also includes an operation display unit 3. The operation display unit 3 is an operation panel having a touch screen. The operation display unit 3 displays software buttons, messages, and the like on the touch screen. The operation display unit 3 is also provided with a plurality of hardware buttons. The operation display unit 3 accepts operations from the user. The user can configure the multifunction device 100 via the operation display unit 3 regarding various jobs, such as an anonymization job, which will be described later.

[0025] The multifunction peripheral 100 includes a control unit 10. The control unit 10 includes a CPU, an ASIC, a memory, and the like. The control unit 10 also includes an image processing circuit. The control unit 10 performs various image processing operations on image data. The control unit 10 also controls the printing of an image onto a sheet S by the printing unit 1, and the reading of an original D by the image reading unit 2.

[0026] The control unit 10 also controls the operation and display unit 3. Specifically, the control unit 10 controls the display operation of the touch screen. The control unit 10 detects operations on software buttons and hardware buttons. The control unit 10 sets jobs based on operations received by the operation and display unit 3 from the user.

[0027] The multifunction peripheral 100 includes a storage unit 101. The storage unit 101 is a non-volatile storage device. An HDD, an SSD, or the like may be used as the storage unit 101. The storage unit 101 is connected to the control unit 10. The control unit 10 writes information to the storage unit 101 and reads information from the storage unit 101.

[0028] The storage unit 101 stores a character recognition program in advance. The control unit 10 performs character recognition processing such as OCR (Optical Character Recognition) processing based on the character recognition program. The control unit 10 subjects the image data obtained by the image reading unit 2 reading the document D to the character recognition processing.

[0029] The multifunction peripheral 100 includes a communication unit 102. The communication unit 102 is an interface for communicatively connecting an external device to the multifunction peripheral 100. The communication unit 102 includes a communication circuit, a communication memory, a communication connector, and the like. The communication unit 102 is connected to the control unit 10. The control unit 10 uses the communication unit 102 to send and receive data to and from the external device.

[0030] The communication unit 102 is communicably connected to an external device via a network NT such as a LAN or the Internet. Although not shown, the communication unit 102 may be directly connected to an external device via a communication cable. An external device connected to the communication unit 102 is, for example, a personal computer 1000 (hereinafter referred to as PC 1000) used by a user of the multifunction peripheral 100. An external device other than the PC 1000 may be communicably connected to the multifunction peripheral 100. By connecting the PC 1000 to the multifunction peripheral 100, image data of the original D obtained by reading the original D with the image reading unit 2 can be transmitted to the PC 1000. This allows the image data of the original D to be saved in the PC 1000.

[0031] <Anonymization process overview> The multifunction peripheral 100 has an anonymization function. In other words, the multifunction peripheral 100 is capable of executing a job related to the anonymization function (hereinafter referred to as an anonymization job). In the anonymization job, a document D containing personal information is read, and an anonymization process is performed on the original image data obtained by reading the document D to anonymize the personal information. As a result, image data for output is generated in which at least a portion of the personal information has been anonymized. The image data for output is image data generated from the original image data, and is image data in which a portion of the original image data has been altered.

[0032] By using the anonymization function, it is possible to obtain image data (i.e., output image data) in which at least part of the personal information of the document D has been anonymized. Then, an image based on the output image data can be printed on a sheet S. In addition, the output image data can be sent to the PC 1000 and saved in the PC 1000.

[0033] The document D containing personal information may be a driver's license, an insurance card, a passport, a medical record (chart), etc. The personal information may be a name, an address, a telephone number, a credit card number, an email address, etc. In the following explanation, the document D containing personal information will be simply referred to as the document D.

[0034] An example of manuscript D is shown in Figure 3. Personal information (item value) and the type of personal information (item name) are written in pairs in manuscript D. For convenience, Figure 3 illustrates each piece of personal information related to name, address, and telephone number.

[0035] When executing an anonymization job, the user sets the document D on the image reading unit 2. In this state, the user performs a start operation for the anonymization job on the operation display unit 3. When the start operation is performed on the operation display unit 3, the control unit 10 starts the anonymization job.

[0036] The flow of an anonymization job will be described below with reference to the flowchart shown in Fig. 4. The flow shown in Fig. 4 starts when a start operation for an anonymization job is performed on the operation display unit 3.

[0037] In step #1, the control unit 10 causes the image reading unit 2 to read the original document D. The image reading unit 2 reads the original document D and generates image data of the read original document D. The image data generated at this time becomes the original image data. The control unit 10 acquires the original image data obtained by the image reading unit 2 reading the original document D.

[0038] In step #2, the control unit 10 performs OCR processing on the original image data and extracts text data from the original image data. As a result, the control unit 10 recognizes character strings present in the original image data. The control unit 10 also recognizes the position (i.e., coordinates) of a character area containing character strings within the original image data. If the document D shown in FIG. 3 is the target of the anonymization job, character strings A, B, and C are extracted as text data.

[0039] In step #3, the control unit 10 performs morphological analysis on the text data extracted from the original image data. That is, the control unit 10 breaks down the text data extracted from the original image data into word units. When the manuscript D shown in FIG. 3 is the target of the anonymization job, the character string A is broken down into word character strings A0, A1, and A2. The character string B is broken down into word character strings B0 and B1. The character string C is broken down into word character strings C0 and C1. Furthermore, the control unit 10 performs morphological analysis on the text data extracted from the original image data to determine the part of speech of each word in the text data (i.e., the broken down character strings).

[0040] In step #4, the control unit 10 performs a personal information extraction process. Specifically, the control unit 10 recognizes whether each word in the text data extracted from the original image data is personal information, and extracts the personal information from the text data. In other words, the control unit 10 extracts named entities from the text data. The personal information extraction process will be described in detail later.

[0041] In step #5, the control unit 10 causes the operation display unit 3 to accept an information selection operation. The information selection operation is an operation for selecting personal information to be anonymized. The control unit 10 subjects the personal information selected by the information selection operation to anonymization processing. Note that the information selection operation may be accepted before the anonymization job starts (i.e., before the document D is read).

[0042] The operation display unit 3 displays a setting screen 31 as shown in Fig. 5 and accepts information selection operations from the user. A plurality of information selection buttons SB1 (software buttons) corresponding to a plurality of types of personal information are arranged on the setting screen 31. The operation display unit 3 accepts a touch operation on any of the information selection buttons SB1 as an information selection operation.

[0043] The control unit 10 detects the information selection button SB1 that has received the information selection operation (i.e., the touched information selection button SB1), and sets the personal information corresponding to the detected information selection button SB1 as the target for anonymization. For example, when an information selection operation is detected for the information selection button SB1 labeled "Name," the control unit 10 recognizes the personal information corresponding to the information selection button SB1 labeled "Name" as information to be anonymized.

[0044] In step #6, the control unit 10 causes the operation display unit 3 to accept a method selection operation. The method selection operation is an operation for selecting a method for anonymizing personal information. The control unit 10 recognizes the method selected in the method selection operation as the anonymization method to be executed. For example, three anonymization methods are prepared: filling in, initializing, and labeling, and any one of the three anonymization methods can be selected. Note that the number of selectable anonymization methods may be two, four, or more.

[0045] The operation display unit 3 displays a setting screen 32 as shown in Fig. 6 and accepts a method selection operation from the user. A plurality of method selection buttons SB2 (software buttons) corresponding to a plurality of anonymization methods are arranged on the setting screen 32. The operation display unit 3 accepts a touch operation on any of the method selection buttons SB2 as a method selection operation.

[0046] The control unit 10 detects the method selection button SB2 that has received the method selection operation (i.e., the touched method selection button SB2), and recognizes the method corresponding to the detected method selection button SB2 as the anonymization method to be executed. For example, when a method selection operation is detected for the method selection button SB2 labeled "fill in," the control unit 10 recognizes that the anonymization method to be executed is fill in.

[0047] In the information selection operation, multiple types of personal information can be selected as information to be anonymized. When multiple types of personal information are selected as information to be anonymized, an anonymization method can be selected for each of the selected multiple types of personal information. In this configuration, the operation display unit 3 accepts a method selection operation for each of the multiple types of personal information selected in the information selection operation. On the setting screen 32 (see FIG. 6) that accepts the method selection operation, the type (i.e., item) of the personal information to be anonymized is displayed in a field 320.

[0048] In step #7, the control unit 10 performs an anonymization process on the original image data. As a result, the control unit 10 generates output image data in which at least a portion of the personal information has been anonymized. The anonymization process is a process of anonymizing the personal information in the target area.

[0049] When generating image data for output, the control unit 10 recognizes an area of ​​the original image data that includes personal information selected in the information selection operation as a target area. That is, the control unit 10 recognizes an area of ​​the original image data that includes personal information to be anonymized as a target area. For example, if a name is selected in the information selection operation, the control unit 10 recognizes an area of ​​the original image data that includes personal information corresponding to the name as a target area.

[0050] As a variant example, predetermined personal information may be set as information to be anonymized by default, in which case the control unit 10 will anonymize the personal information even if the personal information is not selected in the information selection operation.

[0051] After recognizing the target area in the original image data, the control unit 10 anonymizes the personal information in the target area using the method selected in the method selection operation. That is, the control unit 10 performs an anonymization process on the personal information in the target area. The anonymization process will be described in detail later.

[0052] In step #8, the control unit 10 generates a preview image PG (see FIGS. 10 to 13) corresponding to the output image data and displays it on the operation display unit 3. The operation display unit 3 enlarges and displays an area of ​​the preview image PG that includes personal information. The user may be able to specify the portion of the preview image PG to be enlarged and displayed. In this case, the operation display unit 3 enlarges and displays the portion of the preview image PG that the user specifies.

[0053] In step #9, the control unit 10 causes the operation display unit 3 to accept an editing operation to edit the output image data. Note that the display of the preview image PG on the operation display unit 3 continues. That is, when accepting an editing operation, the operation display unit 3 displays the preview image PG. While the preview image PG is being displayed, the control unit 10 determines whether an editing operation has been performed. Editing of the output image data will be described in detail later.

[0054] If the control unit 10 determines that an editing operation has been performed, the process proceeds to step #10. In step #10, the control unit 10 edits the output image data based on the content of the editing operation accepted by the operation display unit 3. In other words, the control unit 10 generates new output image data based on the content of the editing operation accepted by the operation display unit 3.

[0055] Then, the control unit 10 generates a new preview image PG corresponding to the new output image data, and displays the new preview image PG in place of the existing preview image PG on the operation display unit 3. Then, the control unit 10 again causes the operation display unit 3 to accept an editing operation for editing the output image data. That is, the process moves from step #10 to step #8.

[0056] If the control unit 10 determines in step #9 that no editing operation has been performed, the process proceeds to step #11. In other words, if the control unit 10 determines that an operation to instruct continuation of the job without performing an editing operation has been performed on the operation display unit 3, the process proceeds to step #11. After the editing operation (i.e., after the transition from step #10 to step #8), if no additional editing operation is required, an operation to instruct continuation of the job without performing an editing operation is performed on the operation display unit 3.

[0057] In step #11, the control unit 10 causes the output unit to perform output processing of the image data for output. For example, in the settings of an anonymization job, the output method of the image data for output can be selected. The output methods include printing and transmission.

[0058] When printing is selected as the output method, the control unit 10 causes the printing unit 1 to print (in other words, output) an image based on the image data for output onto a sheet S. In this case, the printing unit 1 corresponds to the "output unit," and the output destination is the sheet S.

[0059] When transmission is selected as the output method, the control unit 10 causes the communication unit 102 to transmit (in other words, output) the image data for output to the PC 1000. The image data for output may be converted into PDF data and transmitted to the PC 1000. By transmitting the image data for output to the PC 1000, the image data for output can be saved in the PC 1000. In this case, the communication unit 102 corresponds to the "output unit," and the output destination is the PC 1000.

[0060] In this embodiment, the operation display unit 3 accepts an information selection operation and a method selection operation. This allows the user to arbitrarily select personal information to be anonymized, and also to arbitrarily select a method for anonymizing the personal information. In other words, the user can anonymize desired personal information using a desired method. This improves user convenience.

[0061] <Extraction of personal information> The control unit 10 performs the personal information extraction process in accordance with the flow shown in Fig. 7. The flow shown in Fig. 7 starts after morphological analysis is performed on the text data extracted from the original image data (after the processing of step #3).

[0062] In step #21, the control unit 10 performs a first process of extracting a character string that matches a predetermined regular expression as personal information. The first process is a process of extracting a character string written in a predetermined pattern as personal information.

[0063] For example, examples of personal information consisting of character strings written in a predetermined pattern include a telephone number, a credit card number, an email address, a date (including a date of birth, etc.), and a time. The predetermined regular expression is determined in advance based on character string patterns of at least one (for example, all) of the telephone number, credit card number, email address, date, and time. Therefore, by performing the first process, personal information such as the telephone number, credit card number, email address, date, and time can be extracted.

[0064] In step #22, the control unit 10 performs a second process using a machine learning model for extracting named entities to extract character strings that are presumed to represent predetermined information as personal information. The machine learning model for the second process is a trained named entity extraction model, and is stored in advance in the storage unit 101. In the second process, named entities that were not extracted as personal information in the first process are processed.

[0065] The predetermined information is at least one of a character string indicating a person's name and a character string indicating a place name. For example, the predetermined information is both a character string indicating a person's name and a character string indicating a place name. That is, in the second process, the character string indicating a person's name and the character string indicating a place name are extracted as personal information. Therefore, by performing the second process, it is possible to extract the name and address as personal information. Note that character strings indicating other personal information, such as the person's permanent domicile, gender, occupation, the name of the organization to which the person belongs (for example, the name of the company or school), and the department to which the person belongs, may also be added as the predetermined information.

[0066] When document D shown in FIG. 3 is the target of an anonymization job, character string C1 is extracted as personal information in the first process. Other character strings are not extracted as personal information in the first process. On the other hand, character strings A1, A2, and B1 are extracted as personal information in the second process. Character string C1 is not processed in the second process.

[0067] In this embodiment, personal information such as telephone numbers, credit card numbers, and email addresses is extracted in the first process. Then, personal information that cannot be extracted in the first process (such as name and address) is extracted in the second process. This makes it possible to prevent personal information from being overlooked in extraction. In other words, it is possible to prevent the occurrence of personal information that cannot be anonymized even though it should be anonymized.

[0068] Furthermore, the second process uses a learning model, but the first process does not. Since personal information such as phone numbers, credit card numbers, and email addresses can be extracted in the first process, it is sufficient to extract only other personal information (such as name and address) in the second process. This reduces the calculation cost of the personal information extraction process.

[0069] <Method of anonymization> The three anonymization methods will be specifically described below with reference to FIGS.

[0070] 8 and 9, the area including the character string A1, the area including the character string A2, and the area including the character string B1 are recognized as target areas. That is, in the information selection operation, "Name" and "Address" are selected.

[0071] 8, the anonymization method for the region containing character string A1, the region containing character string A2, and the region containing character string B1 is set to fill in. That is, in the method selection operation, fill in is selected as the anonymization method for each of "Name" and "Address."

[0072] 9, the anonymization method for the region containing character string A1 and the region containing character string A2 is set to initialization, and the anonymization method for the region containing character string B1 is set to labeling. That is, in the method selection operation, initialization is selected as the anonymization method for "Name," and labeling is selected as the anonymization method for "Address."

[0073] When the anonymization method is painting, the control unit 10 performs the anonymization process of painting over at least a portion of the personal information in the target area. For example, at least a portion of the personal information in the target area is replaced with a solid black image.

[0074] 8, the character strings A1 and A2 are blacked out as a result of the anonymization process, and the character string B1 is also blacked out.

[0075] When the anonymization method is initialization, the control unit 10 recognizes the initial letters of the character string that constitutes the personal information in the target area, and performs an anonymization process that makes characters other than the initial letters of the personal information in the target area unrecognizable. For example, when the personal information to be anonymized is a name, initialization can be selected as the anonymization method.

[0076] 9, the "Y" in character string A1 and the "T" in character string A2 are each recognized as initial characters. As a result, all characters in character string A1 after the initial "Y" are made unrecognizable, and all characters in character string A2 after the initial "T" are made unrecognizable.

[0077] For example, as shown in Figure 9, a character string constituting a name as personal information in the target area may be replaced with a character string consisting of the initials of the character string. Although not shown, characters other than the initials of the character string constituting the name as personal information in the target area may be blacked out. Note that if the name as personal information in the target area is in Japanese, the Japanese may be converted into a romanized alphabetic character string, and the personal information in the target area may be replaced with a character string consisting of the initials of the alphabetic character string.

[0078] When the anonymization method is labeling, the control unit 10 recognizes labeling character strings associated with the types of personal information in the target area and performs an anonymization process of replacing the personal information in the target area with the labeling character strings associated with those types. Correspondence information indicating the correspondence between the types of personal information and the labeling character strings is determined in advance and stored in the storage unit 101. The control unit 10 recognizes the labeling character strings associated with the types of personal information in the target area based on the correspondence information.

[0079] For example, the labeling character string "Location" is associated with "Address" as a type of personal information. Therefore, in the example shown in Fig. 9, the character string B1 is replaced from "Tokyo" to "Location".

[0080] In this embodiment, these three anonymization methods are prepared as options for the method selection operation. Regardless of which anonymization method is used to anonymize personal information, the personal information can be reliably protected.

[0081] <Editing image data for output> Editing of image data for output will be specifically described below with reference to Figures 10 to 13. In the following description, it is assumed that the method for anonymizing personal information is painting out.

[0082] The operation display unit 3 accepts a first editing operation and a second editing operation as editing operations. The first editing operation is an operation of designating an area of ​​the preview image PG where anonymized personal information exists as an editing target. The second editing operation is an operation of designating an area of ​​the preview image PG where non-anonymized personal information exists as an editing target.

[0083] Note that the areas that can be specified in the first and second editing operations are not limited to areas containing personal information. For example, the first and second editing operations can specify a non-extraction area, which will be described later. A non-extraction area is an area that contains a noun string. A noun string in a non-extraction area may be a string that constitutes personal information (named entity), or it may be a string that constitutes other information (for example, a type of personal information).

[0084] The first and second editing operations are operations of touching (tapping) an area within the preview image PG. The first and second editing operations may be an operation of touching an area within the preview image PG only once (single tap operation), an operation of touching an area within the preview image PG twice in succession (double tap operation), or an operation of continuing to touch an area within the preview image PG (long press operation).

[0085] When a first editing operation is performed, the control unit 10 determines that the area touched by the first editing operation has been designated as the area to be edited. When a second editing operation is performed, the control unit 10 determines that the area touched by the second editing operation has been designated as the area to be edited. Note that there are cases where only one of the first editing operation and the second editing operation is performed, and there are also cases where both the first editing operation and the second editing operation are performed.

[0086] Then, as shown in Fig. 10, when a first editing operation is performed, the control unit 10 anonymizes the personal information in the area designated by the first editing operation. Fig. 10 shows an example in which an area including a character string C1 is designated by the first editing operation. In Fig. 10, the upper diagram shows the state before the first editing operation, and the lower diagram shows the state after the first editing operation.

[0087] Furthermore, as shown in Fig. 11, when a second editing operation is performed, the control unit 10 returns the area specified by the second editing operation to the state before the anonymization process. Fig. 11 shows an example in which an area including a character string B1 is specified by the second editing operation. In Fig. 11, the upper diagram shows the state before the second editing operation, and the lower diagram shows the state after the second editing operation.

[0088] In this embodiment, the operation display unit 3 displays a preview image PG corresponding to the output image data and accepts editing operations. When adding personal information to be anonymized, all that is required is to perform a touch operation on the area containing the personal information (i.e., a first editing operation). When personal information that does not need to be anonymized has been mistakenly anonymized, all that is required is to perform a touch operation on the area containing the personal information (i.e., a second editing operation). This allows the output image data to be edited with a simple operation. In other words, the personal information to be anonymized can be easily changed. As a result, user convenience is improved.

[0089] For example, due to insufficient accuracy in the personal information extraction process, character strings that are not personal information, such as the character strings A0, B0, and C0 (see Figure 3), may be extracted as personal information and anonymized. In such cases, the anonymization of the personal information can be easily removed by performing a second editing operation on the area containing the erroneously anonymized personal information.

[0090] When accepting an editing operation, the operation display unit 3 enlarges and displays the area including the personal information in the preview image PG, thereby making it easy to specify the area including the personal information when performing an editing operation.

[0091] The display screen when accepting an editing operation will be described below with reference to Figures 12 and 13. For convenience, reference numerals of character strings are omitted in Figures 12 and 13, and Figure 3 is used instead.

[0092] In this embodiment, when accepting an editing operation, the operation display unit 3 accepts a display switching operation. Then, when accepting a display switching operation, the operation display unit 3 displays the anonymized personal information in a recognizable manner and accepts the editing operation. This allows the user to perform editing operations while checking the contents of the anonymized personal information, improving user convenience.

[0093] For example, as shown in Fig. 12, when accepting an editing operation, the operation display unit 3 displays display switching buttons as software buttons together with a preview image PG. The display switching buttons include a first display switching button B11 and a second display switching button B12. Note that Fig. 12 shows an example in which character strings A1, A2, and B1 are anonymized.

[0094] When an anonymized personal information is blacked out to make it unrecognizable (as shown in the upper diagram of FIG. 12), and a touch operation is performed on the first display switching button B11, the operation display unit 3 changes the anonymized personal information to a recognizable state (as shown in the lower diagram of FIG. 12). In this example, the operation on the first display switching button B11 corresponds to a "display switching operation."

[0095] When the first display switching button B11 is operated, the display density of the background in the area containing the anonymized personal information is made lighter than the display density of the text in the area containing the anonymized personal information. For example, the text is black and the background is gray.

[0096] On the other hand, when the second display switching button B12 is touched while the anonymized personal information is recognizable (as shown in the lower diagram of FIG. 12), the operation display unit 3 blacks out the anonymized personal information so that it becomes unrecognizable (as shown in the upper diagram of FIG. 12). This is convenient for the user as it allows them to preview the final output result of the output image data.

[0097] In this embodiment, when accepting an editing operation, the operation display unit 3 accepts a candidate display operation. When accepting the candidate display operation, the operation display unit 3 highlights a non-extraction area including a noun character string that was not extracted as personal information, and accepts the editing operation. Then, when a first editing operation is performed on the non-extraction area, the control unit 10 anonymizes the noun character string in the non-extraction area.

[0098] With this configuration, when a noun string that should have been extracted as personal information is not extracted due to insufficient accuracy in the personal information extraction process, the noun string can be easily identified. Then, when the user wishes to anonymize the noun string, the user can do so with a simple operation of touching the non-extraction area that contains the noun string (i.e., the first editing operation). This improves user convenience.

[0099] For example, as shown in Fig. 13, when accepting an editing operation, the operation display unit 3 displays a candidate display button B20 as a software button together with a preview image PG. When a touch operation is performed on the candidate display button B20, the operation display unit 3 surrounds an area of ​​the preview image PG that includes a noun character string that was not extracted as personal information with a frame image F. That is, the state transitions from the state shown in the upper diagram of Fig. 13 to the state shown in the lower diagram of Fig. 13. As a result, the non-extracted area is highlighted. In this example, the operation on the candidate display button B20 corresponds to a "candidate display operation."

[0100] The highlighting method is not particularly limited. For example, an image different from the frame image F may be added to the non-extraction area as a highlighting image. Furthermore, the display color of the noun character string in the non-extraction area may be a color different from the other character strings (for example, red), or may be bold.

[0101] In this embodiment, when the image reading unit 2 reads an original D, the control unit 10 compares the format of the original D (herein referred to as the first original D) read by the image reading unit 2 with the format of an original D (herein referred to as the second original D) read by the image reading unit 2 before the first original D was read. If the format of the first original D and the format of the second original D are the same, the control unit 10 reflects the anonymization processing performed on the output image data of the second original D in the output image data of the first original D. Furthermore, if the format of the first original D and the format of the second original D are the same, the control unit 10 reflects the editing performed on the output image data of the second original D in the output image data of the first original D. In other words, when generating the output image data of the first original D, if the format of the first original D and the format of the second original D are the same, the control unit 10 anonymizes the same personal information as in the output image data of the second original D.

[0102] For example, every time the image reading unit 2 reads an original D, the control unit 10 generates format information indicating the format of the read original D. Furthermore, every time the control unit 10 generates image data for output, the control unit 10 generates anonymized information indicating anonymized personal information, and stores the format information and the anonymized information of the same original D in the storage unit 101 in association with each other.

[0103] Then, the control unit 10 determines whether the format of the first document D and the format of the second document D are the same, based on the format information of each of the first document D and the second document D. Furthermore, the control unit 10 recognizes the personal information that was anonymized when the output image data of the second document D was generated, based on the anonymization information corresponding to the second document D. As a result, the output image data of the first document D and the output image data of the second document D have the same personal information anonymized.

[0104] With this configuration, when you want to anonymize the same type of personal information in the first manuscript D as you have anonymized in the second manuscript D, you can reliably anonymize the desired personal information without having to perform any editing operations on the output image data of the first manuscript D. This improves user convenience.

[0105] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the description of the above embodiments, and further includes all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0106] 1 Printing section (output section) 2 Image reading unit 3 Operation display section 10 Control Unit 100 Multifunction printers (image processing devices) 102 Communication unit (output unit) 1000 PCs (external equipment) D Manuscript PG Preview Image S seat

Claims

1. an image reading unit that reads a document containing personal information; a control unit that performs an anonymization process on original image data obtained by reading the document by the image reading unit, thereby generating output image data in which the personal information has been anonymized; an operation display unit that displays information and accepts operations; When generating the output image data, the control unit: extracting text data by OCR processing of the original image data; extracting the personal information from the text data; Recognizing an area of ​​the original image data that includes the personal information as a target area; The anonymization process is performed by anonymizing information within the target region; After generating the output image data, the control unit: displaying a preview image corresponding to the output image data on the operation display unit; accepting an editing operation for editing the output image data on the operation display unit; When the editing operation is performed, the control unit edits the output image data based on the content of the editing operation; the operation display unit accepts a first editing operation and a second editing operation as the editing operation; the first editing operation is an operation of designating a non-anonymized area of ​​the preview image as an editing target, the second editing operation is an operation of designating an anonymized area of ​​the preview image as an editing target, When the first editing operation is performed, the control unit performs the anonymization process on the area specified by the first editing operation; When the second editing operation is performed, the control unit returns the area specified by the second editing operation to a state before the anonymization processing.

2. The image processing device according to claim 1 , wherein, when the editing operation is accepted, the operation display unit enlarges and displays an area of ​​the preview image that includes the personal information.

3. When accepting the editing operation, the operation display unit accepts a display switching operation, The image processing device according to claim 1 , wherein when the display switching operation is accepted, the operation display unit accepts the editing operation in a state where the anonymized personal information is displayed in a recognizable manner.

4. When accepting the editing operation, the operation display unit accepts a candidate display operation; When the candidate display operation is received, the operation display unit receives the editing operation in a state in which a non-extraction area including a noun character string that was not extracted as the personal information is highlighted; The image processing device according to claim 1 , wherein when the first editing operation is performed on the non-extraction area, the control unit performs the anonymization process on the non-extraction area.

5. 2. The image processing device of claim 1, wherein when the format of a first document read by the image reading unit is the same as the format of a second document read by the image reading unit before reading the first document, the control unit reflects the anonymization processing performed on the output image data of the second document to the output image data of the first document, and also reflects the editing performed on the output image data of the second document.

6. an output unit that performs output processing of the image data for output; An image processing device according to any one of claims 1 to 5, wherein the output unit is at least one of a printing unit that performs the output process by printing an image based on the output image data onto a sheet, and a communication unit that performs the output process by transmitting the output image data to an external device.

Images