program

Abstract

To allow more precise masking of a portion intended by an operator.SOLUTION: An image processing apparatus of an embodiment of the invention has: a document reading unit that performs reading of a document; a display unit that displays a read image obtained by the reading performed by the document reading unit; an input receiving unit that receives a user's input of information at an arbitrary position on the read image displayed by the display unit; an image processing unit that masks at least part of the read image based on the information received by the input receiving unit; and an image forming unit that performs image formation of the read image masked by the image processing unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention 、 Regarding the program. [Background technology]

[0002] 2. Description of the Related Art When copying using an image processing device, there is a masking printing technique in which the color of part of the output image is changed before printing.

[0003] According to a document disclosing a masking technique, a character string on a document is colored in advance using a line marker or the like, and the masking color is set from an operation panel. Then, when the document is copied, the colored part is masked with the color set from the operation panel and the copy is printed out (see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional masking printing has the problem of being time-consuming, as it requires coloring the character strings on the document in advance with a line marker or the like before reading them.

[0005] The present invention has been made in view of the above, and is capable of masking an intended portion more easily and accurately. Napu The purpose is to provide a program. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a program according to one embodiment of the present invention includes a first display control step of causing a computer to preview a first image acquired in response to a user's instruction on a display unit, and a second display control step of causing the user to move over the first image previewed on the display unit. Touch operations including vertical and horizontal tracing touch operationsand generating a second image in which at least a part of the previewed first image is masked with a line along the path traced on the first image by the touch operation. a second image masked with vertical lines when the user traces the first image vertically with a touch operation, and a second image masked with horizontal lines when the user traces the first image horizontally with a touch operation; and a transmission step of transmitting the previewed second image to an external device when an instruction from a user is received while the second image is being previewed on the display unit. [Effects of the Invention]

[0007] The present invention provides an effect of enabling an operator to more accurately mask an intended portion. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the external configuration of an MFP shown as an example of an "image processing device" according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of the MFP. [Figure 3] FIG. 3 is a diagram showing an example of the functions provided by the main body and the operation panel. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the copy output editing application. [Figure 5] FIG. 5 is a diagram illustrating an example of a plurality of APIs for different operation types provided by a WebAPI service. [Figure 6] FIG. 6 is a diagram showing an example of the specifications of each API. [Figure 7] FIG. 7 is a diagram showing an example of the configuration of request data that the WebAPI communication unit transmits to the main body. [Figure 8] FIG. 8 is a diagram showing an example of a display screen displayed on the operation panel. [Figure 9] FIG. 9 is a diagram showing an example of an overall outline flow of the MFP. [Figure 10]FIG. 10 is a diagram illustrating an example of a processing flow of the operation panel. [Figure 11A] FIG. 11A is a diagram showing an example of a user operation. [Figure 11B] FIG. 11B is a diagram showing an example of a user operation. [Figure 11C] FIG. 11C is a diagram showing an example of a user operation. [Figure 11D] FIG. 11D is a diagram showing an example of a user operation. [Figure 11E] FIG. 11E is a diagram showing an example of a user operation. [Figure 11F] FIG. 11F is a diagram showing an example of a user operation. [Figure 11G] FIG. 11G is a diagram showing an example of a user operation. [Figure 11H] FIG. 11H is a diagram showing an example of a user operation. [Figure 11I] FIG. 11I is a diagram showing an example of a user operation. [Figure 11J] FIG. 11J is a diagram showing an example of a user operation. [Figure 11K] FIG. 11K is a diagram showing an example of a user operation. [Figure 11L] FIG. 11L is a diagram showing an example of a user operation. [Figure 11M] FIG. 11M is a diagram showing an example of a user operation. [Figure 12] FIG. 12 is a diagram showing an example of an output product printed out by the main body. [Figure 13] FIG. 13 is an explanatory diagram of image synthesis. [Figure 14] FIG. 14 is a diagram showing an example of a sequence of communication processing between the operation panel and the main body. [Figure 15] FIG. 15 is a diagram showing an example of a copy / edit operation screen according to the first modification of the first embodiment. [Figure 16] FIG. 16 is a diagram showing an example of the template registration screen. [Figure 17]FIG. 17 is a diagram showing an example of the template calling screen. [Figure 18] FIG. 18 is a diagram illustrating an example of the correspondence table. [Figure 19] FIG. 19 is a diagram illustrating an example of a processing flow for registering an operation panel template. [Figure 20] FIG. 20 is a diagram showing an example of a processing flow for calling a template of an operations panel. [Figure 21] FIG. 21 is a diagram showing an example of a sequence of communication processing between the operation panel and the main body when registering a template. [Figure 22] FIG. 22 is a diagram showing an example of a sequence of communication processing between the operation panel and the main body when a template is acquired. [Figure 23] FIG. 23 is a diagram illustrating an example of a correspondence table according to the second modification of the first embodiment. [Figure 24] FIG. 24 is a diagram showing an example of a processing flow for calling a template of an operations panel. [Figure 25] FIG. 25 is a diagram showing an example of a sequence of communication processing when a template is called between the operation panel and the main body. [Figure 26] FIG. 26 is a diagram illustrating an example of a functional configuration of an operation panel according to the third modification of the first embodiment. [Figure 27] FIG. 27 is a flow diagram of an example of a masking color change process performed by the copy output edit processing unit. [Figure 28] FIG. 28 is a diagram showing an example of an output product when the set color is changed to white. [Figure 29] FIG. 29 is a diagram illustrating an example of the configuration of a copy output editing processing unit of an operation panel according to the second embodiment. [Figure 30] FIG. 30 is a diagram showing an example of a display screen of the operation panel. [Figure 31] FIG. 31 is a diagram illustrating an example of a processing flow of the operation panel. [Figure 32A] FIG. 32A is a diagram showing an example of a user operation. [Figure 32B] FIG. 32B is a diagram showing an example of a user operation. [Figure 32C] FIG. 32C is a diagram showing an example of a user operation. [Figure 33] FIG. 33 is a diagram showing an example of an output product printed out by the main body. [Figure 34] FIG. 34 is a diagram showing an example of a display screen of an operation panel according to the first modification of the second embodiment. [Figure 35] FIG. 35 is a diagram illustrating an example of a processing flow of the operation panel. [Figure 36A] FIG. 36A is a diagram showing an example of a user operation. [Figure 36B] FIG. 36B is a diagram showing an example of a user operation. [Figure 36C] FIG. 36C is a diagram showing an example of a user operation. [Figure 36D] FIG. 36D is a diagram showing an example of a user operation. [Figure 37] FIG. 37 is a diagram showing an example of a display screen of an operation panel according to the second modification of the second embodiment. [Figure 38] FIG. 38 is a diagram illustrating an example of a processing flow of the operation panel. [Figure 39A] FIG. 39A is a diagram showing an example of a user operation. [Figure 39B] FIG. 39B is a diagram showing an example of a user operation. [Figure 39C] FIG. 39C is a diagram showing an example of a user operation. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the present invention will be described with reference to the accompanying drawings. Rup An embodiment of the program will now be described in detail.

[0010] The image processing apparatus according to the embodiment includes a document reading unit, a display unit, an input receiving unit, an image processing unit, and an image forming unit.

[0011] The document reading unit includes a reading unit such as a scanner and reads a document image from the document (i.e., performs scanning). The display unit includes a display unit such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display and displays the scanned image (a scanned image having pixel data, also called a scanned image) read by the document reading unit. The input receiving unit includes an input unit such as a touch panel and receives information input at any position specified by the user on the scanned image displayed on the display unit. The information is masking information (masking pattern) including at least one of position information such as coordinate information (x, y) for the scanned image and color information such as RGB values ​​for the specified coordinates (position). The masking information is generated by a generating unit. As described below, the masking information (masking pattern) can be stored and called as an independent file separate from the scanned image. Note that the color information also includes information for changing a color other than the color of the specified position (e.g., information for changing to white to remove yellowing) and information for changing the color of the specified position to white or a color similar to the surroundings or background (e.g., information for removing pencil markings, etc.). The image processing unit performs image processing to mask the read image based on the masking information at the position received by the input receiving unit. The image forming unit includes a plotter or the like and forms (prints, etc.) the read image masked by the image processing unit.

[0012] The image processing unit performs masking processing, for example, by changing the RGB values ​​of a scanned image acquired in RGB format for the portion to be masked, based on masking information that specifies a pixel (called a "reference pixel") at a position specified by a user in the scanned image being displayed and pixels equivalent thereto as the masking target or as the non-masking target.Specific methods for masking the portion to be masked include, for example, a method in which the user traces a masking pattern on the scanned image being displayed and combines the masking pattern with the scanned image, a method in which pixels in the scanned image that have pixel values ​​equal to or within an approximate range of the pixel value of the position specified by the user in the scanned image being displayed are replaced with the pixel value of the masking, and variations of each of these methods.

[0013] Here, a pixel is the smallest unit that contains color information (tone and gradation) when handling images on a computer. The pixel value is a concept that includes RGB values, brightness information, and color information.

[0014] The following describes the embodiments of each method in detail. Note that the following embodiments and their modifications may be combined as appropriate.

[0015] (First embodiment) 1 is a diagram showing the external configuration of an MFP (Multifunction Peripheral) shown as an example of an "image processing device" according to the first embodiment. The MFP is a device having at least a scanning function and an image forming function.

[0016] 1 has a main body (main body part) 10 and an operation panel 20 as an "operation device." The main body 10 has a scanner unit 10-1 that scans (also called "reads") a document, an image forming unit 10-2 that forms an image on a recording medium such as paper, and the like.

[0017] The operation panel 20 is a user interface that allows the user to instruct the main body 10 to execute a copy and to perform editing operations on the image to be copied and output. The main body 10 and the operation panel 20 are connected for communication via a wired or wireless connection, and transmit and receive operation signals, data, and the like via the communication path 30 (see FIG. 2).

[0018] In the first embodiment, an example is shown in which user operations such as issuing a copy instruction to the main body 10 on the operation panel 20 and editing the image to be copied and output are realized by using an API (Application Programming Interface) provided in the main body 10. Note that as the "user operation for image editing," a masking operation for masking a part of an image is shown.

[0019] Additionally, "API" refers to the use of WebAPI as an example. Unless otherwise specified below, "API" refers to "WebAPI (mainly REST WebAPI)." A REST (Representational State Transfer) WebAPI sends requests (also called "requests" or "request data") using GET (retrieval), POST (registration), PUT (update), and DELETE (deletion) methods and a URL (Uniform Resource Locator) that identifies a resource to a Web (World Wide Web) server, and receives responses (also called "responses" or "response data") from the Web server in JSON (JavaScript (registered trademark) Object Notation), XML (Extensible Markup Language), or similar.

[0020] (Hardware configuration) Fig. 2 is a diagram showing an example of the hardware configuration of MFP 1. As shown in Fig. 2, main body 10 has CPU (Central Processing Unit) 100, which is a first CPU, ROM (Read Only Memory) 101, RAM (Random Access Memory) 102, HDD (Hard Disk Drive) 103, communication I / F 104, connection I / F 105, and engine 106. Each part is connected to each other via system bus 107.

[0021] The CPU 100 is a central processing unit that executes arithmetic processing and control processing. The CPU 100 reads various programs stored in the ROM 101, HDD 103, etc. into the RAM 102, and executes the arithmetic processing and control processing, thereby controlling the main body 10 in an overall manner.

[0022] The ROM 101 is a non-volatile memory that stores fixed programs (e.g., BIOS (Basic Input / Output System)). The RAM 102 is a volatile memory that the CPU 100 uses as a work area for various processes. The HDD 103 is an auxiliary storage device that stores various programs and data. The various programs include a first OS (Operating System), web server software, web API, database management software, and the like.

[0023] The communication I / F 104 is an interface (for example, an Ethernet (registered trademark) board) for connecting the main body 10 to a communication network such as a LAN (Local Area Network) or a VPN (Virtual Private Network).

[0024] The connection I / F 105 is an interface conforming to the USB (Universal Serial Bus) standard or the like, which establishes a communication connection with the operation panel 20 via a communication path 30. Note that the connection with the operation panel 20 is not limited to a USB. Furthermore, the communication path 30 is not limited to a wired one, but may be wireless.

[0025] The engine 106 may be a reading scanner that scans an original document, or a plotter that forms an image on a recording medium such as paper. The reading scanner is provided in the scanner unit 10-1 (see FIG. 1) and outputs scanning data of RGB (Red, Green, Blue) values ​​by scanning. The plotter is provided in the image forming unit 10-2 (see FIG. 1).

[0026] 2 includes a CPU 200, which is a second CPU provided separately from the main body 10, a ROM 201, a RAM 202, a flash memory 203, a communication I / F 204, a connection I / F 205, a touch panel 206, and an LCD 207. Each component is interconnected via a system bus 208.

[0027] The CPU 200 is a central processing unit that executes arithmetic processing and control processing. The CPU 200 reads various programs stored in the ROM 201, flash memory 203, etc. into the RAM 202, and executes arithmetic processing and control processing, thereby controlling the entire operation panel 20 in an integrated manner.

[0028] The ROM 201 is a non-volatile memory that stores fixed programs (e.g., BIOS). The RAM 202 is a volatile memory that the CPU 200 uses as a work area for various processes. The flash memory 203 is an auxiliary storage device that stores a second OS (such as the Android (registered trademark) OS) separate from the main body 10, and apps (such as the Android (registered trademark) apps) that run under the "second OS" by the CPU 200. The OS of the operation panel 20 runs independently of the OS of the main body 10. A copy output editing app, which will be described later, is stored in the flash memory 203 as an app that runs under the OS of the operation panel 20. In this specification, "app" refers to an "application."

[0029] The communication I / F 204 is an interface for connecting the operation panel 20 to a communication network such as a LAN or VPN.

[0030] The connection I / F 205 is an interface for establishing a communication connection with the main body 10 via a communication path 30 .

[0031] The touch panel 206 is a touch sensor that detects the position touched by the operator on the screen of the LCD 207 and notifies the CPU 200 of information about the touch position (position information). In addition to the touch panel 206, hardware keys that notify the CPU 200 of input values, commands, etc. may also be provided.

[0032] The LCD 207 is an example of a display means. The LCD 207 has a liquid crystal display screen, and displays on the liquid crystal display screen the display screen information output by the CPU 200. Note that the display means may use other display methods, such as organic EL, in addition to liquid crystal.

[0033] (Functional configuration) In the first embodiment, the main body 10 and the operation panel 20 of the MFP 1 operate independently under separate operating systems (a first OS and a second OS, respectively). The copy output editing application of the operation panel 20 operates the main body 10 using the WebAPI of the main body 10 under the operating system of the operation panel 20.

[0034] The main body 10 performs various functions by having the CPU 100 load programs stored in the ROM 101, HDD 103, etc. into the RAM 102 and execute them sequentially. The operation panel 20 performs various functions by having the CPU 200 load programs stored in the ROM 201, flash memory 203, etc. into the RAM 202 and execute them sequentially.

[0035] Fig. 3 is a diagram showing an example of the functions performed by the main body 10 and the operation panel 20. Fig. 3 mainly shows a function for instructing the main body 10 to execute a copy from the operation panel 20 and a function for enabling a masking operation.

[0036] As shown in FIG. 3, the main body 10 performs functions such as a connection control unit 11, a communication control unit 12, a storage control unit 13, a WebAPI service 15, a reading scanner control unit (scanner control unit) 16, and a plotter control unit 17.

[0037] The connection control unit 11 controls the connection I / F 105 to establish communication with the operation panel 20 and transmits and receives data to and from the operation panel 20 .

[0038] The communication control unit 12 controls the communication I / F 104 to establish communication with a communication destination and transmit and receive data to and from the communication destination.

[0039] The storage control unit 13 accesses storage areas such as the ROM 101, RAM 102, and HDD 103 to read and write data. A first temporary storage area is allocated in part of the storage area of ​​the RAM 102 to temporarily store a scanned image (read image). Here, the "scanned image" is scanning data output by the reading scanner and saved in an image format such as TIFF (Tagged Image File Format) or JPEG.

[0040] WebAPI service 15 provides various APIs, including an API for receiving a copy execution instruction from operation panel 20 and an API for providing operation panel 20 with a masking operation for an image to be copied. WebAPI service 15 is implemented as a web server on main body 10 of MFP1.

[0041] The Web server communicates with the operation panel 20 and external devices using HTTP (Hypertext Transfer Protocol). In this communication, the Web server processes request data included in the received data and responds with the processing results to the request source.

[0042] The reading scanner control unit 16 controls the reading scanner of the engine 106 according to settings specified by the user. The plotter control unit 17 controls the plotter of the engine 106 according to settings specified by the user.

[0043] The operation panel 20 performs functions such as a connection control unit 21, a communication control unit 22, a memory control unit 23, an input reception unit 24, a display control unit 25, a copy output editing processing unit (copy output editing application, masking application) 26, and a WebAPI communication unit 27.

[0044] The connection control unit 21 controls the connection I / F 205 to establish communication with the main body 10 and transmits and receives data to and from the main body 10. For example, the connection control unit 21 transmits an operation signal to the connection control unit 11 of the main body 10 to instruct the execution of a copy. The connection control unit 21 also transmits and receives information about masking to and from the connection control unit 11 of the main body 10.

[0045] The communication control unit 22 controls the communication I / F 204 to establish communication with a communication destination and transmit and receive data to and from the communication destination.

[0046] The storage control unit 23 accesses storage areas such as the ROM 201, RAM 202, and flash memory 203 to read and write data. A second temporary storage area for temporarily storing a scanned image and a third temporary storage area for temporarily storing a masking pattern are secured in part of the storage area of ​​the RAM 202.

[0047] The input receiving unit 24 receives information on the touch position from the touch panel 206, thereby receiving input of an operation instruction.

[0048] The display control unit 25 outputs display screen information to the LCD 207 .

[0049] The copy output editing processor 26 performs editing processing for copy output in an interactive manner with the operator of the operation panel 20. When outputting copies, the copy output editing processor 26 instructs the display controller 25 to update the display of screen information, receives input information from the operator from the input receiver 24, instructs the WebAPI communication unit 27 to transmit request data to the main unit 10, and receives response data from the main unit 10 from the WebAPI communication unit 27.

[0050] The WebAPI communication unit 27 performs HTTP communication with the main body 10. Specifically, the WebAPI communication unit 27 uses an HTTP client to send a request based on a user input received by the copy output editing processing unit 26 via a UI (User Interface) of the operation panel 20 to the Web server of the main body 10. At that time, the WebAPI communication unit 27 transmits information for calling an API (function) to the WebAPI service 15 of the main body 10 in order to convey an execution instruction from an execution instruction unit 261 (described later) to the main body 10. In the HTTP communication, the WebAPI communication unit 27 appropriately specifies the API of the WebAPI service 15 and receives the execution processing result by that API.

[0051] 4 is a diagram showing an example of the configuration of the copy output editing application. As shown in FIG. 4, the copy output editing application has an execution instruction unit 261, a display unit 262, an input acceptance unit 263, a generation unit 264, and an image processing unit 265.

[0052] The execution instruction unit 261 instructs the main body 10 to perform scanning, printing, transmission, storage, etc. via the WebAPI communication unit 27. Regarding scanning, the execution instruction unit 261 instructs the main body 10 to perform processing to acquire a scanned image in RGB format on the operation panel 20.

[0053] The display unit 262 displays a preview of the scanned image.

[0054] The input receiving unit 263 receives input of information from the user, including the position (coordinates) and color (RGB value).

[0055] The generating unit 264 generates masking information based on the information received by the input receiving unit 263 .

[0056] The image processing unit 265 performs composition by masking the read image by changing the RGB values ​​of the read image based on the masking information.

[0057] The WebAPI communication unit 27 may be provided as a copy output editing application, rather than as a separate application such as a system application included in the service layer of the operation panel 20.

[0058] In the functional configuration of the first embodiment, the reading scanner control unit 16 corresponds to the "document reading unit." The plotter control unit 17 corresponds to the "image forming unit." The input accepting unit 24 and the input accepting unit 263 correspond to the "input accepting unit." The display control unit 25 and the display unit 262 correspond to the "display unit." The image processing unit 265 of the copy output editing processing unit 26 corresponds to the "image processing unit." The generation unit 264 corresponds to the "generation unit." The WebAPI service 15 and the WebAPI communication unit 27 correspond to the "API communication unit," such as the "reading API communication unit," the "image forming API communication unit," the "image reading API communication unit," the "image acquisition API communication unit," the "storage API communication unit," and the "acquisition API communication unit."

[0059] Fig. 5 is a diagram showing an example of a plurality of APIs for different operation types provided by the WebAPI service 15. As shown in Fig. 5, the WebAPI service 15 provides a scan execution API 15-1 (corresponding to an "image reading API communication unit" and "first API"), a scanned image acquisition API 15-2 (corresponding to an "image acquisition API communication unit" and "second API"), an image formation API 15-3 (corresponding to an "image formation API communication unit" and "third API"), a file registration API 15-4 (corresponding to a "storage API communication unit" and "fourth API"), a file acquisition API 15-5 (corresponding to an "acquisition API communication unit" and "fifth API"), etc.

[0060] Fig. 6 is a diagram showing an example of the specifications of each API. Fig. 6 shows, for each type of API, the services provided by the main unit 10, the specified path (URL) of the API, and the method. For example, for the first API 15-1, by sending a request specifying the POST method and the path " / ricoh / scan / job" to the main unit 10 from the operation panel 20, scanning of a document is performed by the main unit 10.

[0061] (Request data configuration) 7 shows the structure of request data sent from the WebAPI communication unit 27 to the main body 10. The response from the main body 10 to the request is in JSON (JavaScript (registered trademark) Data in Object Notation format.

[0062] Fig. 7(a) is a diagram showing an example of the configuration of request data when the WebAPI communication unit 27 instructs the main body 10 to execute scanning. As shown in Fig. 7(a), a POST request is made by specifying the URL of the first API 15-1 (see Fig. 5).

[0063] Fig. 7(b) is a diagram showing an example of the configuration of request data when the WebAPI communication unit 27 instructs the main body 10 to acquire a scanned image. As shown in Fig. 7(b), a GET request is made by specifying the URL of the second API 15-2 (see Fig. 5).

[0064] Fig. 7(c) is a diagram showing an example of the configuration of request data when the WebAPI communication unit 27 instructs the main body 10 to execute image formation. As shown in Fig. 7(c), a POST request is made by specifying the URL of the third API 15-3 (see Fig. 5).

[0065] Fig. 7(d) is a diagram showing an example of the configuration of request data when the WebAPI communication unit 27 registers a masking pattern in the main body 10. As shown in Fig. 7(d), a POST request is made by specifying the URL of the fourth API 15-4 (see Fig. 5).

[0066] Fig. 7(e) is a diagram showing an example of the configuration of request data when the WebAPI communication unit 27 calls a masking pattern from the main body 10. As shown in Fig. 7(e), a GET request is made by specifying the URL of the fifth API 15-5 (see Fig. 5).

[0067] (Screen configuration of the screen displayed on the operation panel 20) Fig. 8 is a diagram showing an example of a display screen displayed on operation panel 20. Fig. 8 shows a copy editing operation screen 1100. Copy editing operation screen 1100 has a scanning operation button display area 1110, a masking operation button display area 1120, an image formation operation button display area 1130, an image display area 1140, etc.

[0068] A scanning operation button display area 1110, a masking operation button display area 1120, and an image formation operation button display area 1130 are provided with various operation buttons.

[0069] The scanning operation button display area 1110 includes, as one of the operation buttons, a scanning execution button 1111. The scanning execution button 1111 is an instruction button for instructing the main body 10 to execute scanning.

[0070] The masking operation button display area 1120 includes a zoom button 1121, a drawing object selection button 1122, a line correction button 1123, and the like. The zoom button 1121 is an instruction button for gradually enlarging or reducing the scanned image (preview image) displayed in the image display area 1140. The drawing object selection button 1122 is a selection button for selecting from multiple drawing objects, the thickness of which is to be specified, using a pull-down menu. The line correction button 1123 is a handwriting assistance button for correcting a path traced on the screen with a finger or the like (or a touch pen or the like) into a predetermined line. The predetermined line may be, for example, a horizontal line, a vertical line, a curved line, or the like.

[0071] The image formation operation button display area 1130 includes an image formation execution button 1131. The image formation execution button 1131 is an instruction button for instructing the main body 10 to execute image formation. In addition, a setting button for setting image formation conditions may be included. The setting button displays a setting screen for image formation conditions by touch operation, and accepts the setting of image formation conditions on the setting screen. The set conditions are sent to the main body 10.

[0072] The display area for other operation buttons includes a home button 1151 for returning to the home screen, a stop button 1152 for closing the screen, and a clear button 1153 for erasing the scanned image.

[0073] The image display area 1140 is provided with an entire display area 1141 and a zoom display area 1142. The entire display area 1141 is an image display area for displaying the entire preview image after scanning. The zoom display area 1142 is an image display area for displaying an enlarged image or a reduced image. Images are displayed in each area using an image viewer or the like.

[0074] (Overall overview flow) Next, the overall outline flow of the MFP 1 will be explained. 9 is a diagram showing an example of an overall outline flow of the MFP 1. First, the MFP 1 starts up a copy output editing application (masking application) on the home screen of the operation panel 20 (SA1).

[0075] Next, the MFP 1 instructs to execute a scan and obtains a scanned image in RGB format (SA2).

[0076] Next, the MFP 1 displays a preview of the acquired scanned image (SA3).

[0077] Next, the MFP 1 receives input of information for the preview (SA4), and generates masking information based on the received information (SA5).

[0078] Next, the MFP 1 performs image processing on the read image based on the masking information (SA6).

[0079] Then, the MFP 1 outputs the scanned image after image processing by printing or saving it as a file (SA7).

[0080] The MFP 1 can also call up masking information in the preview display in step SA3. If masking information is called up (SA8), the MFP 1 skips steps SA4 and SA5 and performs image processing on the scanned image in step SA6 based on the masking information called up in step SA8.

[0081] The MFP 1 can also register the masking information generated in step SA5. When registration is instructed in step SA5, the MFP 1 performs a process (SA6) of performing image processing on the scanned image based on the generated masking information, and a process (SA9) of registering the masking information. After registration, the masking information can be called up and used again in step SA8.

[0082] (Detailed flow) Next, we will explain the detailed flow of the MFP 1 when a copy operation and a masking operation of an image to be copied and output are performed on the operation panel 20. First, we will explain the processing of the operation panel 20 based on the processing flow diagram shown in Fig. 10. In addition, in explaining this processing, we will explain the user's operation procedure and the screen flow of the operation panel 20 with appropriate reference to Figs. 11A to 11M.

[0083] (Operation of operation panel 20) 10 is a diagram showing an example of a processing flow of operation panel 20 from the start of the copy output editing application to the copy output of a document on main body 10. It is assumed that the OS of operation panel 20 has started up and communication connection between operation panel 20 and main body 10 via communication path 30 has been completed.

[0084] First, when the icon 1001 of the copy output editing application on the home screen 1000 (see FIG. 11A) is touched, the copy output editing processing unit 26 is started (S1). Specifically, a program is loaded into the RAM 202, and the copy output editing processing unit 26 is started. Although not shown in the flow chart, the program is safely ended by appropriately accepting a touch operation of the stop button 1152 (see FIG. 8).

[0085] After startup, copy output editing processing unit 26 instructs display control unit 25 to display copy editing operation screen 1100 (see FIG. 8), and waits while being able to accept input from input accepting unit 24 (S2). At this stage, copy editing operation screen 1100 sets only scanning operation button display area 1110 on, so that it accepts input from buttons included in that area, and sets the rest of the area off, so that it does not accept input from buttons (see FIG. 11B).

[0086] The user places a document on the main body 10 (see FIG. 11C), and then touches the scanning execution button 1111 (see FIG. 11D).

[0087] After step S2, the copy output editing processor 26 determines whether a scanned image has not yet been acquired (S3). For example, the copy output editing processor 26 provides a flag area in the RAM 202, and manages whether a scanned image has been acquired or not using the flag area. When a scanned image has been acquired, the flag area is set to "1," and when the default or cleared, the flag area is set to "0." The first time, a scanned image has not yet been acquired.

[0088] If the scanned image has not been acquired (S3: Yes), the copy output editing processing unit 26 determines whether the touched button is the scanning execution button 1111 based on the input information received by the input receiving unit 24 (S4).

[0089] If the scanning execution button 1111 is not touched (S4: No), the copy output editing processor 26 waits until the scanning execution button 1111 is touched.

[0090] When the scanning execution button 1111 is touched (S4: Yes), the copy output editing processing unit 26 instructs the main body 10 to execute scanning (S5).

[0091] After issuing the execution instruction, copy output edit processing unit 26 instructs display control unit 25 to display a wait screen indicating that scanning is in progress (S6). As a result, the screen of operation panel 20 changes to wait screen 1200 (see FIG. 11E) indicating that scanning is in progress.

[0092] After step S6, when the scanned image is sent from the main body 10 as the execution result, the copy output editing processing unit 26 acquires it (S7), and temporarily stores the scanned image in the second temporary storage area.

[0093] Then, the copy output editing processing unit 26 instructs the display control unit 25 to display a screen with the display of the scanned image set to on on the copy editing operation screen 1100 (see Figure 8), and waits while allowing input from the input accepting unit 24 to be accepted (S8).

[0094] Specifically, the copy output editing processor 26 displays the acquired scanned image (preview image) in the entire display area 1141 (see FIG. 8) and the enlarged / reduced display area 1142 (see FIG. 8) of the copy editing operation screen 1100. At this stage, all areas other than the scanning operation button display area 1110 are also set to ON so that input from the buttons in each area is accepted. As a result, the screen of the operation panel 20 returns to the copy editing operation screen 1100 (see FIG. 11F), and the acquired scanned image is displayed as the entire image X1 in the entire display area 1141, and the acquired scanned image is displayed as the enlarged / reduced image X2 in the enlarged / reduced display area 1142.

[0095] The user enlarges the scanned image displayed in the zoom display area 1142 using the zoom button 1121 (see Figure 11G) and masks personal information and the like contained in the scanned image (see Figures 11H, 11I, and 11J).

[0096] After step S8, the copy output edit processing unit 26 determines which button on the copy edit operation screen 1100 has been touched, and operates as follows according to the touched button.

[0097] When copy output edit processing unit 26 determines that zoom button 1121 has been touched (S9: Yes), it instructs display control unit 25 to gradually enlarge or reduce the display of the scanned image in enlargement / reduction display area 1142 in accordance with the type (enlargement or reduction) of zoom button 1121 and the number of touch operations (S10). As a result, the scanned image is enlarged or reduced in enlargement / reduction display area 1142 of operation panel 20 (see FIG. 11G).

[0098] After step S10, the process of the copy output edit processing unit 26 proceeds to step S11.

[0099] Furthermore, when the copy output edit processing unit 26 determines that the drawing object selection button 1122 has been touched (S11: Yes determination), it accepts a drawing input operation for masking in the enlargement / reduction display area 1142 and performs the drawing process (S12).

[0100] Specifically, when one of the multiple drawing objects 1122a displayed in a pull-down menu on the drawing object selection button 1122 (see FIG. 11H) is selected by a touch operation (for example, when the sixth-highest black level with a thickness of "6" is selected out of six levels of black), the drawing function (an example of a "generator") is activated, and the scaling display area 1142 (see FIG. 11I) becomes capable of accepting a drawing input operation for masking. When the user traces the scanned image displayed in a scaled manner in the scaling display area 1142 with a finger or the like, the coordinates of any touched position (an example of "position information") and the direction of movement are detected, and a line with the width of the selected drawing object (black with a thickness of "6") is written and stored along the trajectory in a third temporary storage area. Note that when a line correction button 1123 for a horizontal line, vertical line, curved line, or the like is selected, the line corrected according to the setting is stored.

[0101] If multiple locations on a single scanned image need to be masked at the same time, the user traces each of the multiple locations with a finger or the like, and the information about each traced path is written to the third temporary storage area. In the zoom display area 1142, the traced path with the finger or the like is corrected as needed to a horizontal line, vertical line, curved line, or the like, and displayed as a line (black line with a thickness of "6") L1 (see FIG. 11I). Furthermore, if a different location is traced with a finger or the like, the further traced location is displayed as a line (black line with a thickness of "6") L2 (see FIG. 11J).

[0102] After step S12, the process of the copy output edit processing unit 26 proceeds to step S13.

[0103] The user touches the image formation execution button 1131 (see FIG. 11K) to instruct the output of a copy of the masked document.

[0104] When the copy output editing processing unit 26 determines that the image formation execution button 1131 has been touched (S13: Yes judgment), it composites the masking pattern onto the scanned image (S14) and instructs the main body 10 to perform image formation using the composite image as the target (S15).

[0105] After issuing the execution instruction, copy output editing processing unit 26 instructs display control unit 25 to display a wait screen indicating that image formation is in progress (S16). As a result, the screen of operation panel 20 changes to wait screen 1300 (see FIG. 11L) indicating that image formation is in progress. In main body 10, the masked image is formed on a recording medium such as paper and printed out.

[0106] Thereafter, upon receiving a notification of completion of image formation from main body 10 (S17), copy output edit processing unit 26 instructs display control unit 25 to return the display to the original state (S18). As a result, the screen of operation panel 20 returns to copy edit operation screen 1100 (see FIG. 11M). After step S18, the process of copy output edit processing unit 26 proceeds to step S19.

[0107] Furthermore, when copy output editing processor 26 determines that clear button 1153 has been touched (S19: Yes judgment), it erases the scanned image from the second temporary storage area, and repeats the processing from step S3 (S20). At this time, copy output editing processor 26 sets the flag area of ​​RAM 202 to "0", and further returns the display of copy editing operation screen 1100 to the initial setting, and then repeats the processing from step S3.

[0108] Fig. 12 is a diagram showing an example of an output product printed out by the main body 10. As shown in Fig. 12, the output product P has masking P1 (corresponding to line L1) and masking P2 (corresponding to line L2) applied to a copy of the original.

[0109] 13 is an explanatory diagram of image composition. When the image formation execution button 1131 is touched, the copy output edit processing unit 26 composes a masking pattern on the scanned image by the image processing unit.

[0110] The image processing unit may be one provided by, for example, OpenCV (Open Source Computer Vision Library). In the first embodiment, the scanned image is stored in the main unit 10 in an RGB color mode or in an image format such as TIFF or JPEG, and the scanned image is acquired from the main unit 10 to the operation panel 20 in these modes or image formats. The image processing unit replaces the values ​​(RGB values) of each pixel specified in the masking information (masking pattern) at coordinates on the scanned image with black (0,0,0). That is, as image processing, the color information (RGB values) of the pixel of the scanned image at the position specified by the user on the displayed scanned image preview is converted to the specified color information (e.g., black). This processing allows the masking information to be superimposed or combined on the scanned image.

[0111] (Communication sequence) Next, the communication processing between the operation panel 20 and the main body 10 will be explained. In the first embodiment, the operation panel 20 communicates with the main body 10 and executes copy output on the main body 10 by means of an API provided by the main body 10 (i.e., using the API), and performs masking processing on an image to be copied and output. Here, the communication processing between the operation panel 20 and the main body 10 will be explained, focusing mainly on functions related to API processing, and illustrations and explanations of other functional units will be omitted as appropriate.

[0112] 14 is a diagram showing an example of a sequence of communication processing between the operation panel 20 and the main body 10. First, when a touch operation is performed on the scanning execution button 1111 on the copy editing operation screen 1100 (see FIG. 11B) on the operation panel 20, the copy output editing processing unit 26 instructs the WebAPI communication unit 27 to execute scanning (S101), and the WebAPI communication unit 27 transmits request data (see FIG. 7(a)) requesting execution of the first API 15-1 to the main body 10 (S102).

[0113] As a result, in the main body 10, the WebAPI service 15 accepts the designation of the first API 15-1 and instructs the reading scanner control unit 16 to execute a scan, which is an operation request (S104). The reading scanner control unit 16 controls the reading scanner of the engine 106 to execute scanning (S105). After executing scanning, the reading scanner control unit 16 instructs the storage control unit 13 to save the image (scanned image) read by the reading scanner (S106). The storage control unit 13 manages the scanned image read by the reading scanner using a job ID and writes the scanned image to a first temporary storage area (S107). The WebAPI service 15 receives the job ID from the reading scanner control unit 16 as a result of the scanning execution and transmits the job ID to the WebAPI communication unit 27 as response information.

[0114] Next, when the copy output editing processing unit 26 is notified of the job ID from the WebAPI communication unit 27, it instructs the WebAPI communication unit 27 to acquire the scanned image (S108), and the WebAPI communication unit 27 transmits request data (see FIG. 7(b)) requesting execution of the second API 15-2 to the main body 10 (S109). Note that the notified job ID is set in {job ID} of the request data (see FIG. 7(b)).

[0115] As a result, in the main body 10, the WebAPI service 15 accepts the designation of the second API 15-2 and instructs the storage control unit 13 to acquire a scanned image, which is an operation request (S111). In response to this instruction, the storage control unit 13 reads out the scanned image corresponding to the job ID from the first temporary storage area (S112) and transfers the read scanned image to the WebAPI service 15 (S113). The WebAPI service 15 transmits the scanned image to the WebAPI communication unit 27 (S115). The WebAPI communication unit 27 notifies the copy output editing processing unit 26 that the scanned image has been acquired (S116).

[0116] When the scanned image is acquired, the copy output editing processor 26 accepts an editing operation by the user to apply masking to the scanned image, and creates a masking pattern based on the editing operation (S117).

[0117] Then, when the image formation execution button 1131 on the copy editing operation screen 1100 (see FIG. 11K) is touched, the copy output editing processing unit 26 performs a composition process of the masking pattern (S118). Then, the copy output editing processing unit 26 instructs the WebAPI communication unit 27 to perform image formation (S119), and the WebAPI communication unit 27 transmits request data (see FIG. 7(c)) requesting execution of the third API 15-3 to the main unit 10 (S120). When transmitting the request data, the WebAPI communication unit 27 transfers composite data of the image (also referred to as image composite data or output image) to the main unit 10.

[0118] As a result, in the main body 10, the WebAPI service 15 accepts the specification of the third API 15-3, and first instructs the memory control unit 13 to write the image composite data to the first temporary memory area (S122), and the memory control unit 13 writes the image composite data to the first temporary memory area (S123).

[0119] Next, the WebAPI service 15 instructs the plotter control unit 17 to execute image formation, which is an operation request of the third API 15-3 (S124). The plotter control unit 17 reads the image composite data from the first temporary storage area using the storage control unit 13 (S125), and executes image formation processing based on the image composite data (S126). In this image formation processing, the plotter control unit 17 converts the image composite data from RGB to CMYK data, and forms an image by transferring the dot pattern of the image onto the recording medium, etc.

[0120] When the formation of the image is completed, the plotter control unit 17 notifies the WebAPI service 15 of the completion (S127). Then, the WebAPI service 15 transmits response information indicating the completion notification to the WebAPI communication unit 27 (S129), and the WebAPI communication unit 27 notifies the copy output editing processing unit 26 of the completion of the image formation process (S130).

[0121] In the first embodiment, a scanned image of RGB values ​​is acquired from the main body 10, and an arbitrary position on the scanned image is designated on the operation panel 20 to perform masking. This allows the operator to mask the desired location on the scanned image. Note that in the first embodiment, functions such as a "generation unit" and an "image synthesis unit" are realized on the operation panel 20 by the copy output editing application of the operation panel 20, but it is sufficient that the function realized on the operation panel 20 is at least the "generation unit," and other functions may also be realized by the main body 10.

[0122] (Modification 1 of the first embodiment) Next, an example will be shown in which the masking pattern obtained when masking is performed is modified so that it can be used in subsequent copy output. In Modification 1 of the first embodiment, a masking pattern storage area (an example of a "storage unit") is provided in HDD 103 of main body 10 as a storage area for masking patterns, and operation panel 20 is designed as follows.

[0123] (Screen configuration of operation panel 20) First, we will explain a modified example of the screen configuration of the operation panel 20. Fig. 15 is a diagram showing an example of a copy / edit operation screen according to Modification 1 of the first embodiment. The copy / edit operation screen 2100 shown in Fig. 15 is the copy / edit operation screen 1100 (see Fig. 8) with a template registration button 1133 and a template call button 1124 added.

[0124] The template registration button 1133 is a button for registering the masking pattern written in the third temporary storage area as a template in the main body 10. In this example, the template registration button 1133 calls up a template registration screen (see FIG. 16), and the template registration operation is performed on the template registration screen.

[0125] The template call button 1124 is a button for retrieving a masking pattern registered as a template from the masking pattern storage area of ​​the main body 10. In this example, the template call button 1124 calls a template call screen (see FIG. 17), and the template retrieval operation is performed on the template call screen.

[0126] Fig. 16 is a diagram showing an example of a template registration screen. A template registration screen 2200 shown in Fig. 16 is provided with a template name input field 2201, a masking number input field 2202, a register button 2203, a close button 2204, and the like.

[0127] The template name input field 2201 accepts input of the template name of the masking pattern being processed. The masking number input field 2202 accepts input of the number of masked locations (such as "1 (location)" or "2 (locations)"). The registration button 2203 accepts registration of the masking pattern being processed. The close button 2204 accepts termination of the template registration screen 2200.

[0128] Fig. 17 is a diagram showing an example of a template call screen. A template call screen 3200 shown in Fig. 17 is provided with a template list selection box 3201, a call button 3202, a close button 3203, and the like.

[0129] The template list selection box 3201 displays a list of templates set in the correspondence table (see FIG. 18), here showing management information (serial number, template name, number of maskings, etc.) as an example. If the number of registered templates does not allow for a list display, the entire list can be displayed by scrolling or switching using a scroll bar or display switching button. The template list selection box 3201 also accepts the selection of each template from the list. For example, when the user touches a template from the list, that template is selected. In FIG. 17, an inverted state is shown by a shaded portion 3204 as an example of a selected state.

[0130] The call button 3202 accepts the call of the masking pattern of the selected template. The close button 2203 accepts the termination of the template call screen 3200.

[0131] (Operation panel 20 table) Fig. 18 is a diagram showing an example of a correspondence table stored in flash memory 203 of operation panel 20. Fig. 18 shows template information table T1 as the correspondence table. Template information table T1 is an information table for associating masking patterns of main body 10 with template management information (an example of "bibliographic information").

[0132] In this example, the template information table T1 associates the item "management information" t2 (such as "document ID" t2-1, "template name" t2-2, and "number of maskings" t2-3) with the item "masking pattern file name" t1. The file name of the masking pattern instructed to be registered as a template is set in the item "masking pattern file name" t1. The ID assigned by the main unit 10 when registering the masking pattern is set in the item "document ID" t2-1. The template name of the masking pattern is set in the item "template name" t2-2. The number of masking locations in the masking pattern is set in the item "number of maskings" t2-3.

[0133] (Operation of operation panel 20) Next, the operation of the operation panel 20 when registering and calling up a template will be explained. In this operation, the copy output editing processor 26 also functions as a "registration unit," "selection unit," "acquisition unit," etc. In addition, this operation is based on the processing flow of FIG. 10, and only variations will be explained here. Other overlapping explanations and illustrations will be omitted as appropriate.

[0134] 19 is a diagram showing an example of a processing flow for registering a template on operation panel 20 in Modification 1 of the first embodiment. After the drawing process in step S12, if copy output edit processing unit 26 determines that the template registration button 1133 has been touched (S31: Yes determination), it instructs display control unit 25 to display template registration screen 2200 (see FIG. 16), and waits while ready to accept input from input accepting unit 24 (S32).

[0135] As a result, the template registration screen 2200 is displayed in an active state on the copy edit operation screen 1100 (see FIG. 11J) of the operation panel 20. On the template registration screen 2200, the user inputs template information (template name, number of maskings, etc.) for the masking pattern being processed into the respective input fields, and touches the register button 2203. Note that the template information is input using a software keyboard or a hardware keyboard.

[0136] The copy output edit processing unit 26 accepts input of template information (S33), and upon accepting a touch operation on the registration button 2203 (S34), registers the template information entered on the template registration screen 2200 and the file name of the masking pattern being processed in association with each other in the template information table T1 (see FIG. 18) via the storage control unit 23 (S35).

[0137] Then, the copy output edit processing unit 26 instructs the main unit 10 to register the masking pattern being processed (S36).

[0138] Thereafter, when the main body 10 notifies the copy output edit processing unit 26 of the "document ID" assigned to the masking pattern as a registration result, the copy output edit processing unit 26 accepts it (S37).

[0139] Then, copy output edit processing unit 26 sets the "document ID" in the section corresponding to the file name of the masking pattern in the "document ID" t2-1 column of template information table T1 (see FIG. 18) via storage control unit 23 (S38), and closes template registration screen 2200 (S39). As a result, template registration screen 2200 is closed on operation panel 20, and the display returns to copy edit operation screen 1100 (see FIG. 11J).

[0140] After step S39, the copy output edit processing unit 26 proceeds to the determination in step S13.

[0141] FIG. 20 is a diagram showing an example of a processing flow for calling a template of operation panel 20 according to the first modification.

[0142] After step S3 (No judgment), if the copy output edit processing unit 26 determines that the template call button 1124 has been touched (S41: Yes judgment), it acquires template information from template information table T1 (see FIG. 18) via the storage control unit 23 (S42). Then, the copy output edit processing unit 26 instructs the display control unit 25 to display the template information on the template call screen 3200 (see FIG. 17), and waits while ready to accept input from the input accepting unit 24 (S43).

[0143] As a result, template call screen 3200 is displayed in an active state on copy / edit operation screen 1100 (see FIG. 11F) of operation panel 20. On template call screen 3200, the user touches template list selection box 3201 to select a masking template to be applied to the scanned image being processed, and then touches call button 3202.

[0144] When copy output edit processing unit 26 receives the selection of a template (S44) and receives a touch operation on call button 3202 (S45), it acquires the document ID associated with the selected template from template information table T1 via storage control unit 23 (S46). Then, copy output edit processing unit 26 instructs main unit 10 to acquire a masking pattern for the document ID (S47). Specifically, copy output edit processing unit 26 specifies the document ID to the API and acquires the masking pattern from main unit 10.

[0145] When copy output edit processing unit 26 acquires the masking pattern from main unit 10 (S48), it terminates template call screen 3200 (S49). As a result, template call screen 3200 closes on the screen of operation panel 20, and the screen returns to displaying copy edit operation screen 1100 (see FIG. 11F). Note that while a masking pattern is being acquired from main unit 10, display control unit 25 may be instructed to display a wait screen. In this case, the screen of operation panel 20 displays a wait screen while the masking pattern is being acquired from main unit 10, and after the masking pattern arrives, the screen returns to displaying copy edit operation screen 1100 (see FIG. 11F).

[0146] After acquiring the masking pattern from the main body 10, the copy output edit processing unit 26 instructs the display control unit 25 to display the scanned image in the enlarged / reduced display area 1142 (see FIG. 11F) by replacing each pixel specified in the masking pattern with black RGB values ​​(0,0,0) (S50). That is, in the enlarged / reduced display area 1142 (see FIG. 11F), masking of the arrangement pattern specified in the masking pattern is drawn on the scanned image.

[0147] After step S50, the copy output edit processing unit 26 proceeds to the determination of step S9. If the masked portion of the selected template corresponds to the scanned image, the user touches the image formation execution button 1131. The processing after touching the image formation execution button 1131 is as described in the embodiment. In the first modified example, the masking pattern of the template is used as the masking pattern to be composited with the scanned image, and the composite image is sent to the main unit 10 for image formation. The destination main unit 10 executes image formation based on the composite image.

[0148] If the user wants to change the template because the masking portion of the selected template does not correspond to the scanned image, for example, the user may provide a delete button on the copy / edit operation screen 1100 (see FIG. 11F) and instruct to delete the acquired masking pattern and the masking drawing on the display. In this case, the user repeats the process from step S41 and selects another template in step S44.

[0149] (Communication sequence) Next, the communication process between the operation panel 20 and the main body 10 will be described. Note that this communication process is based on the communication process sequence in Fig. 14, and only modifications will be described here. Other overlapping portions will be omitted from the illustrations and explanations as appropriate.

[0150] 21 is a diagram showing an example of a sequence of communication processing when a template is registered between the operation panel 20 and the main body 10. After creating the masking pattern in step S117, when an instruction to register the template is received, the copy output edit processing unit 26 instructs the WebAPI communication unit 27 to register the masking pattern being processed (S201), and the WebAPI communication unit 27 transmits request data (see FIG. 7(d)) ​​requesting execution of the fourth API 15-4 to the main body 10 (S202). When transmitting the request data, the WebAPI communication unit 27 transfers the masking pattern to the main body 10.

[0151] As a result, in the main body 10, the WebAPI service 15 accepts the designation of the fourth API 15-4 and instructs the storage control unit 13 to register the file, which is the operation request (S204).

[0152] The storage control unit 13 manages the masking pattern by the document ID and stores the masking pattern in the masking pattern storage area (S205). The WebAPI service 15 transmits the file registration result (document ID) to the WebAPI communication unit 27 as response information (S207). The transmitted document ID is notified to the copy output editing processing unit 26 (S208), and is registered in the template information table T1 (see FIG. 18).

[0153] Thereafter, when the user touches the image formation execution button 1131, the masking pattern is combined with the scanned image being processed in step S118. The flow from step S118 onwards is the same as that shown in the embodiment, and therefore a description thereof will be omitted.

[0154] 22 is a diagram showing an example of a communication processing sequence when a template is acquired between the operation panel 20 and the main body 10. After acquiring the scanned image in step S116, the copy output edit processing unit 26 instructs the WebAPI communication unit 27 to acquire the template specified by the user (S301), and the WebAPI communication unit 27 transmits request data (see FIG. 7(e)) requesting execution of the fifth API 15-5 to the main body 10 (S302). Note that the document ID notified when the template was registered is set in {document ID} (see FIG. 7(e)) of the request data.

[0155] As a result, in the main body 10, the WebAPI service 15 accepts the designation of the fifth API 15-5 and instructs the storage control unit 13 to call a file, which is an operation request (S304). In response to this instruction, the storage control unit 13 reads out a masking pattern corresponding to the document ID from the masking pattern storage area (S305) and transfers the read masking pattern to the WebAPI service 15 (S306). The WebAPI service 15 transmits the masking pattern to the WebAPI communication unit 27 (S308). The WebAPI communication unit 27 notifies the copy output editing processing unit 26 that the masking pattern has been acquired (S309).

[0156] When the masking pattern is acquired, the copy output edit processing unit 26 displays the masking pattern on the scanned image, and thereafter, if the user wants to change the template, steps S301 to S309 are repeated.

[0157] Once the template is confirmed and the user touches the image formation execution button 1131, in step S118 the masking pattern of the confirmed template is combined with the scanned image being processed. The flow from step S118 is the same as that shown in the embodiment, so a description thereof will be omitted.

[0158] As described above, in the first modification of the first embodiment, by registering a masking pattern when masking is performed, the registered masking patterns are called up in subsequent copy output, and a masking pattern is selected from the registered masking patterns and combined with the scanned image. Therefore, the user can combine the masking pattern with the scanned image by selecting a masking pattern from the registered masking patterns according to the type of scanned document. The user can omit the operation of marking the scanned image, thereby improving user operability.

[0159] (Modification 2 of the First Embodiment) This shows a modified example in which a masking pattern (priority pattern) that is preferred by an operator when masking a scanned image is preset. Here, an example is shown in which the operator logs in to the operation panel 20 by holding an IC card over the operation panel 20, and the priority pattern is preset based on the authentication information of the operator at the time of login. Hereinafter, it is assumed that the operation panel 20 is equipped with an IC card reader. The IC card reader is connected to the system bus 208, and reads authentication information from the held-up IC card based on instructions from the CPU 200. It is also assumed that the CPU 200 realizes an authentication function unit by executing an authentication program. The authentication function unit compares the authentication information read by the IC card reader with an authentication table in the main unit 10, and if authentication is successful, permits the operator to log in to the system using the operation panel 20.

[0160] (Operation panel 20 table) 23 is a diagram showing an example of a correspondence table stored in flash memory 203 of operation panel 20. Template information table T2 (an example of a correspondence table) shown in FIG. 23 is template information table T1 (see FIG. 18) with an item of "user ID" t2-4 added. User ID information (an example of "personal ID information") is set in the item of "user ID" t2-4. The user ID information is information obtained when the operator logs in with an IC card. Here, for ease of explanation, it is represented by a user name.

[0161] Furthermore, priority pattern identification information for each user is set in "template name" t2-2. Here, for ease of explanation, the name of a masking pattern is shown as the priority pattern identification information. As examples, "driver's license blackout pattern" and "passport blackout pattern" are shown, but the present invention is not limited to these and any other suitable pattern may be set. For example, "health insurance card blackout pattern" and "resident registration card blackout pattern" may be set. In this example, a pattern having a predetermined format that is frequently used for copy output is treated as a priority pattern.

[0162] (Operation of operation panel 20) The operation of operation panel 20 in Variation 2 of the first embodiment differs from the operation in Variation 1 of the first embodiment in that authentication information is used. In Variation 2 of the first embodiment, copy output editing processing unit 26 also operates as an "authentication selection unit." Here, differences from the first embodiment and its Variation 1 will be mainly described, and other common operations will be denoted by the same reference numerals and explanations thereof will be omitted as appropriate.

[0163] First, in step S35 of the registration process shown in FIG. 19, the copy output edit processing unit 26 of Modified Example 2 registers, in association with the template information table T2 (see FIG. 23), the user ID information used by the operator for login authentication, in addition to the template information input to the template registration screen 2200 and the file name of the masking pattern being processed.

[0164] 24 is a diagram showing an example of a process flow for calling a template on operation panel 20 in Modification 2 of the first embodiment. After the copy output editing application is launched in step S1, copy output editing processing unit 26 acquires user ID information of the logged-in operator from the authentication function unit (not shown) (S51), and acquires a document ID associated with the user ID information from template information table T2 via storage control unit 23 (S46). Then, copy output editing processing unit 26 instructs main unit 10 to acquire a masking pattern for the document ID (S47).

[0165] When the copy output edit processing unit 26 acquires the masking pattern from the main unit 10 (S48), it stores the masking pattern in the third temporary storage area (S52).

[0166] Thereafter, the copy output editing processor 26 executes steps S2 to S7 (see FIG. 10) in the same manner as in the embodiment.

[0167] When the scanned image is acquired in step S7, the copy output editing processor 26 instructs the display controller 25 to display the copy editing operation screen 1100 (see FIG. 8), and waits for input from the input receiver 24 (S53). When this display is performed, the copy output editing processor 26 sets the display of the scanned image to ON and displays the acquired scanned image (preview image) and the masking pattern from the third temporary storage area in the zoom display area 1142 (see FIG. 8). As a result, when the scanned image is acquired, the scanned image masked with the preset priority pattern is displayed on the screen of the operation panel 20.

[0168] After the process of step S53, the process returns to step S3, and if the determination is No, the process proceeds to step S9 and thereafter.

[0169] When the user wishes to execute image formation using the priority pattern displayed on the copy edit operation screen 1100, the user touches the image formation execution button 1131. Then, the copy output edit processing unit 26 determines that the image formation execution button 1131 has been touched (S13: Yes determination), and performs a process of combining the masking pattern with the scanned image (S14). Thereafter, the copy output edit processing unit 26 performs the processes of steps S15 to S18 (see FIG. 10) in the same manner as in the embodiment. The other processes have been described in the embodiment, so description thereof will be omitted here.

[0170] (Communication sequence) 25 is a diagram showing an example of a sequence of communication processing when a template is called between operation panel 20 and main body 10. Note that the same reference numerals are used to designate sequences that are common to the first embodiment and its first modification.

[0171] After the copy output editing application is launched, the copy output editing processing unit 26 acquires a document ID (instruction information of a template prioritized by the operator) associated with the user ID information of the operator from the template information table T2, and instructs the WebAPI communication unit 27 to acquire the template (S301). The WebAPI communication unit 27 transmits request data (see FIG. 7(e)) requesting execution of the fifth API 15-5 to the main unit 10 (S302). Note that the document ID is set in {document ID} (see FIG. 7(e)) of the request data.

[0172] As a result, in the main body 10, the WebAPI service 15 accepts the designation of the fifth API 15-5 and instructs the storage control unit 13 to call a file, which is an operation request (S304). In response to this instruction, the storage control unit 13 reads out a masking pattern corresponding to the document ID from the masking pattern storage area (S305) and transfers the read masking pattern to the WebAPI service 15 (S306). The WebAPI service 15 transmits the masking pattern to the WebAPI communication unit 27 (S308). The WebAPI communication unit 27 notifies the copy output editing processing unit 26 that the masking pattern has been acquired (S309).

[0173] When the copy output edit processing unit 26 acquires the masking pattern, it stores it in a third temporary storage area.

[0174] When the operator touches the scanning execution button 1111, the copy output editing processing unit 26 instructs the main body 10 to execute scanning and acquires the scanned image (steps S101 to S116). Note that this operation has already been explained in the embodiment, so explanation here will be omitted.

[0175] When the scanned image is acquired in step S116, the copy output edit processing unit 26 uses the masking pattern in the third temporary storage area to display the scanned image masked with the preset priority pattern.

[0176] Then, when the operator touches the image formation execution button 1131, the copy output edit processing unit 26 combines the masking pattern with the scanned image being processed (S118). The subsequent flow from step S118 is the same as the flow shown in the embodiment, and therefore a description thereof will be omitted.

[0177] Here, as an example, an example has been shown in which template information and user ID information are managed in the same table (template information table T2), but template information and user ID information may be managed in separate tables and associated with each other.

[0178] Although the example shown here shows a process of acquiring a priority pattern from the main body before step S101, the timing of this process is not limited to this example and may be performed at any timing. Furthermore, this process may be performed in the background in parallel with the process of step S101, etc.

[0179] In the second modification of the first embodiment, it is possible to register a masking pattern that is set as a priority for each user in advance. Since frequently used masking patterns are automatically preset for each user, the number of operational steps for masking and the like is reduced, making it possible to further improve user operability.

[0180] (Third modification of the first embodiment) The masking color may be modified so that it can be changed according to the remaining toner amount. Here, an additional configuration for changing the masking color will be described.

[0181] Fig. 26 is a diagram showing an example of the functional configuration of operation panel 20 in Modification 3 of the first embodiment. Fig. 26 shows the functional configuration of operation panel 20 in the first embodiment (see Fig. 3) further provided with a remaining toner amount confirmation unit (remaining toner amount check unit) 31.

[0182] The remaining toner amount confirmation unit 31 requests the main body 10 to transmit remaining toner amount information, and acquires the remaining toner amount information from the plotter control unit 17 .

[0183] 27 is a flow diagram of an example of a masking color change process performed by the copy output editing processor 26. First, the copy output editing processor 26 acquires remaining toner amount information from the remaining toner amount confirmation unit 31 (S401). The remaining toner amount confirmation unit 31 may, for example, periodically request the main body 10 to transmit remaining toner amount information, thereby always acquiring the latest remaining toner amount information, or it may acquire remaining toner amount information from the plotter control unit 17 only when instructed to do so by the copy output editing processor 26.

[0184] Next, the copy output editing processor 26 determines whether the masking color is near toner (S402) based on the acquired remaining toner information. The masking color is generally set to black, but may be red, blue, or the like.

[0185] If the toner is not near end (S402: No), the copy output edit processing unit 26 maintains the masking color setting as is.

[0186] If the toner is near-end (S402: Yes), the copy output edit processing unit 26 changes the masking color setting to another color (S403). Here, the other color is preferably white (RGB value: 255, 255, 255), but is not limited to white as long as it is a color that does not indicate a toner near-end.

[0187] Fig. 28 is a diagram showing an example of an output product when the set color is changed to white. As shown in Fig. 28, masking P1 and masking P2 of output product P are formed by the white background of the paper.

[0188] The above masking color change process may be performed, for example, periodically, or only when an instruction to perform image formation is issued.

[0189] In addition, if the set color is temporarily changed to another color (such as white) due to toner near-end, and then the remaining toner of the original set color is restored by replacing the toner, the flow of the color change process may be modified so that the copy output edit processing unit 26 determines whether the remaining toner of the original set color has recovered, and if so, restores the original set color.

[0190] When masking with a set color (e.g., black), if a large amount of masking is performed when there is little black toner, a large amount of black toner will be consumed, resulting in the toner running out. When the toner runs out, the operation is interrupted and downtime occurs. In Modification 3 of the first embodiment, the operation panel 20 checks the remaining toner in the main body 10, and if the toner is near end, it automatically switches from black to white, for example, or back to the original set color black if there is toner. Therefore, Modification 3 of the first embodiment reduces the number of times a specific color runs out of toner, which has the effect of leading to a reduction in downtime. Furthermore, switching to white toner reduces the amount of toner used, further reducing downtime.

[0191] (Second embodiment) In the second embodiment, another form of processing for masking a scanned image being displayed on an operation panel is shown. Here, the operation panel searches the scanned image for pixels that have pixel values ​​equal to or within an approximate range of the pixel value of the pixel (called the "reference pixel") at the position touched by a finger in the scanned image being displayed, and replaces the pixel found by the search with the pixel value of a masking color.

[0192] Here, the "approximate range" refers to pixel values ​​that are included within a range between an upper limit and a lower limit based on the pixel value at the position touched by a finger. In the following, a pixel having the pixel value (color) at the position touched by a finger and a pixel having a pixel value within the approximate range are referred to as "pixels of the same color."

[0193] Each pixel has independent luminance information for each of the three primary colors (red, green, and blue) as color information. If each primary color has 8 bits (256 gradations), 16,777,216 colors (256 3s) can be represented because there are three primary colors. A pixel value is assigned to each pixel. RGB values ​​range from 0 to 255 for each primary color, i.e., (R, G, B) = (0, 0, 0) to (255, 255, 255). The upper and lower limits can be set appropriately depending on the color and density of the object to be masked (e.g., a handwritten note, as described below). For example, when specifying a range of RGB values ​​belonging to the same color, black and near-black pixel values ​​can be set to a range (R, G, B) = (0, 0, 0) to (50, 50, 50) where all RGB values ​​are greater than or equal to 0 and within a predetermined range, e.g., 50 or less. Alternatively, within a predetermined range of pixel values ​​(each RGB value) at a specified position, pixel values ​​within 25 gradations above and below may be considered similar, and if (R, G, B) = (0, 100, 100) is specified, colors in the range of (R, G, B) = (0, 75, 75) to (25, 125, 125) may be considered similar.

[0194] The following describes the functions and processing of the operation panel (mainly the copy output editing processing unit) that differ from the first embodiment. Note that in the following description, illustrations and descriptions of parts that are common to the first embodiment, such as the hardware configuration of the operation panel and the communication sequence with the main body 10, will be omitted as appropriate.

[0195] In the following, we will use a black and white question paper (where the printed parts such as questions and ruled lines are black and the background is white) as an example of a manuscript, and explain how to erase handwritten notes (such as letters, lines, and figures) included in the manuscript by masking. We will also assume that the handwritten notes are written with a colored pencil such as red or a colored ballpoint pen to distinguish them from the color of the printed parts. Note that the color combinations of the printed parts, background, and handwritten notes are not limited to these. The example shown below may be applied to any color combination as appropriate. Furthermore, the manuscript is not limited to a question paper, and may be replaced as appropriate with any other document that has printed parts and is writable, such as a circular document, a questionnaire, or other format paper.

[0196] Fig. 29 is a diagram showing an example of the configuration of the copy output edit processing unit of the operation panel according to the second embodiment. The copy output edit processing unit 26 shown in Fig. 29 has a search unit 41-1 and a replacement unit 41-2 as functions for performing masking.

[0197] The search unit 41-1 searches within the scanned image to detect pixels that belong to the same color as the pixel in the scanned image currently being displayed that corresponds to the position information received by the input receiving unit 24 (that is, the reference pixel).

[0198] Here, as an example, the conditions for pixels belonging to the same color are set as an allowable range (upper and lower limit ranges) based on the density and color system of the color of the reference pixel (referred to as the "reference color"). When an allowable range is set for the density, for example, a range of several percent is set in the direction of increasing and decreasing the density of each of the R value, G value, and B value of the pixel value of the reference pixel. Note that the allowable range may be set to "0" to cause the search unit 41-1 to search for a pixel value that is the same as the reference pixel.

[0199] The replacement unit 41-2 replaces the pixel values ​​of pixels belonging to the same color detected by the search unit 41-1 with a masking color. A predetermined color is set as the masking color in advance. Here, white, which is generally used for output paper, is set as an example of the masking color.

[0200] In the functional configuration of the second embodiment, the reading scanner control unit 16 corresponds to the "document reading unit." The plotter control unit 17 corresponds to the "image forming unit." The input receiving unit 24 corresponds to the "input receiving unit." The display control unit 25 corresponds to the "display unit." The copy output editing processing unit 26 according to the second embodiment is another example of the "image processing unit," the search unit 41-1 corresponds to the "detection unit," and the replacement unit 41-2 corresponds to the "replacement unit." The WebAPI service 15 and the WebAPI communication unit 27 correspond to various "API communication units."

[0201] (Screen configuration of the screen displayed on the operation panel) FIG. 30 is a diagram showing an example of a display screen of an operation panel according to the second embodiment. FIG. 30 shows a copy / edit operation screen 3100, which is an example of a display screen. The copy / edit operation screen 3100 has a configuration in which the drawing object selection button 1122 and the line correction button 1123 in the masking operation button display area 1120 are deleted from the copy / edit operation screen 1100 (see FIG. 8) according to the first embodiment. Note that the masking operation button display area 1120 may be provided with a setting button for setting a masking color and a setting button for setting a tolerance (an example of a setting means). For example, a color selection button may be provided to accept a color to be designated as the masking color. Furthermore, a button for gradually changing the tolerance range or a numerical setting button may be provided to accept the tolerance range. For ease of explanation, the following description will be given assuming that the masking color is set to white and the tolerance range is set to a predetermined width on another screen.

[0202] In the copy / edit operation screen 3100, the enlargement / reduction display area 1142 is a display area for displaying the scanned image as an enlarged or reduced image, and also receives input of the reference pixel of the scanned image being displayed.

[0203] The signal flow between the copy editing operation screen 3100 and the search unit 41-1 is as follows: The operator touches and designates a part of the scanned image to be masked in the enlarged / reduced display area 1142. The touch-designated position information is acquired by the input receiving unit 24, and the search unit 41-1 accepts the position information as a reference pixel.

[0204] (Operation panel operation) Next, the operation of the operation panel according to the second embodiment will be described. Here, the processing of the operation panel will be described using the processing flow shown in Fig. 31 as an example. Also, Fig. 32A to Fig. 32C will be referred to as appropriate for the user's operation procedures and the screen flow of the operation panel.

[0205] 31 is a diagram showing an example of a processing flow of the operation panel according to the second embodiment. This processing flow is a modification of the masking processing in the processing flow of the operation panel 20 according to the first embodiment (see FIG. 10). Steps S1 to S10 and steps S15 to S19 are substantially the same as those in the processing flow of the operation panel 20 according to the first embodiment. The following describes other processing steps that differ from those in the processing flow of the operation panel 20 according to the first embodiment.

[0206] Following the processing of step S9 (No determination) or step S10, the copy output editing processing unit 26 (search unit 41-1) first determines whether there has been a touch in the zoom display area 1142 of the copy editing operation screen 3100 (see FIG. 32A) (S61). If there has been no touch (S61: No determination), the process proceeds to step S13.

[0207] On the other hand, when the input receiving unit 24 receives position information of the touch position, the copy output editing processing unit 26 (search unit 41-1) determines that a touch has occurred (S61: Yes determination) and sets the pixel value of the pixel corresponding to the position information (corresponding to the reference pixel) as the reference color (S62). For example, if the user touches part of a handwritten note written on a test paper with a red pencil or a red ballpoint pen (part of the note within the dashed frame L3 or L4 in FIG. 32B), the touched position becomes the reference pixel. Note that the scanned image may be enlarged when touching to allow the intended part to be specified accurately. Also, an operation unit for moving the specified position on the screen may be provided so that the specification can be moved from the pixel specified by touch.

[0208] Next, the copy output editing processing unit 26 (searching unit 41-1) generates a copy of the scanned image being displayed (S63), and reads the pixel values ​​of each pixel of the scanned image in a predetermined scanning order for the copy (S64).

[0209] Next, the copy output editing processing unit 26 (search unit 41-1) determines whether the read pixel value belongs to the reference color (S65). For example, it determines whether the R value, G value, and B value of the read pixel value are each within the allowable ranges of the R value, G value, and B value of the reference color. Then, if the R value, G value, and B value of the read pixel value are all within their respective allowable ranges, it determines that the read pixel value belongs to the reference color. In other words, it detects the pixel as belonging to the reference color.

[0210] If the read pixel value belongs to the reference color (S65: Yes), the copy output edit processing unit 26 (replacement unit 41-2) replaces the pixel value of the read pixel with the pixel value of the set masking color (white in this example) (S66).If the read pixel value does not belong to the reference color (S65: No), the copy output edit processing unit 26 (search unit 41-1) skips step S66 and proceeds to step S67.

[0211] Next, the copy output editing processing unit 26 (search unit 41-1) determines whether scanning of all pixels has been completed (S67), and if scanning of all pixels has not been completed (S67: No determination), it proceeds to step S64 and performs similar search and replacement processing on the remaining pixels.

[0212] If scanning of all pixels has been completed (S67: Yes judgment), the copy output editing processing unit 26 (search unit 41-1) replaces the display in the zoom display area 1142 with a masked scanned image (see Figure 32C) (S68) and proceeds to processing of step S13.

[0213] If a portion of the handwritten note remains after a single touch due to a small setting value for the tolerance used to determine pixels belonging to the reference color, the remaining portion of the handwritten note can be erased by repeatedly touching the remaining portion of the handwritten note. In this case, a copy of the last generated scanned image is displayed. When repeatedly touching the remaining portion of the handwritten note, multiple copies of the scanned image may be generated according to the number of touches so that the original state can be restored in the order in which the notes were erased.

[0214] The user instructs the image forming execution button 1131 (see FIG. 32C) to be touched to output a copy of the masked scanned image. When the copy output editing processor 26 determines that the image forming execution button 1131 has been touched (S13: Yes determination), it instructs the main body 10 to execute image formation of the masked image (copy image) being displayed (S15). The subsequent process is substantially the same as in FIG. 10, and therefore a description thereof will be omitted.

[0215] In addition, in the deletion of the scanned image in step S20, the scanned image and all of its copies are deleted.

[0216] 32A to 32C are diagrams showing an example of a user's operation on the operation panel according to the second embodiment. Fig. 32A shows the display state immediately after a zoom operation is performed on an acquired scanned image. The scanned image after the zoom operation (zoomed image X2) is displayed in the zoom display area 1142 of the copy editing operation screen 3100. The scanned image (zoomed image X2) shows, as an example, a printed original containing handwritten notes in two places.

[0217] 32B shows a user touching a handwritten note on the scanned image (enlarged / reduced image X2) with their finger. The user touches a part of the handwritten note included in the scanned image (enlarged / reduced image X2) with their finger. The touched position is on the handwritten note, and is part of one of the two notes enclosed by dashed lines L3 and L4 in FIG. 32B for the purpose of explanation.

[0218] FIG. 32C is a diagram showing an example of the display state of the zoom display area 1142 after masking has been performed. FIG. 32C shows that two handwritten notes have been replaced with a white background and erased with a single touch. If the tolerance range is set appropriately, the two handwritten notes are almost completely detected as targets for masking from the image, and the handwritten notes are erased from the printed manuscript in this manner. Note that if the tolerance range setting is too small, for example, and part of the handwritten note remains with a single touch, it is recommended to repeatedly touch the remaining part of the handwritten note. This will erase the remaining handwritten note, resulting in a state similar to that shown in FIG. 32C, where the handwritten note has been erased.

[0219] (Communication sequence) The communication process between the operation panel and main body 10 according to the second embodiment is substantially the same as the communication process between the operation panel 20 and main body 10 according to the first embodiment (see FIG. 14). However, the processes of step S117 (see FIG. 14) and step S118 (see FIG. 14) on the operation panel are replaced with search and replace processes in the second embodiment. Also, the image transferred in step S120 and the image read in step S125 are replaced with a copy image replaced with a masking color. Further explanation will be omitted here to avoid duplication.

[0220] Fig. 33 is a diagram showing an example of an output printed out by the main body 10. Fig. 33 shows an original Q before masking and an output P obtained by masking the original Q and copying it, so that the state before and after masking can be compared. Fig. 33(a) shows a test sheet on which red notes q1 and q2 have been written, as an example of the original Q. Fig. 33(b) shows an example of the output P after the red notes q1 and q2 shown in Fig. 33(a) have been masked out with the background color.

[0221] In this way, in the second embodiment, the operation panel searches for pixels in the scanned image being displayed that have the same color as the pixel value (color) of the position touched with a finger as the masking target, and replaces the color of the pixels found to have the same color with the masking color. This makes it possible to erase information such as notes added later while leaving the original information (characters, lines, figures, etc.) printed on the document as is.

[0222] (Modification 1 of the second embodiment) A modified example of setting the masking color will be shown. The masking color may be a preset color, or may be set by the user when masking is performed. When the user sets the color, for example, a color selection button may be provided on a setting screen for setting masking settings, and the color to be used for masking may be selected from the setting screen. Here, as another example, a case where the user specifies the masking color from a scanned image will be shown.

[0223] (Screen configuration of the screen displayed on the operation panel) Fig. 34 is a diagram showing an example of a display screen of an operation panel according to Modification 1 of the second embodiment. The copy edit operation screen 4100 shown in Fig. 34 is obtained by adding a masking color setting button 4101 to the masking operation button display area 1120 of the copy edit operation screen 3100 (see Fig. 30) according to the second embodiment.

[0224] The masking color setting button 4101 is an instruction button for allowing the operator to specify, as a masking color, the color used in the scanned image displayed in the enlargement / reduction display area 1142. When the masking color setting button 4101 is set to ON, the enlargement / reduction display area 1142 of the copy / edit operation screen 4100 accepts input of the reference pixel to be masked, as well as input of the pixel to be specified as the masking color.

[0225] The setting in the copy output edit processing unit 26 is switched as follows: For example, the operator touches the masking color setting button 4101 to set it on, and then touches a pixel area (such as the background) to be designated as the masking color of the scanned image being displayed in the enlarged / reduced display area 1142. In this case, the copy output edit processing unit 26 enables the user to accept designation input of the pixel to be set as the masking color by touching the masking color setting button 4101, and when the input accepting unit 24 subsequently acquires position information touched in the enlarged / reduced display area 1142, the replacement unit 41-2 sets the color (pixel value) of that position information as the masking color.

[0226] The order in which touches on the reference pixels to be masked and touches on the pixels to be masked may be set as appropriate. For example, touches on the reference pixels to be masked and touches on the pixels to be masked may be received alternately in this order.

[0227] (Operation panel operation) Next, the operation of the operation panel according to the first modification of the second embodiment when masking is performed will be described. Here, the processing of the operation panel will be described using the processing flow shown in Fig. 35 as an example. Also, Figs. 36A to 36D will be referred to as appropriate for the user's operation procedure and the screen flow of the operation panel.

[0228] Fig. 35 is a diagram showing an example of a processing flow of the operation panel according to Modification 1 of the second embodiment. Fig. 35 shows a modification of the processing from after step S9 (No determination) and step S10 to immediately before step S13 in the processing flow of the operation panel according to the second embodiment (see Fig. 31). Note that in the processing of the modification shown in Fig. 35, the same processes as those in the processing flow of the operation panel according to the second embodiment (see Fig. 31) are assigned the same numbers.

[0229] First, the copy output edit processing unit 26 determines whether the masking color setting button 4101 on the copy edit operation screen 4100 (see FIG. 36A) has been touched (S71).

[0230] When the input receiving unit 24 receives the position information of the masking color setting button 4101, the copy output editing processing unit 26 determines that there has been a touch (S71: Yes judgment) and switches the setting of the masking color setting button 4101 between on and off (S72). When the input receiving unit 24 does not receive the position information of the masking color setting button 4101, the copy output editing processing unit 26 determines that there has been no touch (S71: No judgment) and skips step S72, thereby maintaining the setting of the masking color setting button 4101 between on and off.

[0231] Next, the copy output edit processing unit 26 (search unit 41-1) determines whether or not there has been a touch on the enlargement / reduction display area 1142 of the copy edit operation screen 4100 (see FIG. 36B) (S61).

[0232] When the input receiving unit 24 receives the position information of the touch, the copy output editing processing unit 26 (search unit 41-1) determines that a touch has occurred (S61: Yes judgment) and sets the pixel value of the reference pixel corresponding to the position information as the reference color (S62).

[0233] Next, the copy output edit processing unit 26 (replacement unit 41-2) determines whether the masking color setting button 4101 is set to ON (S73). If the masking color setting button 4101 is set to ON (S73: Yes determination), the copy output edit processing unit 26 (replacement unit 41-2) then further accepts a touch input in the zoom display area 1142 of the copy edit operation screen 4100 (see FIG. 36C) and determines whether there has been a touch (S74).

[0234] When the input receiving unit 24 receives touched position information, the copy output editing processing unit 26 (replacement unit 41-2) determines that a touch has occurred (S74: Yes determination) and specifies the pixel value of the pixel corresponding to the position information as the masking color (S75). For example, if the background of the document is colored, such as light blue or yellow, and the user touches the background of the scanned image, the pixel value of the touched colored position is specified as the masking color. If there is no touch (S74: No determination), the copy output editing processing unit 26 (replacement unit 41-2) waits until there is a touch input. Here, a child's notebook is one example of a document with a colored background. Other examples include straw paper (light brown) and old paper.

[0235] If the masking color setting button 4101 is set to OFF (S73: No), step S75 is skipped, that is, the masking color remains the default (white, etc.).

[0236] Next, the copy output editing processing unit 26 (searching unit 41-1) generates a copy of the scanned image being displayed (S63), and reads pixel values ​​from the copy in a predetermined scanning order (S64).

[0237] Next, the copy output edit processing unit 26 (search unit 41-1) determines whether the read pixel value belongs to the reference color (S65).

[0238] If the read pixel value belongs to the reference color (S65: Yes), the copy output edit processing unit 26 (replacement unit 41-2) replaces the pixel with the pixel value of the masking color (S66). If the read pixel value does not belong to the reference color (S65: No), the copy output edit processing unit 26 (search unit 41-1) skips step S66 and proceeds to step S67.

[0239] Next, the copy output editing processing unit 26 (search unit 41-1) determines whether scanning of all pixels has been completed (S67), and if scanning of all pixels has not been completed (S67: No determination), it proceeds to step S64 and performs similar search and replacement processing on the remaining pixels.

[0240] If scanning of all pixels has been completed (S67: Yes judgment), the copy output editing processing unit 26 (search unit 41-1) replaces the display in the zoom display area 1142 with a masked scanned image (see Figure 36D) (S68) and proceeds to processing of step S13.

[0241] In this modified process, when the scanned image is deleted in step S20 (see FIG. 31), the scanned image and all of its copies are deleted.

[0242] 36A to 36D are diagrams showing an example of a user's operation on the operation panel according to the first modification of the second embodiment. FIG. 36A shows the display state immediately after a zoom operation is performed on an acquired scanned image. The scanned image after the zoom operation (zoomed image X2) is displayed in the zoom display area 1142 of the copy editing operation screen 4100. The scanned image (zoomed image X2) shows, as an example, a printed original containing handwritten notes in two places.

[0243] 36B shows a user touching a handwritten note on the scanned image (enlarged / reduced image X2) with their finger. The user touches a part of the handwritten note included in the scanned image (enlarged / reduced image X2) with their finger. The touched position is on the handwritten note, and is part of one of the two notes enclosed by dashed lines L3 and L4 in FIG. 36B for the purpose of explanation.

[0244] 36C shows a state in which a user touches with a finger a color to be designated as a masking color in the scanned image (enlarged / reduced image X2). Here, as an example, a state in which a finger touches the background of the document included in the scanned image (enlarged / reduced image X2) is shown. The touched position is any part of the background of the document.

[0245] Fig. 36D is a diagram showing an example of the display state of the zoom display area 1142 after masking has been performed. Fig. 36D shows the handwritten note being replaced with the color specified as the masking color by touching, resulting in the erased state. Although not shown, the output printed out by the main body 10 also has an image similar to the display state shown in the zoom display area 1142.

[0246] (Communication sequence) The communication processing between the operation panel and the main body 10 in the variant 1 of the second embodiment is substantially the same as the communication processing between the operation panel and the main body 10 in the second embodiment, and therefore will not be described here.

[0247] In this way, in variant example 1 of the second embodiment, the user specifies a reference pixel and a pixel indicating the masking color, for example, by specifying the reference pixel and then specifying the pixel indicating the masking color, and the masking object including the reference pixel is masked with the specified color in the image.

[0248] Furthermore, if a portion of the handwritten note remains after one touch, the remaining portion of the handwritten note can be erased with the same masking color by repeatedly touching the remaining portion of the handwritten note.

[0249] Although the example shown here alternates between touching the reference pixel of the masking target and touching the pixel of the masking color, the order of touching is not limited to this. For example, if the masking targets are distributed, multiple touches may be received for each masking target, followed by a single touch to set each masking target to a common masking color. In this case, multiple touched locations on the masking target are masked with the same masking color with a single touch.

[0250] Also, after receiving a touch on the reference pixel to be masked, the replacement target may be replaced with a predetermined color that is easy for the user to understand, and then a masking color may be received from the user, and the temporary predetermined color may be replaced with the masking color received by the user.

[0251] In this way, in the first modification of the second embodiment, the user can specify the masking color from the scanned image. Therefore, even if the document has a background color other than white, handwritten notes and the like can be erased according to the background color of the document.

[0252] (Modification 2 of the second embodiment) Here we will show another variation of the masking color setting. Here, we will show how to leave the masking target at the touched position and mask the rest as the masking target. For example, this can be applied to removing discoloration such as yellowing from a document.

[0253] (Screen configuration of the screen displayed on the operation panel) Fig. 37 is a diagram showing an example of a display screen of an operation panel according to Modification 2 of the second embodiment. A copy editing operation screen 5100 shown in Fig. 37 is obtained by adding a designated reversal button 5101 to the masking operation button display area 1120 of the copy editing operation screen 3100 (see Fig. 30) according to the second embodiment.

[0254] The specified reversal button 5101 is an instruction button that instructs that the pixels to be masked in the scanned image displayed in the zoom display area 1142 are the remaining pixels (called "removed pixels") excluding the reference pixel specified and input by the user and the pixels included in the color range of the reference pixel.

[0255] The settings in the copy output editing processor 26 are switched as follows. For example, the operator touches the designated reversal button 5101 to set it on, and then touches an area in the scanned image displayed in the enlarged / reduced display area 1142 that is not to be masked, such as original information on the document (printed ruled lines, etc.). In this case, the copy output editing processor 26 accepts the pixel designated in the enlarged / reduced display area 1142 by touching the designated reversal button 5101 as a non-masking reference pixel that is not to be masked, and sets pixels outside the color range of the reference pixel (pixels such as background pixels) as masking targets. When the input accepting unit 24 acquires position information touched in the enlarged / reduced display area 1142, the search unit 41-1 in the copy output editing processor 26 sets the color (pixel value) of the reference pixel corresponding to the position information as not to be masked.

[0256] (Operation panel operation) Next, the operation of the operation panel according to the second modification of the second embodiment when masking is performed will be described. Here, the processing of the operation panel will be described using the processing flow shown in Fig. 38 as an example. Also, Figs. 39A to 39C will be referred to as appropriate for the user's operation procedure and the screen flow of the operation panel.

[0257] Fig. 38 is a diagram showing an example of a processing flow of the operation panel according to Modification 2 of the second embodiment. Fig. 38 shows a modification of the processing from after step S9 (No determination) and step S10 to immediately before step S13 in the processing flow of the operation panel according to the second embodiment (see Fig. 31). Note that in the processing of the modification shown in Fig. 38, the same processes as those in the processing flow of the operation panel according to the second embodiment (see Fig. 31) are assigned the same numbers.

[0258] First, the copy output edit processing unit 26 determines whether the designated reversal button 5101 on the copy edit operation screen 5100 (see FIG. 39A) has been touched (S81).

[0259] When the input receiving unit 24 receives the position information of the designated reversal button 5101, the copy output editing processing unit 26 determines that a touch has been made (S81: Yes judgment) and switches the setting of the designated reversal button 5101 between on and off (S82). When the input receiving unit 24 does not receive the position information of the designated reversal button 5101, the copy output editing processing unit 26 determines that a touch has not been made (S81: No judgment), skips step S82, and maintains the setting of the designated reversal button 5101 between on and off.

[0260] Next, the copy output edit processing unit 26 (search unit 41-1) determines whether or not there has been a touch on the enlargement / reduction display area 1142 of the copy edit operation screen 5100 (see FIG. 39B) (S61).

[0261] When the input receiving unit 24 receives the position information of the touch, the copy output editing processing unit 26 (search unit 41-1) determines that a touch has occurred (S61: Yes judgment), and then determines whether the designated reversal button 5101 is set to on (S83).

[0262] If the specified reversal button 5101 is set to ON (S83: Yes), the copy output edit processing unit 26 (search unit 41-1) sets the pixel value of the reference pixel received by the input receiving unit 24 to a reference color that is not to be masked (S84). For example, if the user specifies original information of the document (printed ruled lines, etc.), the pixel value of the reference pixel that indicates the specified information is set to a reference color that is not to be masked. Note that if the specified reversal button 5101 is set to OFF (S83: No), the copy output edit processing unit 26 (search unit 41-1) performs the process from step S62 (see FIG. 31). This process has already been described, so its description and illustration will be omitted here.

[0263] Following step S84, the copy output edit processing unit 26 (search unit 41-1) generates a copy of the scanned image being displayed (S85), and reads the pixel values ​​of each pixel of the scanned image in a predetermined scanning order for the copy (S86).

[0264] Next, the copy output edit processing unit 26 (search unit 41-1) determines whether the read pixel value belongs to the reference color (S87).

[0265] If the read pixel value does not belong to the reference color (S87: Yes), the copy output edit processing unit 26 (replacement unit 41-2) replaces the pixel value of the read pixel with the pixel value of the set masking color (the white background in this example) (S88). That is, in this example, pixels other than the original information of the specified document, such as yellowing or dirt in the background, are replaced with the white background. If the read pixel value belongs to the reference color (S87: No), the copy output edit processing unit 26 (search unit 41-1) skips step S88 and proceeds to step S89.

[0266] Next, the copy output editing processing unit 26 (search unit 41-1) determines whether scanning of all pixels has been completed (S89), and if scanning of all pixels has not been completed (S89: No determination), it proceeds to step S86 and performs similar search and replacement processing on the remaining pixels.

[0267] If scanning of all pixels has been completed (S89: Yes judgment), the copy output editing processing unit 26 (search unit 41-1) replaces the display in the zoom display area 1142 with a masked scanned image (see Figure 39C) (S90) and proceeds to processing of step S13.

[0268] If the original information (printed lines, etc.) of the document cannot be selected in one touch and remains, the remaining part may be repeatedly touched to select the remaining original information.

[0269] The user instructs the copy output of the masked scanned image by touching the image formation execution button 1131. When the copy output editing processing unit 26 determines that the image formation execution button 1131 has been touched (S13: Yes determination), it instructs the main body 10 to execute image formation of the masked image (copy image) being displayed (S15). The subsequent process is substantially the same as in FIG. 10, and therefore description thereof will be omitted.

[0270] 39A to 39C are diagrams showing an example of a user's operation on the operation panel according to Modification 2 of the second embodiment. FIG. 39A shows the display state immediately after a zoom operation is performed on an acquired scanned image. The scanned image after the zoom operation (zoomed image X2) is displayed in the zoom display area 1142 of the copy editing operation screen 5100. The scanned image (zoomed image X2) shows, as an example, a scanned image of an old printed original having yellowing Y1.

[0271] 39B shows a user touching the original printed portion (ruled lines, letters, drawings, etc.) of the scanned image (enlarged / reduced image X2) with their finger. The user touches a portion of the printed portion included in the scanned image (enlarged / reduced image X2) with their finger. The touched position is a portion of the printed portion.

[0272] FIG. 39C is a diagram showing an example of the display state of the enlarged / reduced display area 1142 after masking has been performed. FIG. 39C shows the state after the printed portion is left and the other yellowing Y1 and the like are replaced with a white background. In this way, it is also possible to erase the yellowing Y1 of an old manuscript and other stains. Although not shown in the figure, the output printed out by the main body 10 also has an image similar to the display state shown in the enlarged / reduced display area 1142.

[0273] (Communication sequence) The communication processing between the operation panel and the main body 10 in the second embodiment of the present invention is substantially the same as the communication processing between the operation panel and the main body 10 in the second embodiment of the present invention, and therefore will not be described here.

[0274] In this way, in variant 2 of the second embodiment, the original information of the original document (printed lines, etc.) is left intact, and the rest is masked with a white background, etc., thereby removing yellowing of the background, etc.

[0275] (Modification 3 of the second embodiment) In the second embodiment and its variations, the specified line (pencil writing, etc.) is erased by masking it with the masking color of the background, but the background may be masked with the color of the specified line or a color close to it, so that the specified line is relatively erased.

[0276] (Fourth modification of the second embodiment) In the second embodiment and its modified examples, the masked scanned image edited on the operation panel may be output to a recording medium or an external device. In this case, for example, an output button may be provided on the copy / edit operation screen (copy / edit operation screen 3100, 4100, 5100) of the operation panel, and the user may touch the output button after the masking has been applied to the scanned image. When the input accepting unit 24 accepts the touch operation, the copy / output / edit processing unit 26 outputs the masked scanned image to a recording medium or an external device via the storage control unit 23 or the communication control unit 22. For example, the storage control unit 23 or the communication control unit 22 may use the communication I / F 204 (see FIG. 2) implemented on the operation panel as a wireless communication interface such as a USB interface or BLE to externally output and store the masked scanned image in a user terminal or USB flash memory. Alternatively, the masked scanned image may be externally output from the operation panel to a specified external server device via the communication I / F 104 (see FIG. 2) of the main unit 10.

[0277] In variant example 4 of the second embodiment, the output button and copy output editing processing unit 26 become the "external output unit," and it becomes possible to reproduce physically deteriorated media such as paper media neatly as data and re-output them as neat manuscripts, or to erase the handwritten parts of a handwritten format sheet so that they are not visible and reuse the format sheet.

[0278] The programs executed by the image processing devices of each embodiment and each variant example are provided as files in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0279] The programs executed by the image processing devices of the embodiments and modifications may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.The programs executed by the image processing devices of the embodiments and modifications may be provided or distributed via a network such as the Internet.

[0280] Furthermore, the programs of the embodiments and modifications may be provided in a state that they are pre-installed in a ROM or the like.

[0281] Examples of usage scenarios for the applications (including copy output editing applications) shown in the embodiments and modifications will be described below. Note that the applications can also be used in other usage scenarios and business types.

[0282] (1) Users use this application when copying or scanning information, such as personal information, that must be protected from the public eye. For example, when opening an account in the financial industry, bank tellers often use image processing devices such as copiers to copy customer identification documents, such as driver's licenses and health insurance cards. In recent years, the handling of personal information has become increasingly strict. Therefore, while it is necessary to record the fact that a copy has been made, it is also necessary to prevent personal information not required for identity verification, such as the "wearing glasses" information printed on a driver's license or personal identification information that must be protected, such as the My Number on a My Number card, from being printed when copying, printing, or scanning. Therefore, when copying documents containing personal information, a preview image of the document is displayed on the operation panel, allowing the staff member to specify areas to be masked. When copying or scanning the necessary information, the specified areas of the personal information are masked in a dark color such as black, white, or the same color as the background, preventing their visibility. This allows for personal information protection.

[0283] (2) This application is used in cases where completed forms are reused by erasing the completed sections and restoring the format to its pre-filled state. For example, educational facilities such as schools, kindergartens, and cram schools use handwritten homework sheets and final exam forms for student study and academic assessment, and students write their names and answers on the sheets. Because these sheets and test forms are old learning materials, the electronic data itself may no longer exist. A cram school teacher may reuse a completed printout (with permission to copy) that a student already owns. Or, a teacher at an educational institution may not even have the original electronic data (for example, a printed printout delivered by a teaching materials publisher with permission to copy is used as teaching material). In such cases, the paper can be reused by erasing only the written sections of the paper on which the student has handwritten notes and making a copy, or by saving a scanned image. This application can also be used in educational settings. Office workers can scan or copy handouts or meeting materials that they have been given permission to copy from conferences or seminars they have attended, and take them home to share with their colleagues. The process of erasing annotations can be time-consuming, requiring them to erase only the annotations they have made. Using this application, however, a simple operation on the control panel allows users to mask the annotations with white or the same color as the background, making them invisible. This allows users to copy or scan as if the annotations had been erased, allowing for reuse of the format paper.

[0284] (3) This application can be used to remove the yellowing from old, yellowed documents (such as drawings or books) and restore them to like-new condition. For example, in manufacturing, libraries, and other document management facilities, documents that have been stored for a long time and have become discolored or damaged by ultraviolet light can be restored to a more readable state by removing the yellowing and staining. Discolored areas can be whitened and faded text can be blackened to make them clearly readable, allowing the documents to be reused. [Explanation of symbols]

[0285] 10 Main Unit 11 Connection control section 12 Communication control section 13 Memory control unit 15 WebAPI Services 16 Reading scanner control unit 17 Plotter control unit 20 Operation Panel 21 Connection control section 22 Communication control section 23 Memory control unit 24 Input reception section 25 Display control unit 26 Copy output editing processing section 27 WebAPI Communication Department 261 Execution instruction section 262 Display section 263 Input Reception Unit 264 Generator 265 Image Processing Unit [Prior art documents] [Patent documents]

[0286] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-218836

Claims

1. Computer, a first display control step of displaying a preview of a first image acquired in response to an instruction from a user on a display unit; an input receiving step of receiving a masking input by a touch operation including a touch operation of tracing vertically and a touch operation of tracing horizontally by a user on the first image previewed on the display unit; a second display control step of previewing, on the display unit, a second image in which at least a part of the preview-displayed first image is masked with a line along the path traced by the touch operation on the first image, the second image being masked with a vertical line when the user traces the first image vertically with the touch operation, and masked with a horizontal line when the user traces the first image horizontally with the touch operation; a transmission step of transmitting the previewed second image to an external device when an instruction from a user is received while the previewed second image is being displayed on the display unit; to function as program.

2. the first display control step displays an edit operation screen including the first image previewed on the display unit; The editing operation screen displays the first image by enlarging or reducing it in response to an instruction from a user. The program according to claim 1.

3. The editing operation screen can accept designation of the line thickness, and the second display control step causes the display unit to display a preview of the second image masked with the line having the thickness designated on the editing operation screen. The program according to claim 2.

4. the editing operation screen includes a button for accepting designation of a color of the masking; The second display control step displays a preview of the second image masked with the masking color accepted by the button on the display unit. The program according to claim 2.

5. The second display control step displays the second image on a display unit, the second image being masked with a line obtained by correcting the traced trajectory. The program according to claim 2.

6. The editing operation screen includes a clear button, and when the clear button is operated by a user, the first image or the second image previewed on the display unit is deleted. The program according to any one of claims 3 to 5.

7. the first display control step displays an edit operation screen including the first image previewed on the display unit; the editing operation screen includes a masking color setting button, the program further includes a masking color receiving step of receiving, when the masking color setting button is touched, a touch operation on a predetermined position of the first image, including a color to be set as a masking color, from a user; The second display control step displays the second image masked based on the color of the predetermined position received in the masking color receiving step on the display unit. The program according to claim 1.

8. The method further includes an image receiving step of displaying a button for receiving an instruction to perform scanning on the display unit, The first display control step displays the first image acquired by performing a scan when the user operates the button on the display unit. The program according to claim 1.

9. The method further includes a storage step of detecting coordinates and a moving direction of an arbitrary position traced by the user on the first image by the touch operation in the input receiving step, and storing a line along the traced path in a storage area; The second display control step displays the first image and the masking stored in the storage area, thereby causing the display unit to preview the masked second image. The program according to claim 1.

Images