Image processing device, image processing method and program

Abstract

To remove a specified color in image data of a manuscript while suppressing undesirable reduction in image quality.SOLUTION: An image processing device includes an acquiring unit that acquires color image data of a manuscript and a removal unit that removes a specified color in the color image data by replacing a color of a pixel indicating the specified color with a background color. The image processing device further includes a determining unit that determines a graphic area in the color image data. The removal unit does not remove the specified color in the determined graphic area.SELECTED DRAWING: Figure 7B

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Even in today's digital age, there are still many opportunities to write characters and symbols with a writing implement on printed documents (e.g., handouts at meetings). On the other hand, when copying documents later to share with others, it may be desirable to erase the handwritten content. To meet such needs, it has been proposed to incorporate a so-called color dropout function, which removes colors specified by the user from image data of an optically read document, into devices such as scanners, copiers, and multifunction devices (see Patent Documents 1 and 2).

[0003] The color dropout function proposed in Patent Document 1 allows the user to specify the range of colors to be removed on a setting screen that displays a color plane, and converts the colors in the image data that fall within that range to white using a lookup table method. However, this method of uniformly removing the specified colors also removes the same colors that are included in non-handwritten parts, degrading the image quality of the scanned image. For example, when trying to erase characters written in red pen on a document, the red subject in a photograph inserted in the document may also be erased.

[0004] Patent Document 2 proposes that in order to avoid unintended degradation of image quality in photographic areas, halftone dot areas having gradation expression in image data are determined to be photographic areas, and color removal is not performed in the photographic areas. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-188484 A [Patent Document 2] JP 2016-139868 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, photo regions are not the only regions where degradation of image quality due to color removal is an issue. For example, multiple colors may be used in graphic elements such as graphs and figures that are often inserted in meeting materials. However, partial regions of graphic elements have almost uniform density when viewed locally, so they are not determined to be photo regions by the determination method based on gradation expression proposed in Patent Document 2, and are not excluded from the target of color removal. In addition, a user may not want to remove colors from partial regions other than the regions of graphic elements, or may want to remove colors only from specific partial regions.

[0007] The present invention has been made in view of the above problems, In the image of the original To convert the color of a target pixel having a color corresponding to a specified color among pixels included in an area other than a graphic area to a specific color without converting the color of pixels in the graphic area. Furthermore, a preview image can be displayed so that the graphic area can be distinguished as a non-removal area, and the user can modify the non-removal area to a removal area, and the graphic area that has been modified to a removal area can be converted to a specific color. The purpose is to provide a mechanism. [Means for solving the problem]

[0008] According to one aspect, the present invention comprises a reading means for reading an image of a document, a designation means for designating a color, a specification means for designating a graphic area within the image of the document, and a conversion means for converting a color of a target pixel, which has a color corresponding to the color designated by the designation means and is included in an area other than the graphic area, to a specific color without converting the color of the pixels within the graphic area designated by the specification means. a display control means for displaying a preview image based on an image of the document on a display means such that the graphic area specified by the specifying means is distinguishable by a user as a non-removal area, and a receiving means for receiving a correction by the user to change the non-removal area to a removal area; Have The conversion means converts the color of a pixel having a color corresponding to the color designated by the designation means in the graphic area corrected to the removal area to the specific color. An image processing apparatus is provided.

[0009] According to another aspect, there is provided an image processing method executed by an image processing device, comprising the steps of: reading an image of an original; specifying a color; and identifying a graphic area within the read image of the original; displaying a preview image based on an image of the document on a display means so that the specified graphic area can be identified by a user as a non-removal area, and accepting a correction by the user of the non-removal area to a removal area; converting the color of a target pixel having a color corresponding to the designated color among pixels included in an area other than the graphic area to a specific color without converting the color of the pixel in the specified graphic area; converting the color of pixels having a color corresponding to a specified color in the graphic region modified to the removal region to the specified color; There is provided an image processing method comprising the steps of: Effect of the Invention

[0010] According to the present disclosure, the color of a target pixel having a color corresponding to a specified color among pixels included in an area other than a graphic area is converted to a specific color without converting the color of pixels in the graphic area. and displaying a preview image so that the graphic area can be distinguished as a non-removal area, accepting a user's correction of the non-removal area to a removal area, and converting the graphic area corrected to a removal area to a specific color. can be done. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of a basic configuration of a multifunction peripheral. [Diagram 2] 1A and 1B are explanatory diagrams for explaining inconveniences of a conventional color dropout function. [Diagram 3] FIG. 2 is a block diagram showing an example of a logical configuration of the multifunction peripheral according to the first embodiment. [Figure 4] FIG. 2 is an explanatory diagram for explaining an example of a setting GUI that can be displayed on a screen in the first embodiment. [Diagram 5] FIG. 4 is an explanatory diagram for explaining a method for determining a graphic area according to the first embodiment. [Figure 6] 4 is a sequence diagram showing the flow of basic processing executed by the multifunction peripheral together with interactions between a user and the multifunction peripheral; [Figure 7A] 5 is a flowchart showing an example of the flow of image processing according to the first embodiment. [Figure 7B] 6 is a flowchart showing an example of a detailed flow of an area determination process according to the first embodiment. [Figure 8] FIG. 11 is a block diagram showing an example of a logical configuration of a multifunction peripheral according to a second embodiment. [Figure 9]FIG. 11 is an explanatory diagram for explaining an example of a setting GUI that can be displayed on a screen in the second embodiment. [Figure 10] 10 is a flowchart showing an example of a detailed flow of an area setting process according to the second embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of a logical configuration of a multifunction peripheral according to a third embodiment. [Figure 12] FIG. 13 is an explanatory diagram for explaining an example of a setting GUI that can be displayed on a screen in the third embodiment. [Figure 13] 13 is a flowchart showing an example of a detailed flow of an area setting process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0013] <<1. Introduction>> In this specification, an example in which the technology according to the present disclosure is applied to a multi-function peripheral (MFP) will be mainly described. However, the technology according to the present disclosure is not limited to such an example, and is broadly applicable to image processing devices in general (including, for example, copiers, printers, scanners, and facsimiles). Furthermore, unless otherwise specified, each of the components such as the apparatus, device, module, and chip described below may be composed of a single entity, or may be composed of multiple physically different entities.

[0014] <1-1. Basic configuration of the device> FIG. 1 is a block diagram showing an example of a basic configuration of a multifunction device. The multifunction device 100 shown in FIG. 1 includes a CPU 101, a ROM 102, a RAM 103, a storage unit 104, an operation unit 105, a scanner 111, a printer 112, a communication I / F 113, an image processing unit 115, and a bus 120. The CPU (Central Processing Unit) 101 is a processor for controlling the overall functions of the multifunction device 100. The ROM (Read Only Memory) 102 is a non-volatile memory that stores in advance a computer program to be executed by the CPU 101. The ROM 102 may be, for example, a flash memory. The RAM (Random Access Memory) 103 is a volatile memory that provides a working storage area for the CPU 101. The RAM 103 temporarily stores program data, control data, and image data to be processed, for example, when the computer program is executed by the CPU 101. The storage unit 104 is used as a so-called auxiliary storage device, and can store image data of many images and other data. The storage unit 104 may be, for example, a HDD (Hard Disk Drive). The operation unit 105 provides a user interface to the user of the multifunction device 100. The operation unit 105 includes, for example, a display device for displaying information and images, and an input device for receiving user input. The scanner 111 is a reading unit that generates image data by reading an original. Depending on the reading setting, the image data generated by the scanner 111 may be color image data or monochrome image data. The printer 112 is an image forming unit that forms an image based on image data specified in a job on a recording medium. In a copy mode, the image data generated by the scanner 111 is processed by an image processing unit 115 described later, and then an image based on the processed image data is formed by the printer 112 (i.e., the original is copied). In a print mode, an image based on image data specified by a user is formed by the printer 112 on a recording medium (i.e., the image is printed). The communication interface (I / F) 113 is an interface for the multifunction device 100 to communicate with other devices via a network.The image processing unit 115 is a processing circuit for performing various image processing such as color space conversion, size change, and noise reduction on image data output from the scanner 111 and image data input to the printer 112. Functions such as area determination and color removal, which will be described in relation to the embodiments described later, may be realized by either the CPU 101 or the image processing unit 115, or may be realized by both of them working together. The bus 120 is a signal line that mutually connects the CPU 101, ROM 102, RAM 103, storage unit 104, operation unit 105, scanner 111, printer 112, communication I / F 113, and image processing unit 115.

[0015] <1-2. Description of the assignment> The left side of FIG. 2 shows the appearance of an example of a document 210 to be read by the scanner 111. The document 210 is, for example, a handout for a meeting, and includes graphic elements 211 and 212 in addition to text. The graphic element 211 is a pie chart, and the graphic element 212 is a bar graph. Furthermore, the document 210 has handwritten characters 213 and symbols 214 and 215. The handwritten parts are in red, and parts of the graphic elements 211 and 212 are also in red. The right side of FIG. 2 shows an output image 220 in which red is specified as the color to be removed in a conventional color dropout function. In the output image 220, the characters 213 and symbols 214 and 215 are successfully removed, but the red parts of the graphic elements 211 and 212 are missing against the user's intention.

[0016] As will be explained in more detail in the next section, the technology disclosed herein provides a mechanism that can appropriately remove colors specified in the image data of a document while suppressing the degradation of image quality that is undesirable to the user.

[0017] <<2. First embodiment>> <2-1. Functional configuration> 3 is a block diagram showing an example of a logical configuration of the multifunction device 100 according to the first embodiment. Referring to FIG. 3, the multifunction device 100 includes a data acquisition unit 301, a setting unit 302, an area determination unit 303, and a removal unit 304.

[0018] (1) Data Acquisition Section Data acquisition unit 301 acquires color image data to which the color dropout function of this embodiment is to be applied. For example, data acquisition unit 301 may cause scanner 111 to read an original document and acquire color image data generated by scanner 111. Data acquisition unit 301 may also acquire color image data stored in storage unit 104 or color image data received from another device via communication I / F 113. Data acquisition unit 301 outputs the acquired color image data to area determination unit 303 and removal unit 304.

[0019] (2) Setting section The setting unit 302 acquires setting information related to the removal of color from color image data via the operation unit 105, and sets the acquired setting information in the removal unit 304. The setting information related to color removal includes at least a designated color to be removed from the color image data. For example, the setting unit 302 displays a setting GUI (Graphical User Interface) for receiving settings related to color removal on the screen of the operation unit 105, and acquires setting information based on a user input to the setting GUI. Fig. 4 shows an example of the setting GUI that may be displayed on the screen of the operation unit 105 in this embodiment.

[0020] A main menu 410 in the upper left of FIG. 4 is a menu that displays options for the main functions of the multifunction device 100. The main menu 410 includes buttons 411, 412, 413, 414, and 415. Button 411 is a button for calling up a "scan and send" function. Button 412 is a button for calling up a "scan and save" function. Button 413 is a button for calling up a "box" function (use of data already stored in a storage device). Button 414 is a button for calling up a "copy" function. Button 415 is a button for calling up a "print" function. 5 is a button for calling up the "print" function. The color removal mechanism described in this specification can be applied to any of these functions, but here, an example in which it is applied to the "copy" function will be described. When a user operates (e.g., touches) the button 414, the GUI transitions from the main menu 410 to a copy setting menu 420. The copy setting menu 420 includes, for example, color mode (full color / monochrome), magnification, paper size, and number of copies as basic setting items. The copy setting menu 420 also includes, for example, density, single-sided / double-sided, document type, and other settings as advanced setting items. When a user selects "full color" as the color mode and operates the button 421 ("other settings"), the GUI transitions from the copy setting menu 420 to the other setting menu 430. The other setting menu 430 includes buttons for calling up various application settings. For example, when a user operates the button 431 ("color removal"), the GUI transitions from the other setting menu 430 to the color removal setting menu 440.

[0021] The color removal setting menu 440 includes buttons 441, 442, 443, and 444 corresponding to a plurality of colors that can be selected as the color to be removed. The button 441 is a button for selecting red. The button 442 is a button for selecting green. The button 443 is a button for selecting blue. The button 444 is a button for selecting black. By operating a plurality of buttons, two or more colors may be simultaneously selected as the color to be removed. Note that the color candidates shown here are merely examples. More colors, fewer colors, or different colors may be selectable as the color to be removed. Note that the color removal-related settings accepted via the setting GUI are not limited to the example of FIG. 4 and may include further setting items. The further setting items may include, for example, one or more of the range of luminance to be removed, the range of density of the color to be removed, and the designation of the background color. The setting unit 302 sets the color removal-related setting information acquired via the color removal setting menu 440 in the removal unit 304.

[0022] (3) Area determination section The area determination unit 303 determines removal areas in which color removal should be performed within the color image data input from the data acquisition unit 301, or non-removal areas that should be excluded from the target of color removal. In this embodiment, the non-removal areas include at least graphic areas. A graphic area corresponds to an area in which a graphic element is drawn as described with reference to FIG. 2. The area determination unit 303 determines a photographic area within the color image data according to a known photographic area determination technique, and may treat both the graphic area and the photographic area as non-removal areas. In this embodiment, an area that is not determined as a non-removal area becomes a removal area.

[0023] The area determination unit 303 scans pixels in the color image data, which is a two-dimensional pixel array, for example, in a raster scan order, and sets each pixel as a pixel of interest and a pixel set of M rows and N columns including the pixel of interest as a determination window. Here, M and N are integers of 2 or more. The area determination unit 303 then determines whether a determination condition is met that the color variation in the determination window is below a predetermined standard. If the color variation in the determination window is below the predetermined standard, the area determination unit 303 determines that the pixel of interest belongs to a graphic area. Generally, a local area of ​​a graphic element is usually drawn on a recording medium with a uniform density (i.e., as a solid area). However, after being read by the scanner 111, the density may vary between pixels in the read image data depending on the reading accuracy and noise. Therefore, even if the color is not completely uniform, an area in which the color variation is below a predetermined standard is determined to be a graphic area, so that the graphic area can be appropriately determined.

[0024] As an example, the color variation in the judgment window falling below a predetermined standard may include the difference between any color component value of any pixel inside the judgment window and the corresponding average color component value calculated over the entire judgment window falling below a judgment threshold. For example, a portion handwritten with fluorescent ink or pencil has a tendency to have relatively large density unevenness for each pixel. Therefore, by determining whether there is a color component showing a large deviation from the average color component value inside the judgment window according to the above judgment method, such a handwritten portion can be accurately distinguished from a graphic area. Also, for example, the density unevenness of a portion handwritten with a ballpoint pen is not so large, but handwritten characters usually have a narrow line width. Therefore, by using a judgment window (a judgment window with a certain area) that is larger than such a narrow line width, handwritten characters can be accurately distinguished from a graphic area. In this way, the accuracy of the judgment of the graphic area is improved. Note that the color components in the above judgment method may be color components that constitute any color space, such as RGB, CMY, or YUV.

[0025] When the color variation for a certain judgment window falls below the above-mentioned standard, the area determination unit 303 may determine that not only the pixel of interest but also the neighboring pixels included in the judgment window belong to the graphic area. Once a pixel has been determined to belong to a graphic area, the area determination unit 303 may maintain the determination that the pixel belongs to the graphic area regardless of the color variation in other judgment windows. In other words, when two pixel sets (judgment windows) with different pixel of interest include some pixels in common, if the common pixels are determined to belong to the graphic area at least once during scanning, they are determined to belong to the graphic area as the final determination result. This prevents conflicts in judgment results between two or more judgment windows that overlap each other at the boundary portion of a graphic element, and allows for consistent graphic judgment.

[0026] FIG. 5 is an explanatory diagram for explaining the graphic region determination method in this embodiment. The upper part of FIG. 5 shows an enlarged view of the document 210 illustrated in FIG. 2 (however, the upper half of the document 210 is omitted). The lower left and lower right of FIG. 5 show pixel sets included in the determination window at the time when the scanning by the region determination unit 303 reaches the position 501 of the read image data (color image data) of the document 210, together with the respective pixel values. However, the position of the determination window 520 at the lower right advances by one pixel compared to the determination window 510 at the lower left. Here, as an example, N=M=3, and therefore each determination window consists of nine pixels. The three values ​​separated by commas shown inside the square of each pixel in the figure are the color component values ​​(R, G, B) of the pixel in the RGB color space.

[0027] The pixel set of the decision window 510 includes a target pixel 511 located in the center and eight neighboring pixels located to the upper left, upper, upper right, left, right, lower left, lower, and lower right of the target pixel 511. The average color component value (R AVE ,G AVE ,B AVE)=(251, 41, 10). Furthermore, among the color component values ​​of the pixels inside the judgment window, the one that has the largest difference from the corresponding average color component value is the B component of the pixel to the lower left of the pixel of interest 511, and the difference (absolute value) is equal to 12 (=22-10). Here, if the judgment threshold for judging a graphic region is 30, the above difference falls below the judgment threshold. Based on the result of this threshold comparison, the region judgment unit 303 judges that the pixel of interest 511 and the eight neighboring pixels inside the judgment window 510 belong to a graphic region.

[0028] The pixel set of the decision window 520 includes a pixel of interest 521 located in the center and eight neighboring pixels around the pixel of interest 521. The average color component value (R AVE ,G AVE ,B AVE )=(251, 111, 88). Among the color component values ​​of the pixels inside the judgment window, the pixel to the right of the pixel of interest 521 has the largest difference from the corresponding average color component value, and the difference (absolute value) is equal to 159 (=247-88), which exceeds the judgment threshold (=30). Here, the pixel of interest 521 and the neighboring pixels to the upper left, upper, left, lower left, and lower of the pixel of interest 521 have already been judged to belong to the graphic region based on the judgment result for the judgment window 510. Therefore, the region judgment unit 303 maintains the judgment that these pixels belong to the graphic region. On the other hand, the region judgment unit 303 provisionally judges that the neighboring pixels to the upper right, right, and lower right of the pixel of interest 521 do not belong to the graphic region. This provisional judgment result that the pixel does not belong to the graphic region may be overwritten by the result of a new judgment after the scanning progresses.

[0029] The area determination unit 303 repeatedly performs the area determination described above, for example, while sequentially scanning pixels in the color image data. The area determination unit 303 then registers pixels determined to belong to the graphic area as pixels of a non-removal area in memory (for example, by adding the pixels to a corresponding pixel list or by marking the pixels as pixels of a non-removal area). Finally, pixels that are not registered as pixels of a non-removal area constitute a removal area, and are subject to color removal by the removal unit 304, which will be described later.

[0030] 5 shows an example where a 3×3 pixel set constitutes the judgment window (i.e., M=N=3), but the size and shape of the judgment window are not limited to this example. In a modified example, the parameters M and N that affect the size of the judgment window may be set based on the reading resolution of the document read by the scanner 111. For example, even if the width of a handwritten line is the same, the number of pixels corresponding to the line width in the read image data becomes larger if the reading resolution is higher. Therefore, by setting the size of the judgment window larger as the document is read at a higher resolution, the possibility of erroneously determining that the handwritten portion is a non-removal area can be reduced.

[0031] In addition, in FIG. 5, an example is shown in which the pixel of interest moves one pixel at a time during scanning for region determination, but in order to speed up scanning, the pixel of interest is moved every n pixels in the row direction and every m rows in the column direction. Set up may be set (m and n are integers. 1 <m≦M,1<n≦N)。

[0032] (4)Removal part The removal unit 304 replaces the color of pixels indicating the color (hereinafter, referred to as the designated color) set by the setting unit 302 as the color to be removed in the color image data input from the data acquisition unit 301 with the background color, thereby removing the designated color. However, in this embodiment, the removal unit 304 does not remove (convert to background color) the color of pixels determined by the area determination unit 303 to belong to a non-removal area. The non-removal area here includes at least a graphic area. The background color may be defined in advance as, for example, white. The removal unit 304 may also treat a color specified by the user via the operation unit 105 as the background color, or may treat a color automatically recognized in the background area of ​​the color image data as the background color.

[0033] More specifically, while scanning the pixels in the color image data in, for example, raster scan order, the removal unit 304 determines whether each pixel belongs to the non-removal area determined by the area determination unit 303. Then, for a pixel that does not belong to the non-removal area, if the pixel value of the pixel indicates a designated color, the removal unit 304 replaces the pixel value of the pixel with the pixel value of the background color. By repeating such determination and replacement for all pixels in the removal area, it is possible to appropriately remove colors that should be removed in the image data of the document while suppressing color removal that is contrary to the user's intention.

[0034] <2-2. Processing flow> (1) Overall flow 6 is a sequence diagram showing the flow of basic processing executed by the multifunction device 100 together with the interaction between the user and the multifunction device 100. In the following description, a processing step will be abbreviated as S (step).

[0035] First, in S601, the operation unit 105 displays the main menu 410 on the screen. In S602, the user operates a button on the main menu 410 corresponding to the desired function to select the desired function. Here, it is assumed that the user selects the copy function. Next, in S603, the operation unit 105 displays the copy setting menu 420, which is a type of function menu, on the screen. In S604, the user inputs basic settings for the copy function (e.g., color mode, magnification, paper size, and number of copies) in the copy setting menu 420. The setting unit 302 acquires the basic setting values ​​accepted by the operation unit 105 and stores them in memory. Furthermore, in S606, the user operates the button 421 for "other settings" to perform advanced settings (applied settings). Next, in S607, the operation unit 105 displays the other setting menu 430, which is a type of applied setting menu, on the screen. A user who desires to remove a designated color in a copy of an original operates the button 431 for calling a color removal function in S608. Next, in S609, the operation unit 105 displays a color removal setting menu 440 on the screen. In S610, the user selects a color removal-related setting (e.g., one or more colors to be removed) in the color removal setting menu 440. In S611, the setting unit 302 acquires the value of the color removal-related setting accepted by the operation unit 105 and sets it in the removal unit 304. When the user completes the necessary settings, in S612, the user performs a predetermined operation (e.g., pressing a physical start key or touching a start button on a GUI) to instruct the multifunction device 100 to start execution of a job. When the operation unit 105 detects an instruction from the user, the CPU 101 causes the scanner 111 and the printer 112 to start execution of a copy job. In S613, the scanner 111 reads an original and generates color image data. Next, in S614, CPU 101 and image processing unit 115 execute image processing including the above-mentioned area determination and color removal on the generated color image data. In S615, the image data after the image processing is output to printer 112, and an image based on the image data is formed on a recording medium by printer 112.In addition, when the user selects the scan function instead of the copy function, the image data after image processing may be stored in a specified location or sent to a specified destination instead of forming an image on a recording medium. In S616, the result of the job execution is notified to the user on the screen.

[0036] (2) Details of image processing Fig. 7A is a flowchart showing an example of the flow of image processing that may be executed by the multifunction device 100 in S614 of Fig. 6. The image processing here may be realized, for example, by the CPU 101 executing a computer program loaded from the ROM 102 to the RAM 103 in cooperation with the image processing unit 115.

[0037] First, in S701, the setting unit 302 acquires color removal-related setting information including the designation of the color to be removed via the operation unit 105. Then, the setting unit 302 sets the acquired setting information in the removal unit 304. Next, in S703, the data acquisition unit 301 acquires color image data of the document. Then, the data acquisition unit 301 outputs the acquired color image data to the area determination unit 303 and the removal unit 304. Next, in S705, the area determination unit 303 executes an area determination process on the color image data to determine the removal area and the non-removal area in the color image data. A more detailed flow of the area determination process executed here will be further described later. Next, in S731, the removal unit 304 selects one pixel in the color image data as a pixel of interest. Next, in S733, the removal unit 304 determines whether the pixel of interest is a pixel in the non-removal area based on the result of the area determination process. The non-removal area here includes at least a graphic area. If it is determined that the pixel of interest is not a pixel in a non-removal region (is a pixel in a removal region), in S735, the removal unit 304 determines whether the pixel value of the pixel of interest indicates the designated color. For example, if the pixel value of the pixel of interest (a three-dimensional vector consisting of three color component values) is included in a subspace in a color space corresponding to the designated color, the removal unit 304 may determine that the pixel value of the pixel of interest indicates the designated color. If it is determined that the pixel value of the pixel of interest indicates the designated color, the removal unit 304 replaces the color of the pixel of interest with the background color in S737. On the other hand, if it is determined that the pixel value of the pixel of interest does not indicate the designated color, the removal unit 304 does not replace the color of the pixel of interest with the background color. If it is determined that the pixel of interest is a pixel in a non-removal region in S733, the processing steps of S735 and S737 are skipped. Next, in S739, the removal unit 304 determines whether any unprocessed pixels remain in the color image data. If unprocessed pixels remain, the process returns to S731, the next pixel is selected as the pixel of interest, and S733 to S739 are repeated. If unprocessed pixels remain, the image processing of FIG. 7A ends.

[0038] (3) Details of area determination process Fig. 7B is a flowchart showing an example of a detailed flow of area determination processing that may be executed by multifunction device 100 in S705 of Fig. 7A. The area determination processing here may be realized, for example, by CPU 101 executing a computer program loaded from ROM 102 to RAM 103 in cooperation with image processing unit 115.

[0039] First, in S711, the region determination unit 303 selects one pixel in the color image data as a pixel of interest. Next, in S713, the region determination unit 303 sets a judgment window on a pixel set including the pixel of interest. Next, in S715, the region determination unit 303 calculates, for example, average color component values ​​of each of the color components of RGB over the entire judgment window. Next, in S717, the region determination unit 303 determines whether there is a pixel inside the judgment window having a color component value whose difference from the corresponding average color component value exceeds a judgment threshold. If there is no such pixel inside the judgment window, in S719, the region determination unit 303 determines that the pixels inside the judgment window belong to the graphic region. On the other hand, for any color component value of any pixel inside the judgment window, if the difference between the color component value and the corresponding average color component value is below the judgment threshold, the processing step of S719 is skipped. Although not shown in the figure, in addition to determining whether the pixel of interest belongs to a graphic area in S719, the area determination unit 303 may also determine whether the pixel of interest belongs to another type of non-removal area (for example, a photographic area). Next, in S721, the area determination unit 303 determines whether any unprocessed pixels remain in the color image data. If any unprocessed pixels remain, the process returns to S711, the next pixel is selected as the pixel of interest, and S713 to S721 are repeated. If no unprocessed pixels remain, the area determination unit 303 determines in S723 that the pixels that were not determined to belong to a graphic area (and other types of non-removal areas) belong to a removal area. Then, the area determination process of FIG. 7B ends.

[0040] <<3. Second embodiment>> In the first embodiment, an example in which a graphic area is mainly treated as a non-removal area is described. In this section, as a second embodiment, an example in which a user can explicitly specify a non-removal area or a removal area on a screen is described.

[0041] <3-1. Functional configuration> 8 is a block diagram showing an example of a logical configuration of the multifunction peripheral 100 according to the second embodiment. Referring to FIG. 8, the multifunction peripheral 100 includes a data acquisition unit 801, a setting unit 802, and a removal unit 804.

[0042] (1) Data Acquisition Section The data acquisition unit 801 acquires color image data of a document in the same manner as the data acquisition unit 301 according to the first embodiment, and outputs the acquired color image data to a setting unit 802 and a removal unit 804 .

[0043] (2) Setting section The setting unit 802 acquires setting information related to the removal of color from color image data via the operation unit 105, and sets the acquired setting information in the removal unit 804. The setting information related to color removal includes a designated color designated by the user as a color to be removed. In addition, in this embodiment, the setting information related to color removal includes area information indicating a non-removal area or removal area designated by the user. For example, the setting unit 802 displays a GUI for accepting area designation on the screen of the operation unit 105 together with a preview image based on the color image data, and acquires area information based on a detected user input.

[0044] FIG. 9 shows an example of a GUI that may be displayed on the screen of the operation unit 105 in this embodiment. In the second embodiment, when the user operates the button 431 ("Color Removal") in the other setting menu 430 illustrated in FIG. 4, a color removal setting menu 940 illustrated in the upper left of FIG. 9 is displayed. The color removal setting menu 940 includes a check box 945 in addition to the buttons 441, 442, 443, and 444 corresponding to the color options. The check box 945 is a UI for allowing the user to select whether or not to display a preview when the job is started (before printing is performed). When the execution of a copy job is started with the preview display enabled ("Yes") selected, the operation unit 105 displays a preview image 951 based on the color image data of the document read by the scanner 111 on the screen 950. Then, the operation unit 105 accepts the designation of an area by the user on this screen. For example, the designation of an area may be performed by a two-point designation method in which the upper left and lower right vertices of a rectangular area are designated by a drag operation or two touch operations. The area may be specified by a multi-point specification method in which the vertices of the polygonal area are specified in order by a plurality of touch operations. An area 962 specified by a two-point specification method and an area 963 specified by a multi-point specification method are shown in the lower right of FIG. 9. The operation unit 105 displays an object for masking each area superimposed on the preview image 951 so that the specified area can be distinguished by the user. Here, the areas 962 and 963 are specified as non-removal areas that are not subject to the execution of color removal. The setting unit 802 sets the area information acquired through such area specification on the preview image as color removal-related setting information in the removal unit 804.

[0045] (3)Removal section The removal unit 804 removes the designated color by replacing the color of pixels indicating the designated color with the background color in the color image data input from the data acquisition unit 801 based on the area information set by the setting unit 802. In a first example of the second embodiment, the area information indicates a non-removal area specified by the user. In this case, the removal unit 304 does not remove the designated color in the non-removal area, and removes the designated color in the remaining area. In a second example of the second embodiment, the area information indicates a removal area specified by the user. In this case, the removal unit 304 removes the designated color only in the removal area. As described above, the background color may be predefined as, for example, white, may be specified by the user, or may be automatically determined based on the input color image data.

[0046] More specifically, while scanning pixels in the color image data, the removal unit 804 determines whether each pixel belongs to the area indicated by the area information. In the first embodiment, the removal unit 804 replaces a pixel value of a pixel that does not belong to a non-removal area indicated by the area information with a pixel value of the background color when the pixel value of the pixel indicates a designated color. In the second embodiment, the removal unit 804 replaces a pixel value of a pixel that belongs to a removal area indicated by the area information with a pixel value of the background color when the pixel value of the pixel indicates a designated color. By repeating such determination and replacement, it is possible to appropriately remove colors to be removed from the image data of the document while reliably avoiding the execution of color removal contrary to the explicit intention of the user.

[0047] <3-2. Processing flow> The basic process flow executed by the multifunction device 100 in the second embodiment may be substantially the same as the process flow described with reference to Fig. 6. However, in the second embodiment, after the document is read in S613, at the beginning of the image processing in S614, the operation unit 105 displays a preview image and accepts area designation.

[0048] The flow of image processing corresponding to S614 in Fig. 6 in the second embodiment may be substantially similar to the flow described using Fig. 7A. However, in the second embodiment, in S705, instead of the area determination process shown in Fig. 7B, the setting unit 802 performs the area setting process shown in Fig. 10. Also, in S733, it is determined whether or not the pixel of interest is a pixel in a non-removal area based on area information indicating an area designated by the user.

[0049] 10 is a flowchart showing an example of a detailed flow of an area setting process that can be executed by the multifunction device 100 in the second embodiment. The area setting process here can be realized, for example, by the CPU 101 executing a computer program loaded from the ROM 102 to the RAM 103 in cooperation with the image processing unit 115.

[0050] First, in S1001, the setting unit 802 displays a preview image (e.g., a low-resolution image) based on the color image data of the document on the screen of the operation unit 105. Next, in S1003, the operation unit 105 accepts area designation by the user, for example, by the method described with reference to FIG. 9. The area designated here is either a non-removal area or a removal area. Next, in S1005, the setting unit 802 obtains area information indicating the designated area, and sets the obtained area information in the removal unit 804. Then, the area setting process of FIG. 10 ends.

[0051] <<4. Third embodiment>> In the third embodiment, it is determined whether or not to remove the color of each pixel based on both the result of determining the graphic region and an explicit region designation by the user.

[0052] <4-1. Functional configuration> 11 is a block diagram showing an example of a logical configuration of a multifunction peripheral 100 according to the third embodiment. Referring to FIG. 11, the multifunction peripheral 100 includes a data acquisition unit 1101, a setting unit 1102, an area determination unit 1103, and a removal unit 1104.

[0053] (1) Data Acquisition Section A data acquisition unit 1101 acquires color image data of an original in the same manner as the data acquisition unit 301 according to the first embodiment, and outputs the acquired color image data to a setting unit 1102 , an area determination unit 1103 , and a removal unit 1104 .

[0054] (2) Area determination section The area determination unit 1103 determines a non-removal area to be excluded from the target of color removal within the color image data input from the data acquisition unit 1101. In this embodiment, the non-removal area includes at least a graphic area. The area determination unit 1103 may determine the graphic area within the color image data by a method similar to that of the area determination unit 303 according to the first embodiment. The non-removal area may further include a photographic area. The area determination unit 1103 outputs the result of the area determination to the setting unit 1102 and the removal unit 1104.

[0055] (3) Setting section The setting unit 1102 acquires setting information related to the removal of color from color image data via the operation unit 105, and sets the acquired setting information in the removal unit 1104. The setting information related to color removal includes a designated color designated by the user as a color to be removed. In addition, in this embodiment, the setting information related to color removal includes area information indicating a non-removal area or a removal area. For example, the setting unit 1102 displays a preview image on the screen of the operation unit 105 so that the graphic area determined by the area determination unit 1103 can be distinguished by the user on the screen. Then, the setting unit 1102 acquires area information corrected based on a user input detected on the screen.

[0056] FIG. 12 shows an example of a GUI that may be displayed on the screen of the operation unit 105 in this embodiment. In the third embodiment, the operation unit 105 also displays a color removal setting menu 940 having a check box 945 as shown in FIG. 9 on the screen, and allows the user to select whether or not to display a preview when starting a job. When the execution of a copy job is started with the preview display enabled, the operation unit 105 displays a preview image 1251 based on color image data of the document read by the scanner 111 on the screen 1250, as shown on the left in FIG. 12. In this example, in the preview image 1251, areas 1252, 1253, 1254, and 1255 that are determined to be graphic areas are masked so that they can be distinguished by the user. Instead of being masked and concealed, each area may be surrounded by a line indicating the boundary of the area, or a semi-transparent object may be superimposed thereon. Unless the user releases these areas, they are treated as non-removal areas in the subsequent color removal process. The operation unit 105 accepts the user's cancellation or additional designation of a non-removal region on this screen. For example, cancellation of a non-removal region may be performed by a touch operation on a region determined to be a graphic region. Additional designation of a region may be performed by the above-mentioned two-point designation method or multi-point designation method. On the right side of FIG. 12, a preview image 1251 after correction by the user is shown, in which region 1252 is no longer a non-removal region. Meanwhile, region 1256 has been newly and additionally designated as a non-removal region. The setting unit 1102 sets region information acquired as a result of such correction of the non-removal region in the preview image in the removal unit 1104 as color removal-related setting information.

[0057] (4)Removal part The removal unit 1104 removes the designated color by replacing the color of pixels indicating the designated color with the background color in the color image data input from the data acquisition unit 1101 based on the region information set by the setting unit 1102. As described above, the background color may be, for example, white, a color specified by the user, or a color determined automatically.

[0058] For example, the removal unit 1104 treats an area determined by the area determination unit 1103 to be a graphic area, but for which the setting as a non-removal area has not been cancelled by the user, as a non-removal area, and does not remove the designated color in the area. On the other hand, the removal unit 1104 treats an area determined by the area determination unit 1103 to be a graphic area, but for which the setting as a non-removal area has been cancelled by the user, as a removal area, and removes the designated color in the area. Also, the removal unit 1104 treats an area not determined to be a graphic area, but for which the user has added a designation as a non-removal area, as a non-removal area, and does not remove the designated color in the area. On the other hand, the removal unit 1104 treats an area not determined to be a graphic area and for which the designation as a non-removal area has not been added, as a removal area, and removes the designated color in the area. By such a combination of area determination and user designation, it is possible to appropriately remove colors to be removed in the image data of the document while reducing the burden on the user for area designation operations and reliably avoiding the execution of color removal contrary to the user's intention.

[0059] <4-2. Processing flow> The basic process flow executed by the multifunction device 100 in the third embodiment may be substantially similar to the process flow described with reference to Fig. 6. However, in the third embodiment, after the document is read in S613, at the beginning of the image processing in S614, the operation unit 105 displays a preview image and accepts corrections to area information.

[0060] The flow of image processing corresponding to S614 in Fig. 6 in the third embodiment may be substantially the same as the flow described using Fig. 7A. However, in the third embodiment, as shown in Fig. 13, after the area determination process in S705, the setting unit 1102 performs area setting process. Also, in the third embodiment, in S733, it is determined whether or not the pixel of interest is a pixel in a non-removal area based on the result of area determination by the area determination unit 1103 and the result of correction of area information by the user.

[0061] 13 is a flowchart showing an example of a detailed flow of an area setting process that can be executed by the multifunction device 100 in the third embodiment. The area setting process here can be realized, for example, by the CPU 101 executing a computer program loaded from the ROM 102 to the RAM 103 in cooperation with the image processing unit 115.

[0062] FIG. 13 shows that the area determination unit 1103 performs the area determination process (S705) described with reference to FIG. 7B prior to the area setting process. Next, in S1301, the setting unit 1102 displays a preview image on the screen of the operation unit 105 so that an area determined to be a graphic area as a result of the area determination process (i.e., a provisional non-removal area) can be distinguished on the screen. Next, in S1303, the operation unit 105 accepts the release of the non-removal area or the additional designation of a non-removal area based on a user input, as described with reference to FIG. 12. The subsequent process branches depending on whether the operation accepted in S1305 indicates the release of the non-removal area or indicates the additional designation of a non-removal area. When the release of the non-removal area is accepted, in S1307, the operation unit 105 releases the masking of the specified non-removal area on the screen. In this case, the setting for the area is modified from the non-removal area to the removal area. On the other hand, if an additional designation of a non-removal area is accepted, in S1309, the operation unit 105 masks the designated area on the screen as a new non-removal area. In this case, the setting for that area is corrected from a removal area to a non-removal area. Such user interaction continues until the correction of the area information is completed (for example, the user touches the "OK" button) (S1311). When the correction of the area information is completed, in S1313, the setting unit 1102 acquires the corrected area information as setting information related to color removal, and sets the acquired setting information in the removal unit 1104. Then, the area setting process in FIG. 13 ends.

[0063] <<5. Summary>> Up to this point, several embodiments have been described in detail with reference to Figures 1 to 13. In one embodiment, in an image processing device that removes a designated color by replacing pixel colors with a background color in color image data of an original, graphic areas in the color image data are determined, and the determined graphic areas are excluded from targets for color removal. With this configuration, for image data of an original that includes graphic areas, content written in a designated color can be appropriately removed from the image data while suppressing undesirable degradation of image quality.

[0064] In another embodiment, in an image processing device that removes a specified color by replacing the color of a pixel in color image data of an original with a background color, a user's designation of an area in a preview image is accepted, and color removal is controlled based on the area designation. With this configuration, it is possible to appropriately remove colors to be removed from image data while reliably avoiding the execution of color removal contrary to the explicit intention of the user.

[0065] In another embodiment, whether or not to perform color removal is switched based on both the determination of the graphic area and the area designation by the user. With this configuration, for example, if the user wants to remove a specific color even within a graphic area, color removal can be performed as the user desires. Also, for an area that is not a graphic area but that the user wants to exclude from the execution of color removal, the execution of color removal can be avoided.

[0066] <<6. Other embodiments>> The above-mentioned embodiment can also be realized in the form of a process in which a program for realizing one or more functions is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0067] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0068] 100: image processing device (multifunction device), 105: operation unit, 111: reading unit (scanner), 112: image forming unit (printer), 301, 801, 1101: data acquisition unit, 302, 802, 1102: setting unit, 303, 1103: area determination unit, 304, 804, 1104: removal unit

Claims

1. A reading means for reading an image of a document; A designation means for designating a color; means for identifying a graphic area within an image of the document; a conversion means for converting a color of a target pixel, which has a color corresponding to the color specified by said designation means and is included in an area other than said graphic area, into a specified color without converting the color of the pixel in said graphic area specified by said designation means; a display control means for controlling a display means to display a preview image based on an image of the document so that the graphic area specified by the specifying means is distinguishable by a user as a non-removal area; a receiving unit for receiving a correction by the user to change the non-removal area to a removal area; having The image processing device according to claim 1, wherein the conversion means converts the color of a pixel having a color corresponding to the color specified by the designation means in the graphic area corrected to the removal area, into the specific color.

2. 2. The image processing device according to claim 1, wherein the conversion means converts the color of pixels of characters having a color corresponding to the color specified by the designation means to the specified color without converting the color of pixels in the graphic area specified by the specification means.

3. 3. The image processing device according to claim 1, wherein the specifying means determines that the pixel of interest belongs to the graphic area when a color variation in a pixel set including the pixel of interest is smaller than a predetermined standard.

4. 4. The image processing device of claim 3, wherein the color variation in the pixel set being less than the predetermined standard includes, for any color component value of any pixel in the pixel set, the difference between that color component value and a corresponding average color component value calculated over the pixel set being less than a threshold value.

5. 4. The image processing device according to claim 3, wherein the identification means determines that the first pixel and the second pixel belong to the graphic area when color variation in a first pixel set including a first pixel as a pixel of interest and a second pixel as a neighboring pixel is smaller than the predetermined standard, and maintains the determination that the second pixel belongs to the graphic area regardless of color variation in a second pixel set different from the first pixel set and including the second pixel.

6. 6. The image processing apparatus according to claim 3, wherein the size of the pixel set is set based on a reading resolution of the document.

7. The image processing device according to claim 1 , wherein the specific color is white.

8. The receiving means receives a designation of adding a non-removal region by a user, 2. The image processing apparatus according to claim 1, wherein said conversion means does not convert the color of pixels within said designated non-removal region.

9. 9. The image processing apparatus according to claim 1, further comprising a printing means for printing, on a sheet, an image in which the color of the target pixel has been converted to the specific color by the conversion means.

10. An image processing method executed by an image processing device, comprising: Scanning an image of an original; Specify the color, Identifying a graphic area within the scanned image of the original document; displaying a preview image based on the image of the document on a display means so that the specified graphic area can be distinguished by a user as a non-removal area; accepting a modification of the non-removal region to a removal region by the user; converting the color of a target pixel having a color corresponding to the designated color among pixels included in an area other than the graphic area to a specific color without converting the color of the pixel in the specified graphic area; converting the color of pixels having a color corresponding to a specified color in the graphic region modified to the removal region to the specified color; 13. An image processing method comprising:

11. A program for causing a computer of an image processing apparatus to execute the image processing method according to claim 10.

Images