Image processing device, image processing method and program

Abstract

To provide an image processing apparatus and method, and a program that appropriately detect an abnormal pixel area and correct the detected abnormal pixel area.SOLUTION: An image processing apparatus comprising an acquisition unit acquiring an image and a control unit is applied as an image forming apparatus 10. The control unit detects a target pixel in the image acquired by the acquisition unit as a candidate for an abnormal pixel based on a change in a plurality of pixels in a main scanning direction to which the target pixel belongs; when the pixel detected as the candidate for the abnormal pixel has continuity in a sub scanning direction, detects an area in the sub scanning direction including the pixel detected as the candidate for the abnormal pixel as an abnormal pixel area; and corrects the abnormal pixel based on a result of detection of the abnormal pixel area.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and the like. [Background technology]

[0002] Conventionally, when a user uses an image processing device such as a multifunction peripheral, the user transports a document using an automatic document feeder (ADF) and then scans the document using an image reading device such as a scanner. In this case, dust or debris on the glass or mirror can cause so-called scanner streaks in the scanned image.

[0003] Various techniques for detecting and correcting scanner streaks in scanned images have been proposed. For example, one proposed technique determines a line noise pixel based on the displacement of the RGB component readout values ​​relative to neighboring pixels in the document feed direction when the pixels have a difference of a predetermined value or more in the readout values ​​of the RGB components relative to the pixels in the surrounding area, and corrects the readout values ​​of the RGB components of the line noise pixel based on the readout values ​​of the RGB components of the pixels in the surrounding area (see, for example, Patent Document 1). Another proposed technique calculates the density change rate in the main scanning direction, determines potential streak pixels based on the density change rate between a pixel of interest and a pixel a predetermined distance away from the pixel of interest, and determines that a streak image has occurred if there are consecutive potential streak pixels in the sub-scanning direction that exceed a threshold value (see, for example, Patent Document 2). Another proposed technique detects the background color, selects foreign substances to be corrected based on the density, and performs density correction by interpolation based on the detected density (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4591531 [Patent Document 2] Patent No. 5008918 [Patent Document 3] Japanese Patent Application Laid-Open No. 2016-066836 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, scanner streaks may be caused by foreign matter such as paper dust on the glass. Furthermore, paper dust or the like may move when the document passes through the document feeder. Scanner streaks caused by such moving foreign matter may change in their location with each scan. Therefore, the image processing device must detect abnormal pixel areas that constitute scanner streaks from the scanned image in real time and correct the abnormal pixel areas. However, this has not been taken into consideration in the prior art, such as the patent documents mentioned above.

[0006] In view of the above-described problems, an object of the present disclosure is to provide an image processing device and the like that appropriately detects an abnormal pixel region and corrects the detected abnormal pixel region. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the image processing device of the present disclosure includes an acquisition unit that acquires an image, and a control unit, and the control unit detects a target pixel as a candidate abnormal pixel based on pixel changes in multiple pixels in the main scanning direction to which the target pixel belongs, from the image acquired by the acquisition unit, and if the pixels detected as the candidate abnormal pixel have continuity in the sub-scanning direction, detects an area in the sub-scanning direction that includes the pixel detected as the candidate abnormal pixel as an abnormal pixel area, and corrects the abnormal pixel based on the detection result of the abnormal pixel area.

[0008] The image processing method of the present disclosure is characterized by including the steps of: acquiring an image; detecting a pixel of interest in the image as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs; if the pixels detected as the candidate for an abnormal pixel have continuity in the sub-scanning direction, detecting an area in the sub-scanning direction including the pixel detected as the candidate for an abnormal pixel as an abnormal pixel area; and correcting the abnormal pixel based on the detection result of the abnormal pixel area.

[0009] The program of the present disclosure is characterized by causing a computer to realize the following functions: a function of acquiring an image; a function of detecting a target pixel of the image as a candidate for an abnormal pixel based on pixel changes in multiple pixels in the main scanning direction to which the target pixel belongs; a function of detecting, if the pixels detected as the candidate for an abnormal pixel have continuity in the sub-scanning direction, an area in the sub-scanning direction that includes the pixels detected as the candidate for an abnormal pixel as an abnormal pixel area; and a function of correcting the abnormal pixel based on the detection result of the abnormal pixel area. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to provide an image processing device or the like that appropriately detects an abnormal pixel region and corrects the detected abnormal pixel region. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a perspective view of an appearance of an image forming apparatus according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the functional configuration of the image forming apparatus according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a data configuration of parameter information in the first embodiment. [Figure 4] FIG. 3 is a diagram illustrating an example of the data configuration of area separation information in the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a data configuration of a provisional scanner line determination result in the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of the data configuration of a scanner line determination result in the first embodiment. [Figure 7] FIG. 3 is a flowchart showing the flow of main processing in the first embodiment. [Figure 8] FIG. 10 is a flowchart showing the flow of first scanner line detection processing in the first embodiment. [Figure 9] FIG. 10 is a flowchart showing the flow of first scanner line detection processing in the first embodiment. [Figure 10] FIG. 10 is a flowchart showing the flow of second scanner line detection processing in the first embodiment. [Figure 11] FIG. 4 is a flowchart showing the flow of scanner streak correction processing in the first embodiment. [Figure 12] 10A and 10B are diagrams for explaining a case where scanner streaks occur. [Figure 13] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 14] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 15] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 16] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 17] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 18] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 19] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 20] FIG. 4 is a diagram illustrating an example of operation in the first embodiment. [Figure 21] FIG. 10 is a flowchart showing the flow of scanner streak correction processing in the second embodiment. [Figure 22] FIG. 10 is a diagram illustrating an example of operation in the second embodiment. [Figure 23] FIG. 10 is a diagram illustrating an example of operation in the second embodiment. [Figure 24] FIG. 10 is a diagram illustrating an example of operation in the second embodiment. [Figure 25]FIG. 10 is a diagram illustrating an example of operation in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment for carrying out the present disclosure will be described with reference to the drawings. Note that the following embodiment is an example for explaining the present disclosure, and the technical scope of the invention described in the claims is not limited to the following description.

[0013] [1. First embodiment] First, a first embodiment will be described. In the first embodiment, a case will be described in which an image processing device according to the present disclosure is applied to an image forming device 10. The image forming device 10 is an information processing device having a copy function, a scan function, a document print function, etc., and is also called an MFP (Multi-Function Printer / Peripheral, or multifunction device).

[0014] [1.1 Functional Configuration] The functional configuration of an image forming apparatus 10 of this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is an external perspective view of the image forming apparatus 10, and Fig. 2 is a block diagram showing the functional configuration of the image forming apparatus 10.

[0015] As shown in FIG. 2, the image forming apparatus 10 includes a control unit 100, an image input unit 120, an image forming unit 130, a display unit 140, an operation unit 150, a storage unit 160, and a communication unit 190.

[0016] The control unit 100 is a functional unit for controlling the entire image forming apparatus 10. The control unit 100 realizes various functions by reading and executing various programs stored in the storage unit 160, and is configured, for example, by one or more arithmetic units (CPUs (Central Processing Units)). The control unit 100 may also be configured as an SoC (System on a Chip) having multiple functions among those described below.

[0017] The control unit 100 executes a program stored in the storage unit 160 to function as an image processing unit 102 and a region separation processing unit 104 .

[0018] The image processing unit 102 performs various image-related processes. For example, the image processing unit 102 performs sharpening and tone conversion processes on the image of the original document that is read and input by the image input unit 120.

[0019] The region separation processing unit 104 executes region separation processing. Region separation processing is processing for separating an image into predetermined regions. For example, the region separation processing unit 104 separates the image into regions of various types, such as a character region, a line region, a halftone dot region, and a background region. Note that the types of regions separated by the region separation processing unit 104 are not limited to the above-mentioned types, and regions such as halftone dot character regions may also be separated. Furthermore, the region separation processing unit 104 may determine the color of the characters included in the character regions to separate regions such as a region including black characters (black character regions) and a region including chromatic characters such as red (color character regions) from the image.

[0020] The image input unit 120 is an image acquisition unit that acquires an image and inputs the acquired image to the image forming apparatus 10. For example, the image input unit 120 is configured with a document feeder (ADF, Automatic Document Feeder) and a scanner device that reads an image of a document. The scanner device is a device that converts an image into an electrical signal using an image sensor such as a CCD (Charge Coupled Device) or a CIS (Contact Image Sensor), and quantizes and encodes the electrical signal. When the image of the document is read by the scanner device, the image of the document is input to the image forming apparatus 10 as digital data.

[0021] Image forming unit 130 forms (prints) an image on a recording medium such as recording paper. Image forming unit 130 is configured, for example, by a printing device such as a laser printer that uses an electrophotographic method. Image forming unit 130, for example, feeds recording paper from paper feed tray 132 in FIG. 1, forms an image on the surface of the recording paper, and discharges the recording paper from paper discharge tray 134.

[0022] The display unit 140 displays various types of information. The display unit 140 is configured by a display device such as an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, or a micro LED (Light Emitting Diode) display.

[0023] The operation unit 150 accepts operation instructions from a user who uses the image forming apparatus 10. The operation unit 150 is configured with input devices such as key switches (hard keys) and touch sensors. The touch sensor may detect input by contact (touch) using any common detection method, such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitance method. The image forming apparatus 10 may be equipped with a touch panel in which the display unit 140 and the operation unit 150 are integrally formed.

[0024] The storage unit 160 stores various programs and various data necessary for the operation of the image forming apparatus 10. The storage unit 160 is configured by a storage device such as a solid state drive (SSD) or a hard disk drive (HDD), which is a semiconductor memory.

[0025] The storage unit 160 secures a parameter information storage area 162 as a storage area, and further stores region separation information 164, provisional scanner line determination result information 166, and scanner line determination result information 168.

[0026] The parameter information storage area 162 stores information on various parameters (parameter information). For example, as shown in Fig. 3, the parameter information includes a parameter name (e.g., "Th1") and a parameter value (e.g., "80") corresponding to the parameter name.

[0027] In this embodiment, four types of parameter values, Th0, Th1, Th2, and Th3, are stored. The parameter values ​​may be predetermined or may be set by the user. Furthermore, the parameter values ​​may be individual values ​​for each image forming apparatus.

[0028] Th0 is a threshold value for the pixel value when determining that the pixel of interest is a white pixel. Th1 and Th2 are threshold values ​​for the density difference between the pixel of interest and its surrounding pixels. Th0, Th1, and Th2 are used in the first scanner line detection process to detect candidate abnormal pixels. The first scanner line detection process will be described later.

[0029] In this embodiment, the abnormal pixels are pixels that form the scanner lines. Also, in this embodiment, the pixels that are candidates for the abnormal pixels are also called provisional scanner lines.

[0030] Th3 is a threshold value for the continuity in the sub-scanning direction of pixels that are candidates for abnormal pixels, and is used in the second scanner streak detection process described later.

[0031] The region separation information 164 is information indicating the result of the region separation process (region separation result). The region separation information 164 may be any information that can identify the range of a region and the type of the region within the range, and may be, for example, a table that stores values ​​indicating the type of region for each pixel of an image, as shown in FIG.

[0032] Here, the position of a pixel in an image is expressed by coordinates, which are expressed as (x, y) where the upper left pixel of the image is the origin, the number of pixels in the main scanning direction between the origin and the pixel of interest is x, and the number of pixels in the sub-scanning direction is y.

[0033] The value indicating the type of area is, for example, an integer value. In this case, the integer value corresponds to the type of area, such as "0" indicating a character area, "1" a line area, "2" a halftone dot area, and "3" a halftone dot character area. For example, P100 in Figure 4 has a value of "1" for the pixel at position (x,y)=(10,3) in the image, which indicates that the pixel is included in the line area.

[0034] The provisional scanner line determination result information 166 is information indicating, for each pixel of the image, the determination result as to whether or not the pixel is a candidate pixel for an abnormal pixel.

[0035] The provisional scanner line determination result information 166 is a table that stores, for example, a value indicating whether or not each pixel of an image is a candidate for an abnormal pixel, as shown in Fig. 5. In this embodiment, the value stored in the provisional scanner line determination result information 166 is one of the following values. Wh: Indicates that the pixel is not a candidate for an abnormal pixel and is a white pixel. ×: Indicates that the pixel is not a candidate for an abnormal pixel and is a non-white pixel. 1R, 1G, 1B: Indicates that the pixel is a candidate for an abnormal pixel.

[0036] Note that "1R" indicates a pixel whose red component density difference from surrounding pixels exceeds Th1 and whose other color component density difference is less than Th2. "1G" indicates a pixel whose green component density difference from surrounding pixels exceeds Th1 and whose other color component density difference is less than Th2. "1B" indicates a pixel whose blue component density difference from surrounding pixels exceeds Th1 and whose other color component density difference is less than Th2.

[0037] For example, P102 in FIG. 5 indicates that the value for the pixel at the position (x, y)=(10, 3) in the image is "x", and therefore the pixel is not a candidate for an abnormal pixel.

[0038] The scanner streak determination result information 168 is information indicating the result of determining whether or not scanner streaks have occurred on each line (pixel row) in the sub-scanning direction of the image. For example, as shown in FIG. 6, the scanner streak determination result information 168 is a table storing a line number (e.g., "0") indicating the position (row) of the line in the sub-scanning direction of the image and a value (e.g., "Yes") indicating the determination result of whether or not scanner streaks have occurred on that line, in association with each other. The value indicating whether or not scanner streaks have occurred may be any information that can distinguish whether or not scanner streaks have occurred. Therefore, either "Yes" or "No" may be stored, or either "0" or "1" may be stored, or a value other than Null or Null may be stored. For example, in P104 in FIG. 6, the value corresponding to the line number "8" is "Yes," indicating that scanner streaks have occurred on the eighth line of the image.

[0039] The communication unit 190 communicates with external devices via a LAN (Local Area Network) or a WAN (Wide Area Network). The communication unit 190 is configured by, for example, a communication device or a communication module such as a NIC (Network Interface Card) used in a wired / wireless LAN.

[0040] [1.2 Processing flow] The flow of processing executed by the image forming apparatus 10 in this embodiment will be described with reference to Figures 7 to 11. The processing shown in Figures 7 to 11 is executed by the control unit 100 reading out a program stored in the storage unit 160. The processing shown in Figures 7 to 11 is also executed when a user performs an operation to start the execution of a job that reads a document, such as a copy job or a scan job.

[0041] [1.2.1 Main Processing] First, the flow of the main processing will be described with reference to Fig. 7. Control unit 100 acquires an input image (for example, a scanned image of a document) (step S100). The input image is an image input to image forming apparatus 10. For example, control unit 100 controls image input unit 120 to read a document conveyed by a document feeder and acquire a scanned image of the document as an input image. Control unit 100 may acquire the input image from an external device via communication unit 190, or may acquire the input image by reading an image stored in storage unit 160, or may acquire the input image from a storage medium such as a USB (Universal Serial Bus) memory.

[0042] Next, the control unit 100 (segment separation processing unit 104) executes a segmentation process on the input image (step S102). The segmentation processing unit 104 stores the result of the segmentation process in the storage unit 160 as segmentation information 164.

[0043] Next, the control unit 100 executes a first scanner line detection process (step S104). The first scanner line detection process is a process for detecting pixels that are candidates for abnormal pixels from the input image based on pixel changes in multiple pixels in the main scanning direction. For example, the control unit 100 provisionally determines, for each pixel of the input image, whether or not it is a candidate for an abnormal pixel that constitutes a scanner line, based on changes in pixel value (density or luminance) of multiple pixels in the main scanning direction, changes in hue, and changes in the region separation results. The details of the first scanner line detection process will be described later. The control unit 100 stores the results of the first scanner line detection process in the storage unit 160 as provisional scanner line determination result information 166.

[0044] Next, the control unit 100 executes a second scanner line detection process (step S106). The second scanner line detection process is a process for detecting a line in the sub-scanning direction where a scanner line occurs, based on whether or not pixels detected as candidate abnormal pixels have continuity in the sub-scanning direction by evaluating the provisional scanner line determination result information 166. Details of the second scanner line detection process will be described later. The control unit 100 stores the result of the second scanner line detection process in the storage unit 160 as scanner line determination result information 168. In this embodiment, a region formed by a row of pixels constituting a line in the sub-scanning direction where a scanner line occurs is referred to as an abnormal pixel region. In other words, the second scanner line detection process is a process for detecting an abnormal pixel region.

[0045] Next, the control unit 100 executes a scanner streak correction process to correct scanner streaks that have appeared in the input image based on the scanner streak determination result information 168 (step S108). As a result, the control unit 100 corrects abnormal pixels included in the abnormal pixel region based on the detection result of the abnormal pixel region. The scanner streak correction process will be described later.

[0046] Next, control unit 100 outputs the input image corrected in step S108 (step S110). For example, control unit 100 controls image forming unit 130 to form and output the corrected input image. Note that control unit 100 may output the data of the corrected input image by storing it in storage unit 160 or by transmitting it to another device.

[0047] [1.2.2 First scanner muscle detection process] The flow of the first scanner line detection process will be described with reference to Figures 8 and 9. The first scanner line detection process may be executed while the input image is being acquired, or may be executed after the input image is acquired. In this embodiment, as an example of the first scanner line detection process, a case will be described in which pixels that are candidates for abnormal pixels are detected based on changes in pixel values ​​(density) of multiple pixels in the main scanning direction.

[0048] First, the control unit 100 selects one pixel from the pixels of the input image as a pixel of interest (step S120), and sets a range of multiple pixels in the main scanning direction to which the pixel of interest belongs (step S122). For example, the control unit 100 sets a scanner streak detection mask (e.g., 7 pixels in the main scanning direction × 1 pixel in the sub-scanning direction) for the input image, and sets the range in which the scanner streak detection mask is set as the range of multiple pixels in the main scanning direction to which the pixel of interest belongs.

[0049] The scanner streak detection mask is used to set a range that includes a pixel of interest, several consecutive pixels (e.g., three pixels) in the forward direction from the pixel of interest in the main scanning direction, and several consecutive pixels (e.g., three pixels) in the reverse direction from the pixel of interest in the main scanning direction. By using such a scanner streak detection mask, the control unit 100 sets a range of seven consecutive pixels that includes multiple pixels in the main scanning direction to which the pixel of interest belongs.

[0050] Next, the control unit 100 determines whether each pixel included in the range set in step S122, including the pixel of interest, is a white pixel, and sets the value of a flag (Flg) for the white pixel to "Wh" (step S124). Flg is information indicating the characteristics of each pixel included in the range set in step S122.

[0051] For example, when pixel values ​​are expressed as RGB values ​​and the ranges of the values ​​of the R, G, and B components are all from 0 to 255, the control unit 100 determines that a pixel whose R, G, and B component values ​​all exceed Th0 is a white pixel. Note that the control unit 100 may use a value stored in the parameter information storage area 162 as Th0. Also, when the color component values ​​are any value from 0 to 255, the value of Th0 is, for example, "230." Note that when a white pixel is found, the control unit 100 may temporarily store in the storage unit 160 the position of the pixel and the Flg value "Wh" in association with each other.

[0052] Next, the control unit 100 determines whether the Flg value of the pixel of interest is "Wh" (step S126). If the Flg value of the pixel of interest is "Wh", the control unit 100 determines that the pixel of interest is not a candidate for an abnormal pixel but a white pixel (step S126; Yes → step S128). At this time, the control unit 100 stores "Wh" as the value of the position of the pixel of interest in the provisional scanner line determination result information 166.

[0053] On the other hand, if the value of Flg of the pixel of interest is not "Wh", the control unit 100 executes the process shown in Fig. 9. First, the control unit 100 assigns XMIN to a variable X indicating the position of a pixel included in the range set in step S122 of Fig. 8 (step S132). Here, the relationship between the value of the variable X and the pixel position is as follows: (1) When X=0, it indicates the position of the pixel of interest. (2) If X<0, it indicates a position that is the number of pixels away from the pixel of interest in the reverse direction of the main scanning direction, equal to the absolute value of the value of X. For example, if X=-3, it indicates a position that is 3 pixels away from the pixel of interest in the reverse direction of the main scanning direction. (3) If X>0, it indicates a position that is the number of pixels away from the pixel of interest in the forward direction of the main scanning direction, which is the value of X. For example, if X=3, it indicates a position that is three pixels away from the pixel of interest in the forward direction of the main scanning direction.

[0054] Furthermore, XMIN indicates the minimum value that X can take. For example, in step S122 of Fig. 8, if a range is set that includes three consecutive pixels in the forward direction from the pixel of interest in the main scanning direction and three consecutive pixels in the reverse direction from the pixel of interest in the main scanning direction, the range that X can take is from -3 to 3, and XMIN is -3.

[0055] Next, the control unit 100 determines whether the value of variable X is 0 or not (step S134). If the value of variable X is not 0, the control unit 100 determines whether the pixel at the position indicated by the value of variable X (pixel at position X) is a white pixel or not (step S134; Yes → step S136). For example, if the value of Flg for the pixel at position X is "Wh", the control unit 100 determines that the pixel is a white pixel.

[0056] If the pixel at position X is not a white pixel, the control unit 100 acquires the density difference for each color component between the pixel at position X and the pixel of interest (step S136; Yes → step S138). The density difference refers to the difference in value of a specific color component between the two pixels.

[0057] For example, the pixel value of the pixel at position X is (R X ,G X ,B X ), the control unit 100 calculates the density difference for each color component using the following formula, thereby obtaining the density difference for each color component. Concentration difference of R component = |R0-R X | Concentration difference of B component = |G0-G X | G component density difference = |B0-B X | Here, when X=−3, the control unit 100 calculates the density difference for each color component using the following formula. Concentration difference of R component = |R0-R -3 | Concentration difference of B component = |G0-G -3 | G component density difference = |B0-B -3 |

[0058] Next, the control unit 100 determines whether or not the density difference of one color component among the density differences for each color component acquired in step S138 exceeds a threshold value Th1 and the density differences of the other color components are less than a threshold value Th2 (step S140).

[0059] For example, the control unit 100 determines whether any of the following conditions is met. (1)|R0-R X |>Th1 and |G0-G X | <Th2かつ|B0-B X | <Th2 (2)|R0-R X | <Th2かつ|G0-G X |>Th1 and |B0-B X | <Th2 (3)|R0-R X | <Th2かつ|G0-G X | <Th2かつ|B0-B X |>Th1 Here, if X=−3, the control unit 100 determines whether or not any of the following conditions is met. (1)|R0-R -3 |>Th1 and |G0-G -3 | <Th2かつ|B0-B -3 | <Th2 (2)|R0-R -3 | <Th2かつ|G0-G -3 |>Th1 and |B0-B -3 | <Th2 (3)|R0-R -3 | <Th2かつ|G0-G -3 | <Th2かつ|B0-B -3 |>Th1

[0060] Note that Th1 and Th2 may use the values ​​stored in the parameter information storage area 162. If the color component values ​​are any value between 0 and 255, for example, the value of Th1 is "80" (preferably any value between 50 and 150), and Th2 is "15" (preferably any value between 5 and 20).

[0061] If the density difference of one color component exceeds the threshold value Th1 and the density difference of the other color components is less than Th2, the control unit 100 sets the Flg value of the pixel at position X to "1" (step S140; Yes → step S142). The control unit 100 also sets information about the color component that exceeds the threshold value Th1 to the pixel at position X along with the Flg value. In this embodiment, a color component that exceeds the threshold value Th1 is referred to as a specific color component. In this embodiment, the control unit 100 executes the process of step S140 to determine whether any one of the R, G, and B color components is a specific color component, and if any one color component is a specific color component, sets the Flg value to "1".

[0062] For example, when the specific color component is the R component, the control unit 100 associates the value of the variable X with the value of Flg "1R" and temporarily stores them in the storage unit 160. Similarly, when the specific color component is the G component, the control unit 100 associates the value of the variable X with the value of Flg "1G" and temporarily stores them in the storage unit 160. When the specific color component is the B component, the control unit 100 associates the value of the variable X with the value of Flg "1B" and temporarily stores them in the storage unit 160.

[0063] On the other hand, unless the density difference of one color component exceeds the threshold value Th1 and the density difference of the other color components is less than Th2, the control unit 100 sets the Flg value of the pixel at position X to "0" (step S140; No -> step S144). For example, the control unit 100 associates the value of variable X with the Flg value "0" and temporarily stores them in the storage unit 160.

[0064] If the value of X is 0 (step S134; No) or if the pixel at position X is a white pixel (step S136; No), the control unit 100 omits the processes from step S138 to step S144.

[0065] Next, the control unit 100 determines whether the value of the variable X is XMAX (step S146). XMAX indicates the maximum value that X can take. If the value of the variable X is not XMAX, the control unit 100 assigns the value of X+1 to X and returns to step S134 (step S146; No → step S148 → step S134). In this way, the control unit 100 sets the values ​​of Flg for pixels other than the pixel of interest that are included in the range of pixels set in step S122 of FIG. 8.

[0066] On the other hand, if the value of the variable X is XMAX, the control unit 100 determines whether there are one or more pairs of pixels on both sides of the pixel of interest that have an Flg value of "1" and the same specific color element (step S146; Yes → step S150). "On both sides of the pixel of interest" refers to pixels adjacent to the pixel of interest in the forward main scanning direction and pixels adjacent to the pixel of interest in the reverse main scanning direction. The control unit 100 determines whether there are any pixels at any positions adjacent to the pixel of interest in the forward main scanning direction and any positions adjacent to the pixel of interest in the reverse main scanning direction that have an Flg value of "1" and the same specific color element. If there are any pixels with an Flg value of "1" and the same specific color element, the control unit 100 determines that there are one or more pairs of pixels on both sides of the pixel of interest that have an Flg value of "1" and the same specific color element.

[0067] For example, if there are pixels with Flg values ​​of "1R" on both sides of the pixel of interest, the control unit 100 determines that there is a pair of pixels with Flg=1 and the same specific color component.

[0068] If there is one or more pairs of pixels on both sides of the pixel of interest that have an Flg value of "1" and the same specific color element, the control unit 100 determines that the pixel of interest is a candidate for an abnormal pixel as a provisional scanner line determination result for the pixel of interest (step S150; Yes → step S152). Here, the control unit 100 stores information on the specific color element of the one or more pairs of pixels on both sides of the pixel of interest that have an Flg value of "1" and the same specific color element in provisional scanner line determination result information 166 for the pixel of interest.

[0069] For example, if there is one or more pairs of pixels with an Flg value of "1R" on both sides of the pixel of interest, the control unit 100 stores "1R" as the position value of the pixel of interest in the provisional scanner line determination result information 166. From the information "1R" stored in the provisional scanner line determination result information 166, the control unit 100 can thereby identify that the pixel in question is a candidate for an abnormal pixel and that the density difference of the red component between the pixel in question and its surrounding pixels exceeds Th1. The information on the specific color element stored in the provisional scanner line determination result information 166 is the specific color component corresponding to the pixel detected as a candidate for an abnormal pixel.

[0070] On the other hand, if Flg=1 and there are no pixels on either side of the pixel of interest that have the same specific color component, the control unit 100 determines that the pixel of interest is not a candidate for an abnormal pixel but a non-white pixel as a provisional scanner line determination result for the pixel of interest (step S150; No→step S154). In this case, the control unit 100 stores "x" as the value of the position of the pixel of interest in the provisional scanner line determination result information 166.

[0071] Returning to FIG. 8, after executing any one of the processes of step S128 in FIG. 8, step S152 in FIG. 9, and step S154 in FIG. 9, the control unit 100 determines whether or not all pixels of the input image have been determined to be candidates for abnormal pixels (step S130). If the determination has been made for all pixels, the control unit 100 ends the first scanner line detection process (step S130; Yes). On the other hand, if the determination has not been made for all pixels, the control unit 100 returns to step S120 (step S130; No → step S120). In this way, the control unit 100 detects candidates for abnormal pixels from the input image.

[0072] The control unit 100 may execute the first scanner line detection process while reading an image of a document. For example, the control unit 100 may execute the first scanner line detection process on the image of the read line at the point when the control unit 100 has read one line or a predetermined number of lines of the image of the document in the main scanning direction. In this case, the control unit 100 selects a pixel of interest from pixels included in the image of the read line in step S120 of FIG. 8, and determines whether or not the determination has been performed on all pixels included in the image of the read line in step S130 of FIG. 8. In this way, the control unit 100 can execute the first scanner line detection process in real time.

[0073] [1.2.3 Second scanner muscle detection process] The flow of the second scanner line detection process will be described with reference to Fig. 10. The second scanner line detection process may be executed after the first scanner line detection process, or may be executed in parallel with the first scanner line detection process when the input image is acquired.

[0074] First, the control unit 100 selects one line in the sub-scanning direction from the input image (step S160). For example, the control unit 100 sets a variable indicating the line in the sub-scanning direction to L, and assigns a predetermined value to L (for example, 0, 1, 2, ..., where the range of L is 0≦L≦the number of pixels in the main scanning direction of the input image−1). In the following description, the line in the sub-scanning direction selected in step S160 will be referred to as the "line of interest." In other words, the line of interest is a row of pixels whose longitudinal direction is the sub-scanning direction, and is composed of pixels whose x-coordinate value is L.

[0075] Next, the control unit 100 evaluates the provisional scanner line determination result for the line of interest. First, the control unit 100 determines whether or not there is a pixel that is a candidate for an abnormal pixel on the line of interest (step S162). For example, the control unit 100 reads the determination result for the Lth column from the provisional scanner line determination result information 166, and if any of "1R," "1G," or "1B" is included, it determines that there is a pixel that is a candidate for an abnormal pixel.

[0076] If there is a candidate abnormal pixel, the control unit 100 determines whether there is a single color component whose density difference exceeds Th1 between the candidate abnormal pixel and its surrounding pixels (step S162; Yes → step S164). That is, the control unit 100 determines whether the specific color components corresponding to the pixels detected as candidate abnormal pixels included in the target line are the same. For example, if the values ​​of the provisional scanner line determination results for the pixels included in the target line include two or more of "1R," "1G," and "1B," the control unit 100 determines that there is not a single color component whose density difference exceeds Th1. That is, the control unit 100 determines that although there is a candidate abnormal pixel in the target line, the color components whose density difference exceeds Th1 are not the same color component.

[0077] If there is more than one color component whose density difference exceeds Th1, the control unit 100 changes the provisional scanner line determination result for the candidate abnormal pixel included in the line of interest to a non-white pixel that is not a candidate abnormal pixel (step S164; No → step S166). That is, the control unit 100 changes the provisional scanner line determination result for the candidate abnormal pixel in the line of interest to "X." As a result, the control unit 100 excludes pixels present on a line whose density difference exceeds Th1 is more than one color component (the specific color component corresponding to the pixel detected as the candidate abnormal pixel is not the same) from the candidate abnormal pixel. Note that if the control unit 100 determines in step S164 that there is one color component whose density difference exceeds Th1, it omits the processing in step S166 (step S164; Yes).

[0078] Next, the control unit 100 determines whether there is an isolated candidate abnormal pixel on the line of interest (step S168). Here, an isolated candidate abnormal pixel refers to a candidate abnormal pixel that is not adjacent to other candidate abnormal pixels, or a candidate abnormal pixel where the number of consecutive candidate abnormal pixels is less than a predetermined threshold (e.g., 5).

[0079] If there is a pixel that is a candidate for an isolated abnormal pixel, the control unit 100 changes the provisional scanner line determination result for the pixel that is a candidate for an isolated abnormal pixel to a non-white pixel that is not a candidate for an abnormal pixel (step S168; Yes → step S170). That is, the control unit 100 changes the provisional scanner line determination result for the pixel that is a candidate for an isolated abnormal pixel within the line of interest to "X". As a result, the control unit 100 excludes the pixel that is a candidate for an isolated abnormal pixel from the candidate for an abnormal pixel. Note that if there is no pixel that is a candidate for an isolated abnormal pixel in step S168, the control unit 100 omits the processing in step S170 (step S168; No).

[0080] Next, the control unit 100 determines whether or not the candidate abnormal pixels on the line of interest are continuous in the sub-scanning direction (step S172). A case where there is continuous in the sub-scanning direction means that the candidate abnormal pixels on the line of interest are intermittently present in the sub-scanning direction. A case where there are intermittently present candidate abnormal pixels on the line of interest are when the candidate abnormal pixels are arranged with white pixels sandwiched between them. In other words, when the candidate abnormal pixels are alternately arranged with white pixels sandwiched between them, or when the candidate abnormal pixels are consecutive with white pixels sandwiched between them, the candidate abnormal pixels are continuous in the sub-scanning direction.

[0081] In step S172, the control unit 100 determines whether the number of consecutive white pixels and candidate abnormal pixels on the line of interest is equal to or greater than the threshold value Th3. The value of Th3 may be a number of pixels equivalent to 10% of the number of vertical pixels of the input image (preferably, a number of pixels equivalent to any ratio between 5% and 20% of the number of vertical pixels of the input image), or a specific value (e.g., "200"). In this case, the control unit 100 reads the determination result of the line of interest from the provisional scanner line determination result information 166 and determines whether the number of consecutive pixels determined as "Wh" and the number of consecutive pixels determined as candidate abnormal pixels (either "1R," "1G," or "1B") is equal to or greater than the threshold value Th3. If the number of consecutive pixels determined as "Wh" and the number of consecutive candidate abnormal pixels determined as candidate abnormal pixels is equal to or greater than the threshold value Th3, the control unit 100 determines that the candidate abnormal pixels are continuous in the sub-scanning direction on the line of interest.

[0082] If the candidate abnormal pixels on the line of interest are continuous in the sub-scanning direction, the control unit 100 determines that the line of interest is a line where a scanner streak has occurred (step S172; Yes → step S174). This allows the control unit 100 to identify candidate abnormal pixels included in the line where a scanner streak has occurred as abnormal pixels. The control unit 100 can also identify a line (pixel row) in the sub-scanning direction that includes an abnormal pixel as an abnormal pixel area. At this time, the control unit 100 stores the value of X, which indicates the row number of the line of interest, and "Yes" in the scanner streak determination result information 168, associating them with each other.

[0083] On the other hand, if the candidate abnormal pixels on the line of interest are not continuous in the sub-scanning direction, the control unit 100 determines that the line of interest is a line on which no scanner streaks occur (step S172; No → step S176). At this time, the control unit 100 stores the value of X indicating the number of columns on the line of interest in association with "No" in the scanner streak determination result information 168. Note that even if the control unit 100 determines in step S162 that the line of interest does not have any candidate abnormal pixels, it also determines that the line of interest is a line on which no scanner streaks occur (step S162; No → step S176).

[0084] Next, the control unit 100 determines whether or not all the lines of the input image have been selected (step S178). If all the lines of the input image have been selected, the control unit 100 ends the processing shown in Fig. 10 (step S178; Yes). On the other hand, if all the lines of the input image have not been selected, the control unit 100 returns to step S160 (step S178; No → step S160).

[0085] When the control unit 100 executes the second scanner streak detection process in parallel with the first scanner streak detection process, it only needs to execute the second scanner streak detection process each time a predetermined number (e.g., 200) of lines of the document image are read in the main scanning direction. In this way, the control unit 100 can determine in real time the lines on which scanner streaks have appeared.

[0086] [1.2.4 Scanner streak correction processing] The flow of the scanner streak correction process will be described with reference to Fig. 11. First, the control unit 100 determines whether or not scanner streaks have appeared in the input image (step S180). For example, the control unit 100 reads the scanner streak determination result information 168, and if the scanner streak determination result contains one or more "Yes", it determines that scanner streaks have appeared in the input image. Note that if there are no scanner streaks in the input image, the control unit 100 ends the process shown in Fig. 11 (step S180; No).

[0087] If scanner streaks are present in the input image, the control unit 100 selects one line in the sub-scanning direction where the scanner streaks are present (step S180; Yes → step S182). For example, the control unit 100 obtains a position where the scanner streak determination result is "Yes" from the scanner streak determination result information 168, and selects a line in the sub-scanning direction at that position.

[0088] Next, the control unit 100 identifies the image pattern (region separation result) around the scanner line (step S184). For example, while referring to the region separation information 164, the control unit 100 identifies, for each pixel in the line selected in step S182, the image pattern of the pixel adjacent to either the left (in the reverse direction of the main scanning direction) or the right (in the forward direction of the main scanning direction) of the pixel as the image pattern around the scanner line at that pixel. Note that, for each pixel in the line selected in step S182, the control unit 100 may identify the image pattern around the scanner line at that pixel based on the image pattern of a region of a predetermined size adjacent to the left or right of the pixel. For example, the control unit 100 sets two regions of 25 pixels, five pixels vertically and five pixels horizontally, to the left and right of a certain pixel (pixel of interest) in the line selected in step S182, so that they are adjacent to the pixel of interest. In this case, the control unit 100 counts up the image patterns of the pixels (50 pixels in total) within the set region and specifies the most common image pattern as the image pattern surrounding the pixel of interest. Note that the control unit 100 may specify the image pattern surrounding the scanner line for each pixel within the line selected in step S182, or may specify the image pattern surrounding one scanner line for the selected line.

[0089] Next, the control unit 100 determines whether the surrounding image pattern is a halftone dot area or a line area (step S186). If the surrounding image pattern is a halftone dot area or a line area, the control unit 100 replaces the pixel value of the scanner line with the average value of the pixels surrounding that pixel (peripheral pixels) (step S186; Yes → step S188). For example, the control unit 100 sets 25-pixel areas, 5 pixels vertically and 5 pixels horizontally, to the left and right of the pixel to be corrected (target pixel), so that they are adjacent to the target pixel, and the pixels included in these areas are designated as peripheral pixels. At this time, the control unit 100 acquires pixel values ​​for each pixel from the peripheral pixels and designates the average of the acquired pixel values ​​as the average value of the peripheral pixels.

[0090] On the other hand, if the surrounding image pattern is not a halftone dot area or a line area, the control unit 100 determines whether the surrounding image pattern is a character area (step S186; No → step S190). If the surrounding image pattern is a character area, the control unit 100 replaces the pixel value of the scanner line with the pixel value of the pixel adjacent to that pixel (adjacent pixel value) (step S190; Yes → step S192). For example, the control unit 100 sets the pixel value of the pixel adjacent to either the left or right of the pixel to be corrected as the adjacent pixel value.

[0091] If the surrounding image pattern is not a character area in step S190, the control unit 100 replaces the pixel value of the scanner line using a predetermined method (step S190; No → step S194). An area where the surrounding image pattern is neither a halftone dot area nor a line area and is not a character area (so-called "other" area) is, for example, a photographic image area. If the surrounding image pattern is "other," the control unit 100 may replace the pixel value of the scanner line based on the average pixel value of the surrounding pixels, as in the case where the surrounding area is a halftone dot area or line area, or may replace the pixel value based on the pixel value of adjacent pixels, as in the case where the surrounding area is a character area. The method of replacing pixel values ​​in step S194 may be predetermined or may be set by the user. The control unit 100 may also replace the pixel value of the scanner line using a method other than the method of replacing pixel values ​​based on the average value or the method of replacing pixel values ​​based on the pixel values ​​of adjacent pixels.

[0092] The control unit 100 performs the processes from step S186 to step S194 for each pixel in the line selected in step S182. This allows the control unit 100 to switch the correction method (replacement method) for the abnormal pixels included in the abnormal pixel region based on the region separation information surrounding the abnormal pixel region. The control unit 100 can also appropriately correct the abnormal pixels by replacing the pixel values ​​of the abnormal pixels with different pixel values ​​depending on the correction method for the abnormal pixels. The control unit 100 may replace the pixel values ​​of all pixels included in the line selected in step S182, or may replace only the pixel values ​​of the abnormal pixels included in the line (pixels whose provisional scanner line determination result is "1R," "1G," or "1B")

[0093] Next, the control unit 100 determines whether or not all lines with scanner streaks have been selected (step S196). If all lines with scanner streaks have been selected, the control unit 100 ends the process shown in Fig. 11 (step S196; Yes). On the other hand, if all lines with scanner streaks have not been selected, the control unit 100 returns to step S182 (step S196; No → step S182).

[0094] [1.3 Example of operation] Next, an example of the operation of this embodiment will be described. First, a scanner streak corrected by the image forming apparatus 10 of this embodiment will be described. Fig. 12 is a diagram showing a case where a white streak (a streak caused by loss of one component color) occurs.

[0095] P1 in FIG. 12 indicates the document feed direction. Document A100 is transported in the direction of P1 by a document feeder. Document A100 reflects light P2 from light source A102. The reflected R component light R100, G component light G100, and B component light B100 are read by image sensor A104 such as a CCD. This allows image forming apparatus 10 to obtain an image of document A100.

[0096] Now, suppose there is a foreign object A108, such as a piece of paper or dust, between the original A100 and the glass A106 located below the original A100. If the foreign object A108 is a small, white object, light P2 from the light source A102 is not absorbed by the original A100 and is reflected by the foreign object A108. If the foreign object A108 is small, light of one color (for example, light R100 of the R component) is always reflected, and this light of one color is always read by the image sensor A104. In this way, light of a predetermined color component is always read by the image sensor A104, and as a result, pixels appearing to be missing the predetermined color component appear in the input image.

[0097] Next, an example of the operation of the first scanner streak detection process will be described with reference to Fig. 13 to Fig. 18. Fig. 13(a) is a diagram showing a scanned image (input image) of a document, and the squares in the diagram represent pixels. Area E100 represents a halftone dot area. Also, scanner streaks E102 represent scanner streaks. In this way, scanner streaks may appear, for example, on halftone dot areas.

[0098] Furthermore, an area E104 indicates a scanner line detection mask. An area E106 indicates a pixel of interest. To determine whether the pixel of interest is a scanner line, the image forming apparatus 10 performs a process of acquiring density differences for each color component of the scanner line detection mask.

[0099] FIG. 13(b) is an enlarged view of the scanner streak detection mask. Here, the pixel where X=0 (pixel marked with *) is the pixel of interest. The pixels located from X=-3 to X=-1 are consecutive pixels in the reverse direction from the pixel of interest in the main scanning direction. The pixels located from X=1 to X=3 are consecutive pixels in the forward direction from the pixel of interest in the main scanning direction. In this way, the image forming apparatus 10 sets a range of multiple pixels in the main scanning direction to which the pixel of interest belongs.

[0100] Fig. 14 shows the provisional scanner streak determination results for a halftone dot area when no scanner streaks have occurred. Fig. 14(a) is an enlarged view of a halftone dot area in an input image, with the squares in the figure representing pixels. Area E110 represents a scanner streak detection mask. Area E112 represents a pixel of interest. As shown in P110, rectangles with a white background represent white pixels. As shown in P112, rectangles with a non-white background represent non-white pixels (halftone dot pixels with some color).

[0101] Figure 14(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in Figure 14(b), there are no pixels on either side of the pixel of interest where Flg=1. Therefore, the provisional scanner determination result for the pixel of interest is "X."

[0102] All pixels in the input image are judged to be candidates for abnormal pixels, resulting in the judgment results shown in Fig. 14(c). Note that, for lines where scanner streaks do not occur, provisional scanner streak judgment results such as "Wh" or "x" continue in the sub-scanning direction, as shown in area E114.

[0103] FIG. 15 shows the results of a provisional scanner streak determination for a dot region when scanner streaks have occurred. FIG. 15(a) is an enlarged view of a dot region in an input image, with the squares in the figure representing pixels. Area E120 represents a scanner streak detection mask. Area E122 represents a pixel of interest. Note that cross-hatched pixels, such as P120 in FIG. 15(a), represent dot pixels of a color that is not substantially the same as the color of the surrounding dots. In other words, within the dot region of the input image, as shown in P120, dot pixels of a color that is not substantially the same as the color of the surrounding dots appear consecutively in the sub-scanning direction, and scanner streaks are caused by these pixels.

[0104] Fig. 15(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in area E124 and area E126 in Fig. 15(b), there are pixels where Flg=1 and the specific color component is red (R) (pixels where the Flg value is "1R") in the pixels consecutive from the pixel of interest in the forward main scanning direction and the pixels consecutive from the pixel of interest in the reverse main scanning direction. Therefore, the provisional scanner line determination result for the pixel of interest is "1R."

[0105] 15(c) is obtained by determining whether or not all pixels in the input image are candidates for abnormal pixels. Note that, in the line where the scanner streak occurs, as shown in area E128, provisional scanner streak determination results, which indicate that the pixels are candidates for abnormal pixels with the same color component, such as "Wh" or "1R," whose density difference exceeds Th1, continue in the sub-scanning direction.

[0106] Figure 16 shows the results of provisional scanner streak determination for a character region when no scanner streaks have occurred. Figure 16(a) is an enlarged view of a character region in an input image, with the squares in the figure representing pixels. Area E130 represents the scanner streak detection mask. Area E132 represents the pixel of interest. As shown in P130, rectangles with a white background represent white pixels, and as shown in P132, rectangles with a black background represent black pixels.

[0107] Figure 16(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in Figure 16(b), there are no pixels on either side of the pixel of interest where Flg=1. Therefore, the provisional scanner determination result for the pixel of interest is "X."

[0108] All pixels in the input image are judged to be candidates for abnormal pixels, and the judgment result shown in Fig. 16(c) is obtained. As shown in Fig. 16(c), when there are no scanner streaks, the provisional scanner streak judgment results such as "Wh" and "x" continue in the sub-scanning direction on every line.

[0109] FIG. 17 shows the provisional scanner streak determination results for a character region when scanner streaks have occurred. FIG. 17(a) is an enlarged view of a character region in an input image, with the squares in the figure representing pixels. Area E140 represents a scanner streak detection mask. Area E142 represents a pixel of interest. Note that cross-hatched pixels, such as P140 in FIG. 17(a), are pixels of a color different from black or white (e.g., red). In other words, within the character region of the input image, as shown in P140, pixels of a color different from the colors of the surrounding pixels appear consecutively in the sub-scanning direction, and these pixels cause scanner streaks.

[0110] 17(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in FIG. 17(b), among the pixels consecutive from the pixel of interest in the forward main scanning direction and the pixels consecutive from the pixel of interest in the reverse main scanning direction, there are pixels where Flg=1 and the specific color component is red (R) (pixels where the Flg value is "1R"). Therefore, the provisional scanner line determination result for the pixel of interest is "1R."

[0111] 17(c) is obtained by determining whether or not all pixels in the input image are candidates for abnormal pixels. In addition, in the line where the scanner streak occurs, as shown in area E144, provisional scanner streak determination results, which indicate that the pixels are candidates for abnormal pixels with the same color component, such as "Wh" or "1R," whose density difference exceeds Th1, continue in the sub-scanning direction.

[0112] Also, Fig. 18 is a diagram showing a case where ruled lines of a table or the like extend in the sub-scanning direction, and the squares in the diagram represent pixels. Fig. 18(a) is an enlarged view of a portion of the input image where ruled line L150 extends in the sub-scanning direction. Area E150 in Fig. 18(a) represents a scanner streak detection mask. Area E152 represents a pixel of interest. Here, the pixel of interest is assumed to be a pixel that constitutes the ruled line.

[0113] FIG. 18(b) shows the Flg value for each pixel in the scanner streak detection mask. As shown in FIG. 18(b), the pixel of interest is not a white pixel, so the Flg value for the pixel of interest is not Wh. Furthermore, there are no pixels on either side of the pixel of interest where Flg=1. Therefore, the provisional scanner streak determination result for the pixel of interest is "X," and the pixel is determined not to be a candidate for an abnormal pixel. As a result, ruled lines (e.g., colored lines) included in the input image are not determined to be lines where the color has been removed by the scanner streak (so-called missing lines).

[0114] Next, an example of the operation of the scanner streak correction process will be described with reference to Figures 19 and 20. Figure 19 is a diagram showing an example of the operation of correcting scanner streaks on halftone dots, where the line shown in area E128 in Figure 15(c) is corrected. Here, it is assumed that area E128 in Figure 15(c) is determined to be a line where scanner streaks occur.

[0115] FIG. 19(a) shows the result of scanner line correction for text performed on an abnormal pixel (a pixel for which the provisional scanner line determination result is "1R") included in a line containing a scanner line. Scanner line correction for text is a correction that replaces the pixel value of the scanner line with the pixel value of the pixel adjacent to that pixel. Here, area E160 in FIG. 19(a) indicates the area of ​​the line containing the scanner line (the abnormal pixel area). When scanner line correction for text is performed on an abnormal pixel on a halftone dot, the color of the corrected abnormal pixel matches the color of the surrounding pixels, but a visual gap occurs due to a shift in the periodicity of the halftone dots.

[0116] FIG. 19(b) shows the result of halftone dot scanner streak correction performed on an abnormal pixel included in a line containing a scanner streak. Halftone dot scanner streak correction replaces the pixel value of the scanner streak with the average value of its surrounding pixels. Area E162 in FIG. 19(b) indicates the area of ​​the line containing the scanner streak (the abnormal pixel area). When halftone dot scanner streak correction is performed on an abnormal pixel on a halftone dot, the color of the corrected abnormal pixel matches the color of the surrounding colored pixels, reducing the appearance of a color gap.

[0117] Fig. 20 shows an example of the operation of correcting scanner streaks on a character, where the line shown in area E144 in Fig. 17(c) is corrected. Here, it is assumed that area E144 in Fig. 17(c) is determined to be a line where scanner streaks occur.

[0118] Figure 20(a) shows a case where scanner streak correction for text is performed on an abnormal pixel included in a line where a scanner streak has occurred. Area E170 in Figure 20(a) indicates the area of ​​the line where the scanner streak has occurred (the abnormal pixel area). When scanner streak correction for text is performed on an abnormal pixel on a character, the pixel value of the abnormal pixel becomes the same as the pixel value of the surrounding pixels (black or white), and sharpness is maintained.

[0119] 20(b) shows the result of performing scanner streak correction for halftone dots on an abnormal pixel included in a line where a scanner streak has occurred. Area E172 in FIG. 20(b) indicates the area of ​​the line where a scanner streak has occurred (the abnormal pixel area). When scanner streak correction for halftone dots is performed on an abnormal pixel on a character, the pixel value of the abnormal pixel is replaced with a color that takes into account the pixel values ​​(density) of the surrounding pixels, resulting in an insufficient correction.

[0120] In this way, the pixel values ​​of the scanner streaks are replaced according to the results of the area separation around the scanner streaks. For example, when correcting scanner streaks in character areas, the pixel values ​​are replaced with the pixel values ​​of adjacent pixels, thereby achieving correction that maintains sharpness. Furthermore, when correcting scanner streaks in line or halftone dot areas, the pixel values ​​are replaced with the average value of the pixels around the streaks, thereby minimizing gaps with the surrounding area. In this way, scanner streaks are appropriately corrected.

[0121] In the above description, it is assumed that the number of scanned images (input images) of documents input to the image forming apparatus 10 is one. If a plurality of documents are input to the image forming apparatus 10, the image forming apparatus 10 performs the above-described processing on all scanned images (input images) of the documents. As a result, the image forming apparatus 10 stores region separation information 164, provisional scanner streak determination result information 166, and scanner streak determination result information 168 for the input image, and corrects scanner streaks based on this information.

[0122] In the above description, the input image is divided into a line area, a halftone dot area, a character area, and a background area, and the method of correcting the scanner streaks is switched based on whether the type of the area around the scanner streaks is a line / halftone dot area or a character area. However, even if the type of the area around the scanner streaks is an area other than the above-mentioned types of areas, the image forming apparatus 10 may switch the method of correcting the scanner streaks depending on the type of the area.

[0123] In the above description, in step S184 of FIG. 11, the control unit 100 identifies the image pattern around the scanner streak by referring to the region separation information 164. However, the control unit 100 may identify the image pattern around the scanner streak without using the region separation information 164. For example, the control unit 100 may identify the pixel pattern by pattern matching. Alternatively, the control unit 100 may identify the pixel pattern based on the periodicity of the brightness of pixels in a pixel block around the scanner streak (e.g., a region of 20 pixels vertically and horizontally). Note that when correcting the scanner streak without referring to the region separation information 164, the control unit 100 may omit the region separation process in step S102 of FIG. 7.

[0124] As described above, the image forming apparatus of this embodiment makes it possible to detect scanner streaks in a scanned image in real time while reading an image of a document. Furthermore, the image forming apparatus of this embodiment corrects scanner streaks according to the image pattern (region separation result) around the scanner streaks. Conventionally, scanner streaks have been corrected by uniformly replacing the pixel values ​​of the scanner streaks with the pixel values ​​of pixels around the scanner streaks. This has resulted in gaps due to deviations in the periodicity of the halftone dots, or insufficient correction. However, the image forming apparatus of this embodiment can solve these problems by performing correction according to the image pattern of the scanner streaks.

[0125] [2. Second Embodiment] Next, a second embodiment will be described. In the second embodiment, in addition to the processing in the first embodiment, processing for correcting the region separation result is executed. In this embodiment, FIG. 11 of the first embodiment is replaced with FIG. 21. Note that the same functional units and processing are assigned the same reference numerals, and the description thereof will be omitted.

[0126] The flow of scanner streak correction processing in this embodiment will be described with reference to Fig. 21. If the image pattern around the scanner streak is a halftone dot area or a line area (step S186; Yes), after processing step S188, control unit 100 corrects the area separation results corresponding to the pixels whose pixel values ​​have been corrected to a halftone dot area or a line area (step S200). For example, control unit 100 changes the values ​​corresponding to the pixels whose pixel values ​​have been corrected, among the values ​​stored in area separation information 164, to values ​​corresponding to the image pattern identified in step S184.

[0127] On the other hand, if the image pattern around the scanner line is a character area (step S190; Yes), after processing step S192, control unit 100 corrects the region separation result corresponding to the pixel whose pixel value has been corrected to a character area (step S202). For example, control unit 100 changes the value corresponding to the pixel whose pixel value has been corrected in region separation information 164 to a value corresponding to a character area.

[0128] Furthermore, if the image pattern around the scanner line is not a halftone dot area or a line area, or a character area, the control unit 100 performs the process of step S194 (step S190; No → step S194). Next, the control unit 100 corrects the area separation information of the pixel by changing the area separation result corresponding to the pixel whose pixel value has been corrected to a value corresponding to the image pattern identified in step S184 (step S204).

[0129] Here, the region separation process for the input image is performed before the scanner line correction process. Therefore, it is possible that the results of the region separation process for the scanner line pixels may not be appropriate. In response to this, the control unit 100 can correct the region separation information 164 by performing the above-mentioned process.

[0130] By correcting the region separation information 164, the control unit 100 can more appropriately perform image processing when performing image processing using the region separation information 164. Examples of image processing include color correction, black generation UCR (Under Color Removal) processing, filter processing, and screen processing. In addition, processing for halftone dot regions is performed on halftone dot regions, and processing to emphasize halftone dot characters is performed on halftone dot character regions. By performing image processing using the corrected region separation information 164, the control unit 100 can reduce the influence of scanner streaks and obtain appropriate image processing results.

[0131] Next, an example of the operation of this embodiment will be described. Fig. 22(a) is an enlarged view of a scanned halftone dot image. It is assumed that no scanner streaks are present in the scanned image. Fig. 22(b) is a view showing region separation information for Fig. 22(a). In Fig. 22(b), a region shown with halftone dots indicates that the region is a halftone dot region. As shown in Fig. 22(b), the region separation information indicates that the entire region is a halftone dot region.

[0132] FIG. 23(a) is an enlarged view of a halftone dot scanned image. Assume that scanner streaks have appeared in the scanned image. The scanner streaks have appeared in the input image, as shown in area E200. FIG. 23(b) is a diagram showing region separation information for FIG. 23(a). In FIG. 23(b), regions shown with dots indicate that they are halftone dot regions, and regions shown with cross-hatching indicate that they are color-on-dot text regions. As shown in FIG. 23(b), the region separation information includes area E206, which is a "color-on-dot text region" caused by scanner streaks, in addition to halftone dot region E202 and halftone dot region E204. That is, a color-on-dot text region caused by scanner streaks exists on a halftone dot region.

[0133] Here, the pixel values ​​of the pixels of the scanner streak are corrected by the scanner streak correction process, as shown in area E208 in Figure 23(c). At the same time, the area separation result in the sub-scanning direction where the scanner streak occurred is also corrected. Figure 23(d) is a diagram showing the area separation information after the area separation result has been corrected. As shown in Figure 23(d), the area separation information indicates a halftone dot area as a whole.

[0134] FIG. 24(a) is an enlarged view of a scanned image of text. It is assumed that the scanned image does not have scanner streaks. FIG. 24(b) is a view showing region separation information for FIG. 24(a). In FIG. 24(b), regions shown in black indicate text regions, and regions shown in white indicate background regions. As shown in FIG. 24(b), the region separation information indicates that regions E220, E222, E224, and E226 are black text regions.

[0135] FIG. 25(a) is an enlarged view of a scanned image of a character. Assume that scanner streaks appear in the scanned image. The scanner streaks appear in the input image, as shown in region E230. FIG. 25(b) shows region separation information for FIG. 25(a). In FIG. 25(b), regions shown in black indicate character regions, regions shown in white indicate background regions, and regions shown in halftone dots indicate color character regions. As shown in FIG. 25(b), the region separation information includes, for example, region E236, which is a color character region, in addition to region E232 and region E234. That is, a color character region due to the influence of scanner streaks exists on a black character region.

[0136] Here, the scanner streak correction process corrects the pixel values ​​of the pixels of the scanner streak, as shown in area E240 in FIG. 25(c). At the same time, the area separation result in the sub-scanning direction where the scanner streak occurred is also corrected. FIG. 25(d) is a diagram showing the area separation information after the area separation result has been corrected. The area separation information is corrected as shown in FIG. 25(d). For example, area E236, which was a "color character area" in FIG. 25(b), becomes a "black character area" as shown in area E242 in FIG. 25(d). As a result, the area determination results for all pixels included in area E244 become "black character area."

[0137] In this way, the image forming apparatus of this embodiment corrects scanner streaks and also corrects the segmentation results, thereby enabling the image forming apparatus of this embodiment to appropriately perform image processing using the segmentation results.

[0138] 3. Third Embodiment Next, a third embodiment will be described. In addition to the processing in the first embodiment, the third embodiment calculates the amount of change in density between a pixel of interest and pixels other than the pixel of interest, and executes processing to detect pixels where the amount of change exceeds a predetermined threshold for two components.

[0139] In this embodiment, if the determination in step S140 of the first scanner detection process shown in Figure 9 is No, the control unit 100 determines whether the density difference between the pixel value of the pixel at position X and the pixel value of the target pixel satisfies the condition that the density difference of two of the color components exceeds threshold value Th1 and the density difference of the remaining color component is less than threshold value Th2. For example, the control unit 100 determines whether any of the following conditions is satisfied: (1)|R0-R X |>Th1 and |G0-G X |>Th1 and |B0-B X | <Th2 (2)|R0-R X |>Th1 and |G0-G X | <Th2かつ|B0-B X |>Th1 (3)|R0-R X | <Th2かつ|G0-G X |>Th1 and |B0-B X |>Th1

[0140] For example, when X=−3, the control unit 100 determines whether or not any of the following formulas is satisfied. (1)|R0-R -3 |>Th1 and |G0-G -3 |>Th1 and |B0-B -3 | <Th2 (2)|R0-R -3 |>Th1 and |G0-G -3 | <Th2かつ|B0-B -3 |>Th1 (3)|R0-R -3 | <Th2かつ|G0-G -3 |>Th1 and |B0-B -3 |>Th1

[0141] If any of the above formulas is satisfied, i.e., if the difference between the values ​​of two color components exceeds Th1 and the difference between the values ​​of the remaining color component is less than Th2, the control unit 100 sets the Flg value for the pixel at X=-3 to 1. On the other hand, if none of the above formulas is satisfied, the control unit 100 sets the Flg value for the pixel at X=-3 to 0.

[0142] Note that when Flg=1, the control unit 100 also stores information about color components that exceed the threshold value Th1. For example, if the color components that exceed Th1 are red and green, the control unit 100 sets the Flg value to "1RG." Similarly, if the color components that exceed Th1 are red and blue, the control unit 100 sets the Flg value to "1RB," and if the color components that exceed Th1 are green and blue, the control unit 100 sets the Flg value to "1GB."

[0143] In addition, if there are pixels on both sides of the pixel of interest whose Flg values ​​are "1R", "1G", "1B", "1RG", "1RB", or "1GB", the control unit 100 sets the provisional scanner line determination result for the pixel of interest to 1 and also retains information on the color component set as the Flg value.

[0144] Furthermore, in the second scanner line detection process, the control unit 100 also performs processing on the premise that the provisional scanner line determination result for a pixel of the provisional scanner line includes any one of "1R," "1G," "1B," "1RG," "1RB," or "1GB." For example, in step S154, if the provisional scanner line determination result value for a pixel included in the line of interest includes two or more of "1R," "1G," "1B," "1RG," "1RB," or "1GB," the control unit 100 determines that the pixel includes provisional scanner line determination results for multiple color components.

[0145] In this way, with the image forming apparatus of this embodiment, even if the dust or dirt on the glass or mirror is relatively large and causes the color to be lost in the two components (by reflecting light), scanner streaks can be corrected.

[0146] [4. Modifications] The present invention is not limited to the above-described embodiments and may be modified in various ways. In other words, embodiments obtained by combining technical means modified appropriately within the scope of the present invention are also within the technical scope of the present invention. For example, while the above description describes an image processing device according to the present disclosure implemented as an image forming device, the present disclosure may also be applied to an image reading device such as a scanner. Furthermore, an image processing device according to the present disclosure may be incorporated into a server device to provide an image correction service. In this case, the server device acquires an image from another device, performs a first scanner line detection process, a second scanner line detection process, and a scanner line correction process on the acquired image, and then transmits the corrected image to the other device.

[0147] Furthermore, although the above-described embodiments are described separately for convenience of explanation, they may of course be combined within a technically feasible range. For example, by combining the second and third embodiments, an image processing device can be realized that detects, as a candidate for an abnormal pixel, a pixel in which the amount of change in density between the target pixel and a pixel other than the target pixel exceeds a predetermined threshold value for only two components, and corrects the region separation result for the candidate for an abnormal pixel.

[0148] In the above description, the method for detecting a candidate abnormal pixel uses the density difference between the pixel of interest and its surrounding pixels, but other methods may be used as long as they detect a change between the pixel of interest and its surrounding pixels. For example, it may be determined whether the pixel of interest is a candidate abnormal pixel based on a change in hue between the pixel of interest and pixels other than the pixel of interest, a change in luminance between the pixel of interest and pixels other than the pixel of interest, or whether the pixel of interest is a candidate abnormal pixel based on whether the region separation results between the pixel of interest and pixels other than the pixel of interest are the same or different.

[0149] In addition, the programs that run on each device in the embodiments are programs that control the CPU, etc. (programs that make a computer function) so as to realize the functions of the above-described embodiments. Information handled by these devices is temporarily stored in a temporary storage device (e.g., RAM) during processing, and then stored in various storage devices such as ROMs (Read Only Memories) and HDDs, and is read, modified, and written by the CPU as needed.

[0150] Here, the recording medium for storing the program may be any of semiconductor media (e.g., ROM, non-volatile memory card, etc.), optical recording media / magneto-optical recording media (e.g., DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray (registered trademark) Disc), etc.), magnetic recording media (e.g., magnetic tape, flexible disk, etc.), etc. Furthermore, not only are the functions of the above-described embodiments realized by executing the loaded program, but the functions of the present invention may also be realized by processing in cooperation with an operating system or other application programs, etc., based on instructions from the program.

[0151] Furthermore, when distributing the program on the market, the program can be stored on a portable recording medium and distributed, or transferred to a server computer connected via a network such as the Internet. In this case, the storage device of the server computer is of course included in the present invention. [Explanation of symbols]

[0152] 10 Image forming device 100 control section 102 Image processing section 104 Region separation processing unit 120 Image input unit 130 Image forming unit 140 Display section 150 Operation section 160 Storage section 162 Parameter information storage area 164 Area separation information 166 Provisional Scanner Line Judgment Result Information 168 Scanner muscle determination result information 190 Communications Department

Claims

1. An acquisition unit that acquires an image and a control unit, The control unit Detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs, from among the image acquired by the acquisition unit; If the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels is detected as an abnormal pixel region; correcting the abnormal pixels based on the detection result of the abnormal pixel region; If the pixels detected as candidates for the abnormal pixel are present intermittently in a row of pixels whose longitudinal direction is the sub-scanning direction, the pixels detected as candidates for the abnormal pixel are determined to be continuous in the sub-scanning direction, and the row of pixels is detected as an abnormal pixel region.

1. An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the control unit determines that the pixels detected as candidates for abnormal pixels are continuous in the sub-scanning direction when the pixels detected as candidates for abnormal pixels are arranged side by side with a white pixel in between in a row of pixels whose longitudinal direction is the sub-scanning direction.

3. An image acquisition unit that acquires an image and a control unit, The control unit Detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs, from among the image acquired by the acquisition unit; If the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels is detected as an abnormal pixel region; correcting the abnormal pixels based on the detection result of the abnormal pixel region; For each pixel other than the pixel of interest included in the plurality of pixels, a density difference between the pixel other than the pixel of interest and the pixel of interest is obtained for each color component; The color component whose density difference exceeds a threshold is determined as a specific color component of a pixel other than the target pixel, When the specific color component of a pixel at a position in the forward direction of the main scanning direction relative to the target pixel among the plurality of pixels is the same as the specific color component of a pixel at a position in the reverse direction of the main scanning direction relative to the target pixel among the plurality of pixels, the target pixel is detected as a candidate for the abnormal pixel.

1. An image processing device comprising:

4. The control unit if the specific color component of a pixel among the plurality of pixels at a position in the forward direction of the main scanning direction relative to the target pixel is the same as the specific color component of a pixel among the plurality of pixels at a position in the reverse direction of the main scanning direction relative to the target pixel, the specific color component is determined to be the specific color component corresponding to the pixel detected as the candidate for the abnormal pixel, 4. The image processing device according to claim 3, wherein, when the specific color components corresponding to the pixels detected as candidates for the abnormal pixel included in a pixel row in the sub-scanning direction are not the same, each pixel included in the pixel row in the sub-scanning direction is excluded from the candidates for the abnormal pixel.

5. An acquisition unit that acquires an image and a control unit, The control unit Detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs, from among the image acquired by the acquisition unit; If the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels is detected as an abnormal pixel region; correcting the abnormal pixels based on the detection result of the abnormal pixel region; A method for replacing pixels included in the abnormal pixel region is switched according to region separation information around the abnormal pixel region.

1. An image processing device comprising:

6. The control unit If the periphery of the abnormal pixel region is a halftone dot or line region, the pixels of the abnormal pixel region are replaced with pixels based on an average value of pixel values ​​of the pixels surrounding the abnormal pixel region; If the periphery of the abnormal pixel region is a character region, the pixels of the abnormal pixel region are replaced with pixels based on the pixel values ​​of pixels adjacent to the abnormal pixel region.

6. The image processing device according to claim 5,

7. The control unit replacing information corresponding to the replaced pixel in the region separation information with information on a region surrounding the pixel; Execute image processing based on the region separation information 7. The image processing device according to claim 6,

8. 8. The image processing apparatus according to claim 1, wherein the acquisition unit acquires an image from a document conveyed by a document feeder.

9. acquiring an image; detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes among a plurality of pixels in the main scanning direction to which the pixel of interest belongs in the image; detecting, when the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels as an abnormal pixel region; correcting the abnormal pixels based on the detection result of the abnormal pixel region; a step of detecting, when pixels detected as candidates for abnormal pixels are present intermittently in a pixel row having a sub-scanning direction as its longitudinal direction, determining that the pixels detected as candidates for abnormal pixels are continuous in the sub-scanning direction and detecting the pixel row as an abnormal pixel region; An image processing method comprising:

10. On the computer, The ability to acquire images; a function of detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs in the image; a function of detecting, when the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels as an abnormal pixel region; a function of correcting the abnormal pixel based on the detection result of the abnormal pixel region; a function of detecting, when pixels detected as candidates for abnormal pixels exist intermittently in a pixel row having a sub-scanning direction as its longitudinal direction, the pixels detected as candidates for abnormal pixels as being continuous in the sub-scanning direction, and detecting the pixel row as an abnormal pixel region; A program characterized by realizing the above.

Images