Image processing device, control method and program

Abstract

To provide an image processing apparatus and the like that can appropriately remove streaks from an image.SOLUTION: An image processing apparatus comprises an input unit that inputs an image of a document and a control unit. The control unit removes streaks from the image by using a first method, removes streaks from the image by using a second method different from the first method, and executes crop processing on the image from which the streaks are removed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and the like. [Background technology]

[0002] In an image processing device such as a multifunction peripheral, a document is transported by an automatic document feeder (SPF, Single Pass Feeder), and an image of the document is read. If components of the image processing device, such as a document cover or document presser, are gray, the document is read with the background outside the document gray. In this case, streaky noise may be included in the area outside the document in the read image (scanned image) due to dirt on the document reading surface or uneven color of the sheet metal. When the document range is detected for such a scanned image, the detected document range may include the area containing noise. As a result, if the document is cropped normally on a scanned image containing streaky noise, the cropping process will be performed including the background outside the document. This results in an image (output image) that includes the background, which is inconvenient for the user.

[0003] The above example will be described with reference to FIG. 29. FIG. 29(a) is a diagram showing an example of a scanned image P900. In FIG. 29(a), P1 indicates the direction in which the document is read (main scanning direction), and P2 indicates the direction in which the document is fed (sub-scanning direction). Here, as shown by E900 and E902 in FIG. 29(a), streak-like noise may occur in the scanned image. Note that, since streak-like noise generally occurs in the document feed direction, the direction of the streak is known. FIG. 29(b) is a diagram showing a document range L910 detected based on the scanned image P900. As shown in FIG. 29(b), the document range L910 is detected including the background due to the streak-like noise. As such, the streak-like noise may adversely affect the detection of the document range, resulting in erroneous detection of the document range. Furthermore, an image cropped based on the erroneously detected document range may prevent appropriate image processing, such as blank page determination.

[0004] To address these issues, technologies have been proposed for detecting and correcting streak-like noise from scanned images. For example, one proposed technology determines whether a pixel at each pixel position in the main scanning direction is a streak candidate based on its brightness value, and counts the number of pixels determined to be streak candidates in the sub-scanning direction to determine whether the pixel at each pixel position is a streak (see, for example, Patent Document 1). Another proposed technology calculates an average value (line signal value) of pixel values ​​constituting a line for each position in the main scanning direction from image data extracted from an area where streaks are to be detected, and uses the line signal value to detect streaks (see, for example, Patent Document 2). Another proposed technology uses information about the pixel values ​​constituting a line to determine whether a pixel is a sheet pixel or a background pixel based on the pixel values ​​of the pixels included in the line, and determines whether each line is a medium line or a background line (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-29762 [Patent Document 2] Patent No. 6566794 [Patent Document 3] Japanese Patent Application Publication No. 2019-208136 Summary of the Invention [Problem to be solved by the invention]

[0006] Conventional techniques sometimes fail to properly remove streaks, for example, because they do not properly distinguish between edge pixels that constitute streak-like noise and edge pixels of the original document. For example, Patent Document 1 describes a method for determining whether an area is inside or outside a document by determining that pixel values ​​below a specified value of the pixel value level corresponding to the guide plate area are shadows, and then using the detected shadows to determine whether the area is inside or outside the document. However, depending on the state of the document and the transport situation, shadows may not appear, making it impossible to distinguish whether the streaks are inside or outside the document. This can result in erroneous removal of edge pixels of the document that should have been retained. Furthermore, methods that count edge pixels may delete edge pixels of the document if the streaks overlap with the edge pixels of the document. For example, as shown in the scanned image P920 in Figure 30(a), an image including an image area E920 of the original document and an area E922 of the background outside the document may contain streak-like noise E924 that overlaps the edge of the document. Figure 30(b) illustrates a vertical difference image P930 corresponding to the scanned image P920. The streak-like noise E924 in Figure 30(a) appears as edge pixels E930 in the difference image P930. When the edge pixels E930 are deleted based on the difference image P930, the edge that forms the bottom side of the document is also deleted, as shown in the difference image shown in Figure 30(c). In this way, there has been a problem in that an image that the user does not want may be output when image processing such as cropping is performed in a state in which the edge pixels of the document have been lost.

[0007] In view of the above-described problems, an object of the present disclosure is to provide an image processing device and the like that can appropriately remove streaks from an image. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the image processing device of the present disclosure includes an input unit that inputs an image of an original document, and a control unit, and the control unit is characterized in that it removes streaks from the image using a first method, removes streaks from the image using a second method different from the first method, and performs crop processing on the image from which the streaks have been removed.

[0009] The control method of the present disclosure is a control method for an image processing device, and is characterized by including the steps of removing streaks from an input image using a first method, removing streaks from the image using a second method different from the first method, and performing a cropping process on the image from which the streaks have been removed.

[0010] The program of the present disclosure is characterized by causing a computer to realize the following functions: removing streaks from an input image using a first method; removing streaks from the image using a second method different from the first method; and performing a crop process on the image from which the streaks have been removed. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to provide an image processing device and the like that can appropriately remove streaks from an image. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram showing the overall configuration of an image forming apparatus according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a functional configuration of the image forming apparatus according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing a data structure of parameter information in the first embodiment. [Figure 4] FIG. 3 is a diagram showing a data structure of streak position information in the first embodiment. [Figure 5] FIG. 4 is a diagram showing a data structure of edge pixel tally information in the first embodiment. [Figure 6] FIG. 3 is a flowchart showing the flow of main processing in the first embodiment. [Figure 7] FIG. 4 is a flowchart showing the flow of a first streak detection process in the first embodiment. [Figure 8] FIG. 4 is a flowchart showing the flow of a first streak removal process in the first embodiment. [Figure 9] FIG. 10 is a flowchart showing the flow of second streak detection processing in the first embodiment. [Figure 10] FIG. 10 is a flowchart showing the flow of second streak removal processing in the first embodiment. [Figure 11] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 12] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 13] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 14] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 15] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 16] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 17] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 18] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 19] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 20] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 21] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 22] FIG. 3 is a diagram illustrating an example of operation in the first embodiment. [Figure 23] FIG. 10 is a flowchart showing another processing flow of the second streak detection processing. [Figure 24] FIG. 10 is a flowchart showing another processing flow of the second streak removal processing. [Figure 25] FIG. 10 is a diagram showing a data structure of parameter information in the second embodiment. [Figure 26] FIG. 10 is a flowchart showing the flow of main processing in the second embodiment. [Figure 27] FIG. 10 is a diagram illustrating an example of operation in the second embodiment. [Figure 28]FIG. 11 is a flowchart showing the flow of main processing in the third embodiment. [Figure 29] FIG. 1 is a diagram showing a conventional example. [Figure 30] FIG. 1 is a diagram showing a conventional example. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment for carrying out the present disclosure will be described with reference to the drawings. Note that the following embodiment is an example for explaining the present disclosure, and the technical scope of the invention described in the claims is not limited to the following description.

[0014] [1. First embodiment] First, a first embodiment will be described. In the first embodiment, a case will be described in which an image processing device according to the present disclosure is applied to an image forming device 10. The image forming device 10 is an information processing device having a copy function, a scan function, a document print function, etc., and is also called an MFP (Multi-Function Printer / Peripheral, or multifunction device).

[0015] [1.1 Functional Configuration] The functional configuration of an image forming apparatus 10 of this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is an external perspective view of the image forming apparatus 10, and Fig. 2 is a block diagram showing the functional configuration of the image forming apparatus 10.

[0016] As shown in FIG. 2, the image forming apparatus 10 includes a control unit 100, an image input unit 120, an image forming unit 130, a display unit 140, an operation unit 150, a storage unit 160, and a communication unit 190.

[0017] The control unit 100 is a functional unit for controlling the entire image forming apparatus 10. The control unit 100 realizes various functions by reading and executing various programs stored in the storage unit 160, and is configured, for example, by one or more arithmetic units (CPUs (Central Processing Units)). The control unit 100 may also be configured as an SoC (System on a Chip) having multiple functions among those described below.

[0018] The control unit 100 executes the programs stored in the storage unit 160 to function as an image processing unit 102, an edge detection unit 104, a document range detection unit 106, and a skew determination unit 108.

[0019] The image processing unit 102 performs various image-related processes. For example, the image processing unit 102 performs sharpening and tone conversion processes on an image input by the image input unit 120 (hereinafter referred to as an "input image").

[0020] The edge detection unit 104 detects edges from an input document. For example, the edge detection unit 104 selects each pixel of the input image as a pixel of interest and, for each pixel of interest, obtains the difference in brightness between the pixel of interest and a pixel adjacent to the pixel of interest. In this case, when the difference exceeds a predetermined threshold, the edge detection unit 104 detects the pixel of interest as a pixel constituting an edge (hereinafter referred to as an "edge pixel"). The edge detection unit 104 may also generate a difference image in which edge pixels are designated as white pixels and non-edge pixels are designated as black pixels. In other words, the difference image may be a binary image composed of pixels with pixel values ​​(brightness) of 0 or 1, or a grayscale image composed of pixels with pixel values ​​(brightness) ranging from 0 to 255. In this embodiment, the difference image will be described as a grayscale image in which white pixels have a brightness of 255 and black pixels have a brightness of 0.

[0021] In this embodiment, a difference image (main scanning direction difference image) generated based on the difference in brightness between pixels adjacent in the main scanning direction is called a vertical difference image. Edges extending in the left-right direction (edges corresponding to streaks and the top and bottom edges of the document) are detected using the vertical difference image. Note that in this embodiment, streaks (abnormal pixels) refer to edges included in the input image that are included in an area outside the document image (outside the document). Also, in this embodiment, a difference image (sub scanning direction difference image) generated based on the difference in brightness between pixels adjacent in the sub scanning direction is called a horizontal difference image. Edges extending in the up-down direction (edges corresponding to the left and right edges of the document) are detected using the horizontal difference image.

[0022] The edge detection unit 104 may generate a difference image by applying an edge detection filter, such as a Prewitt filter or a Sobel filter, to the input image. For example, the edge detection unit 104 may apply an edge detection filter to the input image in the vertical direction (main scanning direction) to generate a difference image in the vertical direction, and may apply an edge detection filter to the input image in the horizontal direction (sub-scanning direction) to generate a difference image in the horizontal direction. The edge detection unit 104 may also modify the difference image by performing binarization processing on the difference image or applying a high-pass filter to the difference image so that edge pixels in the difference image become white pixels. In this way, the edge detection unit 104 may generate a difference image using a known method.

[0023] The document range detection unit 106 detects the range in which the document image appears (document range) from the input image. The document range detection unit 106 detects the document range using, for example, the method described in Patent Document 2. Note that the document range detection unit 106 may determine that, among the edges detected from the input image, the edge closest to the edge of the input image is the edge corresponding to the document edge, and detect a rectangular area that contacts the edge corresponding to the document edge as the document range. In other words, the document range detection unit 106 may detect the document range using an existing method.

[0024] The skew determination unit 108 determines whether or not a document has been skewed, and determines the angle of the document that has been read. For example, in an image forming apparatus 10 in which a document presser or a document cover are provided with multiple light-emitting APS (Auto Paper Selector) sensors in the main scanning direction, and a light-receiving APS sensor that receives light from the light-emitting APS sensors is provided on the document surface side, the skew determination unit 108 uses the sensors to determine whether or not a document has been skewed. For example, if there is a difference in the light detection times of the light-receiving APS sensors, the skew determination unit 108 determines that a document has been skewed.

[0025] The skew determination unit 108 may determine whether or not the document is skewed based on edges detected from the input image. For example, the skew determination unit 108 may obtain the document angle based on the document range detected by the document range detection unit 106, or may determine that the document is skewed if the document angle exceeds a predetermined angle.

[0026] The image input unit 120 inputs an image to the image forming apparatus 10. For example, the image input unit 120 is configured with a scanner device or the like that reads a document placed on a document table. The image input unit 120 may also be configured with an automatic document feeder (SPF, Single Pass Feeder) and a scanner device that reads an image of a document transported by the automatic document feeder. The scanner device is a device that converts an image into an electrical signal using an image sensor such as a CCD (Charge Coupled Device) or a CIS (Contact Image Sensor), and quantizes and encodes the electrical signal. When the scanner device reads the image of the document, the image of the document is input to the image forming apparatus 10 as digital data.

[0027] Image forming unit 130 forms (prints) an image on a recording medium such as recording paper. Image forming unit 130 is configured, for example, by a printing device such as a laser printer that uses an electrophotographic method. Image forming unit 130, for example, feeds recording paper from paper feed tray 132 in FIG. 1, forms an image on the surface of the recording paper, and discharges the recording paper from paper discharge tray 134.

[0028] The display unit 140 displays various types of information. The display unit 140 is configured by a display device such as an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, or a micro LED (Light Emitting Diode) display.

[0029] The operation unit 150 accepts operation instructions from a user who uses the image forming apparatus 10. The operation unit 150 is configured with input devices such as key switches (hard keys) and touch sensors. The touch sensor may detect input by contact (touch) using any common detection method, such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitance method. The image forming apparatus 10 may be equipped with a touch panel in which the display unit 140 and the operation unit 150 are integrally formed.

[0030] The storage unit 160 stores various programs and various data necessary for the operation of the image forming apparatus 10. The storage unit 160 is configured by a storage device such as a solid state drive (SSD) or a hard disk drive (HDD), which is a semiconductor memory.

[0031] The memory unit 160 allocates the following memory areas: an input image memory area 162 for storing an input image; a differential image memory area 164 for storing a differential image; a parameter information memory area 166; a streak position information memory area 168; and an edge pixel aggregate information memory area 170.

[0032] The parameter information storage area 166 stores information (parameter information) that associates a parameter name with a parameter value corresponding to the parameter name. For example, as shown in FIG. 3, the parameter information includes a parameter name (e.g., "ROW_STREAK_MAXSIZE") and a parameter value (e.g., "10") corresponding to the parameter name.

[0033] In this embodiment, the parameter information storage area 166 stores the following parameters: (1)ROW_STREAK_MAXSIZE ROW_STREAK_MAXSIZE indicates the threshold for the allowable number of streaks present in the input image. (2) SEARCH_MAX SEARCH_MAX indicates a threshold value for the number of search pixels when edge pixels that form streaks are searched for from the image edge of the differential image in the first streak detection process described later. (3) SEARCH_STOP SEARCH_STOP indicates the threshold number of pixels at which streak removal is stopped when streaks are removed from an input image in the first streak removal process described later. (4)HIST_MAX HIST_MAX indicates a threshold value for the number of edge pixels contained in a line of interest when it is determined that a line has a streak in the second streak removal process described later.

[0034] The parameter values ​​corresponding to the above-mentioned parameter names may be expressed numerically or as a percentage such as "1% of CMAX." The parameter values ​​may be set in advance or may be set by the user.

[0035] Also, CMAX is the maximum value of the column number (column width) of the vertical difference image. The column number is the number of horizontal pixels from the top left pixel of the image, which is the origin (0,0), to the pixel of interest. In other words, the column number corresponds to x when the position of a pixel in the image is expressed as coordinates (x,y), where x is the number of horizontal pixels from the origin to the pixel of interest and y is the number of vertical pixels. The row number corresponds to y in the coordinates (x,y).

[0036] In the following description, the parameter value corresponding to the parameter name "ROW_STREAK_MAXSIZE" will be written as ROW_STREAK_MAXSIZE. Similarly, the parameter value corresponding to the parameter name "ROW_STREAK_MAXSIZE" will be written as ROW_STREAK_MAXSIZE. Furthermore, the parameter value corresponding to the parameter name "SEARCH_STOP" will be written as SEARCH_STOP. Furthermore, the parameter value corresponding to the parameter name "HIST_MAX" will be written as HIST_MAX.

[0037] The streak position information storage area 168 stores information (streak position information) about the line (position) where the streak exists. The streak position information includes, for example, an index number (e.g., "0") and a line number (e.g., "259"), as shown in Fig. 4.

[0038] The index number is a consecutive number assigned to identify streak position information. The index number is, for example, an integer equal to or greater than 0. In this embodiment, a row is a set of pixels (pixel group) with the same row number, and is a group of pixels that are consecutive in the same direction as the sub-scanning direction. A row with row number "259" indicates a pixel group whose coordinates are anywhere from (0,259) to (CMAX,259).

[0039] The edge pixel tally information storage area 170 stores, for each row of the input image, information (edge ​​pixel tally information) that tally the number of edge pixels included in a row of interest. The edge pixel tally information includes, for example, a row number (e.g., "0") and the number of edge pixels (e.g., m "0"), as shown in FIG.

[0040] The communication unit 190 communicates with external devices via a LAN (Local Area Network) or a WAN (Wide Area Network). The communication unit 190 is configured by, for example, a communication device or a communication module such as a NIC (Network Interface Card) used in a wired / wireless LAN.

[0041] [1.2 Processing flow] The flow of processing executed by image forming apparatus 10 in this embodiment will be described with reference to Figures 6 to 10. The processing shown in Figures 6 to 10 is executed by control unit 100 reading out a program stored in storage unit 160. The processing shown in Figures 6 to 10 is also executed when a user performs an operation to start execution of a job that reads a document, such as a copy job or a scan job.

[0042] [1.2.1 Main Processing] First, the flow of the main processing will be described with reference to Fig. 6. The control unit 100 controls the image input unit 120 to read an original document and acquire a scanned image (input image) of the original document (step S100). At this time, the control unit 100 stores the input image in the input image storage area 162.

[0043] Note that image forming apparatus 10 may color components such as a document cover and a document presser gray so that the background portion outside the document is read in gray when the document is read. Image input unit 120 may also read the document and the area outside the document (background portion) including the outside of the document by making the reading range larger than the area where the document is placed. In this way, by making the reading background of the document gray and increasing the reading range, image forming apparatus 10 can read the image of the document under conditions that allow appropriate edge detection of the boundary between the inside and outside of the document.

[0044] Next, the control unit 100 (edge ​​detection unit 104) detects edges of the input image (step S102). At this time, the edge detection unit 104 generates a difference image and stores the difference image in the difference image storage area 164.

[0045] Next, the control unit 100 executes a first streak detection process and a first streak removal process to remove first-method streaks from the input image (step S104 → step S106). Note that in this embodiment, the first method will be described as a method in which edges detected near sides parallel to the main scanning direction in the input image are detected as streaks and the detected streaks are removed, and a vertical difference image and a horizontal difference image are used in combination to remove streaks.

[0046] Furthermore, the control unit 100 executes a second streak detection process and a second streak removal process to remove streaks by a second method different from the first method (step S108 → step S110). Note that in this embodiment, the second method will be described as a method that uses a pixel count histogram of edge pixels to detect, as streaks, pixels included in rows where the pixel count of edge pixels is greater than a threshold value HIST_MAX, and removes the detected streaks.

[0047] In this way, the control unit 100 detects streaks from the input image using two types of methods and removes the streaks detected by each method from the input image. Details of the first streak detection process, first streak removal process, second streak detection process, and second streak removal process will be described later.

[0048] Next, the control unit 100 (document area detection unit 106) detects the document area from the input image from which the streaks have been removed (step S112). The control unit 100 (skew determination unit 108) then performs skew determination (step S114). For example, the skew determination unit 108 determines whether or not skew has occurred in the document, and obtains the angle of the document.

[0049] Next, the control unit 100 (image processing unit 102) performs skew correction and cropping on the input image from which the streaks have been removed (step S116). For example, if the image processing unit 102 determines in step S114 that the document is skewed, it corrects the skew of the document by rotating the input image in the opposite direction to the tilt by the angle of the document. Furthermore, the image processing unit 102 crops the input image based on the document range detected in step S112.

[0050] Next, the control unit 100 outputs the input image that has been subjected to skew correction and cropping (step S118). For example, the control unit 100 controls the image forming unit 130 to form and output the corrected input image. Note that the control unit 100 may output the data of the corrected input image by storing it in the storage unit 160 or by transmitting it to another device.

[0051] [1.2.2 First streak detection process] Next, the flow of the first streak detection process will be described with reference to Fig. 7. The first streak detection process is a process for detecting the row (position) where a streak has occurred based on the vertical difference image. In the following description, P[r,c] indicates the pixel at row r and column c (the pixel at coordinates (c,r)) in the vertical difference image.

[0052] First, the control unit 100 assigns 0 to variables R and tmp (step S200), and also assigns 0 to variable C (step S202).

[0053] Next, the control unit 100 accesses (references) P[R,C] and P[R,CMAX-C] (step S204). Furthermore, the control unit 100 determines whether or not either the pixel value of P[R,C] or the pixel value of P[R,CMAX-C] is equal to 255, i.e., whether or not it is an edge pixel (step S206).

[0054] If either the pixel value of P[R,C] or the pixel value of P[R,CMAX-C] is equal to 255, the control unit 100 stores the value of the variable R as streak position information in the streak position information storage area 168 (step S206; Yes → step S208). For example, the control unit 100 generates streak position information in which the value of the variable tmp is the index number and the value of the variable R is the line number, and stores the streak position information in the streak position information storage area 168.

[0055] In this way, by performing the processes in steps S204 and S206, the control unit 100 uses the vertical difference image to detect edges near sides parallel to the main scanning direction (the left and right edges of the vertical difference image). Specifically, the control unit 100 accesses pixels that are C pixels away in the sub-scanning direction from the left and right edges of the vertical difference image, and if at least one of the two accessed pixels is an edge pixel, the control unit 100 detects the edge pixel as a streak. Then, the control unit 100 stores the row containing the streak as streak position information.

[0056] Next, the control unit 100 determines whether the value of the variable tmp is equal to ROW_STREAK_MAXSIZE (step S210). If the value of the variable tmp is equal to ROW_STREAK_MAXSIZE, the control unit 100 executes error processing (step S210; Yes). For example, as error processing, the control unit 100 may display a message indicating an error on the display unit 140, or terminate the processing shown in FIG. 7 and the processing from step S106 onwards shown in FIG. 6. In this way, when the control unit 100 detects streaks of ROW_STREAK_MAXSIZE or more from the input image, it can notify the user of information such as that an abnormality has occurred.

[0057] On the other hand, if the value of the variable tmp is not equal to ROW_STREAK_MAXSIZE, the control unit 100 adds 1 to the value of the variable tmp and assigns the result to the variable tmp (step S210; No->step S212). That is, the control unit 100 increments the variable tmp.

[0058] Furthermore, in step S206, if neither the pixel value of P[R,C] nor the pixel value of P[R,CMAX-C] is 255, the control unit 100 determines whether the value of the variable C is equal to SEARCH_MAX (step S206; No → step S214). If the variable C is not equal to SEARCH_MAX, the control unit 100 increments the variable C and returns to step S204 (step S214; No → step S216 → step S204). In this way, by performing the processes of steps S204 and S206 while incrementing the variable C, the control unit 100 can search for edge pixels from the left and right ends of the vertical difference image in the sub-scanning direction.

[0059] After executing the process of step S212 or when it is determined in step S214 that the value of the variable C is equal to SEARCH_MAX (step S214; Yes), the control unit 100 determines whether the value of the variable R is equal to RMAX (step S218), where RMAX is the maximum value of the row number (row width) of the vertical difference image.

[0060] 7 (step S218; Yes). On the other hand, if the value of variable R is not equal to RMAX, control unit 100 increments variable R and returns to step S202 (step S218; No → step S220 → step S202).

[0061] [1.2.3 First streak removal process] Next, the flow of the first streak removal process will be described with reference to Fig. 8. The first streak removal process is a process for removing streaks detected by the first streak detection process by using both the vertical difference image and the horizontal difference image. In the following description, pixel P[R,C] indicates the pixel at row R and column C in the vertical difference image, and pixel Q[R,C] indicates the pixel at row R and column C in the horizontal difference image.

[0062] First, the control unit 100 assigns 0 to a variable tmp (step S300). Then, the control unit 100 acquires streak position information having an index number of tmp from the streak position information storage area 168 (step S302), and assigns the value of the line number included in the acquired streak position information to a variable R (step S304).

[0063] Next, the control unit 100 assigns 0 to the variable C (step S306) and accesses the pixel P[R,C] (step S308). Furthermore, the control unit 100 accesses the pixel Q[R,C] and determines whether the pixel value of the pixel Q[R,C] is equal to 255, i.e., whether the pixel is an edge pixel (step S310).

[0064] If the pixel value of pixel Q[R,C] is not equal to 255, the control unit 100 assigns 0 to the pixel value of pixel P[R,C] (step S310; No→step S312). That is, the control unit 100 replaces the pixel at row R and column C with a pixel other than an edge pixel.

[0065] At this time, control unit 100 reads the input image acquired in step S100 from input image storage area 162, and replaces the pixel at row R, column C of the input image with a pixel whose color is based on the colors of the surrounding pixels. For example, control unit 100 replaces the color of the pixel at row R, column C of the input image by substituting a pixel value corresponding to the background color of the input document or an average value of the surrounding pixel values ​​for the pixel value of the pixel at row R, column C of the input image. In this way, even if the pixel at row R, column C of the input image is an edge pixel, control unit 100 can remove the edge pixel by replacing the color of the edge pixel.

[0066] Note that the control unit 100 may change the pixel value of the pixel at row R, column C of the input image when the pixel at row R, column C in the vertical difference image is 255 (when the pixel is an edge pixel). In this way, the control unit 100 can replace the color of the edge pixel only when the pixel at row R, column C of the input image is an edge pixel.

[0067] Next, the control unit 100 increments the value of the variable C (step S314), and determines whether the value of the variable C is equal to SEARCH_STOP (step S316). If the value of the variable C is not equal to SEARCH_STOP, the control unit 100 returns to step S308 (step S316; No→step S308).

[0068] In this way, the control unit 100 increments the value of the variable C and determines in step S310 whether the pixel value of pixel Q[R,C] is equal to 255. If the pixel value of pixel Q[R,C] is 255, the pixel is considered to be an edge corresponding to the left edge of the document. Therefore, by processing step S310, the control unit 100 detects the edge pixel closest to the edge (left edge) of the horizontal difference image as the pixel of the left edge of the document. Furthermore, the control unit 100 repeatedly executes step S312 until the left edge of the document is detected, thereby replacing pixels in the Rth row containing the streak that are included between the edge (left edge) parallel to the main scanning direction and the edge (left edge) of the document with other pixels. In this way, the control unit 100 can replace only pixels outside the document in the left portion of the input image with other pixels.

[0069] On the other hand, if the value of variable C is equal to SEARCH_STOP, the control unit 100 assigns 0 to variable C (step S316; Yes→step S318). Note that even if the control unit 100 determines in step S310 that the pixel value of Q[R,C] is equal to 255, it executes the process in step S316 (step S310; Yes→step S318).

[0070] Next, the control unit 100 accesses pixel P[R, CMAX-C] (step S320). The control unit 100 also accesses pixel Q[R, CMAX-C] and determines whether the pixel value of pixel Q[R, CMAX-C] is equal to 255 (step S322).

[0071] If the pixel value of pixel Q[R, CMAX-C] is not equal to 255, the control unit 100 assigns 0 to the pixel value of pixel P[R, CMAX-C] (step S322; No→step S324). The process in step S324 is the same as the process in step S312.

[0072] Next, the control unit 100 increments the value of the variable C (step S326) and determines whether the value of the variable C is equal to SEARCH_STOP (step S328). If the value of the variable C is not equal to SEARCH_STOP, the control unit 100 returns to step S320 (step S328; No→step S320).

[0073] In this way, the control unit 100 determines whether the pixel value of pixel Q[R, CMAX-C] is equal to 255 in step S322 while incrementing the value of variable C. As a result, the control unit 100 detects the edge pixel closest to the edge (right edge) of the horizontal difference image as the pixel at the right edge of the document. Furthermore, the control unit 100 repeatedly executes step S324 until the right edge of the document is detected, and therefore replaces pixels in the Rth row containing streaks that are included between the side (right edge) parallel to the main scanning direction and the edge (right edge) of the document with other pixels. This allows the control unit 100 to replace only pixels outside the document in the right portion of the input image with other pixels. Furthermore, by performing the pixel replacement process until the edge of the document is detected in both the left and right portions of the input image, the control unit 100 can appropriately remove streaks outside the document even if the positions of the edge of the document are different in the left and right portions of the input image.

[0074] On the other hand, when the value of the variable C is equal to SEARCH_STOP, the control unit 100 determines whether the value of the variable tmp plus 1 is equal to the total number of pieces of streak position information stored in the streak position information storage area 168 (step S328; Yes → step S330). Note that even when the control unit 100 determines in step S322 that the pixel value of pixel Q[R, CMAX-C] is equal to 255, it also executes the process in step S330 (step S322; Yes → step S330).

[0075] 8 (step S330; Yes), if the value obtained by adding 1 to the value of the variable tmp is equal to the total number of pieces of streak position information stored in the streak position information storage area 168. On the other hand, if the value obtained by adding 1 to the value of the variable tmp is not equal to the total number of pieces of streak position information stored in the streak position information storage area 168, the control unit 100 increments the variable tmp and returns to step S302 (step S330; No → step S332 → step S302).

[0076] [1.2.4 Second streak removal process] Next, the flow of the second streak detection process will be described with reference to Fig. 9. The second streak detection process is a process for generating a histogram of the number of edge pixels for each row. In the following description, pixel P[R,C] refers to the pixel in row R and column C in the vertical difference image.

[0077] First, the control unit 100 assigns 0 to a variable R (step S400). The control unit 100 also assigns 0 to a variable C and assigns 0 to a variable tmp (step S402).

[0078] Next, the control unit 100 accesses the pixel P[R,C] (step S404) and determines whether the pixel value of the pixel P[R,C] is equal to 255 (step S406).

[0079] If the pixel value of pixel P[R,C] is equal to 255, the control unit 100 increments the variable tmp (step S406; Yes → step S408). If the pixel value of pixel P[R,C] is not equal to 255, the control unit 100 omits the process in step S408 (step S406; No). That is, the control unit 100 counts the number of edge pixels detected from the vertical difference image for each position (row) in the sub-scanning direction.

[0080] Next, the control unit 100 increments the variable C (step S410) and determines whether the value of the variable C is equal to CMAX (step S412). If the value of the variable C is not equal to CMAX, the control unit 100 returns to step S404 (step S412; No→step S404).

[0081] On the other hand, if the value of variable C is equal to CMAX, the control unit 100 stores the value of tmp as edge pixel tally information in the edge pixel tally information storage area 170 (step S414). For example, the control unit 100 generates edge pixel tally information in which the value of variable R is the row number and the value of variable tmp is the number of edge pixels, and stores the edge pixel tally information in the edge pixel tally information storage area 170. In this way, the number of edge pixels for each row is stored as edge pixel tally information. Furthermore, the edge pixel tally information stored in the edge pixel tally information storage area 170 becomes information indicating a histogram of the number of edge pixels.

[0082] Next, the control unit 100 increments the variable R (step S416) and determines whether the value of the variable R is equal to RMAX (step S418). If the value of the variable R is equal to RMAX, the control unit 100 ends the processing shown in Fig. 9 (step S418; Yes). On the other hand, if the value of the variable R is not equal to RMAX, the control unit 100 returns to step S402 (step S418; No → step S402).

[0083] [1.2.5 Second streak removal process] Next, the flow of the second streak removal process will be described with reference to Fig. 10. The second streak removal process is a process for detecting the row (position) where a streak occurs based on the histogram generated by the second streak detection process, and removing the detected streak. In the following description, pixel P[R,C] indicates the pixel at row R and column C in the vertical difference image.

[0084] First, the control unit 100 assigns 0 to a variable R (step S500). The control unit 100 also assigns 0 to a variable C (step S502).

[0085] Next, the control unit 100 determines whether the number of edge pixels in the Rth row is greater than HIST_MAX (step S504). If the number of edge pixels in the Rth row is greater than HIST_MAX, the control unit 100 determines that a streak has occurred in the Rth row of the input image, and detects the pixels included in the Rth row as a streak.

[0086] If the number of edge pixels in the Rth row is greater than HIST_MAX, the control unit 100 accesses pixel P[R,C] (step S504; Yes → step S506) and assigns 0 to the pixel value of pixel P[R,C] (step S508). Note that the process in step S508 is the same as the process in step S312 in Fig. 8. Furthermore, the control unit 100 increments the variable C (step S510).

[0087] Next, the control unit 100 determines whether the value of the variable C is equal to CMAX (step S512). If the value of the variable C is not equal to the value of CMAX, the control unit 100 returns to step S506 (step S512; No→step S506).

[0088] In this way, by repeatedly executing the processes from step S506 to step S510, the control unit 100 can replace the pixels included in the line in which a streak is detected with other pixels. As a result, even if a pixel included in the line in which a streak is detected is an edge pixel, the control unit 100 can remove the edge pixel by replacing the color of the edge pixel.

[0089] If the control unit 100 determines in step S504 that the number of edge pixels in the Rth row is equal to or less than HIST_MAX, it omits the processes from step S506 to step S512 (step S504; No).

[0090] Next, the control unit 100 increments the variable R (step S514) and determines whether the value of the variable R is equal to the value of RMAX (step S516). If the value of the variable R is equal to the value of RMAX, the control unit 100 ends the processing shown in Fig. 10 (step S516; Yes). On the other hand, if the value of the variable R is not equal to the value of RMAX, the control unit 100 returns to step S502 (step S516; No → step S502).

[0091] [1.3 Example of operation] Next, an example of operation of this embodiment will be described. Note that in the explanation of the example of operation, a case will be described in which the scanned image P900 shown in Fig. 27(a) is acquired as the input image.

[0092] FIG. 11 is a diagram showing a difference image of an input image. FIG. 11(a) is a vertical difference image P100 of a scanned image P900. E100 and E102 in FIG. 11(a) indicate edge pixels of E900 and E902, which are streak-like noises that have occurred in the scanned image P900. In this way, the streak-like noises that have occurred in the input image are shown as edge pixels in the vertical difference image. Based on the edge pixels in the vertical difference image, the row (position) where the streak has occurred is detected.

[0093] Figure 11(b) is a horizontal difference image P110 of the scanned image P900. Regions R110 and R112 in Figure 11(b) are ranges that include edge pixels of the document. In this embodiment, the range of edge pixels (streaks) to be removed is determined depending on the state of the pixels in the horizontal difference image that correspond to the line in which the streak was detected (pixels E100 and E102 in Figure 11(b)).

[0094] Fig. 12(a) is a diagram showing an area E110 including edge pixels in the vertical difference image P100 shown in Fig. 11(a). Note that in Fig. 12(a), pR indicates the row direction, and pC indicates the column direction. In the following description, the row number of a pixel of interest is denoted by r, and the column number of a pixel of interest is denoted by c. Here, as shown in Fig. 12(a), the position of c=0 is the left edge position of the vertical difference image, and the position of c=CMAX is the right edge position of the vertical difference image.

[0095] FIG. 12B shows an example of the operation of the first streak detection process. When the row of interest is the Rth row (r=R), edge pixels are searched for among the pixels included in E120 in FIG. 12B (pixels in the Rth row). At this time, pixels P[R,C] and P[R,CMAX-C] are accessed while the value of variable C changes from 0 to SEARCH_MAX. As a result, edge pixels are searched for from the pixel at the left end (c=0) of the vertical difference image toward P3, and edge pixels are also searched for from the right end (c=CMAX) of the vertical difference image toward P4. For example, when the column of interest is the Cth column (c=C), it is determined whether pixel P120 located in the Cth column and pixel P122 located in the (CMAX-C)th column in the Rth row are edge pixels. Furthermore, if an edge pixel is detected while the value of variable C changes from 0 to SEARCH_MAX, the row number (value of variable R) where the edge pixel exists is stored as streak position information.

[0096] 13A and 13B are diagrams showing an example of the operation of determining the range in which edge pixels (streaks) are to be removed in the first streak removal process. FIG. 13A shows the horizontal difference image P110 shown in FIG. 11B and the position corresponding to the region E110 shown in FIG. 12A. In FIG. 13A, pR indicates the row direction, and pC indicates the column direction. The position of c=0 is the left edge position of the horizontal difference image, and the position of c=CMAX is the right edge position of the horizontal difference image. The positions of c=0 and c=CMAX are the same in the vertical difference image and the horizontal difference image.

[0097] Fig. 13(b) is an enlarged view of area E110 shown in Fig. 13(a). In the first streak removal process, edge pixels are searched for in the direction of P5 among pixels in rows where edge pixels exist. For example, when the row where an edge pixel exists is the Rth row (r=R), edge pixels are searched for among the pixels included in E130 in Fig. 13(b) (pixel group in the Rth row).

[0098] First, the value of variable C changes from 0 to SEARCH_STOP, searching for an edge pixel from the pixel at the left end (c=0) of the horizontal difference image toward P5. At this time, the pixel at row R and column C in the input image is replaced with a pixel whose color is based on the colors of the surrounding pixels until an edge pixel is found. In the example of Figure 13(b), the edge pixel is at position c=6. Therefore, in the input image, the pixels at row R, c=0 to c=5, are replaced with pixels whose color is based on the colors of the surrounding pixels.

[0099] Next, as the value of variable C changes from 0 to SEARCH_STOP, an edge pixel is searched for in the direction of P6, starting from the pixel at the left end (c=CMAX) of the horizontal difference image. At this time, the pixel in row R (CMAX-C) of the input image is replaced with a pixel whose color is based on the color of the surrounding pixels until an edge pixel is found. In the example of Figure 13(b), there is no edge pixel between the position c=CMAX and the position c=CMAX-SEARCH_STOP. Therefore, in the input image, the pixel located in the Rth row between c=CMAX and c=CMAX-SEARCH_STOP is replaced with a pixel whose color is based on the color of the surrounding pixels.

[0100] Fig. 14 is a diagram showing an example of the operation of the first streak removal process. E120 and E124 in Fig. 14 show examples of vertical difference images. E122 and E126 in Fig. 14 show horizontal difference images. E120 and E122 show difference images before the first streak removal process is performed, and E124 and E126 show difference images after the first streak removal process is performed.

[0101] As shown in E120, in the vertical difference image, edge pixels are included in the row r=R from positions c=0 to c=3. Therefore, a streak is detected in the row r=R. On the other hand, as shown in E122, in the horizontal difference image, an edge pixel appears in the column c=6 in the row r=R. Therefore, in the row r=R, the edge pixels appearing from c=0 to c=5 are removed.

[0102] As a result, edge pixels present at c=0 to c=5 in the row where r=R are removed, as shown at E124 in Fig. 14. Note that the horizontal difference image is only referenced for removing streaks, and therefore, as shown at E122 and E126, there is no change before and after the streak removal.

[0103] Fig. 15 is a diagram showing an example of the operation of the second streak detection process. Fig. 15(a) is a diagram showing an example of an input image P130. Fig. 15(b) shows a difference image P132, which is a difference image of the vertical direction of the input image P130. Here, as shown by E132 in the difference image P132, edge pixels appear in the horizontal direction from one end of the input image to the other.

[0104] Figure 15(c) is a histogram showing the number of edge pixels per row for the difference image P132. The vertical axis of the histogram indicates the row number, and the horizontal axis indicates the number of pixels contained in the row corresponding to the row number. The dotted line in Figure 15(c) indicates the number of pixels corresponding to HIST_MAX. Here, as shown in E134, the number of edge pixels in the row shown in E132 in Figure 15(b) is greater than HIST_MAX. Therefore, streaks are detected from the row corresponding to E132 in Figure 15(b).

[0105] Fig. 16 is a diagram showing an example of the operation of the second streak removal process. Fig. 16 is an enlarged view of part E132 in Fig. 15(b). Here, in the second streak detection process, assume that the row number of the row with the number of pixels greater than HIST_MAX is R. In this case, edge pixels are removed all at once from the pixel group included in the Rth row (E136 in Fig. 16).

[0106] FIG. 17 is a diagram showing the advantages of using two streak detection methods. FIG. 17(a) is a diagram showing an example of a streak that is not detected by the first streak detection process but is detected by the second streak removal process. FIG. 17(a) is a diagram showing an example of an input image P150. A streak R150 appears in input image P150. However, as shown in E150 and E151 in FIG. 17, the portion where the streak occurs is located at a distance of SEARCH_MAX or more from the edge of the input image (scanner image). In this case, the streak R150 is not detected by the first streak detection process.

[0107] Generally, due to the characteristics of scanners, streaks often occur from the edges of the input image. However, in rare cases, streaks may occur away from the edges of the input image due to the optical conditions of the scanner. In this case, even if the edge of the streak is searched from the edge of the input image, as in the first streak detection process, the position of the streak will not be detected, and as a result, the streak will not be removed. However, by using a method using a histogram, as in the second streak detection process, it is possible to detect the position of the streak even if it does not occur from the edge of the input image. Streaks detected in this way are removed by the second streak removal process.

[0108] Furthermore, by searching for the edge of the streak from the edge of the input image using the first streak detection process, it is possible to remove only the edge of the streak without removing the edge of the document. Furthermore, even if the edge of the document overlaps with the streak, the edge of the document can be left intact.

[0109] For example, FIG. 17(b) shows a vertical difference image P152 after the first streak removal process and the second streak removal process are performed on the vertical difference image shown in FIG. 30(b). The streak appearing at E930 in FIG. 30(b) is removed by the first streak removal process, which removes edge pixels from the edge of the input image to the position of the edge pixel of the document or the position of SEARCH_STOP. At this time, a maximum of SEARCH_STOP × 2 edge pixels are removed from the streak appearing at E930. Subsequently, streaks are detected by the second streak detection process. However, if the number of edge pixels in E930 (maximum CMAX - SEARCH_STOP × 2) is less than HIST_MAX, no streaks are detected from the row (position) of E930. As a result, the difference image after the second streak removal process retains some edge pixels on the bottom side of the document, as shown at E152 in FIG. 17(b). Leaving the edge pixels of the document will prevent any adverse effects on subsequent image processing.

[0110] FIG. 18 shows an example of processing performed by the image forming apparatus 10 according to this embodiment. Streaks appear at the edges of the input image P160. In this case, edge detection processing results in a vertical difference image P162 and a horizontal difference image P164. The first streak detection processing detects the positions of streaks E160 and E161, and the streaks are removed based on the horizontal difference image P164. That is, the horizontal difference image P164 is indirectly used to search for the area in which the streaks should be removed. As a result, streaks occurring outside the original document are removed, and a vertical difference image P166 is obtained. The vertical difference image P166 does not contain any rows containing edge pixels equal to or greater than HIST_MAX. Therefore, no streaks are detected in the second streak detection processing, and a vertical difference image P168 is obtained after the second streak removal processing. In this way, the streaks E160 and E161 occurring outside the original document are removed from the input image P160.

[0111] 19 shows another example of processing by the image forming apparatus 10 in this embodiment. A streak E170 appears in an input image P170. In this case, edge detection processing is performed to obtain a vertical difference image P172 and a horizontal difference image P174. Here, as shown in FIG. 20, there are cases where the pixel group on the Rth row (pixels included in E179 in FIG. 20) satisfies the following (1) and (2): (1) The position of the edge pixel P179a, which is the pixel at the left end of the streak, is to the right of SEARCH_MAX. In other words, the value of c, which indicates the horizontal position of the edge pixel P179a, is greater than SEARCH_MAX. (2) The position of the edge pixel P179b, which is the pixel at the right end of the streak, is to the left of (CMAX-SEARCH_MAX). In other words, the value of c, which indicates the horizontal position of the edge pixel P179b, is less than (CMAX-SEARCH_MAX). In such a case, the streak that occurs on the Rth row is not removed by the first streak removal process.

[0112] If the streak E170 is not removed by the first streak detection process, a vertical difference image P176 is obtained. Here, in the vertical difference image P176, the row in which the streak E170 occurs contains edge pixels equal to or greater than HIST_MAX. Therefore, the position of the streak corresponding to E170 is detected in the second streak detection process, and after the second streak removal process is performed, a vertical difference image P178 is obtained. The streak has been removed from the vertical difference image P178, as shown in E178. In this way, the streak E170 is removed from the input image P170. That is, if the position of the pixel at the edge of the streak is greater than SEARCH_MAX and less than (CMAX - SEARCH_MAX), the streak E170 in FIG. 19 is not removed by the first streak removal process. However, even in such a case, the streak is removed by the second streak removal process if the total number of edge pixels in the row in which the streak occurs is equal to or greater than HIST_MAX.

[0113] FIG. 21 shows another example of processing performed by the image forming apparatus 10 according to this embodiment. A streak E180 appears in an input image P180. In this case, a vertical difference image P182 and a horizontal difference image P184 are obtained by edge detection processing. The streak E180 is located at a distance equal to or greater than SEARCH_MAX from the edge of the input image. In this case, the streak E180 is not detected by the first streak detection processing. Therefore, after the first streak removal processing, a vertical difference image P186 is obtained. The streak E180 remains in the vertical difference image P186. The streak E180 is detected by the second streak detection processing. The streak E180 is then removed by the second streak removal processing, resulting in a vertical difference image P188. As shown in E188, the streak has been removed from the vertical difference image P188. In this way, the streak E180 is removed from the input image P180.

[0114] 22 shows another example of processing by the image forming apparatus 10 in this embodiment. A streak E190 appears in the input image P190, overlapping the edge of the bottom side of the document P191. In this case, a vertical difference image P192 and a horizontal difference image P194 are obtained by edge detection processing. The streak E190 is detected by the first streak detection processing, and a portion of the streak E190 is removed by the first streak removal processing. For example, as shown by E196 and E197 in the vertical difference image P196 after the first streak removal processing, edge pixels appearing in the range from the edge of the document to SEARCH_STOP are removed.

[0115] Next, a second streak detection process is performed to detect positions where the number of edge pixels is equal to or greater than HIST_MAX. Here, the position of the partially removed streak E190 is not detected. As a result, after the second streak removal process is performed, a vertical difference image P198 is obtained. For example, some edge pixels remain in the vertical difference image P198. These edge pixels are the edge pixels on the bottom edge of the document. In other words, some of the streak E190 is removed from the input image P190 without removing all of the edge pixels on the bottom edge of the document.

[0116] It should be noted that, apart from the above description, the order of steps may be changed or some steps may be omitted as long as there is no contradiction. For example, the control unit 100 may execute the first streak detection process and the second streak detection process, and then execute the first streak removal process and the second streak removal process. Alternatively, the control unit 100 may execute the second streak detection process and the second streak removal process, and then execute the first streak detection process and the first streak removal process. Alternatively, the first streak detection process and the first streak removal process and the second streak detection process and the second streak removal process may be executed in parallel.

[0117] Furthermore, the control unit 100 may detect a line in which a streak has occurred in the second streak detection process. In this case, the control unit 100 executes the process shown in Fig. 23 as the second streak detection process instead of the process shown in Fig. 9. Note that the same processes as those shown in Fig. 10 are denoted by the same reference numerals, and their explanations will be omitted.

[0118] 23, the control unit 100 stores the edge pixel summary information (edge ​​pixel histogram information) by executing the processes from step S400 to step S418. Next, the control unit 100 deletes the streak position information from the streak position information storage area 168 (step S450), and assigns 0 to the variable R (step S452).

[0119] Next, if the number of edge pixels in the Rth row is greater than HIST_MAX, the control unit 100 stores the value of variable R as streak position information (step S454; Yes → step S456). The process of step S454 is the same as the process of step S208 in FIG. 7. The streak position information stored in this manner is information indicating the row (position) containing the streak detected based on the pixel count histogram of edge pixels, which is the second method. Note that if the value of variable tmp is equal to or less than HIST_MAX, the control unit 100 omits the process of step S454 (step S454; No).

[0120] Next, control unit 100 increments variable R (step S458) and determines whether the value of variable R is equal to RMAX (step S460). If the value of variable R is equal to RMAX, control unit 100 ends the processing shown in Fig. 23 (step S460; Yes), and if the value of variable R is not equal to RMAX, control unit 100 returns to step S454 (step S460; No → step S454).

[0121] When the control unit 100 executes the second streak detection process shown in Fig. 23, it executes the process shown in Fig. 24 as the second streak removal process instead of the process shown in Fig. 10. Note that the same processes as those shown in Fig. 11 are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0122] First, the control unit 100 assigns 0 to a variable tmp (step S550). Then, the control unit 100 acquires streak position information having an index number of tmp from the streak position information storage area 168 (step S552), and assigns the value of the line number included in the acquired streak position information to a variable R (step S554).

[0123] Next, the control unit 100 executes the processes of step S502 and steps S506 to S512. If the value of the variable C is equal to CMAX, the control unit 100 determines whether the value obtained by adding 1 to the value of the variable tmp is equal to the total number of pieces of streak position information stored in the streak position information storage area 168 (step S556). If the value obtained by adding 1 to the value of the variable tmp is equal to the total number of pieces of streak position information stored in the streak position information storage area 168, the control unit 100 ends the process shown in Fig. 24 (step S556; Yes). On the other hand, if the value obtained by adding 1 to the value of the variable tmp is not equal to the total number of pieces of streak position information stored in the streak position information storage area 168, the control unit 100 increments the variable tmp and returns to step S552 (step S556; No → step S558 → step S502).

[0124] In this way, even when the control unit 100 executes the processes described in FIGS. 23 and 24, it is possible to remove streaks from the input image based on the histogram of the number of edge pixels.

[0125] Furthermore, the control unit 100 may expand edge pixels extending in the main scanning direction in the horizontal difference image in the main scanning direction. In other words, the control unit 100 extends edges extending in the vertical direction. By doing so, the control unit 100 can more easily detect edge pixels at the corners of the document among the edge pixels of the document in steps S308 and S318 of the first streak detection process.

[0126] The method of storing information may also be changed as appropriate. For example, the streak position information may be stored as a variable-length array that stores the row numbers where pixels corresponding to the streaks exist. For example, in step S208 of FIG. 7, the control unit 100 may assign the value of R to the tmp-th element of the variable-length array ROW_Streak (ROW_Streak[tmp]←R). Furthermore, the edge pixel tally information may also be stored as an array that stores the number of edge pixels existing in the r-th row of pixel P[r, c] of the vertical difference image. For example, in step S414 of FIG. 9, the control unit 100 may assign the value of tmp to the R-th element of the array hist (hist[R]←tmp).

[0127] In the above description, whether or not a streak exists is determined based on whether the pixel value of the pixel of interest in the vertical difference image is 255. However, the control unit 100 may determine that a streak exists when the pixel value of the pixel of interest is equal to or greater than a predetermined threshold. For example, in step S206 of FIG. 7, the control unit 100 may determine that the condition in step S206 is satisfied if the pixel value of either pixel P[R,C] or pixel P[R,CMAX-C] is equal to or greater than a predetermined threshold. Similarly, in step S406 of FIG. 9, the control unit 100 may determine that the condition in step S406 is satisfied if the pixel value of pixel P[R,C] is equal to or greater than a predetermined threshold. Note that, similarly, in step S308 of FIG. 8, the control unit 100 may determine that the condition in step S308 is satisfied if the pixel value of the pixel of interest in the horizontal difference image is equal to or greater than a predetermined threshold. That is, the control unit 100 may use a condition other than the pixel value being 255 as the condition for detecting an edge.

[0128] The first and second methods described above are merely examples, and the control unit 100 may remove streaks from the input image using other methods.

[0129] In this way, the image forming apparatus of this embodiment can remove only the streak edges without removing the edges of the document. In particular, by performing two streak detection processes and a streak removal process, this embodiment can detect and remove streaks in various states, such as when the streak edge is far from the edge of the input image or coincides with the edge of the document.

[0130] As described above, the image forming apparatus of this embodiment can enhance the reliability of streak removal by combining two methods: a streak detection method that uses both vertical and horizontal differential images, and a streak detection method using a histogram method. That is, the image forming apparatus of this embodiment performs two types of streak detection / removal processing in addition to conventional skew correction / cropping processing. This allows the image forming apparatus to appropriately output an image based on an input image. Furthermore, the image forming apparatus of this embodiment uses a streak removal method that distinguishes between inside and outside the document and accurately separates and processes them, thereby removing only edge pixels outside the document. In this way, the image forming apparatus of this embodiment can preserve the edges of the document as much as possible by removing edge pixels outside the document. As a result, the image forming apparatus of this embodiment can accurately detect edges (document edges) in cropping processing, perform appropriate image processing, and output the processed image without causing any inconvenience to the user.

[0131] [2. Second Embodiment] Next, a second embodiment will be described. The second embodiment differs from the first embodiment in that, when images of multiple documents are acquired, the range to be searched in the second and subsequent documents is limited based on streak information detected in the first document. Note that this embodiment replaces FIG. 3 of the first embodiment with FIG. 25 and FIG. 6 with FIG. 26, respectively. Note that the same processes are denoted by the same reference numerals, and their explanations will be omitted.

[0132] In this embodiment, the parameter information storage area 166 stores parameter information including a parameter value (e.g., "10") corresponding to the parameter name "Streak_Buf" in addition to the parameter information described in the first embodiment. Streak_Buf is a value indicating the processing range for streak detection and streak removal for images of the second and subsequent pages of a document. In the following description, the parameter value corresponding to the parameter name "Streak_Buf" will be referred to as Streak_Buf.

[0133] The flow of the main processing in this embodiment will be described with reference to Fig. 26. In this embodiment, after a differential image is generated in step S102, it is determined whether the document read in step S100 is the second or subsequent document (step S600).

[0134] If the document is the second or subsequent page, the control unit 100 reduces the range (processing range) for performing the first streak detection process, first streak removal process, second streak detection process, and second streak removal process based on the position of the streak detected in the first document (step S600; Yes → step S602). For example, the control unit 100 reduces the processing range to just Streak_Buf lines before and after the streak position. In this case, when the streak position (line number) is R, the control unit 100 reduces the processing range to from the (R-Streak_Buf)th line to the (R+Streak_Buf)th line. This allows the control unit 100 to detect or remove streaks from the images of the second or subsequent pages of the document, starting from the periphery of the streak position detected in the image of the first document.

[0135] If the scanned document is not the second or subsequent document, i.e., if it is the first document, the control unit 100 omits the process in step S202 (step S602; No). In this case, the range for executing the first streak removal process, the second streak detection process, and the second streak removal process is all the lines of the input image.

[0136] Furthermore, in this embodiment, before executing the first streak detection process, the control unit 100 deletes the streak position information stored in the streak position information storage area 168. This prevents the control unit 100 from removing edge pixels from the input image of the document to be processed based on information about streaks detected from documents other than the document to be processed.

[0137] Next, after executing the second streak removal process in step S110, the control unit 100 determines whether the document read in step S100 is the first document (step S604).

[0138] If the read document is the first document, the control unit 100 stores information about the positions of the streaks (for example, line numbers) detected in the processes from step S104 to step S110 in the storage unit 160 (step S604; Yes → step S606). Note that the control unit 100 can obtain information about the positions of the streaks in the first document by reading the information about the positions of the streaks detected from the first document in step S602.

[0139] On the other hand, if the read document is not the first document, the control unit 100 omits the process in step S606 (step S604; No).

[0140] After outputting the image in step S108, the control unit 100 determines whether all the originals have been read (step S608). If all the originals have been read, the control unit 100 ends the process shown in FIG. 26 (step S608; Yes). On the other hand, if all the originals have not been read, the control unit 100 returns to step S100 (step S608; No → step S100).

[0141] An example of operation of this embodiment will be described with reference to Fig. 27. Fig. 27(a) is a diagram showing an input image P200 corresponding to the first sheet of original. Here, it is assumed that a streak E200 is detected in the input image P200. The image forming apparatus 10 of this embodiment stores the position of the streak E200 detected from the input image P200 of the first sheet of original.

[0142] 27(b) is a diagram showing an input image P210 corresponding to the second document. For input images of the second and subsequent documents, image forming apparatus 10 sets the processing range for streak detection and streak removal to only the lines (E210) corresponding to the Streak_Buf lines before and after the line (position) of the streak E200 detected in input image P200 of the first document.

[0143] Similarly, for the input images of the third and subsequent documents, the image forming apparatus 10 sets only the lines corresponding to the Streak_Buf lines before and after the position of the streak E200 (the lines corresponding to E210 shown in Figure 27(b)) as the processing range for streak detection and streak removal.

[0144] In this way, the image forming apparatus of this embodiment uses the position of streaks detected in the first document to detect and remove streaks in the input images of the second and subsequent documents. Specifically, for the input images of the second and subsequent documents, the image forming apparatus performs the same streak detection and removal processes as those performed on the input image of the first document, with the processing range being several lines before and after the position of the streak detected in the first document. This allows the image forming apparatus to reduce the processing time required for streak detection and removal.

[0145] 3. Third Embodiment Next, a third embodiment will be described. Unlike the first embodiment, the third embodiment is an embodiment in which streaks are detected and removed based on a reduced difference image. Note that this embodiment replaces FIG. 6 of the first embodiment with FIG. 28. Note that the same functional units and processes are denoted by the same reference numerals, and their description will be omitted.

[0146] The flow of the main processing in this embodiment will be described with reference to Fig. 28. In this embodiment, after generating a difference image, the control unit 100 reduces the difference image (step S700). For example, the control unit 100 reduces both the vertical length and horizontal length of the difference image to one-fourth, thereby reducing the overall size of the difference image to one-fourth (25%).

[0147] In this embodiment, the first streak detection process, the first streak removal process, the second streak detection process, and the second streak removal process are performed using a reduced difference image.

[0148] An example will be described in which a difference image reduced to one-fourth its original size is used. In the first streak detection process, the control unit 100 detects the position of a streak based on the reduced difference image. At this time, the control unit 100 calculates the position of the streak in the input image, taking into account the reduction ratio of the difference image relative to the input image. Specifically, if the control unit 100 detects a streak in the Rth row of the reduced difference image, it determines that the streak occurs in rows (4×R) to (4×R+3) of the input image. Furthermore, in the first streak removal process, when the control unit 100 searches for an edge pixel in the reduced horizontal difference image starting from position c=0, if the pixel at position c=C is an edge pixel, it sets the edge removal range in the input image to be from column 0 to column (4×C).

[0149] Similarly, in the second streak removal process, if the number of edge pixels in the Rth row is equal to or greater than HIST_MAX, the control unit 100 determines that streaks have occurred in rows (4×R) to (4×R+3) of the input image, and removes the edge pixels from those rows. In this way, when detecting and removing streaks from the input image, the control unit 100 determines the row in which the streak has occurred and the range in which the streak will be removed, taking into account the reduction rate of the difference image relative to the input image.

[0150] In this way, the image forming apparatus of this embodiment reduces the size of the differential image before executing the streak detection process and the streak removal process, thereby improving the processing speed and reducing the memory usage.

[0151] [4. Modifications] The present invention is not limited to the above-described embodiments, and various modifications are possible. In other words, embodiments obtained by combining technical means that are appropriately modified within the scope of the gist of the present invention are also included in the technical scope of the present invention.

[0152] Although the above-described embodiments are described separately for convenience of explanation, they may be combined within the scope of technical feasibility. For example, the second embodiment and the third embodiment may be combined. In this case, the image forming apparatus performs streak detection processing and streak removal processing based on the reduced differential image, and for input images of the second and subsequent pages of a document, the processing range is reduced based on the position of the streak detected from the first page of a document. This allows the image forming apparatus to shorten the processing time for removing streaks from input images.

[0153] In the above-described embodiment, the image processing device according to the present disclosure is configured as an image forming device. However, the image processing device according to the present disclosure may also be applied to an image reading device such as a scanner, or to a program or plug-in that performs image correction. Furthermore, the image processing device according to the present disclosure may be provided as an image correction service by being configured as a server device. In this case, the image forming device or image reading device reads an image of a document and then transmits the read image as an input image to the image correction service. The image correction service performs edge detection processing, first streak detection processing, first streak removal processing, second streak detection processing, and second streak removal processing on the received input image, and transmits the input image after the processing to the image forming device or image reading device that transmitted the input image. The image forming device or image reading device then performs image processing such as skew processing or crop processing on the image received from the image correction service.

[0154] In addition, the programs that run on each device in the embodiments are programs that control the CPU, etc. (programs that make a computer function) so as to realize the functions of the above-described embodiments. Information handled by these devices is temporarily stored in a temporary storage device (e.g., RAM) during processing, and then stored in various storage devices such as ROMs (Read Only Memories) and HDDs, and is read, modified, and written by the CPU as needed.

[0155] Here, the recording medium for storing the program may be any of semiconductor media (e.g., ROM, non-volatile memory card, etc.), optical recording media / magneto-optical recording media (e.g., DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray (registered trademark) Disc), etc.), magnetic recording media (e.g., magnetic tape, flexible disk, etc.), etc. Furthermore, not only are the functions of the above-described embodiments realized by executing the loaded program, but the functions of the present invention may also be realized by processing in cooperation with an operating system or other application programs, etc., based on instructions from the program.

[0156] Furthermore, when distributing the program on the market, the program can be stored in a portable recording medium and distributed, or transferred to a server computer connected via a network such as the Internet. In this case, the storage device of the server computer is of course included in the present invention. [Explanation of symbols]

[0157] 10 Image forming device 100 control section 102 Image processing section 104 Edge detection unit 106 Document range detection unit 108 Skew judgment unit 120 Image input unit 130 Image forming unit 140 Display section 150 Operation section 160 Storage section 162 Input image storage area 164 Differential Image Storage Area 166 Parameter information storage area 168 Line position information storage area 170 Edge pixel count information storage area 190 Communications Department

Claims

1. An input unit for inputting an image of a document and a control unit are provided, The control unit removing streaks from the image by a first method; removing streaks from the image by a second method different from the first method; performing a cropping process on the image from which the streaks have been removed by the first method and the second method; As the second method, the number of edge pixels included in each row in the image is tallied, pixels included in a row in which the number of edge pixels is greater than a predetermined threshold are detected as streaks, and the detected streaks are removed.

1. An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the control unit removes streaks included in rows in which the number of edge pixels exceeds a predetermined threshold by replacing pixels included in the rows in which the number of edge pixels exceeds a predetermined threshold.

3. An input unit for inputting an image of a document and a control unit are provided, The control unit removing streaks from the image by a first method; removing streaks from the image by a second method different from the first method; performing a cropping process on the image from which the streaks have been removed by the first method and the second method; generating a difference image in the main scanning direction based on the difference in pixel values ​​of pixels adjacent in the main scanning direction, and a difference image in the sub-scanning direction based on the difference in pixel values ​​of pixels adjacent in the sub-scanning direction; The streaks are removed using the difference image in the main scanning direction and the difference image in the sub-scanning direction.

1. An image processing device comprising:

4. 4. The image processing apparatus according to claim 3, wherein the control unit removes the streaks by using the reduced difference image in the main scanning direction and the reduced difference image in the sub-scanning direction.

5. 5. The image processing device according to claim 1, wherein the control unit, as the first method, detects edges detected near sides parallel to a main scanning direction in the image as streaks, and removes the detected streaks.

6. The control unit 6. The image processing apparatus according to claim 5, wherein edge pixels detected based on a difference in pixel values ​​between pixels adjacent in the main scanning direction in the image are detected as the edges.

7. 6. The image processing device according to claim 5, wherein the control unit removes the streak by replacing, among pixels constituting a line including the streak detected near an edge parallel to the main scanning direction in the image, pixels included from the edge parallel to the main scanning direction to an edge of the document.

8. The image processing device according to claim 7, characterized in that the control unit detects, among edge pixels detected in the image based on the difference in pixel values ​​of adjacent pixels in the sub-scanning direction, the edge pixel that is closest to the side parallel to the main scanning direction as the pixel at the edge of the document.

9. The control unit When a plurality of images of the originals are input and the streaks are to be removed from the images of the second and subsequent sheets of the originals, the streaks are detected from the lines surrounding the line including the streaks detected from the image of the first sheet of the original, and the detected streaks are removed.

5. The image processing device according to claim 1, wherein the image processing device is a computer.

10. A control method for an image processing device, comprising: removing streaks from an input image by a first method; removing streaks from the image by a second method different from the first method; performing a cropping process on the image from which the streaks have been removed by the first method and the second method; The second method includes a step of counting the number of edge pixels included in each row in the image, detecting pixels included in a row in which the number of edge pixels is greater than a predetermined threshold as streaks, and removing the detected streaks; A control method comprising:

11. On the computer, a function of removing streaks from an input image by a first method; a function of removing streaks from the image by a second method different from the first method; a function of performing a cropping process on the image from which the streaks have been removed by the first method and the second method; As the second method, a function of tallying the number of edge pixels included in each row in the image, detecting pixels included in a row in which the number of edge pixels is greater than a predetermined threshold as streaks, and removing the detected streaks; A program characterized by realizing the above.

12. A control method for an image processing device, comprising: removing streaks from an input image by a first method; removing streaks from the image by a second method different from the first method; performing a cropping process on the image from which the streaks have been removed by the first method and the second method; generating a difference image in the main scanning direction based on a difference in pixel values ​​between pixels adjacent in the main scanning direction, and a difference image in the sub-scanning direction based on a difference in pixel values ​​between pixels adjacent in the sub-scanning direction; removing the streaks using the difference image in the main scanning direction and the difference image in the sub-scanning direction; A control method comprising:

13. On the computer, a function of removing streaks from an input image by a first method; a function of removing streaks from the image by a second method different from the first method; a function of performing a cropping process on the image from which the streaks have been removed by the first method and the second method; a function of generating a difference image in the main scanning direction based on the difference in pixel values ​​of pixels adjacent in the main scanning direction, and a difference image in the sub-scanning direction based on the difference in pixel values ​​of pixels adjacent in the sub-scanning direction; a function of removing the streaks using the difference image in the main scanning direction and the difference image in the sub-scanning direction; A program characterized by realizing the above.

Images