Image processing device, image processing method and program

Abstract

To obtain a high-quality skew corrected image without increasing a processing time of the entire skew correction.SOLUTION: An image processing device sets an allowable error for skew correction according to the type of a scanned image of a document, and performs skew correction on the basis of the set allowable error.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a skew correction technique. [Background technology]

[0002] A deskew (tilt correction) technique is known that rotates an image scanned from a tilted document to obtain a horizontal scanned image. Deskew is performed by generating tilted grid point coordinates according to the skew angle of the document, and then sampling pixels from the input image based on the generated grid point coordinates.

[0003] To quickly skew a scanned image, a hardware circuit that performs skew correction using grid point coordinates generated by a so-called DDA (Digital Differential Analyzer) is sometimes used. The hardware circuit uses the top left coordinate of the scanned image as the starting coordinate of the DDA, and the skew angle of the scanned image as the tilt of the DDA, and generates coordinates using the DDA to generate grid point coordinates contained in a diagonally tilted line that are used for skew correction.

[0004] Digital numerical calculations are performed in hardware circuits, and due to the nature of digital numerical calculations, which round ideal values ​​to a finite number of bits, the numbers that can be handled contain quantization error. When grid point coordinates are generated using the DDA described above, errors accumulate in the calculated grid point coordinates due to the cumulative addition of gradients that contain this error. If grid point coordinates are generated using the DDA described above starting from the leftmost pixel of the scanned image and moving to the right, the coordinate error increases as you move away from the leftmost pixel to the right.

[0005] Regarding a technique for suppressing the accumulation of coordinate errors, Patent Document 1 discloses a technique for reducing the accumulation of coordinate errors to less than one pixel when the accumulation of coordinate errors generated by a DDA reaches one pixel or more by determining a division point during the coordinate generation process by the DDA and correcting the slope at the determined division point. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-269576 Summary of the Invention [Problem to be solved by the invention]

[0007] Incidentally, there are various types of images that are the target of the above-mentioned skew correction process, each of which requires different degrees of accuracy in skew correction, such as CAD drawings, natural images, etc. Therefore, with the technology of Patent Document 1, depending on the type of image, the skew-corrected image may have low image quality, or a high-quality skew-corrected image may be obtained, but the processing time may increase. [Means for solving the problem]

[0008] An image processing device according to one aspect of the present disclosure is an image processing device that performs skew correction on a scanned image of an original, the image processing device including: a setting unit that sets an allowable error for the skew correction according to a type of the scanned image; and a correction unit that performs the skew correction based on the allowable error. The apparatus includes a determining unit that determines a size for dividing the scanned image into a plurality of regions based on the allowable error set by the setting unit and the skew angle of the scanned image, and a dividing unit that divides the scanned image by the size, and the correcting unit performs skew correction on the plurality of regions into which the scanned image is divided by the size. It is characterized by: [Effects of the Invention]

[0009] According to the present disclosure, a high-quality skew-corrected image can be obtained without increasing the overall processing time for skew correction. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of an image processing device. [Figure 2] FIG. 10 is a block diagram showing an example of the configuration of a skew correction circuit. [Figure 3] FIG. 10 is a diagram illustrating skew correction. [Figure 4] FIG. 10 is a diagram illustrating sampling of grid point pixels. [Figure 5] 10 is a flowchart showing the flow of processing executed by the image processing apparatus. [Figure 6] 10 is a table showing the relationship between skew correction modes and allowable errors. [Figure 7] FIG. 10 is a diagram illustrating a method for calculating a coordinate error. [Figure 8] FIG. 10 is a diagram illustrating a method for calculating grid point coordinates using DDA. [Figure 9] FIG. 10 is a diagram illustrating an example in which an input image is divided into rectangular regions. [Figure 10] FIG. 10 is a diagram illustrating resetting of initial coordinates for each rectangular area. [Figure 11] FIG. 10 is a diagram illustrating overhead of the skew correction process. [Figure 12] 10 is a flowchart showing the flow of processing executed by the image processing apparatus. [Figure 13] FIG. 10 is a diagram illustrating an example in which an input image is divided into rectangular regions. [Figure 14] 10 is a flowchart showing the flow of processing executed by the image processing apparatus. [Figure 15] FIG. 10 is a diagram illustrating an example in which an input image is divided into rectangular regions. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments for carrying out the technology of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the technology of the present disclosure according to the claims, and not all of the combinations of features described in the following embodiments are necessarily essential to the solution of the technology of the present disclosure.

[0012] [Embodiment 1] (Configuration of image processing device) 1 is a block diagram showing an example of the hardware configuration of an image processing device according to this embodiment. The image processing device 100 according to this embodiment has a CPU 101, a RAM 102, a ROM 103, a skew correction circuit 104, a scanning unit 105, a user input unit 106, and an I / F (interface) unit 107. These are also connected to each other via a system bus 108. Furthermore, the I / F unit 107 is connected to the system bus 108, and can send and receive data to and from external devices via a network.

[0013] The CPU 101 is a central processing unit that controls the entire image processing device 100 and executes a control program loaded into a random access memory (RAM) 102 or a read-only memory (ROM) 103. The scanning unit 105 scans an original document using an optical sensor and stores the resulting image data (digital image data) in the RAM 102. When the control program is executed, the skew correction circuit 104 performs image processing for skew correction using the image data stored in the RAM 102 by the scanning unit 105 as an input image. The skew correction circuit 104 performs image processing for skew correction on the input image data stored in the RAM 102 and stores output image data resulting from the skew correction in the RAM 102. The control program transmits the output image data stored in the RAM 102 to an external device via the interface (I / F) unit 107. The user input unit 106 includes a display device with display and input functions, such as a touch panel, and input devices, such as a keyboard and a mouse, and is used to notify the user of information and receive instructions from the user.

[0014] (Configuration of skew correction circuit) 2 is a block diagram showing an example of the configuration of the skew correction circuit 104. The skew correction circuit 104 has an image input unit 201, an SRAM 202, a grid point coordinate generation unit 203, a pixel sampling unit 204, and an image output unit 205. The skew correction circuit 104 also has the following multiple registers and operates according to the settings of these registers. A register that indicates the RAM address where the input image is stored (hereafter referred to as ADDR_IN) A register that indicates the RAM address where the output image is stored (hereafter referred to as ADDR_OUT) Register indicating the slope of the skew correction (hereinafter referred to as Δx, Δy) A register that indicates the top left starting coordinates (hereafter referred to as X00, Y00) of the rectangular area where skew correction is performed. A register that indicates the width (hereafter referred to as BW) and height (hereafter referred to as BH) of the rectangular area where skew correction is performed. The skew correction circuit 104 also has a startup register, and starts image processing for skew correction by writing data into the startup register after setting various registers.

[0015] The image input unit 201 reads pixel data of a rectangular area to be subjected to skew correction from the data of the input image stored in the input image address ADDR_IN of the RAM 102, and stores the data in the SRAM 202. Here, the size of the rectangular area corresponds to the range to be subjected to image processing in one activation of the skew correction circuit 104.

[0016] The SRAM 202 temporarily stores pixel data of a rectangular area and supplies the necessary pixel data in accordance with a read instruction from the pixel sampling unit 204 .

[0017] The grid point coordinate generation unit 203 generates grid point coordinates for sampling pixels from a rectangular area using the DDA described above. The sampling grid point coordinates are a tilted coordinate system for performing skew correction, and are calculated based on the start point coordinates X00, Y00, the slope Δx, Δy, and the width BW and height BH of the rectangular area. DDA is a technology for calculating coordinates included in a straight line. The DDA technology uses the start point coordinate of a line and the slope, which is the difference value, to cumulatively add the slope to the start point coordinate to determine the coordinates included in the line. As such, DDA has the advantage of being a simple process of adding the slope to the start point coordinate, allowing for high-speed processing.

[0018] The pixel sampling unit 204 reads out, from the SRAM 202, pixel values ​​at coordinates corresponding to the sampling grid point coordinates generated by the grid point coordinate generation unit 203. When the coordinates of a sampling grid point are coordinates between pixels in the input image, the pixel sampling unit 204 obtains the pixel value of the sampling grid point by linear interpolation from four input pixels surrounding the coordinates of the sampling grid point.

[0019] The image output unit 205 writes the output pixels read by the pixel sampling unit 204 in order from the output image address ADDR_OUT of the RAM 102. When the image output unit 205 completes writing of all output pixels in the rectangular area, it issues (outputs) an end interrupt to the CPU 101.

[0020] (skew correction) 3 is a diagram illustrating skew correction, showing an input image 301 before skew correction and an output image 302 after skew correction.

[0021] As shown in Fig. 3, skew correction is a process in which an input image 301, which is tilted at a skew angle θ with respect to the horizontal direction, is subjected to image rotation image processing to output a horizontal output image 302. The input image 301 is a digital image that is read by the scanning unit 105 using an optical sensor and stored in RAM 102. When reading the original, the scanning unit 105 also detects the skew angle θ of the original using an angle sensor, and records information related to the detected skew angle θ of the original together with the input image 301 in RAM 102.

[0022] (Sampling of pixels at grid point coordinates) 4A and 4B are diagrams for explaining the sampling of pixels at grid point coordinates in skew correction, with Fig. 4A showing the entire image and Fig. 4B showing an enlarged view of the dashed-dotted line area in Fig. 4A. In Figs. 4A and 4B, the white circles indicate grid point coordinates of the input image, and the dotted circles indicate sampling grid point coordinates in skew correction. White circles A, B, C, and D indicate the coordinates of four input pixels surrounding the sampling grid point coordinate P.

[0023] As shown in Figure 4(a), skew correction is performed by generating grid point coordinates that are tilted diagonally along the skew angle (for example, angle θ in Figure 3) and sampling pixels from the input image. White circles indicate the grid point coordinates of the input image. Black circles indicate the sampling grid point coordinates for skew correction, and a horizontal image can be obtained by creating an output image by sampling the pixels of the black circles.

[0024] (skew correction processing) FIG. 5 is a flowchart showing the flow of skew correction processing executed by the image processing device. Each symbol S below indicates a step in the flowchart. This also applies to FIG. 5 and subsequent figures. Skew correction is performed by the CPU 101 and the skew correction circuit 104 working together. Of the processes from S501 to S509, the process of S506 is executed by the skew correction circuit 104, and the other processes from S501 to S505 and S507 to S509 are executed by the CPU 101.

[0025] In S501, CPU 101 accepts a user instruction via user input unit 106 to select a mode that determines the accuracy of skew correction. The user instruction includes, for example, an instruction to select the character / line drawing mode or the natural image mode for each page of a document. In other words, the user instruction can also be said to include an instruction to select the character / line drawing mode or the natural image mode for each page of a document. The character / line drawing mode is, for example, a mode (first mode) set for a page on which at least one of characters or line drawings is drawn. The natural image mode is, for example, a mode (second mode) set for a page that does not correspond to either characters or line drawings, that does not include characters or line drawings, and on which a natural image is drawn. Therefore, the user selects a mode corresponding to the type of scanned image. In other words, in S501, the type of scanned image can also be said to be specified by the user. Note that an instruction to select the character / line drawing mode or the natural image mode may be accepted not for each page but for each document (each document).

[0026] In S502, the CPU 101 selects the allowable error Et, which is the accuracy of the grid point coordinates for skew correction, according to the mode selected in the processing of S501. Note that the allowable error Et may be set in advance for each mode, or may be set at the time of selection.

[0027] (Relationship between skew correction mode and tolerance Et) FIG. 6 is a table showing an example of setting the allowable error. In this table 600, an allowable error 620 is set in correspondence with a skew correction mode 610. For example, as shown in FIG. 6, in table 600, the allowable error (Et) 620 is set to 0.25 pixels for the character / line drawing mode of the skew correction mode 610, and the allowable error (Et) 620 is set to 0.50 pixels for the natural image mode of the skew correction mode 610. Note that the allowable error (Et) is not limited to the value shown in table 600 described above, and may be set with a precision appropriate for the image processing device. The allowable error for the character / line drawing mode may be set to a smaller value than the allowable error for the natural image mode, and the allowable error for the natural image mode may be set to a larger value than the allowable error for the character / line drawing mode. The reason for setting the magnitude of the allowable error in this manner is that the precision of the grid point coordinates required for skew correction differs depending on the type of scanned image. For example, if the scanned image is a text or line drawing such as a CAD drawing, distortion caused by changes in image structure due to skew is easily noticeable, and highly accurate skew correction is required. On the other hand, if the scanned image is a natural image such as a photograph, even slight errors in coordinate accuracy in skew correction are less noticeable, and highly accurate skew correction may not be required. Furthermore, if the tolerance used in skew correction is set to a fixed value, there is a problem that the coordinate accuracy of the skew correction may be excessive or insufficient depending on the type of image. Therefore, in S502, the coordinate accuracy (tolerance) of the skew correction is changed (set) depending on the type of scanned image, so that skew correction can be performed with optimal accuracy in S506, which will be described in detail later.

[0028] In S503, CPU 101 divides the input image into a plurality of rectangular regions. Here, in the skew correction circuit 104 of this embodiment, the rectangular region size is determined so that the coordinate error E that occurs when grid point coordinates are generated by DDA using the rectangular region as a processing region is equal to or less than the allowable error Et selected in S502. In other words, CPU 101 determines the rectangular region size (the width BW and height BH of the rectangular region) that satisfies the allowable error.

[0029] <How to calculate the rectangular area size> FIG. 7 is a diagram explaining a method for calculating coordinate errors that occur within a rectangular area. First, as shown in FIG. 7, consider a rectangular area with ideal accuracy, where the starting coordinate (grid point coordinate) P00 is (X00, Y00), and the skew angle is θ, with a width of BW and a height of BH. With ideal accuracy, the lattice point coordinates (X_Ideal, Y_Ideal) of the bottom right pixel of the rectangular area are expressed by the following formula. Note that the following formula is expressed in a coordinate system where the upper left of the image is the origin (0, 0), the right direction is the positive X axis, and the downward direction is the positive Y axis. The same applies to formula (1) and subsequent formulas.

[0030]

number

[0031] As mentioned above, in digital numerical calculations such as hardware processing, ideal values ​​are rounded to a finite number of bits, which means that the numerical values ​​that can be handled contain quantization errors. When the trigonometric functions sinθ and cosθ have quantization errors, if the quantization errors of the trigonometric functions are δx and δy, the slopes Δx and Δy containing the quantization errors are expressed by the following formulas and are defined as having the following relationship:

[0032]

number

[0033] The following describes how the skew correction circuit 104 of this embodiment generates coordinates using the DDA with the gradients Δx and Δy, including the quantization error. FIG. 8 is a diagram illustrating a method for calculating grid point coordinates using the DDA. As shown in FIG. 8, when a start point coordinate P00 (X00, Y00) and gradients Δx and Δy are given, the grid point coordinate P01 (X01, Y01) located to the right of the start point coordinate P00 is calculated as (X00 + Δx, Y00 + Δy). Furthermore, the grid point coordinate P10 (X10, Y10) located below the start point coordinate P00 is calculated as (X00 + Δy, Y00 + Δx). Therefore, when the skew correction circuit 104 generates coordinates using the DDA, the grid point coordinates (X_DDA, Y_DDA) of the bottom right pixel of the rectangular area are expressed by the following formula:

[0034]

number

[0035] Here, the grid point coordinates (X_DDA, Y_DDA) of the bottom right pixel of the pixels corresponding to the four corners of the rectangular area described above will be described in detail with reference to FIG. 10, which will be described later. FIG. 10 shows an example in which three rectangular areas BLOCK0, BLOCK1, and BLOCK2 are lined up to the right. The white circles indicate grid points when no division of the rectangular area (rectangular division) is performed, and the black circles indicate grid points when the initial coordinates (grid point coordinates of the upper left pixel of the rectangular area) are reset for each rectangular area. Referring to BLOCK0 in FIG. 10 (an example in which the width BW of the rectangular area is 3 pixels and its height BH is 4), the grid point coordinates corresponding to the above-mentioned "grid point coordinates (X_DDA, Y_DDA) of the bottom right pixel of the rectangular area" indicate the following grid points: In other words, it indicates the grid point located BW times to the right (three times to the right in the example of Figure 10) from the starting coordinate P00 and BH times down (four times down in the example of Figure 10), which is the bottom right grid point of the rectangular area BLOCK0.

[0036] Since the coordinates have the above-mentioned relationship, the coordinate error E occurring between the coordinate generation with ideal accuracy and the coordinate generation by the DDA in the skew correction circuit 104 of this embodiment is expressed by the following formula.

[0037]

number

[0038] Here, E0 = √(δx^2 + δy^2). E0 is the quantization error that occurs when a trigonometric function that inputs the skew angle θ is quantized using the register bit width of the skew correction circuit 104. Since the quantization error is a value that differs depending on the input value, E0 is a value that differs depending on the skew angle θ.

[0039] Therefore, if the allowable error within the rectangular area selected in S502 is Et, the width BW of the rectangular area is expressed by the following formula.

[0040]

number

[0041] Here, the height BH of the rectangular area is a predetermined fixed value, and the formula is described as a formula for calculating the width BW of the rectangular area that satisfies the given tolerance Et. Note that the method for calculating the rectangular area size is an example and is not limited to this. It is also possible to calculate combinations of the width BW and height BH of the rectangular area under the condition that the tolerance Et is satisfied, and then select from these a size that is convenient for image processing. Alternatively, other calculation formulas such as those shown below may be used.

[0042] (Other calculation formula 1) It may be calculated using the following formula, where BW=BH and the width BW and height BH of the rectangular area are equal in size.

[0043]

number

[0044] Since the size of the rectangular area is equal in the vertical and horizontal directions, and the accumulation of coordinate errors generated by DDA within the rectangular area is isotropic, the image quality is better than when the width BW of the rectangular area and the height BH of the rectangular area are different (BW≠BH).

[0045] (Other formula 2) In this embodiment, the rectangular area size (width BW and height BH of the rectangular area) is calculated under the condition that the coordinate error E, which occurs at the bottom right pixel within the rectangular area between coordinate generation with ideal accuracy and coordinate generation by DDA, is equal to or less than the allowable error Et. This is because the coordinate error E is greatest at the bottom right pixel within the rectangular area.

[0046] (Other methods) The coordinate error E caused by the DDA in the x direction at the top right pixel in the rectangular area 右上 From this, the width BW of the rectangular region may be calculated.

[0047] In the ideal accuracy, the grid point coordinates of the upper right pixel of the rectangular area (X_Ideal 右上 ,Y_Ideal 右上 ) is calculated using the following formula:

[0048]

number

[0049] When coordinate generation is performed using DDA, the grid point coordinates (X_DDA, Y_DDA) of the upper right pixel of the rectangular area are expressed by the following formula.

[0050]

number

[0051] The coordinate error E occurring in the upper right pixel of the rectangular area is expressed by the following formula:

[0052]

number

[0053] Therefore, if the allowable error within the rectangular area selected in S501 is Et, the width BW of the rectangular area is expressed by the following formula.

[0054]

number

[0055] Similarly, the coordinate error E 左下 The height BH of the rectangular area may be calculated from the

[0056] In ideal accuracy, the grid point coordinates of the bottom left pixel of the rectangular area (X_Ideal 左下, Y_Ideal 左下 ) is calculated using the following formula:

[0057]

number

[0058] When coordinates are generated using DDA, the grid point coordinates (X_DDA, Y_DDA) of the bottom left pixel of the rectangular area are expressed by the following formula:

[0059]

number

[0060] The coordinate error E occurring at the bottom left pixel of the rectangular area is expressed by the following formula:

[0061]

number

[0062] Therefore, if the allowable error within the rectangular area selected in S501 is Et, the height BH of the rectangular area is expressed by the following formula.

[0063]

number

[0064] In this method, BW and BH can be calculated independently, which simplifies the calculation.

[0065] An example of rectangular division of an input image will now be described with reference to the drawings. Fig. 9 is a diagram showing an example of dividing an input image into rectangular regions corresponding to the character / line drawing mode or the natural image mode. Fig. 9 shows an example in which the rectangular region sizes (rectangular region width BW and height BH) for dividing the input image into rectangular regions are calculated using the above-mentioned formula in S503.

[0066] For example, Figure 9(a) shows an example of calculating the width BW of a rectangular region in which the skew correction tolerance Et in the character / line drawing mode is 0.25 pixels or less. Assume that the input image has a skew angle θ1 of 9 degrees. The ideal values ​​of the trigonometric functions representing the skew angle, cosθ1 and sinθ1, are 0.9877 and 0.1564, respectively. Assume that the registers representing the skew correction gradients Δx and Δy in the skew correction circuit 104 can only store up to 9 decimal bits. Due to rounding of the register bits, the gradient register settings Δx and Δy in the skew correction circuit 104 are 0.9863 and 0.1562, respectively. The gradient register settings Δx and Δy in the skew correction circuit 104 have a quantization error E0 of 0.0014 (δx = 0.0014, δy = 0.0002), which is the difference from the theoretical values ​​cosθ1 and sinθ1. When grid point coordinates are generated by DDA using the gradient register settings Δx and Δy, which include the quantization error, the size of the rectangular area where the coordinate error E of the grid point coordinates is equal to or less than the allowable error Et of 0.25 pixels is calculated using the formula explained in S503 above. Under the condition that the height BH of the rectangular area is 128 pixels, the width BW of the rectangular area can be calculated as 129 pixels.

[0067] Figure 9(b) shows an example of calculating the width BW of a rectangular region that satisfies the skew correction tolerance Et of 0.50 pixels or less in natural image mode. The skew angle θ1 of the input image is assumed to be 9 degrees, the same as in Figure 9(a). The skew register settings Δx and Δy of the skew correction circuit 104 contain a quantization error E0 of 0.0014 (δx = 0.0014, δy = 0.0002). When grid point coordinates are generated using the DDA using the skew register settings Δx and Δy, including this quantization error, the size of the rectangular region that ensures the coordinate error E of the grid point coordinates is less than the tolerance Et of 0.50 pixels is calculated using the formula described in S503 above. Given that the height BH of the rectangular region is 128 pixels, the width BW of the rectangular region can be calculated as 341 pixels.

[0068] As shown in the examples of Figures 9(a) and 9(b) above, the optimal rectangular area size (rectangular area width BW and height BH) can be calculated using the formula explained in S503 above. Note that the optimal rectangular area size (rectangular area width BW and height BH) for dividing the input image differs depending on the value of the allowable error Et determined by the skew correction mode. In the character / line drawing mode, the allowable error Et is set smaller than in the natural image mode, so the optimal rectangular area size (rectangular area width BW and height BH) is smaller.

[0069] FIG. 9(c) shows an example of calculating the width BW of a rectangular region where the skew correction tolerance Et in the character / line drawing mode is 0.25 pixels or less. Note that the skew angle θ2 of the input image is 9.9 degrees, which is a different condition from FIG. 9(a). The ideal values ​​of the trigonometric functions representing the skew angle, cosθ2 and sinθ2, are 0.9851 and 0.1719, respectively. Assume that the registers representing the skew correction gradients Δx and Δy in the skew correction circuit 104 can only store up to 9 decimal bits. Due to rounding of the register bits, the gradient register settings Δx and Δy of the skew correction circuit 104 become 0.9843 and 0.1718, respectively. The tilt register settings Δx and Δy of the skew correction circuit 104 contain a quantization error E0 of 0.0007 (δx = 0.0007, δy = 0.0000), which is the difference from the theoretical values ​​cosθ2 and sinθ2. When grid point coordinates are generated by DDA using the tilt register settings Δx and Δy, including this quantization error, the rectangular area size in which the coordinate error E of the grid point coordinates is less than the allowable error Et of 0.25 pixels is calculated using the formula explained in S503 above. Given that the height BH of the rectangular area is 128 pixels, the width BW of the rectangular area can be calculated as 314 pixels.

[0070] Figure 9(d) shows an example of calculating the width BW of a rectangular area that satisfies the skew correction tolerance Et of 0.50 pixels or less in natural image mode. The skew angle θ2 of the input image is 9.9 degrees, as in Figure 9(c). The skew register settings Δx and Δy of the skew correction circuit 104 contain a quantization error E0 of 0.0007 (δx = 0.0007, δy = 0.0000). When grid point coordinates are generated using the DDA using the skew register settings Δx and Δy that include this quantization error, the rectangular area size that ensures the coordinate error E of the grid point coordinates is less than the tolerance Et of 0.50 pixels is calculated using the formula described in S503 above. Given that the height BH of the rectangular area is 128 pixels, the width BW of the rectangular area can be calculated as 666 pixels.

[0071] As shown in the examples of Figures 9(c) and 9(d), the optimal rectangular area size (rectangular area width BW and height BH) can be calculated using the formula explained in S503 above. Note that different skew angles θ result in different quantization error E0, resulting in different optimal rectangular area sizes (rectangular area width BW and height BH) for dividing the input image. In Figures 9(a) and 9(b), the skew angle θ1 is 9 degrees, and the quantization error E0 is 0.0014. In Figures 9(c) and 9(d), the skew angle θ2 is 9.9 degrees, and the quantization error E0 is 0.0007. In the cases of Figures 9(a) and 9(b), the quantization error E0 is large, so the optimal rectangular area sizes BW and BH are small.

[0072] As described above, the optimal rectangular area size (width BW and height BH of the rectangular area) for skew correction varies depending on the allowable error Et and skew angle θ determined by the skew correction mode, so it is desirable to calculate it using the formula described in S503 above.

[0073] In S504, register settings are made in the skew correction circuit 104 for each rectangular area divided in S503. Here, we will explain a method for improving the coordinate accuracy of skew correction by setting the skew correction slopes Δx and Δy to be the same for all rectangular areas and resetting the start point coordinate P00 (X00, Y00) in FW for each rectangular area. Specifically, registers BW and BH indicating the width and height of the rectangular area calculated in S503 and quantized slopes Δx and Δy are set according to the register bit width of the skew correction circuit 104. The start point coordinate P00 (X00, Y00) is calculated for each rectangular area using the following formula:

[0074]

number

[0075] Here, ib indicates the rectangular area number (ib=0,1,2,...) in the X direction (rightward from the starting point coordinate), and jb indicates the rectangular area number (jb=0,1,2,...) in the Y direction (downward from the starting point coordinate).

[0076] (Resetting the initial coordinates for each rectangular area) FIG. 10 illustrates how the coordinate accuracy of skew correction can be improved by resetting the starting coordinate P00 (X00, Y00) for each rectangular area. FIG. 10 shows a rectangular area BLOCK0 with rectangular area number 0, a rectangular area BLOCK1 with rectangular area number 1 to the right of BLOCK0, and a rectangular area BLOCK2 with rectangular area number 2 to the right of BLOCK0. The dashed lines passing through the white circles, which represent grid points when no rectangular area division (rectangular division) is performed, deviate from the ideal skew correction angle indicated by the thick line as they move to the right. In contrast, the black circles, which represent grid points when the initial coordinates are reset for each rectangular area, represent coordinates closer to the ideal skew correction angle. Here, the skew correction circuit sets the same slope Δx and Δy for all rectangular areas, and resets the starting coordinate P00 (X00, Y00) for each rectangular area to improve the accuracy of the grid point coordinates for skew correction. However, this is not a limitation. A method may be used in which the accuracy of the grid point coordinates for skew correction is improved by correcting the inclinations Δx and Δy for each rectangular area.

[0077] In S505, the CPU 101 writes the start register of the skew correction circuit 104, and the skew correction circuit 104 starts skew correction.

[0078] In S506, the skew correction circuit 104 performs skew correction on the input image data stored in the RAM 102 in accordance with the register setting in S504, and stores the output image data after the skew correction in the RAM 102. The skew correction is performed on a processing unit basis for each rectangular area obtained by dividing the input image into a plurality of areas.

[0079] The flow of processing in the skew correction circuit 104 will be explained below.

[0080] First, the image input unit 201 reads pixel data of a rectangular area to be subjected to skew correction from the RAM 102 in accordance with the input image address ADDR_IN, and stores the read pixel data of the rectangular area in the SRAM 202 .

[0081] Next, the grid point coordinate generator 203 generates grid point coordinates for sampling pixels from the input image along the skew angle according to the register settings of the rectangular area's starting coordinates X00, Y00, the inclination Δx, Δy, the rectangular area's width BW, and its height BH. This grid point coordinate generation uses DDA coordinate generation. As shown in FIG. 4, when the grid point coordinate P00 (X00, Y00) that is the starting point within the rectangular area and the inclination Δx, Δy are given, the grid point coordinate P01 (X01, Y01) to the right of the grid point coordinate P00 is calculated as (X00 + Δx, Y00 + Δy). Similarly, the grid point coordinate P10 (X10, Y10) below the grid point coordinate P00 is calculated as (X00 + Δy, Y00 + Δx). Similar coordinate generation is repeated to generate sampling grid point coordinates for the rectangular area with the number of pixels BW × BH to be skew-corrected.

[0082] Next, the pixel sampling unit 204 reads out the pixel values ​​of the sampling grid point coordinates from the SRAM 202. When the coordinates of the sampling grid point are coordinates between pixels in the input image, the pixel value of the sampling grid point is calculated by linearly interpolating the coordinates of the sampling grid point from the four input pixels surrounding the sampling grid point. As shown in FIG. 4(b), when the coordinates of sampling grid point P are coordinates between pixels A, B, C, and D, the pixel value is expressed by the following formula:

[0083]

number

[0084] Here, n and m are the distances from the sampling grid point to the surrounding input pixel, and the pixel value P of the coordinates of the sampling grid point is found.

[0085] Finally, the image output unit 205 writes the output pixels sequentially from the output image address ADDR_OUT of the RAM 102. When the image output unit 205 completes writing of all output pixels within the rectangular area, it issues (outputs) an end interrupt to the CPU 101.

[0086] In S507, the CPU 101 waits for an end interrupt from the skew correction circuit 104.

[0087] In S508, the CPU 101 clears the end interrupt register from the skew correction circuit 104. When the end interrupt register is cleared, the process proceeds to S509.

[0088] In S509, CPU 101 checks (determines) whether skew correction has been completed for all rectangular areas of the input image. If the determination result indicates that there are unprocessed rectangular areas and that skew correction has not been completed for all rectangular areas of the input image (NO in S509), the process returns to S504, and the series of processes from S503 to S509 are executed for the unprocessed rectangular areas. On the other hand, if the determination result indicates that skew correction has been completed for all rectangular areas (YES in S509), the flow shown in FIG. 5 ends.

[0089] As described above, the allowable error of the skew correction coordinates is selected (set) according to the type of scanned image, and the input image is divided into rectangular areas that serve as processing units for skew correction based on the selected allowable error and skew angle. Skew correction is performed by resetting the registers of the skew correction circuit for each rectangular area using the firmware so as to correct coordinate errors that occur in the skew correction circuit.

[0090] (skew correction processing overhead) Figure 11 illustrates the overhead of skew correction processing. In Figure 11, the upper part shows the case where the rectangular area (image division) size is relatively small. The lower part shows the case where the rectangular area (image division) size is relatively large. Note that in both the upper and lower parts, the processing time required for register calculation and setting in the skew correction circuit and the end interrupt time of the skew correction circuit are the same. The dotted area indicates the processing time by the FW, and the white area indicates the processing time by the HW (skew correction circuit). As shown in Figure 11, when the rectangular area size is reduced, the coordinate accuracy of skew correction increases, but the number of register settings by the FW increases, and the FW processing time increases accordingly. On the other hand, when the rectangular area size is increased, the coordinate accuracy of skew correction decreases, but the number of register settings by the FW decreases, and the FW processing time decreases accordingly. In this embodiment, by selecting an allowable error according to the type of scanned image for each page of the document and appropriately changing the size of the rectangular area that is the processing unit for skew correction, it is possible to obtain a high-quality skew-corrected image without increasing the overall processing time for skew correction.In other words, it is possible to obtain a high-precision image as a corrected scanned image while suppressing an increase in the processing time required for skew correction for the entire scanned image.

[0091] Although the above description describes mode selection based on a user instruction in S501 of this embodiment, the mode selection method is not limited to this. The type of input image may be automatically determined by image analysis, and mode selection may be performed based on the determination result. Automatically determining the type of input image eliminates the need for user instruction for mode selection. Various methods can be used for automatic determination. Using the number of colors in the image determination area (scanned image), an image may be determined to be a text / line drawing if the number of colors is less than a predetermined value, and a natural image if the number of colors is greater than the predetermined value. Alternatively, an edge detection filter such as a Sobel filter may be used. Alternatively, attribute information generated by other image processing means, corresponding to the image determination area (scanned image), and indicating the type of image may be used. When an "edge detection filter" is used, an image may be determined to be a text / line drawing if an edge is detected in the image determination area (scanned image), and a natural image if no edge is detected. When "attribute information" is used, an image may be determined to be a text / line drawing if the attribute information indicates that the image contains text / line drawings, and a natural image if the attribute information indicates that the image does not contain text / line drawings.

[0092] In addition, in S501, one mode is selected from two modes, the text / line drawing mode and the natural image mode, and the skew correction tolerance is set based on the selected mode. However, this is not limited to this. Three or more modes with different skew correction tolerances may be provided, and a mode may be selected according to the type of image. By providing multiple modes for selecting the tolerance, skew correction can be performed with optimal accuracy depending on the type of document. For example, three modes may be provided: a first text / line drawing mode (high accuracy), a second text / line drawing mode (medium accuracy), and a natural image mode. In this case, the tolerances Et for the first text / line drawing mode, the second text / line drawing mode, and the natural image mode may be set to 0.13 pixels, 0.25 pixels, and 0.50 pixels, respectively.

[0093] Alternatively, the text / line drawing mode may be divided into two modes, such as a line drawing mode (CAD drawing mode), a text mode, and a natural image mode, resulting in three modes. In this case, the allowable error Et for the line drawing mode, text mode, and natural image mode may be 0.13 pixels, 0.25 pixels, and 0.50 pixels, respectively. The allowable error value may also be different for the line drawing mode and the text mode. The reason for providing these three modes is that CAD drawings (design drawings) require particularly high-precision line drawing, and there is a stronger demand for high-precision line drawing than for text. Furthermore, even if coordinate errors occur in short lines such as text, the errors are slight and inconspicuous. On the other hand, even a slight coordinate error in a long straight line such as a line drawing gradually accumulates along the long line, resulting in a larger error at the end of the line.

[0094] Furthermore, while the above description has treated text and line drawing regions as a single classification, it is also possible to provide classifications for horizontal and vertical line regions. Even for the same line drawing, the degree to which skew is perceived varies depending on the angle. Specifically, the human eye has high accuracy in recognizing the angle of horizontal and vertical lines, and even slight skew in horizontal and vertical lines is easily perceived by the human eye. On the other hand, if a line is originally oblique, such as at 45 degrees, a slight change in the angle is less noticeable. This is known as the oblique effect. Therefore, it is also possible to provide separate classifications for horizontal and vertical lines, assigning a smaller tolerance to areas containing horizontal and vertical lines, and performing highly accurate skew correction.

[0095] [Embodiment 2] In this embodiment, a mode will be described in which the allowable error for skew correction is changed (set) for each image discrimination area obtained by dividing one page into small areas. Note that the image processing device and skew correction circuit according to this embodiment have the same configuration as the image processing device shown in Figure 1 and the skew correction circuit shown in Figure 2 described in the first embodiment above, and therefore a description thereof will be omitted.

[0096] (skew correction processing) FIG. 12 is a flowchart showing the flow of the skew correction process executed by the image processing device according to this embodiment.

[0097] In S1201, the CPU 101 of the image processing device of this embodiment analyzes the image type (text / line image area, natural image / single tone area) for each image discrimination area obtained by dividing one page of image into small areas. The image discrimination area is assumed to be, for example, a small area obtained by dividing one page of image into 16 pixel by 16 pixel units. The method (size) of dividing the image discrimination area is not limited to this and can be determined arbitrarily. Various methods can be used to analyze the image type. The number of colors in the image discrimination area may be used to determine that the area is a text / line image area if the number of colors is equal to or less than a predetermined value. Alternatively, an edge detection filter such as a Sobel filter may be used. Alternatively, attribute information generated by other image processing means may be used.

[0098] In S1202, the CPU 101 assigns an allowable error to each image discrimination region according to the image type determined in S1201. In S502 of the first embodiment described above, the allowable error was the same for all images on one page, but in S1202, the allowable error can be made different for each image discrimination region obtained by dividing the image on one page into small regions.

[0099] In S1203, CPU 101 divides the input image into rectangular regions with a rectangular region size that keeps the coordinate error of the DDA below the allowable pixel limit for each image discrimination region. Similar to the first embodiment, the calculation formula described in S503 of the first embodiment is used to determine the rectangular region size that satisfies the allowable error. In this embodiment, the allowable error differs for each image discrimination region, so the rectangular region size can also differ for each image discrimination region of the input image.

[0100] Here, an example of rectangular division of an input image in which the rectangular region size is changed for each image discrimination region will be described with reference to the drawings. Fig. 13 is a diagram showing an example of dividing an input image into rectangular regions in S1203. A relatively small value of allowable error is assigned to a character region 1311 and a line drawing region 1312, which are image discrimination regions in the input image, so the rectangular region sizes are relatively small. On the other hand, a relatively large value of allowable error is assigned to a natural image region 1313 and a single tone region 1314, which are also image discrimination regions in the input image, so the rectangular region sizes are also relatively large.

[0101] The processing from S504 to S509 shown in FIG. 12 is the same as the processing from S504 to S509 in the first embodiment described above, and therefore a description thereof will be omitted.

[0102] As described above, according to this embodiment, by analyzing the image type for each image discrimination area within a page and changing the allowable error for skew correction based on the analysis results, it is possible to control the accuracy of skew correction for each image discrimination area and obtain a skew-corrected image of higher image quality. Furthermore, even for an image in which text / line drawing areas and natural image areas are mixed within the same page, it is possible to determine the allowable error for skew correction according to the image type for each image discrimination area, and skew correction can be performed with an accuracy corresponding to the image type.

[0103] Furthermore, while the above description has treated text and line image regions as a single classification, it is also possible to provide classifications for horizontal and vertical line regions. This is because the degree to which the same line image is perceived as skew varies depending on its angle. Specifically, the human eye has high accuracy in recognizing the angles of horizontal and vertical lines, and even slight skew in horizontal and vertical lines is easily perceived by the human eye. On the other hand, if the line is originally at an angle such as 45 degrees, a slight change in the angle is less noticeable. This is known as the oblique effect. For this reason, it is also possible to provide separate classifications for horizontal and vertical lines, assigning smaller tolerances, and perform skew correction with high accuracy.

[0104] In this case, since even slight deviations from the horizontal or vertical axis of horizontal and vertical lines are noticeable, the allowable error may be set to a small value, such as 0.13 pixels, in order to perform skew correction with high precision. In character and line image areas that include other characters and diagonal lines, slight changes in angle are less noticeable, so the allowable error may be set to a larger value, such as 0.25 pixels, compared to that for horizontal and vertical lines.

[0105] [Embodiment 3] In this embodiment, a mode will be described in which the allowable error for skew correction is set to a relatively small value for a user-selected area specified via the user input unit 106. Note that the image processing device and skew correction circuit according to this embodiment have the same configuration as the image processing device shown in Fig. 1 and the skew correction circuit shown in Fig. 2 described in the first embodiment above, and therefore a description thereof will be omitted.

[0106] (skew correction processing) Fig. 14 is a flowchart showing the flow of skew correction processing executed by the image processing apparatus according to this embodiment. Note that the processing of S1203 and S504 to S509 shown in Fig. 14 is the same as the processing of S1203 and S504 to S509 in the second embodiment, and therefore description thereof will be omitted.

[0107] In S1401, the CPU 101 accepts a designation of an important area in an input image as a user-selected area by a user operation via the user input unit 106 such as a touch panel. That is, in S1401, an input operation by the user is accepted.

[0108] In S1402, CPU 101 assigns an allowable error to each image discrimination region in the input image. The user-selected region specified by the user in S1401 is treated as a single image discrimination region and assigned a small allowable error value, while other regions are assigned an allowable error value larger than the allowable error assigned to the user-selected region.

[0109] (Input image including image discrimination region) 15 is a diagram showing an example of dividing an input image into rectangular regions in S1203 of this embodiment. A small value of allowable error is assigned to a user-selected region 1511 in an input image 1510, so the rectangular region size is relatively smaller than other regions 1512 indicating text, line drawing regions, etc. A large value of allowable error is assigned to other regions 1512, so the rectangular region size is also relatively larger than that of the user-selected region 1511.

[0110] As described above, according to this embodiment, the following effects can be obtained by accepting a user's operation to designate an important area in an input image as the user selection area 1511 via the user input unit 106. That is, by treating the user selection area 1511 as a single image discrimination area, allocating a relatively small value of allowable error, and performing skew correction with high precision, it is possible to obtain a high-quality skew-corrected image that the user desires.

[0111] The above describes a case where the image discrimination area in the input image is classified into two areas: user-selected area and other area, but this is not limited to this, and the image discrimination area in the input image may be classified into three or more areas.

[0112] The method for specifying a user-selected area according to this embodiment and the method for classifying by image type according to the second embodiment described above may be used in combination to classify areas such as user-selected text and line drawing areas and user-selected natural image and single-tone areas, and assign different allowable errors to each image discrimination area. In this case, by assigning a particularly small allowable error value to the user-selected text and line drawing areas among the user-selected areas, a skew-corrected image of higher image quality can be obtained.

[0113] [Other embodiments] The present disclosure can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program. The present disclosure can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0114] 100 Image processing device 101 CPU

Claims

1. An image processing device that performs skew correction on a scanned image of a document, a setting unit for setting a tolerance for the skew correction in accordance with the type of the scanned image; a correction unit that performs skew correction based on the allowable error; a determining means for determining a size for dividing the scanned image into a plurality of regions based on the allowable error set by the setting means and the skew angle of the scanned image; a dividing means for dividing the scanned image into parts according to the size; and The correction means performs skew correction on the plurality of areas into which the scanned image is divided according to the size.

1. An image processing device comprising:

2. The setting means sets the allowable error for each page or each document.

2. The image processing device according to claim 1, wherein:

3. The setting means If the scanned image includes a straight line, a first tolerance is set; If the scanned image does not contain the straight line, a second tolerance is set.

3. The image processing device according to claim 2.

4. The setting means If the scanned image includes text or line art, setting the first tolerance; 4. The image processing apparatus according to claim 3, wherein the second allowable error is set when the scanned image does not include the characters and the line drawing.

5. 5. The image processing apparatus according to claim 4, wherein said setting means sets said second allowable error when said scanned image is a natural image.

6. 6. The image processing apparatus according to claim 3, wherein the setting means sets a third tolerance when the scanned image includes a horizontal line or a vertical line.

7. 2. The image processing apparatus according to claim 1, wherein the setting means sets the allowable error for each discrimination area obtained by dividing the scanned image into small areas.

8. The setting means If the discrimination region includes a straight line, a first tolerance is set; If the discrimination region does not include the straight line, a second tolerance is set.

8. The image processing device according to claim 7,

9. The setting means If the discrimination region includes a character or a line drawing, the first allowable error is set; 9. The image processing apparatus according to claim 8, wherein the second allowable error is set when the discrimination region does not include the character and the line drawing.

10. 10. The image processing apparatus according to claim 9, wherein said setting means sets said second allowable error when said discrimination region is a natural image region or a single tone region.

11. The setting means If the discrimination region includes a horizontal line or a vertical line, a third tolerance is set.

11. The image processing device according to claim 8, wherein the image processing device is a computer.

12. The first tolerance is smaller than the second tolerance.

10. The image processing device according to claim 3, 4, 8 or 9.

13. 12. The image processing apparatus according to claim 6, wherein the third allowable error is smaller than the first allowable error and the second allowable error.

14. further comprising an input means for accepting an input operation by a user; The setting means setting a fourth tolerance for an area of ​​the scanned image designated by the input means; A fifth tolerance is set for an area of ​​the scanned image that is not specified by the input means.

10. The image processing device according to claim 3, 4, 8 or 9.

15. The fourth tolerance is smaller than the fifth tolerance.

15. The image processing device according to claim 14.

16. A specifying unit for specifying the type of the scanned image by a user operation is provided.

16. The image processing device according to claim 1, wherein the image processing device is a processor.

17. an analysis means for analyzing the type of the scanned image; The setting means sets the allowable error in accordance with the analysis result by the analysis means.

16. The image processing device according to claim 1, wherein the image processing device is a processor.

18. 18. The image processing device according to claim 17, wherein the analyzing means analyzes the type of the scanned image based on the number of colors in the scanned image, edges detected in the scanned image, or attribute information corresponding to the scanned image and indicating the type of image.

19. An image processing method for performing skew correction on a scanned image of a document, comprising: a setting step of setting a tolerance for the skew correction according to the type of the scanned image; a correction step of performing skew correction based on the tolerance; a determining step of determining a size for dividing the scanned image into a plurality of regions based on the allowable error set in the setting step and the skew angle of the scanned image; a dividing step of dividing the scanned image into parts according to the size; Including, The image processing method is characterized in that, in the correcting step, skew correction is performed on the plurality of areas into which the scanned image is divided according to the size.

20. A program for causing a computer to function as the image processing device according to any one of claims 1 to 18.

Images