Image processing device, binarization method and program

Abstract

To provide a technique for binarizing a multi-valued image while suppressing voids.SOLUTION: An image processing device comprises: a threshold calculation section which calculates, based on pixel values of an input image having a gradation number M, N-1 (M>N>2) pieces of first thresholds; an N-value section which compares the N-1 pieces of first thresholds with the pixel values of the input image and generates an N-valued image from the input image; and a binarization section which generates a binarized image by binarizing a pixel of interest based on a pixel value for each first local region of the input image including the pixel of interest and a gradation value for each second local region of the N-valued image. The second local region includes the first local region and is larger than the first local region.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to an image processing device, a binarization method, and a program. [Background technology]

[0002] Image data scanned by an image processing device using its scanner function may be a multi-value image. A multi-value image is an image that has pixel values ​​of three or more levels, such as grayscale or color images, rather than just two pixel values, 1 and 0, as in black and white images. While multi-value images can express a wide range of gradations, they can also be difficult for users to read because text and other elements are buried in the gray background, which can reduce the accuracy of OCR (Optical Character Recognition / Reader) processing.

[0003] A technique for binarizing a multi-value image is known (see, for example, Patent Document 1). Patent Document 1 discloses a configuration in which, in order to obtain a binary image in which only the detection target is properly extracted without causing white gaps in a part of the detection target that should be black pixels, the multi-value image is converted into a ternary image using two threshold values, intermediate value pixels in the ternary image are set as unknown pixels, and the unknown pixels are judged as black or white based on the state of neighboring pixels, thereby converting the image into a binary image. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has a problem in that it is not possible to simultaneously remove noise, output the edges of light characters on a white background as black pixels, and prevent the white characters from being blurred.

[0005] In view of the above problems, the present invention aims to provide a technology for binarizing a multi-value image by simultaneously removing noise, outputting the edges of light characters on a white background as black pixels, and preventing the white characters from being blurred. [Means for solving the problem]

[0006] In view of the above problems, the present invention provides an image processing device comprising: a threshold calculation unit that calculates N-1 (M>N>2) first thresholds based on pixel values ​​of an input image having M gradations; an N-value conversion unit that compares the N-1 first thresholds with the pixel values ​​of the input image to generate an N-valued image from the input image; and a binarization unit that generates a binarized image by binarizing the pixel of interest based on pixel values ​​for each first local region of the input image that includes a pixel of interest and gradation values ​​for each second local region of the N-valued image, wherein the second local region includes the first local region and is larger than the first local region. [Effects of the Invention]

[0007] The present invention provides a technology for binarizing a multi-valued image by simultaneously removing noise, outputting the edges of light characters on a white background as black pixels, and preventing the white characters from being blurred. [Brief explanation of the drawings]

[0008] [Figure 1] 10A and 10B are diagrams illustrating a comparison of processing results of global binarization and adaptive binarization; [Figure 2] FIG. 1 is a diagram illustrating an image processing device or an image processing system that is an example of a device that performs binarization processing. [Figure 3] FIG. 1 illustrates an example of a hardware configuration of an image processing apparatus. [Figure 4] FIG. 1 is a functional block diagram of an example of image processing for generating a binarized image (black and white image) from an input image. [Figure 5] FIG. 4 is a functional block diagram illustrating an example of a function of a threshold calculation unit. [Figure 6] FIG. 10 is a diagram showing an example of a histogram created by a histogram creating unit. [Figure 7] FIG. 10 is a flowchart illustrating an example of processing performed by a discriminant analysis unit to determine three thresholds t0, t1, and t2. [Figure 8] FIG. 2 is a functional block diagram illustrating an example of a function of a binarization unit. [Figure 9]FIG. 2 is a diagram illustrating a pixel of interest and a local region. [Figure 10] 10 is a diagram illustrating a pixel of interest, a second local region, and a white pixel block / black pixel block determination result storage unit. FIG. [Figure 11] 10 is an example of a functional block diagram illustrating a function of a white background determination unit. [Figure 12] 10A and 10B are diagrams showing examples of a four-valued image that is determined by a white background determination unit to be a white background 1, a white background 2, or no white background. [Figure 13] FIG. 10 is a flowchart illustrating an example of a binarization process performed by an output unit. [Figure 14] FIG. 10 is a diagram illustrating condition 1. [Figure 15] FIG. 10 is a flowchart illustrating an example of the overall processing flow in which the image processing device binarizes an input image. [Figure 16] FIG. 10 is a diagram illustrating an example of a four-valued image. [Figure 17] FIG. 17 is a diagram showing an example of a binarized image obtained by binarizing the quaternary-valued image of FIG. 16 using an existing technique. [Figure 18] 17 is a diagram showing an example of a binarized image obtained by binarizing the quaternary-valued image of FIG. 16 through the binarization process of this embodiment. FIG. [Figure 19] (a) shows an example of a four-valued image, (b) and (c) show binarized images obtained by adaptive binarization, and (d) shows a binarized image obtained by the binarization method of this embodiment. [Figure 20] 10A and 10B are diagrams showing examples of a four-valued image that is determined to be a white background 1, 2, or not a white background. [Figure 21] FIG. 21 is a diagram showing an example of a binarized image obtained by binarizing the four-valued image of FIG. 20 using thresholds such that the gradation value 0 is converted to 0 (white) and the gradation values ​​1 to 3 are converted to 1 (black). [Figure 22] FIG. 21 is a diagram showing an example of a binarized image obtained by binarizing the four-valued image of FIG. 20 using thresholds such that gradation values ​​0 to 1 are converted to 0 (white) and gradation values ​​2 to 3 are converted to 1 (black). [Figure 23]FIG. 21 is a diagram showing an example of a binarized image obtained by binarizing the four-valued image of FIG. 20 using thresholds such that gradation values ​​0 to 2 are converted to 0 (white) and gradation value 3 is converted to 1 (black). [Figure 24] 21 is a diagram showing an example of a binarized image obtained by binarizing the quaternary-valued image of FIG. 20 using the binarization method of this embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image processing device and a binarization method performed by the image processing device will be described below as an example of an embodiment of the present invention with reference to the accompanying drawings.

[0010] <Types of binarization> First, let's briefly explain the types of binarization. There are two types of binarization: global binarization and adaptive binarization. Global thresholding is a process of binarizing the entire image using the same threshold. There are two types of global thresholding: one that uses a fixed threshold (simple thresholding) and one that uses a value calculated from the overall image characteristics depending on the image (a typical example is binarization using discriminant analysis). Adaptive thresholding is a process of binarizing a local area using a threshold that is the average value + offset value of the local area. The threshold value varies depending on the local area. The offset value may be a constant (for example, 5 to 20 for 256 levels) or may be determined as a percentage of the average value (for example, 10%).

[0011] Adaptive thresholding has the advantage of being able to binarize light text or text on a dark background without crushing it, but it has the disadvantage of introducing defects such as white spaces that do not exist in the input image due to threshold switching within the image.

[0012] For this reason, when converting a multi-value image to a binary image, there are problems such as black pixels remaining around characters on a dark background, or switching between binary processes occurring within a background of uniform density, resulting in an image with a noticeable, unnatural appearance.

[0013] Figure 1 compares and explains the processing results of simple binarization and adaptive binarization. Figure 1(a) shows an input image (multi-valued image). The input image 110 includes an area 111 with text written on a dark background. Here, we will explain simple binarization, but the problems with simple binarization are common to global binarization, although to different degrees.

[0014] FIG. 1(b) shows the processing result 112 of simple binarization of region 111. With simple binarization, both the background and the characters become black, resulting in blurred characters. FIG. 1(c) shows the processing result 113 of region 111 binarized using adaptive binarization. With adaptive binarization, the characters do not become blurred. However, even with the processing result 113, black pixels (gray) remain in the background, making the characters difficult to see.

[0015] It is possible to turn black background pixels, such as those in the processing result 113, into white pixels by adjusting the offset value for adaptive binarization. However, this can result in an unnatural processing result for a specific input image, such as that shown in Figure 1(d). In the input image 120 shown in Figure 1(d), the text area is covered by a larger gray area than the text area. Figures 1(e) and 1(f) show the adaptive binarization of this input image 120. If the offset value is set so that the background gray area becomes white, the dense text with a high density of black pixels becomes broken, making it difficult to read, as shown in Figure 1(e). Conversely, if the offset value is set to improve readability, the adaptive threshold switching within the uniformly dense background creates white gaps, resulting in an unnatural binarized image, as shown in Figure 1(f).

[0016] Furthermore, as will be described later with reference to FIGS. 20 to 23, binarization inherently has the problem that it is difficult to achieve both the legibility of light characters on a white background and the legibility of outline characters.

[0017] <Outline of the binarization process of this embodiment> Therefore, the image processing apparatus of this embodiment performs the following when binarizing a multi-valued input image: A. "Calculate N-1 (M>N>2) thresholds based on the overall characteristics of an input image with M gradations, and apply the N-1 thresholds to the input image to generate an N-valued image of the entire input image." B. "Generate a binarized image by binarizing the entire input image based on the features of each first local region of the input image and the features of each second local region of the N-valued image." C. "Furthermore, in B, the second local region is larger than the first local region." B can reproduce light characters or characters on a dark background, and A generates an N-valued image without unnatural noise or white spaces, and the result is reflected in B, so that the final output binary image can also have little noise or white spaces.

[0018] C makes it possible to determine whether the background color is a dark white background area from the characteristics of the second local area, which is a reference area of ​​a relatively wide fixed size, and the result is reflected in the binarized image, so that conversion to a binary image can be performed quickly without crushing the white characters.

[0019] In other words, the binarization method of this embodiment can simultaneously reproduce light characters and characters on a dark background, reduce the occurrence of noise and white spaces, and prevent white characters from being blurred.

[0020] In this embodiment, the explanation will be given mainly on the assumption that M=256 and N=4, but by setting N=8 or the like, binarization that is even less likely to produce white spots and the like is possible.

[0021] <Terminology> Binarization is the process of converting a multi-tone image into two colors, black and white (0, 1). If there are two colors, 0 and 1 can be assigned to a color other than black and white.

[0022] The number of gradations M is the number of gradations of the input image, and in this embodiment, 256 gradations will be described as an example.

[0023] N-1 is the number of thresholds that is one less than N to create an N-valued image. In this embodiment, as an example, the N-valued image is a four-valued image.

[0024] The first local region is a region including a predetermined number of pixels centered around a pixel of interest to be binarized. In this embodiment, the first local region is, for example, a region of 7×7 pixels.

[0025] The second local region is a region including a predetermined number of pixels centered around a pixel of interest to be binarized, and is larger than the first local region. In this embodiment, the first local region is, for example, a region of 51 × 7 pixels.

[0026] <Configuration example> FIG. 2 shows an image processing device 20 or an image processing system 100, which is an example of a device that performs binarization processing. The image processing device 20 in FIG. 2(a) is a device used by a user that combines multiple different functions, such as a multifunction peripheral (MFP). The image processing device 20 has at least a scanner function. A scanner is a device or function that converts images, documents, etc. into digital still images for communication or recording. In this embodiment, the digital still images may be color or monochrome, but multi-level images with higher gradations than binary can be generated. The digital still images may also be snapshots of a video.

[0027] In addition to the scanner function, the image processing device 20 may also have a fax function, a print function, a copy function, etc. The image processing device 20 may also be called an image forming device, a printing device, a printer, a scanner device, etc.

[0028] 2(a) can, by itself, scan an original image to generate a multi-tone input image and perform binarization processing on the input image. The binarization processing is performed as a preprocessing for OCR, for example, but the binarization processing may be performed independently of OCR.

[0029] On the other hand, as shown in Fig. 2(b), generation of an input image and the binarization process of this embodiment may be performed by different devices. Fig. 2(b) shows an example of an image processing system 100 in which an information processing device 40 performs the binarization process. This image processing system 100 includes an information processing device 40 and an image processing device 20. The information processing device 40 and the image processing device 20 are connected to each other so as to be able to communicate with each other via a LAN or Wi-Fi (registered trademark) within the facility, a USB cable, or the like.

[0030] When a user places an original document on the image processing device 20 and executes scanning, the image processing device 20 transmits a multi-tone input image to the information processing device 40 via the network N. The information processing device 40 receives the multi-tone input image generated by the image processing device 20 scanning the original document, and can perform binarization processing on the input image.

[0031] 2(c), the binarization process may be executed as part of a workflow process. A workflow is a series of processes that combine and execute multiple processes (e.g., scanning, saving to a cloud, or sending by email). For example, a service is known in which an information processing system 60 performs predetermined processing on image data generated by a device reading a document, and then saves the image data in a cloud or sends it by email.

[0032] FIG. 2(c) shows an image processing system 100 that executes a workflow. The image processing system 100 includes an information processing system 60 and an image processing device 20. The information processing device 40 may or may not be included in the image processing system 100. The information processing system 60 and the image processing device 20 are communicatively connected via a wide-area network N1 such as the Internet. The image processing device 20 is installed in a facility such as a company, and is connected to a network N2 installed in the facility. The network N2 may be a LAN, Wi-Fi (registered trademark), wide area Ethernet (registered trademark), or a mobile phone network such as 4G, 5G, or 6G.

[0033] The information processing system 60 may be realized by one or more computers. The information processing system 60 may be realized by cloud computing or by a single information processing device. Cloud computing refers to a form in which resources on a network are used without being aware of specific hardware resources. The information processing system 60 may exist on the Internet or on-premise.

[0034] The image processing device 20 and the information processing system 60 may execute a web application. A web application is an application that runs through cooperation between a program written in a programming language (for example, JavaScript (registered trademark)) that runs on a web browser and a program on the web server side. In contrast, an application that cannot be executed unless it is installed on the image processing device 20 is called a native application. In this embodiment, the application executed on the image processing device 20 may be either a web application or a native application.

[0035] The information processing system 60 generates screen information for displaying a screen of a Web application on the image processing device 20. The screen information is a program written in HTML, XML, a scripting language, CSS (Cascading Style Sheet), etc. The structure of a Web page is mainly specified by HTML, the behavior of the Web page is defined by the scripting language, and the style of the Web page is specified by CSS.

[0036] 2(c), the image processing device 20 generates an input image using a scanner function, performs binarization processing, and transmits the image to the information processing system 60 via networks N1 and N2. The information processing system 60 then performs, for example, OCR processing, and transmits the image by email or stores it in the cloud.

[0037] Alternatively, the binarization process may be performed by the information processing system 60. The image processing device 20 generates an input image using a scanner function and transmits the input image to the information processing system 60 via the networks N1 and N2. The information processing system 60 performs binarization process on the received input image and executes the subsequent workflow.

[0038] The information processing device 40 can also accept the execution of a workflow, but may also be used by the user to perform settings related to the workflow (license assignment, initial settings, etc.).

[0039] 2(c) does not have to be scanned by the image processing device 20. For example, the image processing device 20 may be a digital camera or a smartphone, and the information processing device 40 or the information processing system 60 may perform binarization processing on an input image generated by the image processing device 20 by capturing an image of a document or the like. The information processing device 40 or the information processing system 60 may also perform binarization processing on any image on a network.

[0040] In the following description, unless otherwise specified, it is assumed that the image processing device 20 in FIG. 2(a) performs the binarization process.

[0041] <Hardware configuration example> Fig. 3 is a diagram showing an example of the hardware configuration of an image processing device 20 according to an embodiment of the present invention. As shown in Fig. 3, the image processing device 20 includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network I / F 950. Here, the image processing device 20 may be an image forming device, an MFP, or a multifunction peripheral / product / printer.

[0042] Of these, the controller 910 has a CPU 901, which is the main part of the computer, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an ASIC (Application Specific Integrated Circuit) 906, a local memory (MEM-C) 907, an HDD controller 908, and an HD 909, and is configured such that the NB 903 and the ASIC 906 are connected by an AGP (Accelerated Graphics Port) bus 921.

[0043] Of these, the CPU 901 controls the entire image processing device 20. The NB 903 is a bridge for connecting the CPU 901 with the MEM-P 902, the SB 904, and the AGP bus 921, and includes a memory controller that controls reading and writing to the MEM-P 902, a PCI (Peripheral Component Interconnect) master, and an AGP target.

[0044] The MEM-P 902 comprises a ROM 902a, which is memory for storing programs and data that realize the functions of the controller 910, and a RAM 902b, which is used for expanding the programs and data and as a drawing memory during memory printing. The programs stored in the RAM 902b may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0045] The SB 904 is a bridge for connecting the NB 903 with PCI devices and peripheral devices. The ASIC 906 is an integrated circuit (IC) for image processing applications that has hardware elements for image processing and serves as a bridge connecting the AGP bus 921, PCI bus 922, HDD controller 908, and MEM-C 907. The ASIC 906 includes a PCI target and AGP master, an arbiter (ARB) that forms the core of the ASIC 906, a memory controller that controls the MEM-C 907, multiple direct memory access controllers (DMACs) that perform image data rotation using hardware logic, and a PCI unit that transfers data between the scanner unit 931, printer unit 932, and fax unit 933 via the PCI bus 922. A USB (Universal Serial Bus) interface or an IEEE 1394 (Institute of Electrical and Electronics Engineers) interface may be connected to the ASIC 906.

[0046] The short-distance communication circuit 920 has a card reader 920a for reading user authentication information stored on an IC card or the like.

[0047] The operation panel 940 has a touch panel 940a and a numeric keypad 940b that accept inputs from the user. The touch panel 940a displays a setting screen for the image processing device 20 and the like.

[0048] <About the function> Next, the binarization function of the image processing device 20 will be described in detail with reference to Fig. 4. Fig. 4 is a functional block diagram showing image processing performed up to the generation of a binarized image (black and white image) from an input image. Note that the input image is an RGB image with 256 gradations of 8 bits per color, but it may be higher, such as 10 bits per color, or it may be lower.

[0049] The image processing device 20 has a smoothing unit 11, a gray processing unit 12, and a multi-value image processing unit 13. The multi-value image processing unit 13 also has a threshold calculation unit 14, an N-value conversion unit 15, and a binarization unit 16. Each of these functional units of the image processing device 20 is a function or means realized by the CPU 901 executing instructions included in one or more programs installed in the image processing device 20. Alternatively, each functional unit may be realized by an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), a hardware circuit module, or the like.

[0050] The smoothing unit 11 smooths the input image using a spatial filter. Smoothing filters include averaging filters, Gaussian filters, median filters, maximum value filters, and minimum value filters, and any of these may be used appropriately. Smoothing reduces the effects of noise and reduces the number of black halftone dots remaining after binarization.

[0051] The gray processing unit 12 converts the input image (RGB image) into a gray image (8 bits, 256 gradations) using a conversion formula from RGB to brightness. The gray processing unit 12 converts into a gray image using, for example, a conversion formula for YCbCr to Y signals used in JPEG. Since 0 corresponds to black and 255 corresponds to white in brightness Y, the gray processing unit 12 inverts black and white to convert into an image signal where 0 corresponds to white and 255 corresponds to black. This processing is for the convenience of subsequent processing.

[0052] The multi-value image processing unit 13 converts the 8-bit, 256-level gray image after graying into a 1-bit, 2-level black-and-white image and outputs it. In this embodiment, the conversion is explained as a black-and-white image where 0 is white and 1 is black, but if black-and-white inversion is not performed in the previous stage (gray processing unit 12), the data logic in the following explanation will be reversed.

[0053] The threshold calculation unit 14 calculates three thresholds (in the case of a four-valued image, N=4, so N-1=3) from the overall image characteristics of the input image. The method of calculating the thresholds will be described later.

[0054] The N-value conversion unit 15 converts each pixel value from 256 gradations to 4 gradations (four-valued image) by applying the three thresholds calculated by the threshold calculation unit 14.

[0055] The binarization unit 16 converts the binarization result of the pixel of interest into 0 or 1 based on the features of the first local region of the gray image and the features of the second local region of the quaternary image, and outputs the binarized result. The binarization unit 16 repeatedly performs the same process by shifting the position of the pixel of interest in the gray image or the quaternary image by one pixel at a time.

[0056] <<Calculating the threshold>> Next, a method for calculating a threshold value performed by the threshold value calculation unit 14 will be described with reference to Fig. 5 to Fig. 7. Fig. 5 is a functional block diagram illustrating the function of the threshold value calculation unit 14. The threshold value calculation unit 14 has a histogram creation unit 21 and a discriminant analysis unit 22.

[0057] The histogram creation unit 21 creates a frequency (histogram) for each gradation of the input image. An example of the histogram is shown in FIG. 6. Creating the histogram makes it easier for the discriminant analysis unit 22 to calculate three thresholds. The discriminant analysis unit 22 calculates the three thresholds by performing a predetermined process on the histogram. Details will be explained with reference to FIG. 7.

[0058] FIG. 6 is an example of a histogram created by the histogram creation unit 21. As shown in FIG. 6, the horizontal axis represents pixel values ​​(0 to 256), and the vertical axis represents frequency (the number of pixels for each pixel value). The discriminant analysis unit 22 can easily detect a minimum value or a value near the minimum value of the histogram as a threshold, which makes it easier for the N-value conversion unit 15 to convert similar pixel values ​​into four gradation values. FIG. 6 shows thresholds t0, t1, and t2 (an example of a first threshold) determined by the discriminant analysis unit 22.

[0059] Fig. 7 is a flowchart illustrating the process of determining three thresholds t0, t1, and t2 by the discriminant analysis unit 22. Fig. 7 shows the loop process of the discriminant analysis unit 22 shown in Fig. 5. The discriminant analysis unit 22 performs the discriminant analysis method for calculating the three thresholds three times by changing the reference range of the histogram.

[0060] The discriminant analysis method is a technique for finding the threshold that maximizes the degree of separation. The degree of separation is calculated using the inter-class variance and the intra-class variance. Discriminant analysis will be explained later.

[0061] First, in the first loop, the discriminant analysis unit 22 calculates a threshold value t1 by applying the discriminant analysis method within the range of gradation values ​​[0, 255], that is, over the entire histogram (S1).

[0062] In the second loop, the discriminant analysis unit 22 calculates the threshold value t0 by applying the discriminant analysis method to the partial histogram within the range of gradation values ​​[0, t1-1], that is, on the white side (S2).

[0063] In the third loop, the discriminant analysis unit 22 calculates the threshold value t2 by applying the discriminant analysis method to the partial histogram within the range of gradation values ​​[t1, 255], i.e., on the black side (S3). Note that S2 and S3 can be performed in the reverse order.

[0064] <<Discriminant analysis method>> The discriminant analysis method will be explained. It is assumed that the discriminant analysis unit 22 determines the threshold value t from 0≦t≦255. The range of 0≦t is set as the white class, the number of pixels is ω1, the mean is m1, and the variance is δ1. Similarly, the range of t≦255 is set as the black class, the number of pixels is ω2, the mean is m2, and the variance is δ2. Then, the number of pixels of the entire image is set as ω t , the average is m t , variance is δ t Let's say.

[0065] The within-class variance is defined by equation (1).

[0066]

number

[0067]

number

[0068]

number

[0069]

number

[0070]

number

[0071] Here, we will explain that the thresholds calculated by the discriminant analysis method are applied as they are, but the N-value conversion unit 15 may also add an offset value given as a constant to the thresholds t0 to t2 before converting them to four values, as is commonly done.

[0072] <Binarization> Next, the binarization process will be described with reference to Figs. 8 to 15. Fig. 8 is a functional block diagram illustrating the function of the binarization unit 16. The binarization unit 16 has an average value calculation unit 31, a minimum value calculation unit 32, a pixel number counting unit 33, a white background determination unit 34, and an output unit 35. A gray image with 256 gradations is input to the average value calculation unit 31 and the minimum value calculation unit 32. A four-valued image with four gradations is input to the pixel number counting unit 33 and the white background determination unit 34.

[0073] 9 is a diagram illustrating a pixel of interest 201 and a first local region 301. In this embodiment, the first local region 301 is a 7×7 pixel area centered around the pixel of interest 201. The 7×7 size of the first local region 301 is just an example, and the first local region 301 may be smaller or larger.

[0074] The average value calculation unit 31 refers to the first local region 301 of the 256-level gray image and calculates the average value of 49 pixels.

[0075] The minimum value calculation unit 32 references the first local region 301 of the input image and finds the minimum value of the 49 pixels. The difference between the average value and the minimum value is used to determine whether or not an edge exists. An edge is the boundary between a bright part (white) and a dark part (black) in an image.

[0076] The pixel number counting unit 33 refers to the first local region 301 of the quaternary image and counts the number of pixels for each gradation of 0, 1, 2, and 3. The number of pixels with gradation value 0 is cnt0, the number of pixels with gradation value 1 is cnt1, the number of pixels with gradation value 2 is cnt2, and the number of pixels with gradation value 3 is cnt3.

[0077] The white background determination unit 34 determines whether the second local region (51 × 7 pixels) shown in FIG. 10 is white background 1 (white background on a light background), white background 2 (white background on a dark background), or something else. Note that the pixel of interest 201 of the first local region 301 and the pixel of interest 201 of the second local region are the same, and for each first local region 301, it is determined whether the surrounding second local region is white background 1, white background 2, or something else. In other words, the second local region includes the first local region and is larger than the first local region. Note that a part of the first local region may be outside the range of the second local region.

[0078] The outline background determination unit 34 may determine whether the second local region is outline background 1 or something else, or outline background 2 or something else. Even with this determination, it is possible to simultaneously remove noise, output the edges of light characters on a white background as black pixels, and prevent the outline characters from being blurred. As will be described with reference to FIG. 13 , by determining whether the second local region is outline background 1 or something else, it is possible to determine whether condition 2 is met, and by determining whether the second local region is outline background 2 or something else, it is possible to determine whether condition 3 is met.

[0079] <White background detection> The outline background determination will be described with reference to Fig. 10. First, Fig. 10(a) is a diagram illustrating a pixel of interest 201 and a second local region 302. In this embodiment, the second local region 302 is defined as 51 x 7 pixels centered around the pixel of interest 201. The second local region 302 being defined as 51 x 7 pixels is an example, and the second local region 302 may be smaller or larger, but is defined as a region larger than the first local region 301.

[0080] 51 pixels in the width direction corresponds to approximately 4.3 mm when the image processing device 20 reads a document at a typical reading resolution of 300 dpi. Because the second local region 302 is a region for analyzing the state of the background of a character, it is preferable that the size of the second local region 302 be approximately the same as or slightly larger than one character. Therefore, the preferred size of the second local region 302 varies depending on the reading resolution and character size. Therefore, as an example, the size of the second local region 302 is approximately twice the size of the first local region 301 but less than two characters. The white background determination unit 34 may estimate the character size (number of pixels) from an existing description in the input image and dynamically determine the size of the second local region 302 to be approximately the same as or slightly larger than one character.

[0081] Also, while FIG. 10(a) shows 51 pixels elongated horizontally in the drawing, the second local region 302 may be 51 pixels elongated vertically. It is preferable that the 51 pixels are aligned in the direction of the line sensor, which reduces memory consumption. Therefore, the orientation of the 51 pixels is unrelated to the direction in which the characters are written. Also, the second local region 302 may be square or rectangular, instead of being horizontally or vertically elongated.

[0082] The white background determination unit 34 determines whether the second local region 302 of the quaternary image is a white background region by referring to the second local region 302. A white background region refers to a region where the background color is darker than the character color (reverse character region), such as a region with white characters written on a black background.

[0083] For this determination, a local region 101 is set in the second local region 302. The local region 101 is an even smaller local region (an example of a third local region) than the second local region 302, and is, for example, 3 × 7 pixels. The local region 101 is used to count the number of pixels for each gradation of 0, 1, 2, and 3. While the determination of whether the second local region is a white background 1, 2, or other is performed for the entire second local region, the local region 101 is used to determine whether a cluster of 0, 1, 2, and 3 pixels exists within the second local region 302. In this way, the white background determination unit 34 can determine whether there are white pixels or black pixels as a cluster (i.e., background), rather than whether there are 0, 1, 2, and 3 pixels in the entire second local region.

[0084] 10(b) shows the white pixel block / black pixel block determination result storage unit for storing the result of determining whether the local region 101 is a white pixel block or a black pixel block. The result of the white pixel block / black pixel block determination is 49 x 1 pixels, as it is determined while shifting the local region 101 by one pixel. 49 is the value obtained by subtracting 2 from both ends from 51, which is the width of the second local region 302.

[0085] 11 is a functional block diagram illustrating the function of the white background determination unit 34. The white background determination unit 34 has a pixel number counting unit 41, a white pixel block / black pixel block determination unit 42, a white pixel block / black pixel block counting unit 43, and a background type determination unit 44. A four-level, four-valued image is input to the pixel number counting unit 41.

[0086] The pixel counting unit 41 counts the number of pixels for each gradation of 0, 1, 2, and 3, with reference to a local region 101 within the second local region in the quaternary image. The pixel counting unit 41 acquires the pixel count result in the local region 101 while moving the local region 101 to the right by one pixel at a time. The number of pixels with gradation value 0 is designated wcnt0, the number of pixels with gradation value 1 is designated wcnt1, the number of pixels with gradation value 2 is designated wcnt2, and the number of pixels with gradation value 3 is designated wcnt3. wcnt0 to wcnt3 are values ​​associated with the local region 101.

[0087] The white pixel block / black pixel block determination unit 42 determines whether the local region 101 is white pixel block 1, black pixel block 1, white pixel block 2, or black pixel block 2 based on the pixel count result and the following determination conditions: Therefore, the local region 101 is classified as white pixel block 1, black pixel block 1, white pixel block 2, or black pixel block 2. Note that n_th1 is a threshold parameter for determining whether the local region is a white pixel block or a black pixel block, and is set to a value smaller than the total number of pixels in the local region 101 (3×7=21) (for example, n_th1=15). If wcnt0 > n_th1, the local region 101 is determined to be a white pixel block 1. If wcnt2 + wcnt3 > n_th1, the local region 101 is determined to be black pixel block 1. If wcnt0 + wcnt1 > n_th1, the local region 101 is determined to be a white pixel block 2. If wcnt3 > n_th1, the local region 101 is determined to be a black pixel block 2. In descending order of density, the relationship is black pixel block 2 > black pixel block 1 > white pixel block 2 > white pixel block 1.

[0088] The white pixel block / black pixel block determining unit 42 stores the white pixel block / black pixel block determination results determined for each local region 101 in each storage pixel 102 in the white pixel block / black pixel block determination result storage unit.

[0089] The white pixel block / black pixel block counting unit 43 counts the number of pixels in white pixel block 1, black pixel block 1, white pixel block 2, and black pixel block 2, respectively, by referring to the 49 x 1 pixel white pixel block / black pixel block judgment result storage unit shown in Figure 10(b). The number of pixels in white pixel block 1 is block_cnt_wt1 The number of pixels in black pixel block 1 is block_cnt_bk1 The number of pixels in white pixel block 2 is block_cnt_wt2 The number of pixels in black pixel block 2 is set to block_cnt_bk2.

[0090] The background type determination unit 44 determines whether the pixel of interest 201 in the second local region 302 is a pixel of white background 1, a pixel of white background 2, or a pixel of a background region that falls into neither category, based on the following determination conditions. Examples of white backgrounds 1 and 2 are shown in FIG. 12. n_th2 is a threshold parameter for determining the background type, and is set to a value smaller than the number of pixels, 49×1 pixels, shown in FIG. 10(b) (for example, n_th2=10). If block_cnt_wt1 ≦ block_cnt_bk1 and block_cnt_bk1 > n_th2, it is determined to be white background 1. If block_cnt_wt2 ≦ block_cnt_bk2 and block_cnt_bk2 > n_th2, it is determined to be white background 2. If the character is not determined to be either white background 1 or 2, it is determined not to be white background (i.e., the inside of the character is determined to be darker than the background). This determination makes it possible to determine whether the second local region 302 has a white background (white characters) and whether the background is light or dark.

[0091] In this way, the white-out background determination unit 34 can quickly determine whether an image is a white-out background in the second local region 302 of a relatively small fixed size, and can use the information on whether an image is a white-out background to prevent white-out characters from being crushed.

[0092] 12 shows examples of a four-valued image that is determined by the white background determination unit 34 to be white background 1, 2, or no white background. FIG. 12(a) is an example of a four-valued image that is determined to be no white background (text on a white background). As shown in FIG. 12(b), if there are more pixel blocks with gradation value 2 (dark gray) or gradation value 3 (black) than pixel blocks with gradation value 0 (white), the image is determined to be white background 1 (an example of a first white background). As shown in FIG. 12(c), if there are more pixel blocks with gradation value 3 (black) than pixel blocks with gradation value 0 (white) or gradation value 1 (light gray), the image is determined to be white background 2 (an example of a second white background).

[0093] <Binarization process and conditions 1-3> Next, the binarization flow and conditions 1 to 3 will be explained based on Fig. 13. Conditions 1 to 3 are conditions for determining whether to convert a pixel of interest in a gray image to 0 or 1. Conditions 1 to 3 will be described in detail later.

[0094] 13 is a flowchart illustrating the binarization process performed by the output unit 35. The output unit 35 determines whether or not Condition 1 (an example of the first condition) is met (S11).

[0095] If the determination in step S11 is Yes, the output unit 35 performs binarization using binarization method 1 (S12). Binarization method 1 will be described later.

[0096] If the determination in step S11 is No, the output unit 35 determines whether or not Condition 2 (an example of the second condition) is met (S13).

[0097] If the determination in step S13 is Yes, the output unit 35 performs binarization using binarization method 2 (S14). Binarization method 2 will be described later.

[0098] If the determination in step S13 is No, the output unit 35 determines whether or not Condition 3 (an example of the third condition) is met (S15).

[0099] If the determination in step S15 is Yes, the output unit 35 performs binarization using binarization method 3 (S16). Binarization method 3 will be described later.

[0100] If the determination in step S15 is No, the output unit 35 performs binarization using binarization method 4 (S17). Binarization method 4 will be described later.

[0101] In FIG. 13, condition 3 is determined after condition 2, but condition 3 may be determined before condition 2.

[0102] <Regarding Condition 1> FIG. 14 is a diagram illustrating condition 1. Condition 1 is that the count result for the same gradation value as the pixel of interest in the quaternary image is 2 or less, and only two of cnt0, cnt1, cnt2, and cnt3 have a value of 1 or greater (there are only two types of gradation values ​​within the first local region). In other words, there are only pixels with the same gradation around the pixel of interest (within the first local region), and there is at most one pixel around with the same gradation value as the pixel of interest. Therefore, condition 1 is a condition for detecting isolated point noise.

[0103] Note that the count result of the same gradation value as the gradation value of the pixel of interest being 2 (an example of the second threshold) or less is one example, and this threshold may be 3, etc. Also, the threshold is set to 2 when the first local region 301 has 49 pixels, and if the first local region 301 is wider, the threshold used for comparison with the count result of the same gradation value as the pixel of interest will also be larger. If the first local region 301 is narrower, the threshold used for comparison with the count result of the same gradation value as the pixel of interest will be smaller.

[0104] 14(a), the count result of the number of pixels having the same gradation value as the pixel of interest 91 (gradation value 1) is 2. Also, cnt0 = 47, cnt1 = 2, cnt2 = 0, and cnt3 = 0. Therefore, the first local region 301 in FIG. 14(a) satisfies condition 1.

[0105] 14(b), the count result of the number of pixels having the same gradation value as the pixel of interest 92 (gradation value 2) is 1. Also, cnt0=0, cnt1=0, cnt2=1, and cnt3=48. Therefore, the first local region 301 in FIG. 14(b) satisfies condition 1.

[0106] If condition 1 is satisfied, the output unit 35 performs binarization using binarization method 1. Binarization method 1 is a process in which if the value of another gradation in the quaternary image that is different from that of the target pixel is 0 or 1, the binarization result of the target pixel is set to 0, and if it is 2 or 3, the binarization result is set to 1. In other words, the output unit 35 binarizes the target pixel of the quaternary image so that the gradation values ​​of the target pixel and its surrounding pixels are the same. Since the number of pixels with the same gradation as the target pixel is two or less, the target pixel is unified to the gradation value with the largest value within the first local region 301. This allows the output unit 35 to eliminate isolated point noise.

[0107] Fig. 14(c) is a binarized image obtained by binarizing the quaternary image of Fig. 14(a) using binarization method 1. In Fig. 14(a), the value of the other gradation level different from that of the target pixel 91 is 0, so in Fig. 14(c), the binarization result of the target pixel 91 is 0. Note that the first local region 301 in Fig. 14(c) shows a state in which the pixel immediately below the target pixel 91 has been similarly binarized in the next processing.

[0108] 14(d) is a binarized image obtained by binarizing the quaternary image of FIG. 14(b) using binarization method 1. In FIG. 14(b), the other gradation value different from that of the pixel of interest 92 is 3, so in FIG. 14(d), the binarization result of the pixel of interest 92 is 1.

[0109] <Regarding Condition 2> Condition 2 is (i) the gradation value of the pixel of interest in the four-valued image is 1 (an example of a first predetermined value), and (ii) the absolute value of the difference between the average value and the minimum value of the first local region 301 in the grayscale image is equal to or greater than a threshold value (an example of a third threshold value); and (iii) The gradation value of the pixel of interest in the grayscale image is greater (darker) than the minimum value of the first local region 301 plus the parameter den_th.

[0110] The threshold value of (ii) may be, for example, 10, but may also be less than 10 or more than 10. Condition 2 (ii) is a condition for noise removal and is not essential in this embodiment.

[0111] Furthermore, the parameter den_th in (iii) is variable depending on the result of the white background determination. For example, the parameter den_th is set to, for example, 50 (dark gradation value) if the second local region 302 is white background 1, and to a smaller value, for example, 20 (light gradation value) if the second local region 302 is white background 2 or is not white background. 50 and 20 are merely examples. The larger (i.e., darker) the parameter den_th, the less likely it is to meet condition (iii), and the smaller (lighter) the parameter den_th, the more likely it is to meet condition (iii). For example, if the second local region 302 is determined to be white background 1, it becomes less likely to meet condition 2, and the process proceeds to the determination of condition 3. If the second local region 302 is determined to be white background 2 or is not white background, it becomes more likely to meet condition 2, and is subject to binarization under condition 2. Note that white background 2 does not satisfy condition 2 in (i), and therefore is more likely to meet condition 2 if it is not a white background.

[0112] Condition 2 (i) requires that the pixel of interest be relatively light, but condition 2 (ii) also excludes pixels of interest that are not simply light and should not be converted to black. Condition 2 (ii) is a condition for preventing noise on a white background that is not desired to be converted to black (the absolute value of the difference between the average and minimum values ​​does not exceed a threshold value (e.g., 10)). Examples of such noise include edges with relatively gradual changes in density. For example, the traces of a line in a scanned paper document would be equivalent (excluded by condition 2). Condition 2 (iii) is a condition for preventing pixels with a gradation value of 1 that encroach on the inside of the white-out character in Figure 12(b) from meeting condition 2, and for converting the pixels with a gradation value of 1 to 0 using binarization method 4 (preventing the white-out character from being blurred).

[0113] In other words, condition 2 is a condition for converting edges that are noticeable to a certain extent but not quite noise (edges with a certain intensity or greater) to black. Therefore, condition 2 makes it possible to convert edges with relatively gradual density changes to 0 (noise removal), and to convert edges that are not noticeable to a certain extent but are not noise to 1.

[0114] In addition, by applying a variable parameter according to the result of the blank background determination in condition 2 (iii), the output unit 35 can apply a variable parameter according to the result of the blank background determination in condition 2 (iii). Even if the gradation values ​​of both the light character on the white background and the character on the blank background satisfy condition 2 (i), the edge of the light character on the white background (FIG. 12(a)) is likely to meet condition 2 and can be relatively easily converted to "1". On the other hand, the edge of the character on the blank background 1 (FIG. 12(b)) is unlikely to meet condition 2 (nor does it meet condition 3), so it can be relatively easily converted to "0" by binarization method 4.

[0115] If condition 2 is met, the output unit 35 binarizes the gray image using binarization method 2. Binarization method 2 is adaptive binarization. Adaptive binarization binarizes the pixel of interest using an offset value given by adding the average value to a constant as a threshold. Note that the threshold value for binarization method 2 is a relatively small value (a light gradation value) compared to condition 3 described below, in order to convert non-noise edges to black (for example, to improve the reproducibility of light characters on a white background).

[0116] <Regarding condition 3> Condition 3 is (i) In the four-valued image, the gradation value of the pixel of interest is 2 (an example of a second predetermined value), and (ii) The result of the white background determination is white background 2.

[0117] Condition 3 is a condition for, for example, binarizing pixels that are originally white on a gray background (e.g., white background) to 0. Condition 3 is also a condition for binarizing pixels with a gradation value of 2 that have eroded the inside of the outlined characters in Figure 12(c) to 0 (to prevent the outlined characters from being crushed).

[0118] If condition 3 is met, the output unit 35 binarizes the gray image using binarization method 3. Binarization method 3 is also adaptive binarization. The threshold value applied in binarization method 3 is set to a larger value than in binarization method 2 in order to improve the reproducibility of outline characters. As described above, the offset value applied in binarization method 2 is set to a relatively small value in order to improve the reproducibility of light characters on a white background (which we would like to convert to black).

[0119] For example, the pixel with gradation value 2 that encroaches on the inside of the white-out character in Figure 12(c) satisfies condition 3, so it is converted to white by adaptive binarization using a relatively large offset value. On the other hand, the pixel with gradation value 2 that occupies the background in Figure 12(b) does not satisfy condition 3 (it is not white-out background 2) and does not satisfy condition 2 either (it is white-out background 1 but not gradation value 1), so it is converted to black by the following binarization method 4.

[0120] <If conditions 1 to 3 are not met> If the pixel of interest does not meet any of conditions 1 to 3, the output unit 35 binarizes it using binarization method 4. An example of a case where the pixel does not meet any of conditions 1 to 3 (i.e., it is not isolated point noise, it is not a fairly noticeable edge, and it is not a gray background) is when all the pixels in the first local region 301 in the four-valued image have the same gradation value.

[0121] In binarization method 4, if the gradation value of the pixel of interest in the quaternary image is 0 or 1, the binarization result is 0, and if it is 2 or 3, the binarization result is 1. This process corresponds to binarizing the pixel value of the pixel of interest to a value that is uniquely determined for the gradation value within the first local region 301. In other words, a pixel of interest that does not meet conditions 1 to 3 is forcibly converted to 1 or 0. Binarization method 4 generates a binarized image that is the same as binarizing a grayscale image using threshold value t1 calculated by discriminant analysis.

[0122] <Overall flow of binarization process> FIG. 15 is a flowchart illustrating the overall process flow for binarizing an input image by the image processing device 20.

[0123] First, the histogram creating unit 21 creates a frequency (histogram) for each gradation of the input image (S101).

[0124] Next, as shown in the flowchart of FIG. 7, the discriminant analysis unit 22 determines three thresholds t0, t1, and t2 (S102).

[0125] Next, the N-value conversion unit 15 converts the 256-level gray image into four-level values ​​using the threshold values ​​t0, t1, and t2 (S103).

[0126] Next, the binarization unit 16 acquires features for each first local region 301 of the input image (S104). The first local region 301 is, for example, a region of 7 × 7 pixels. The acquired features include the average value of the pixels in the first local region 301 calculated by the average value calculation unit 31, the minimum value of the first local region 301 calculated by the minimum value calculation unit 32, and the number of pixels for each gradation of 0, 1, 2, and 3 counted by the pixel number counting unit 33.

[0127] Next, the outline background determination unit 34 acquires features for each second local region 302 of the quaternary image (S105). The second local region 302 is, for example, a region of 51 × 7 pixels. The acquired features are the outline background determination results. Note that steps S104 and S105 can be performed in the reverse order.

[0128] Next, the binarization unit 16 generates a binarized image according to the flowchart of FIG. 13 (S106).

[0129] <Effects of binarization according to conditions 2 and 3> FIG. 16 shows an example of a quaternary valued image, and FIG. 17 shows a binarized image obtained by binarizing the quaternary valued image of FIG. 16 using existing technology. In the case of a quaternary valued image like that of FIG. 16, if binarization is performed using only discriminant analysis, an existing binarization technology (i.e., binarizing all pixels using threshold value t1), the binarized image shown in FIG. 17 will result. The binarization result of FIG. 17 has the following defects: The light part 130 in FIG. 16(a) is binarized to white pixels, and the character (the number "60") becomes unclear as shown in FIG. 17(a). In Figure 16(b), the entire character is light, so when it is binarized, the entire character becomes white pixels (the character is deleted), as shown in Figure 17(b). The background in Figure 16(c) is dark, so when it is binarized, the entire image becomes black pixels, as shown in Figure 17(c).

[0130] In contrast, Fig. 18 shows a binarized image obtained by binarizing the quaternary image of Fig. 16 using the binarization method of this embodiment. First, in Fig. 18(a), the light portion 130 of Fig. 16(a) is reproduced as a solid black, which is the effect of binarization according to condition 2. That is, the pixel of interest in the quaternary image of Fig. 16(a) with a gradation value of 1 (light portion 130) satisfies condition 2 because it is surrounded by black pixels. Because the threshold value of binarization method 2 is relatively low, the pixel of interest with a gradation value of 1 becomes 1 (black) in the binarization process.

[0131] Similarly, in Figure 18(b), the light text (gradation value 1) is reproduced in solid black due to the effect of binarization in accordance with condition 2. In the quaternary-valued image in Figure 16(b), the pixel of interest with a gradation value of 1 (the pixel corresponding to the number 60) is surrounded by white background, so conditions 2(i) to (iii) are satisfied. Additionally, since the quaternary-valued image in Figure 16(b) is determined not to have a white background, the parameter den_th takes a small value (for example, 20), satisfying condition (iii). Because the threshold value of binarization method 2 is relatively low, the pixel of interest with a gradation value of 1 becomes 1 (black) in the binarization process.

[0132] The reason why the characters in Figure 18(c) are reproduced without being blurred is due to the effect of binarization in accordance with condition 3. That is, the pixel of interest in the quaternary image in Figure 16(c) with a gradation value of 2 satisfies conditions 3(i) and 3(ii). That is, the quaternary image in Figure 16(c) is determined to be white background 2. Because the threshold value of binarization method 3 is relatively high, the pixel of interest with a gradation value of 2 becomes 0 (white) in the binarization process. The pixel of interest in the quaternary image with a gradation value of 3 (the background of the number 60) does not meet conditions 1 to 3, so it becomes black in binarization method 4.

[0133] In this embodiment, N-value conversion is explained as four-value conversion where N=4, but if N=8, for example, the number of conditions will increase and the number of conditional branches will also increase in Fig. 13. This will increase the possibility of making the conditions and the binarization process appropriate for the conditions more appropriate.

[0134] <Effect of binarization when conditions 1 to 3 are not met> With reference to FIG. 19, the effect of binarization method 4, which is performed when none of conditions 1 to 3 is met, will be described. FIG. 19(a) is an example of a quaternarized image. When the image processing device 20 applies adaptive binarization to the quaternarized image of FIG. 19(a), the binarization results shown in FIGS. 19(b) and 19(c) are obtained. Both FIGS. 19(b) and 19(c) are binarization results obtained by adaptive binarization, but the offset values ​​reflected in the thresholds are different. In either case, noise will appear, causing the background to become uneven, and white spaces will appear inside the characters. On the other hand, Fig. 19(d) shows the binarization result by binarization method 4 of this embodiment. In Fig. 19(d), the background is uniform and no white space occurs inside the characters. This is the effect of applying binarization equivalent to the discriminant analysis method when none of conditions 1 to 3 is met.

[0135] <Effect of white background detection> Next, the effect of the white background determination will be explained. Fig. 20 shows an example of a quaternary valued image. Fig. 20(a) is a quaternary valued image determined not to be a white background, Fig. 20(b) is a quaternary valued image determined to be a white background 1, and Fig. 20(c) is a quaternary valued image determined to be a white background 2.

[0136] First, it will be explained that there is no common threshold value that can be used to binarize the quaternary image of Figures 20(a) to 20(c) while maintaining legibility. Figure 21 shows a binarized image of the quaternary image of Figure 20 binarized with a threshold value that converts gradation value 0 to 0 (white) and gradation values ​​1 to 3 to 1 (black). Figure 22 shows a binarized image of the quaternary image of Figure 20 binarized with a threshold value that converts gradation values ​​0 to 1 to 0 (white) and gradation values ​​2 to 3 to 1 (black). Figure 23 shows a binarized image of the quaternary image of Figure 20 binarized with a threshold value that converts gradation values ​​0 to 2 to 0 (white) and gradation value 3 to 1 (black).

[0137] 21 to 23 show that there is no common threshold value that can convert light text on a white background as in FIG. 20(a), white text on a gray background as in FIG. 20(b), and white text on a black background as in FIG. 20(c) into a legible binarized image. For light text on a white background as in FIG. 20(a), the binarization result in FIG. 21(a) is desirable, while in FIGS. 22(a) and 23(a), the characters are broken up, significantly impairing legibility. For white text on a gray background as in FIG. 20(b), the binarization result in FIG. 22(b) is desirable, while in FIGS. 21(b) and 23(b), the white text is eroded by the background color, causing the characters to appear blurred, or conversely, the entire text is blown out, significantly impairing legibility. For white characters on a black background such as that shown in Figure 20(c), the binarization result shown in Figure 23(c) is desirable; in Figures 21(c) and 22(c), the white characters are eroded by the background color, causing the characters to become blurred and significantly impairing their legibility.

[0138] In contrast, Fig. 24 shows a binarized image obtained by binarizing the quaternary image of Fig. 20 using the binarization method of this embodiment. In Fig. 24(a), the light portion 211 of Fig. 20(a) with a gradation value of 1 is reproduced in black, and in Fig. 24(b), the light portion 212 of Fig. 20(b) with a gradation value of 1 is reproduced in white. This is the effect of binarization in accordance with condition 2. Under condition 2(iii), variable parameters are applied in accordance with the result of the white-out background determination, so the light portion 211 of Fig. 20(a) to which parameters that are likely to match condition 2 are applied is reproduced in black, even though it has the same gradation value of 1. In other words, because the threshold value for adaptive binarization under condition 2 is relatively small, the light portion 211 can be reproduced in black. On the other hand, the light part 212 in FIG. 20(b) has the same gradation value of 1, but is determined to be white background 1 and therefore is applied with parameter den_th, which is unlikely to meet condition 2. Therefore, it does not meet condition 2 and is not binarized by the adaptive binarization of condition 2. Since the light part 212 in FIG. 20(b) is determined to be white background 1, it does not meet condition 3(ii) either, and is therefore likely to be converted to "0" by binarization method 4.

[0139] Furthermore, in FIG. 24(b), the dark portion 213 with a gradation value of 2 in FIG. 20(b) is reproduced in black, and in FIG. 24(c), the dark portion 214 with a gradation value of 2 in FIG. 20(c) is reproduced in white. This is the effect of binarization in accordance with condition 3. First, the dark portion 213 with a gradation value of 2 in FIG. 20(b) satisfies condition 3(i), but does not satisfy condition 3(ii) because it is determined to be white background 1. Therefore, the dark portion 213 is converted to black using binarization method 4. On the other hand, the four-valued image in FIG. 20(c) (where the pixel of interest is the dark portion 214) is determined to be white background 2, so it satisfies conditions 3(i) and (ii). Because the adaptive binarization threshold in condition 3 is relatively large, the dark portion 214 can be reproduced in white. If condition 3 were not met, the dark portion 214 in Figure 20(c) would be treated as , and the dark portion 214 with gradation value 2 would be reproduced in black by binarization method 4. Therefore, condition 3 makes it possible to reproduce in white the outline characters in Figure 20(c), which correspond to the outline background 2.

[0140] <Major Effects> The binarization method of this embodiment can simultaneously reproduce light characters or characters on a dark background, reduce the occurrence of noise and white spaces, and prevent white characters from being blurred.

[0141] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0142] The configuration examples shown in Fig. 4 and the like are divided according to main functions to make it easier to understand the processing by the image processing device 20. The method of dividing the processing units and their names do not limit the present invention. The processing by the image processing device 20 can be divided into even more processing units depending on the processing content. Furthermore, the processing can be divided so that one processing unit includes even more processes.

[0143] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0144] <Aspect> [Aspect 1] a threshold calculation unit that calculates N-1 (M>N>2) first thresholds based on pixel values ​​of an input image with M gradations; an N-value conversion unit that compares pixel values ​​of the input image with N-1 first thresholds to generate an N-value image from the input image; a binarization unit that generates a binarized image by binarizing the pixel of interest based on a pixel value for each first local region of the input image including the pixel of interest and a gradation value for each second local region of the N-ary image, The image processing device is characterized in that the second local region includes the first local region and is larger than the first local region. [Aspect 2] In the N-ary image, if a count result of pixel values ​​that are the same as the gradation value of the pixel of interest in the first local region centered on the pixel of interest is equal to or less than a second threshold value, and if a first condition is satisfied that there are only two types of gradation values ​​in the second local region, 2. The image processing device according to aspect 1, wherein the binarization unit binarizes the pixel of interest according to another gradation value different from the gradation value of the pixel of interest. [Aspect 3] Whether the background color is a first white background in which the background color is darker than the character color based on the gradation value of the second local region, or other than that, or a white background determination unit that determines whether the background is a second white background having a background density darker than that of the first white background, or whether the background is something other than the second white background; The gradation value of the pixel of interest in the N-ary image is a first predetermined value, and If a second condition is satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a gradation value that is darker when the second local region is the first white background than when the second local region is not the first white background, the binarization unit binarizes the input image by adaptive binarization; 3. The image processing device according to aspect 1 or 2. [Aspect 4] Whether the background color is a first white background in which the background color is darker than the character color based on the gradation value of the second local region, or other than that, or a white background determination unit that determines whether the background is a second white background having a background density darker than that of the first white background, or whether the background is something other than the second white background; If a third condition is satisfied that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background, 3. The image processing device according to aspect 1 or 2, wherein the binarization unit binarizes the input image by adaptive binarization. [Aspect 5] Whether the background color is a first white background in which the background color is darker than the character color based on the gradation value of the second local region, or other than that, or a white background determination unit that determines whether the background is a second white background having a background density darker than that of the first white background, or whether the background is something other than the second white background; The gradation value of the pixel of interest in the N-ary image is a first predetermined value, and If a second condition is satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a gradation value that is darker when the second local region is the first white background than when the second local region is not the first white background, the binarization unit binarizes the input image by adaptive binarization; If a third condition is satisfied that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background, the binarization unit binarizes the input image by adaptive binarization; the threshold value for adaptive binarization when the second condition is satisfied is a lighter gradation value than the threshold value for adaptive binarization when the third condition is satisfied; 3. The image processing device according to aspect 1 or 2. [Aspect 6] Whether the background color is a first white background in which the background color is darker than the character color based on the gradation value of the second local region, or other than that, or a white background determination unit that determines whether the background is a second white background having a background density darker than that of the first white background, or whether the background is something other than the second white background; In the N-ary image, the count result of pixel values ​​that are the same as the gradation value of the pixel of interest within the first local region centered on the pixel of interest is equal to or less than a second threshold value, and the first condition that there are only two types of gradation values ​​within the second local region is not satisfied; The gradation value of the pixel of interest in the N-ary image is a first predetermined value, and a second condition is not satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a darker gradation value when the second local region is the first white background than when the second local region is not the first white background; If the third condition that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background is not satisfied, 2. The image processing device according to aspect 1, wherein the binarization unit binarizes the pixel of interest of the N-ary image using a threshold value that is uniquely determined for the gradation value within the local region. [Aspect 7] the white background determination unit counts pixels having values ​​from 0 to N-1 in a third local region in the second local region; determining whether or not there is a cluster of white pixels or a cluster of black pixels in the third local region based on the counting result; Based on the number of the clusters of white pixels and the clusters of black pixels in the second local region, the second local region is determined to be a first white background in which the background color is darker than the character color, or is otherwise, or 7. The image processing device according to any one of aspects 3 to 6, wherein the image processing device determines whether the background is a second blank background having a higher background density than the first blank background, or whether the background is something other than the first blank background. [Aspect 8] the input image includes white-out characters, 8. The image processing device according to any one of aspects 1 to 7, wherein the second local region includes at least one of the outline characters. [Aspect 9] 9. The image processing device according to any one of aspects 1 to 8, wherein the threshold calculation unit creates a histogram of gradation values ​​and detects a value in the vicinity of a minimum value of the histogram as the first threshold. [Aspect 10] 10. The image processing device according to aspect 9, wherein the threshold calculation unit calculates the first thresholds by changing a reference range of the histogram and applying a discriminant analysis method. [Aspect 11] Whether the background color is a first white background in which the background color is darker than the character color based on the gradation value of the second local region, or other than that, or a white background determination unit that determines whether the background is a second white background having a background density darker than that of the first white background, or whether the background is something other than the second white background; In the N-ary image, if a count result of pixel values ​​that are the same as the gradation value of the pixel of interest in the first local region centered on the pixel of interest is equal to or less than a second threshold value, and if a first condition is satisfied that there are only two types of gradation values ​​in the second local region, the binarization unit binarizes the pixel of interest according to another gradation value different from the gradation value of the pixel of interest; The first condition is not satisfied, the gradation value of the pixel of interest in the N-ary image is a first predetermined value, and If a second condition is satisfied, that is, the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a gradation value that is darker when the second local region is the first white background than when the second local region is not the first white background, the binarization unit binarizes the input image by adaptive binarization; If the second condition is not satisfied, and the gradation value of the pixel of interest in the N-ary image is a second predetermined value, and the second local region is the second white background, a third condition is satisfied: the binarization unit binarizes the input image by adaptive binarization using a threshold value that is darker than the threshold value used for adaptive binarization when the second condition is satisfied; If the first condition, the second condition, and the third condition are not satisfied, 2. The image processing device according to aspect 1, wherein the binarization unit binarizes the pixel of interest of the N-ary image using a threshold value that is uniquely determined for the gradation value within the local region. [Explanation of symbols]

[0145] 11 Smoothing section 12 Gray processing section 13 Multi-value image processing section 14 Threshold calculation unit 15 N-value conversion section 16 Binarization section 34 White background detection section [Prior art documents] [Patent documents]

[0146] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-072858

Claims

1. a threshold calculation unit that calculates N-1 (M>N>2) first thresholds based on pixel values ​​of an input image with M gradations; an N-value conversion unit that compares pixel values ​​of the input image with N-1 first thresholds to generate an N-value image from the input image; a binarization unit that generates a binarized image by binarizing the pixel of interest based on a pixel value for each first local region of the input image including the pixel of interest and a gradation value for each second local region of the N-ary image, The image processing device according to claim 1, wherein the second local region includes the first local region and is larger than the first local region.

2. In the N-ary image, when a count result of pixel values ​​that are the same as the gradation value of the pixel of interest in the first local region centered on the pixel of interest is equal to or less than a second threshold value, and when a first condition that only two types of gradation values ​​exist in the second local region is satisfied, The image processing device according to claim 1 , wherein the binarization unit binarizes the pixel of interest according to another gradation value different from the gradation value of the pixel of interest.

3. Whether the background is a first white background in which the background color is darker than the character color, or whether the background is a first white background in which the background color is darker than the character color, based on the gradation value of the second local region, or a white background determination unit that determines whether the background is a second white background having a background density darker than the first white background, or whether the background is something other than the second white background; The gradation value of the pixel of interest in the N-ary image is a first predetermined value, and If a second condition is satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a gradation value that is darker when the second local region is the first white background than when the second local region is not the first white background, the binarization unit binarizes the input image by adaptive binarization; 3. The image processing device according to claim 1 or 2.

4. Whether the background is a first white background in which the background color is darker than the character color, or whether the background is a first white background in which the background color is darker than the character color, based on the gradation value of the second local region, or a white background determination unit that determines whether the background is a second white background having a background density darker than the first white background, or whether the background is something other than the second white background; If a third condition is satisfied that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background, The image processing device according to claim 1 , wherein the binarization unit binarizes the input image by adaptive binarization.

5. Whether the background is a first white background in which the background color is darker than the character color, or whether the background is a first white background in which the background color is darker than the character color, based on the gradation value of the second local region, or a white background determination unit that determines whether the background is a second white background having a background density darker than the first white background, or whether the background is something other than the second white background; The gradation value of the pixel of interest in the N-ary image is a first predetermined value, and If a second condition is satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a gradation value that is darker when the second local region is the first white background than when the second local region is not the first white background, the binarization unit binarizes the input image by adaptive binarization; If a third condition is satisfied that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background, the binarization unit binarizes the input image by adaptive binarization; the threshold value for adaptive binarization when the second condition is satisfied is a lighter gradation value than the threshold value for adaptive binarization when the third condition is satisfied; 3. The image processing device according to claim 1 or 2.

6. Whether the background is a first white background in which the background color is darker than the character color, or whether the background is a first white background in which the background color is darker than the character color, based on the gradation value of the second local region, or a white background determination unit that determines whether the background is a second white background having a background density darker than the first white background, or whether the background is something other than the second white background; In the N-ary image, a count result of pixel values ​​that are the same as the gradation value of the pixel of interest within the first local region centered on the pixel of interest is equal to or less than a second threshold value, and the first condition that there are only two types of gradation values ​​within the second local region is not satisfied; The gradation value of the pixel of interest in the N-ary image is a first predetermined value, and a second condition is not satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a darker gradation value when the second local region is the first white background than when the second local region is not the first white background; If the third condition that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background is not satisfied, 2. The image processing apparatus according to claim 1, wherein the binarization unit binarizes the pixel of interest of the N-ary image with a threshold value that is uniquely determined for the gradation value within the local region.

7. the white background determination unit counts pixels having values ​​from 0 to N-1 in a third local region in the second local region; determining whether or not there is a cluster of white pixels or a cluster of black pixels in the third local region based on the counting result; Based on the number of the clusters of white pixels and the number of the clusters of black pixels in the second local region, the second local region is determined to be a first blank background in which the background color is darker than the character color, or is otherwise, or The image processing device according to claim 3 , wherein the image processing device determines whether the background is a second blank background having a density darker than the first blank background, or whether the background is something other than the first blank background.

8. the input image includes white-out characters, The image processing device according to claim 1 , wherein the second local region includes at least one of the outline characters.

9. 2. The image processing apparatus according to claim 1, wherein the threshold calculation unit creates a histogram of gradation values ​​and detects a value near a minimum value of the histogram as the first threshold value.

10. The image processing device according to claim 9 , wherein the threshold calculation unit calculates a plurality of first thresholds by applying a discriminant analysis method while changing a reference range of the histogram.

11. Whether the background is a first white background in which the background color is darker than the character color, or whether the background is a first white background in which the background color is darker than the character color, based on the gradation value of the second local region, or a white background determination unit that determines whether the background is a second white background having a background density darker than the first white background, or whether the background is something other than the second white background; In the N-ary image, when a count result of pixel values ​​that are the same as the gradation value of the pixel of interest in the first local region centered on the pixel of interest is equal to or less than a second threshold value, and when a first condition that only two types of gradation values ​​exist in the second local region is satisfied, the binarization unit binarizes the pixel of interest according to another gradation value different from the gradation value of the pixel of interest; The first condition is not satisfied, the gradation value of the pixel of interest in the N-ary image is a first predetermined value, and If a second condition is satisfied that the gradation value of the pixel of interest in the input image is darker than a variable threshold, and the variable threshold is a gradation value that is darker when the second local region is the first white background than when the second local region is not the first white background, the binarization unit binarizes the input image by adaptive binarization; If the second condition is not satisfied, and a third condition is satisfied that the gradation value of the pixel of interest in the N-ary image is a second predetermined value and the second local region is the second white background, the binarization unit binarizes the input image by adaptive binarization using a threshold value that is darker than the threshold value used for adaptive binarization when the second condition is satisfied; If the first condition, the second condition, and the third condition are not satisfied, 2. The image processing apparatus according to claim 1, wherein the binarization unit binarizes the pixel of interest of the N-ary image with a threshold value that is uniquely determined for the gradation value within the local region.

12. A binarization method performed by an image processing device, comprising: A process of calculating N-1 (M>N>2) first threshold values ​​based on pixel values ​​of an input image with M gradations; a process of comparing pixel values ​​of the input image with N-1 first thresholds to generate an N-ary image from the input image; generating a binarized image by binarizing the pixel of interest based on a pixel value for each first local region of the input image including the pixel of interest and a gradation value for each second local region of the N-ary image; A binarization method, wherein the second local region includes the first local region and is larger than the first local region.

13. An information processing device a threshold calculation unit that calculates N-1 (M>N>2) first thresholds based on pixel values ​​of an input image with M gradations; an N-value conversion unit that compares pixel values ​​of the input image with N-1 first thresholds to generate an N-value image from the input image; a binarization unit that generates a binarized image by binarizing the pixel of interest based on a pixel value for each first local region of the input image including the pixel of interest and a gradation value for each second local region of the N-ary image, The second local region includes the first local region and is larger than the first local region.

Images