Image processing apparatus and control method for image processing apparatus

Abstract

To provide an image processing apparatus capable of highly accurately identifying an original document area.SOLUTION: An image processing apparatus including a control unit is provided The control unit obtains an image obtained by reading an original document, extracts feature points from the image, identifies a plurality of feature points aligned in a predetermined direction in the image as feature points corresponding to edges that define a range of the original document, sets an area outside the original document range based on the identified feature points, and removes the feature points in the set area outside the original document range.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an image processing device and the like. [Background technology]

[0002] In general, in image processing devices such as flatbed scanners and multifunction printers (MFPs), an image is generated by an image sensor capturing an image of a document placed on the reading surface of a document platen and covered with a document cover. At this time, the image sensor outputs an image that includes not only the document but also the inner surface of the document cover as the background. Therefore, in order to generate an image of the document, it is necessary to identify the range of the document from the output of the image sensor.

[0003] In relation to the present disclosure, Patent Document 1 discloses extracting edges of a document by performing binarization processing having first and second threshold judgment values. Patent Document 2 discloses sampling the edge detection result at a first interval in a first direction to extract a first boundary point group, sampling at a second interval in the first direction to extract a second boundary point group, determining a noise removal condition based on the first boundary point group, and removing boundary points that satisfy the noise removal condition from the second boundary point group as noise. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-044635 [Patent Document 2] JP 2020-149148 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present disclosure is to provide an image processing device capable of identifying an original region with high accuracy. [Means for solving the problem]

[0006] The present disclosure provides an image processing device that includes a control unit that acquires an image obtained by reading a document, extracts feature points from the image, identifies a plurality of feature points aligned in a predetermined direction in the image as feature points corresponding to edges that define a range of the document, sets an area outside the document range based on the identified feature points, and removes the feature points outside the set document range.

[0007] The present disclosure also provides a control method for an image processing device, which includes obtaining an image obtained by reading an image, extracting feature points from the image, identifying a plurality of feature points aligned in a predetermined direction in the image as feature points corresponding to edges that define a range of the original document, setting an area outside the original document range based on the identified feature points, and removing the feature points outside the set original document range. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an image processing device capable of identifying an original region with high accuracy. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram showing an example of a document imaged by an image processing apparatus; [Diagram 2] FIG. 2(A) is an example of an image generated by imaging an original document against a gray background and having feature points extracted, and FIG. 2(B) is an example of an image generated by imaging an original document against a white background and having feature points extracted. [Diagram 3] 1 is a block diagram of an image processing device according to a first embodiment of the present disclosure. [Figure 4] 1A and 1B are diagrams illustrating an example of a document imaged by the image processing apparatus according to the first embodiment. [Diagram 5]2 is a diagram for explaining an original image generated by the image processing apparatus according to the first embodiment by capturing an image of a document, and feature points extracted by executing a feature point extraction process on the original image. FIG. [Figure 6] 5A to 5C are diagrams for explaining how the image processing apparatus according to the first embodiment specifies an outside-document area based on a side of an original image in the sub-scanning direction. [Figure 7] 4A to 4C are diagrams for explaining an original image after the image processing apparatus according to the first embodiment deletes pixels in an area outside the document. [Figure 8] 5A to 5C are diagrams for explaining how the image processing apparatus according to the first embodiment specifies an outside-document area based on a side in the sub-scanning direction of a tilted document image. [Figure 9] 4 is a flowchart for explaining the operation of the image processing device according to the example of the first embodiment. [Figure 10] 10 is a flowchart for explaining removal of unnecessary edge information from among operations of the image processing device according to the example of the first embodiment. [Figure 11] FIG. 11A is an example of an original image, and FIG. 11B is an example after tone correction. [Figure 12] FIG. 12(A) is an example after scaling, and FIG. 12(B) is an example after filtering of horizontal edges. [Figure 13] FIG. 13(A) is an example after vertical edge filtering has been performed, and FIG. 13(B) is an example after horizontal edge detection. [Figure 14] FIG. 14(A) is an example after vertical edge detection, and FIG. 14(B) is an example after horizontal noise removal. [Figure 15] FIG. 15(A) is an example after noise removal in the vertical direction, and FIG. 15(B) is an example after the outer edge in the horizontal direction has been extracted. [Figure 16] FIG. 16A shows an example after the vertical outer edge has been extracted, and FIG. 16B shows an example after the horizontal and vertical outer edges have been combined. [Figure 17]FIG. 11 is a diagram for explaining the shape of a document used in the second embodiment of the present disclosure. [Figure 18] Figure 18(A) is a diagram for explaining the area outside the original document range determined using the method of the first embodiment for an original image generated by capturing an image of the original document in Figure 17, and Figure 18(B) is a diagram for explaining the state after removal of unnecessary edge information in the first embodiment has been performed on the original image in Figure 18(A). [Figure 19] Figure 19(A) is a diagram for explaining the area outside the original range determined using the method of the second embodiment for an original image generated by capturing an image of the original in Figure 17, and Figure 19(B) is a diagram for explaining the state after removal of unnecessary edge information in the second embodiment has been performed on the original image in Figure 19(A). [Figure 20] FIG. 20(A) shows an example of an original image for which tilt is detected in step S15 of FIG. 9 and feature points extracted from the original image, FIG. 20(B) shows the original image and feature points of FIG. 20(A) after tilt correction of edge information in step S17, and FIG. 20(C) is an example of a histogram of vertical edges generated based on the feature points of FIG. 20(B) after tilt correction of edge information. [Figure 21] FIG. 21(A) shows an example of an original image for which tilt detection is performed in step S15 of FIG. 9 and feature points extracted from the original image, FIG. 21(B) is a diagram for explaining frequency counting performed without tilt correction for the feature points of FIG. 21(A), and FIG. 21(C) is an example of a histogram of vertical edges generated by counting the frequencies as in FIG. 21(B). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Prior to the description of the embodiment, the relationship between the difference in background color and the feature points of the detected edges will be described. FIG. 1 is a diagram showing an original 1, which is an example of an original imaged by an image processing device. Here, the image processing device is provided with a line image sensor in which image pickup elements are arranged in a line along the main scanning direction. Hereinafter, in this specification, a feature point is a pixel extracted from an image including the original 1 as the outer periphery side of the original (or the edge of the original), and is expressed by the coordinates of the feature point. Also, the feature point extraction process is a process of extracting feature points from an image. For example, the feature point extraction process extracts pixels with the maximum and minimum X values ​​as feature points on each straight line that passes through an image and is parallel to the X axis, with the sub-scanning direction as the X axis and the main scanning direction as the Y axis, and outputs the coordinates of the feature points. Also, the feature point extraction process extracts pixels with the maximum and minimum Y values ​​as feature points on each straight line that passes through an image and is parallel to the Y axis, and outputs the coordinates of the feature points.

[0011] Original 1 is a rectangular piece of paper having sides 3, 5, 7, and 9. Front page 11 is printed on original 1. Also, stains 13, 15, and 17 are attached to original 1. As shown in the figure, original 1 is arranged with its long side aligned along the sub-scanning direction (direction in which the original is transported) of the image processing device, and its short side aligned along the main scanning direction (direction perpendicular to the direction in which the original is transported) of the image processing device.

[0012] FIG. 2(A) is an example of an image generated by imaging an original document using a gray background. Original image 20 is made up of gray background 20a and original document image 21. Gray background 20a is a background image and has stain 20b. Stains 20b correspond to stains attached to gray background 20a or the optical system that generated original image 20, and are unrelated to original document 1. Original document image 21 is an image corresponding to original document 1. Original document image 21 has sides 23, 25, 27, 29, front 31, stains 33, 35, 37, which correspond respectively to sides 3, 5, 7, 9, front 11, stains 13, 15, 17 of original document 1.

[0013] Now, consider performing a feature point extraction process on the original image 20. In the original image 20, the gray background 20a and the original image 21, which is an image of the original 1 drawn on a white piece of paper, have different colors, so the boundary between the gray background 20a and the white piece of paper is extracted as a strong feature point. By performing the feature point extraction process by extracting only such strong feature points and not extracting weak feature points, weak feature points derived from stains and the like are not extracted. As a result, when feature points are extracted from the original image 20, sides 3, 5, 7, and 9 are extracted as sides 23, 25, 27, and 29 in almost the same form. Therefore, sides 23, 25, 27, and 29 can be easily detected as the boundary between the gray background 20a and the original image 21. At this time, the stain 20b in the gray background 20a is not extracted or is extracted as a weak feature point, so it has almost no effect on the detection of sides 23, 25, 27, and 29.

[0014] In this way, when generating an original image of a white piece of paper, the use of a gray background makes it easy to extract feature points corresponding to the edges of the original, so the gray background is suitable for identifying the original area. However, when performing image generation that does not require the identification of the original area, the gray background has the following problems. Some pieces of white paper have punched holes, some have holes like loose-leaf paper, and some have rounded corners, or so-called rounded corners. When an image of this type of original white piece of paper is generated using a gray background, an original image is generated that includes the gray background exposed from the holes or the rounded corners. This exposed gray background is unrelated to the contents of the original drawn on the white piece of paper, and it is preferable that it is not included in the original image. As a method for avoiding this problem, it is possible to use a gray background when it is necessary to identify the original area when generating an original image, and to use a white background when it is not necessary to identify the original area. However, this method requires a configuration that can switch between the gray background and the white background, which leads to a problem of complex device configuration and increased costs. In view of these circumstances, an object of the present disclosure is to provide image processing capable of identifying an original area with high accuracy even when a white background is used.

[0015] FIG. 2(B) is an example of an image generated by capturing an image of an original using a white background. Original image 40 is made up of white background 40a and original image 41. White background 40a is a background image and has stain 40b. Stains 40b correspond to stains attached to white background 40a and the optical system used to generate original image 40. Original image 41 is an image corresponding to original 1. Original image 41 has pixel groups 43, 45, 47, 49, front 51, stains 53, 55, 57, which correspond respectively to sides 3, 5, 7, 9, front 11, stains 13, 15, 17 of original 1.

[0016] In the original image 40, the document 1 is drawn on a white piece of paper, so when detecting the boundary between the document image 41, which is the image, and the white background 40a, i.e., the feature points corresponding to the edges of the document 1, it is necessary to extract not only strong feature points but also weak feature points. The feature points extracted in this way include a mixture of feature points corresponding to edges and feature points corresponding to stains. Feature points corresponding to such stains, especially feature points corresponding to stains outside the document such as stain 40b, can cause erroneous detection of the edges of the document 1.

[0017] When feature point extraction processing is performed on the original image 41, the pixels corresponding to sides 5 and 9 are extracted as pixel groups 45 and 49 almost as they are. On the other hand, pixels corresponding to sides 3 and 7 are mixed, with some extracted as feature points and others not. The pixels correctly extracted as pixels corresponding to side 3 are pixel group 43, and the pixels incorrectly extracted are pixel group 51a, which is the upper frame line of table 51 in the figure, pixel group 51b, which is part of the frame line of table 51, stain 53, the upper half of stains 55 and 57, and stain 40b. Similarly, the pixels correctly extracted as pixels corresponding to side 7 are pixel group 47, and the pixels incorrectly extracted are pixel group 51c, which is the lower frame line of table 51 in the figure, and the lower half of stains 55 and 57.

[0018] In this way, when an image of an original is formed using a white background and a feature point extraction process is performed, the accuracy of extracting feature points varies depending on the direction of the side to be extracted. A general image forming apparatus (multifunction machine) including an image processing apparatus 60 described later includes a line image sensor and a group of light source elements. The line image sensor includes a plurality of image pickup elements arranged along the main scanning direction (or a first straight line). The group of light source elements includes a plurality of light source elements arranged along the sub-scanning direction (or a second straight line perpendicular to the first straight line). For this reason, light is irradiated obliquely onto the sides 5 and 9 in the main scanning direction (the direction along the line image sensor). It tends to be easy to accurately extract corresponding feature points from the images of the sides 5 and 9 generated in this way. On the other hand, light is irradiated almost directly above the sides 3 and 7 in the sub-scanning direction. It tends to be difficult to extract corresponding feature points from the images of the sides 3 and 7 generated in this way. In addition, with respect to the sides 3 and 7 in the sub-scanning direction, images other than the sides 3 and 7 tend to be erroneously extracted as feature points corresponding to the sides 3 and 7. Images that are erroneously detected as feature points include those depicted within the document, such as part of the front 51 (pixel groups 51a, 51b), stains within the document (53, 55, 57), and stain 40b outside the document.

[0019] In the present disclosure, feature points that have been erroneously extracted from outside the document range are removed based on feature points corresponding to edges in the main scanning direction, which enable feature points to be extracted with greater accuracy, thereby improving the detection accuracy of the document range that is subsequently detected based on the feature points.

[0020] [1. First embodiment] 3 is a block diagram of an image processing device according to the first embodiment of the present disclosure. The image processing device 60 is, for example, a multifunction device, a multifunction printer (MFP), or a scanner. The image processing device 60 includes a display unit 61, an operation unit 63, an image input unit 65, an image forming unit 67, a communication unit 69, a connection unit 71, a storage unit 73, and a control unit 75.

[0021] The display unit 61 displays images and characters. For example, it is configured with a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) panel. The display unit 61 may be a standalone display device, or may further include an externally connected display device.

[0022] The operation unit 63 accepts operation input from a user. For example, the operation unit 63 is configured with hardware keys and software keys. The operation unit 63 also includes task keys for executing tasks such as fax transmission and image reading, and a stop key for canceling an operation. The operation unit 63 may also include physical operation keys such as task keys, a stop key, a power key, and a power saving key.

[0023] The image input unit 65 reads an image (original) and outputs it as image data. The image input unit 65 is configured with a general scanner (image input device). The scanner may be a flatbed scanner or a sheet-through scanner. The image input unit 65 may input image data from an external storage medium such as a USB memory, or may receive an image via a network. The image input unit 65 includes a document table 65a, a document cover 65b, and a document transport unit 65c. The image input unit 65 includes a line image sensor and a light source element group. The line image sensor includes a plurality of imaging elements arranged along the main scanning direction (or a first straight line). The light source element group includes a plurality of light source elements arranged along the sub-scanning direction (or a second straight line perpendicular to the first straight line).

[0024] The document table 65a is a table portion on which a document is placed when imaging. The document table 65a is provided with a glass plate (not shown) on which a document is placed, and a line image sensor (not shown) is provided below the glass plate. The document is placed so that the surface to be read faces the glass plate. The line image sensor is a sensor in which multiple imaging elements are arranged in a straight line. The line image sensor is disposed along one side of the glass plate. The longitudinal direction of the line image sensor is the main scanning direction. The direction perpendicular to the longitudinal direction of the line image sensor is the sub-scanning direction.

[0025] The document cover 65b is a cover that covers the glass plate and the document after the document is placed on the document table 65a. The line image sensor captures an image of the document with the document cover 65b closed. Therefore, when capturing an image of the document, the document cover 65b becomes the background of the document. Therefore, the document cover 65b is also called the background part. The background part of the document cover 65b is white.

[0026] The document transport unit 65c is a so-called document feeder, and transports the document between the document cover 65b and the glass plate of the document table 65a with the document cover 65b closed before the line image sensor captures the document. The document transport unit 65c also discharges the document after imaging from between the document cover 65b and the glass plate of the document table 65a. The document transport unit 65c transports the document along the sub-scanning direction of the line image sensor. The document transport unit 65c may have other configurations. For example, when the scanner of the image input unit 65 is a sheet-through scanner, the document transport unit may transport the paper to be scanned one after another using a transport roller or the like, and have the paper read by the line image sensor.

[0027] Image forming unit 67 forms (prints) an image on a medium such as copy paper based on image data. The printing method of image forming unit 67 is arbitrary, and may be, for example, any of an inkjet printer, a laser printer, a thermal transfer printer, etc. Image forming unit 67 may be a monochrome printer or a color printer. Image forming unit 67 may include a paper feed mechanism that supplies the medium, a transport mechanism that transports the medium, a sorter mechanism that sorts the medium after the image is formed, etc.

[0028] The communication unit 69 connects to a network. For example, the communication unit 69 is configured with an interface that can connect to a wired LAN (Local Area Network), a wireless LAN, or an LTE (Long Term Evolution) network. When the communication unit 69 is connected to a network, it is connected to other devices or an external network. In addition, the communication unit 69 may be an interface that performs short-range wireless communication such as NFC (Near field communication) or Bluetooth (registered trademark).

[0029] The connection unit 71 connects the image processing device 60 to other devices. For example, the connection unit 71 is a USB interface to which a USB memory or the like is connected. Furthermore, the connection unit 71 may be an interface such as HDMI (registered trademark) other than the USB interface.

[0030] The storage unit 73 stores various programs and various data necessary for the operation of the image processing device 60. The storage unit 73 includes a recording device capable of temporary storage, such as a dynamic random access memory (DRAM), and a non-temporary recording device, such as a solid state drive (SSD) composed of a semiconductor memory and a hard disk drive (HDD) composed of a magnetic disk. Although the storage unit 73 is configured as one unit for convenience of explanation, it may be configured as separate devices for each purpose, such as an area used for executing programs (main storage area), an area for saving programs and data (auxiliary storage area), an area used for caching, etc.

[0031] The control unit 75 controls the entire image processing device 60. The control unit 75 is composed of one or more control devices and control circuits, and is composed of, for example, a CPU (Central Processing Unit) and a SoC (System on a Chip). The control unit 75 can realize each function by reading out a program stored in the storage unit 73 and executing the process.

[0032] 4 is a diagram showing an example of a document imaged by the image processing device 60 according to the first embodiment. The document 80 is a rectangular medium (e.g., A-size or B-size copy paper, etc.) on which document contents such as characters, symbols, figures, images, etc. are drawn, and in this case, it is a business card. The document 80 has sides 81, 83, 85, and 87. The image processing device 60 creates an image with the longitudinal direction 89 as the sub-scanning direction (transport direction) and the lateral direction 91 as the main scanning direction.

[0033] FIG. 5 is a diagram for explaining an original image generated by the image processing device 60 according to the first embodiment by capturing an image of an original 80, and feature points extracted by executing a feature point extraction process on the original image. The original image 100 is composed of a white background 100a and an original image 100b. The white background 100a corresponds to the background of the original cover 65b. The original image 100b is an image corresponding to the original 80, and has edges 101, 103, 105, and 107 that correspond to the sides 81, 83, 85, and 87 of the original 80, respectively. The feature point group 111 is a set of feature points extracted from the edge 101. The feature point group 113 is a set of feature points extracted from the edge 103. The feature point group 115 is a set of feature points extracted from the edge 105. The feature point group 117 is a set of feature points extracted from the edge 107. The feature point group 119 is a collection of feature points that are erroneously extracted due to noise (e.g., white background 100a or dirt on the optical system of the image input unit 65) that occurs when imaging using the image input unit 65, and does not correspond to the original document 80.

[0034] For convenience of drawing, the feature points 111, 113, 115, and 117 are drawn outside the edges 101, 103, 105, and 107, respectively, but in reality they overlap approximately one side of the periphery of the edges 101, 103, 105, and 107.

[0035] As described above, feature points corresponding to sides along the main scanning direction are extracted with higher accuracy than feature points corresponding to sides along the sub-scanning direction. Therefore, pixels corresponding to sides along the main scanning direction are extracted as feature point groups 113 and 117 arranged in a continuous straight line. On the other hand, pixels corresponding to sides along the sub-scanning direction are not extracted as a continuous straight line. Feature point groups 111 and 115 extracted from the edges 101 and 105 are both extracted as a set of discontinuous points. In particular, feature point group 111 is not entirely on a single straight line, and some of it is extracted as a position deviating from the straight line. Also, feature point group 119 is erroneously extracted as a feature point corresponding to edge portion 101.

[0036] FIG. 6 is a diagram for explaining that the image processing device 60 according to the first embodiment specifies the outside of the document range based on the sides in the main scanning direction of the document image 100b. The control unit 75 specifies the outside of the document range 131, 133, 135, and 137, for example, as follows, using only the feature point groups 113 and 117 in the main scanning direction. That is, the feature point groups 113 and 117 are extracted as feature points corresponding to a part of the edge that defines the document range. The feature point groups 113 and 117 are feature points corresponding to two sides of the document 80 that are approximately parallel to the main scanning direction. The outside of the document range is specified based on the feature point groups 113 and 117. The sides 141, 143, 145, and 147 are sides that form the periphery of the original image 100, respectively.

[0037] The control unit 75 determines that the straight line connecting the end points 113R, 117R on the right side in the transport direction of the feature point groups 113, 117 and the rectangular area on the right side in the transport direction of this straight line in the original image 100 are outside the document range 131.

[0038] The control unit 75 determines the group of feature points 113 and a rectangular area in the original image 100 that is ahead of the group of feature points 113 in the transport direction as being outside the document range 133 .

[0039] The control unit 75 determines that the straight line connecting the left end points 113L and 117L of the feature point groups 113 and 117 in the transport direction and a rectangular area on the left side of this straight line in the transport direction in the original image 100 are outside the document range 135.

[0040] The control unit 75 determines the feature point group 117 and a rectangular area in the original image 100 that is behind the feature point group 117 in the transport direction as being outside the document range 137. In this manner, the outside document ranges 131, 133, 135, and 137 are determined. Please note that these determinations do not use the feature point groups 111 and 115 in the sub-scanning direction. When determining whether the document ranges 131, 133, 135, and 137 are outside the document range, only the feature point groups 113 and 117 in the main scanning direction, which have high detection accuracy, are used.

[0041] 7 is a diagram for explaining the original image after the image processing device 60 according to the first embodiment deletes pixels outside the document range 131, 133, 135, and 137. Compared with FIG. 6, the feature point groups 111, 115, and 119 have been deleted.

[0042] 7, the control unit 75 executes a general process for identifying the document range based on the original image 100 from which pixels outside the document range 131, 133, 135, and 137 have been deleted. According to the first embodiment, the pixels outside the document range are removed in advance as noise, and then the document range is detected, so that the document range can be detected more accurately without being affected by noise outside the document range.

[0043] Fig. 8 corresponds to Fig. 6 and is a diagram for explaining how the image processing device 60 according to the first embodiment identifies the outside of the document range based on the sides of the tilted document image 100b in the main scanning direction. For example, when a user places a document on the document table 65a, the document may be placed so that each side of the document is tilted with respect to the main scanning direction / sub-scanning direction, or the document image 100b may be generated with the document tilted due to a malfunction of the document transport unit 65c. Even in such cases, the outside of the document range is identified as follows.

[0044] Feature point groups 113 and 117 are extracted as feature points corresponding to a part of the edge that defines the range of the document. The feature point groups 113 and 117 are feature points corresponding to two sides of the document 80 that are approximately parallel to the main scanning direction. In the case of FIG. 8, the straight line formed by the feature point groups 113 and 117 is inclined obliquely. Even in such a case, the feature points corresponding to the two sides of the document 80 that are approximately parallel to the main scanning direction are extracted, for example, as follows. When the lower left corner of the original image 100 in the figure is set as the origin, the sub-scanning direction is the X-axis, and the main scanning direction is the Y-axis, a straight line that passes through point (a, 0) on the X-axis and has an inclination angle of θ is considered, and a straight line group consisting of straight lines with a and θ changed little by little is considered. On the front side of the sub-scanning direction (transport direction) of the original image 100, a straight line with the most matching feature points is extracted from the straight line group, and a feature point that matches the straight line is extracted as a feature point corresponding to the front side of the transport direction of the two sides of the document 80 that are approximately parallel to the main scanning direction. Similarly, of the two sides approximately parallel to the main scanning direction of the document 80, the feature point corresponding to the rear side in the transport direction is extracted. Then, the area other than the rectangular area having the end points 113L, 113R, 117L, and 117R of the feature point groups 113 and 117 as vertices is specified as outside the document range.

[0045] (Example) An example of the first embodiment will be described below. In this example, a general document range detection process is performed, and pixels outside the document range described in the first embodiment are deleted.

[0046] FIG. 9 is a flowchart for explaining the operation of the image processing device 60 according to the first embodiment. The control unit 75 reads an original document using the image input unit 65, generates an original image including a white background, and performs graying on the original image (step S1). Color images are converted to grayscale images. The conversion method does not need to be a specific conversion method, but for example, conversion is performed by adding RGB signals at a ratio of 3:6:1. In this case, the R (red) signal is Rx, the G (green) signal is Gx, and the B (blue) signal is Bx, and the grayed signal G is expressed as G=Rx0.3+Gx0.6+Bx0.1.

[0047] Next, the control unit 75 performs gradation correction on the original image grayed out in step S1 (step S3). Here, gradation correction is performed so that the edges of the document can be easily detected. The gradation correction curve used for the correction does not need to be a specific one, but it is necessary to use a gradation correction curve that does not reduce the number of gradations in the parts corresponding to the white background and the white pieces of paper of the document.

[0048] Next, the control unit 75 performs scaling on the original image that has been subjected to the gradation correction in step S3 (step S5). In order to achieve both detection accuracy and processing time, scaling is preferably performed by reducing the image to about 75 dpi, but is not limited to this reduction ratio.

[0049] Next, the control unit 75 performs a filter process on the original image scaled in step S3 (step S7). In this embodiment, the filter process is performed so that the characteristics of the edges of the document are not easily erased in order to reduce the influence of noise. Specifically, a filter process that performs a smoothing process only in the vertical direction or the horizontal direction is applied, but the filter is not limited to this type of filter.

[0050] Next, the control unit 75 performs edge detection on the original image that has been subjected to the filter process in step S7 (step S9). In this embodiment, a portion where the absolute value of the difference between the pixel values ​​of adjacent pixels exceeds a predetermined value is detected as an edge, but the present invention is not limited to such an edge detection method.

[0051] Next, the control unit 75 performs noise removal on the original image that has been subjected to the filter process in step S9 (step S11). In this embodiment, the process removes streaky noise corresponding to dirt on the sensor, so that the noise does not affect subsequent processes, but the noise removal is not limited to this type of process.

[0052] Next, the control unit 75 extracts contour edge information from the original image that has been subjected to noise removal in step S11 (step S13). Edge information is sequentially searched for in the main scanning direction or the sub-scanning direction, and the first edge information and the last edge information are extracted as contour edge information. The contour edge information includes feature points.

[0053] Next, the control unit 75 executes tilt detection based on the contour edge information extracted in step S13 (step S15). In this embodiment, a histogram of the deviation amount of the coordinate values ​​of the contour edges on two scanning lines separated by a predetermined number of lines is generated, and the tilt corresponding to the deviation amount of the part where the frequency of the histogram becomes a peak is obtained. Specifically, the tilt angle can be obtained by calculating atan (deviation amount ÷ predetermined number of lines), but the method is not limited to this.

[0054] Next, the control unit 75 performs tilt correction for each feature point of the contour edge information extracted in step S13 based on the tilt detected in step S15 (step S17). In this embodiment, the edge information is rotated using a rotation matrix, but the present invention is not limited to this method.

[0055] Next, the control unit 75 removes unnecessary edge information feature points from the outline edge information feature points that have been inclined in step S17 (step S19), as will be described in detail later with reference to FIG.

[0056] Finally, the control unit 75 detects the document range based on the contour edge information after removing the feature points of the unnecessary edge information in step S19 (step S21). In this embodiment, the document range is detected by obtaining a rectangular range that circumscribes the remaining contour edge information, but the present invention is not limited to this method.

[0057] FIG. 10 is a flow chart for explaining the removal of unnecessary edge information in the operation of the image processing device 60 according to the first embodiment. In step S17, the control unit 75 generates a histogram of edge information based on the inclination-corrected outer edge information (step S31). Next, the control unit 75 counts up the bins with the highest frequency in the histogram generated in step S31 (step S33). Next, the control unit 75 counts up the distribution of coordinates included in the top bins counted in step S33 (step S35). Next, the control unit 75 specifies a coordinate range corresponding to the outside of the document range based on the inclination-corrected coordinate standard based on the distribution counted in step S35 (step S37). Since the outside of the document range is specified based on the inclination-corrected outer edge information, if the document is inclined, the outside of the document range specified here is also inclined. Next, the control unit 75 removes the inclination-corrected outer edge information included in the outside of the document range specified in step S37 (step S39).

[0058] An example of an original image that has been subjected to steps S1 to S13 in FIG. 9 will be given. FIG. 11(A) is an example of an original image. FIG. 11(B) shows the state after the gradation correction in step S3 is performed on the original image in FIG. 11(A). FIG. 12(A) shows the state after the scaling in step S5 is performed on the original image in FIG. 12(B). FIG. 12(B) shows the state after the filtering process in step S7 is performed on the horizontal edges on the original image in FIG. 12(A). FIG. 13(A) shows the state after the filtering process in step S7 is performed on the vertical edges on the original image in FIG. 12(A). FIG. 13(B) shows the state after the edge detection in step S9 is performed on the horizontal edges on the original image in FIG. 12(B). FIG. 14(A) shows the state after the edge detection in step S9 is performed on the vertical edges on the original image in FIG. 13(A). FIG. 14(B) shows the state after noise removal in step S11 is performed on noise in the horizontal direction for the original image of FIG. 13(B). FIG. 15(A) shows the state after noise removal in step S11 is performed on noise in the vertical direction for the original image of FIG. 14(A). FIG. 15(B) shows the state after contour edge information extraction in step S13 is performed on contour edges in the horizontal direction for the original image of FIG. 14(B). FIG. 16(A) shows the state after contour edge information extraction in step S13 is performed on contour edges in the vertical direction for the original image of FIG. 15(A). FIG. 16(B) shows the state generated by combining the horizontal contour edges of FIG. 15(B) and the vertical contour edges of FIG. 16(A) in step S13.

[0059] In the above-mentioned first embodiment and its examples, feature points are extracted from an original image including an original image, and the original range and outside of the original range of the original image are determined based on the feature points. In this way, the amount of calculation required can be reduced by determining whether the original range is within the original range or outside of the original range based on the feature points. However, instead of determining whether the original range is within the original range or outside of the original range based on the feature points, the original range and outside of the original range may be determined based on the image data of the original image.

[0060] [2. Second embodiment] The second embodiment will be described. In the first embodiment, the document is drawn on a rectangular medium, but the second embodiment differs in that the document is not rectangular, but has rounded corners, that is, a rectangle with rounded corners. The following describes only the differences in configuration and processing from the first embodiment.

[0061] 17 is a diagram for explaining the shape of a document 200 used in the second embodiment of the present disclosure. The document 200 is surrounded on all four sides by line segments 201, 203, 205, and 207. The line segment 203 is also called the first side, and the line segment 207 is also called the second side. The document 200 has a rectangle with rounded corners, and the line segments 201 and 203 are connected by rounded corners 211, the line segments 203 and 205 are connected by rounded corners 213, the line segments 205 and 207 are connected by rounded corners 215, and the line segments 207 and 201 are connected by rounded corners 217. For convenience of explanation, the radius of the rounded corners is made extremely large.

[0062] The dotted line 221 is provided for explanation purposes and may not be drawn in reality. The dotted line 221 connects the end points 203R and 207R on the right hand side in the conveying direction of the line segment 203 and the line segment 207. Similarly, the dotted line 231 is provided for explanation purposes and may not be drawn in reality. The dotted line 231 connects the end points 203L and 207L on the left hand side in the conveying direction of the line segment 203 and the line segment 207. An object 225 is drawn in an area 223 surrounded by the rounded corner 211, the line segment 201, the rounded corner 217, and the dotted line 221. Similarly, an object 235 is drawn in an area 233 surrounded by the rounded corner 213, the line segment 205, the rounded corner 215, and the dotted line 231. An object 243 is drawn in a rectangular area 241 surrounded by a dotted line 221, a line segment 203, a dotted line 231, and a line segment 207. The objects 225, 235, and 243 are a part or the whole of any character, symbol, figure, image, or the like.

[0063] Fig. 18(A) is a diagram for explaining the outside of the document range determined by using the method of the first embodiment for an original image 300 generated by capturing an image of the document 200 in Fig. 17. Fig. 18(A) includes the original image 300 and feature points extracted by performing feature point extraction processing on the original image 300.

[0064] The original image 300 has a white background 300a and an original image 300b. Feature point groups 301, 303, 305, and 307 are sets of feature points extracted as corresponding to the periphery of the original image 300b. The feature point group 301 corresponds to the rounded corner 211, the line segment 201, and the rounded corner 217 of the original 200. The feature point group 303 corresponds to the line segment 203 of the original 200. The feature point group 305 corresponds to the rounded corner 213, the line segment 205, and the rounded corner 215 of the original 200. The feature point group 307 corresponds to the line segment 207 of the original 200. In an area 323 corresponding to the area 223 of the original 200, there is an object 325 corresponding to the object 225 of the original 200. In an area 333 corresponding to the area 233 of the original 200, there is an object 335 corresponding to the object 235 of the original 200. The feature points 345 are noise that occurs when the original 200 is imaged, and do not correspond to the original 200 .

[0065] Consider a case where unnecessary edge information is removed from such an original image 300 in the same manner as in the first embodiment. In this case, based on the feature point groups 303 and 307, the outside of the document range 351, 353, 355, and 357 are determined, and these pixels outside the document range are removed. However, the outside of the document range 351 includes the feature point group 345, the feature point group 301, and the object 325, and the outside of the document range 355 includes the feature point group 305 and the object 335. Therefore, the feature point groups 301 and 305 corresponding to the sides surrounding the outside of the objects 325 and 335, which should not be removed originally, are also removed, and an image as shown in FIG. 18(B) is obtained. From this image, even the areas including the objects 325 and 335 are determined to be outside the document range, and the desired result is not obtained.

[0066] FIG. 19(A) is a diagram for explaining the outside of the document range determined by using the method of the second embodiment for the original image generated by capturing the document of FIG. 17. FIG. 19(A) includes an original image 400 and feature points extracted by performing feature point extraction processing on the original image 400. The original image 400 generated by imaging the document 200 is composed of a white background 400a and a document image 400b corresponding to the document 200. The white background 400a corresponds to the background of the document cover 65b. The feature point groups 401, 403, 405, and 407 are sets of feature points extracted as corresponding to the outer edges of the document image 400b. The feature point group 401 corresponds to the rounded corner 211, the line segment 201, and the rounded corner 217. The feature point group 403 corresponds to the line segment 203. The feature point group 405 corresponds to the rounded corner 213, the line segment 205, and the rounded corner 215. Feature point group 407 corresponds to line segment 207. Objects 425, 443, and 435 correspond to objects 225, 243, and 235, respectively. Feature point group 445 is noise that occurs when imaging is performed by image input unit 65, and does not correspond to original 200.

[0067] In the second embodiment, when determining the outside of the document range, instead of using the feature point groups 403 and 407 as they are as one side of a rectangle forming the outside of the document range, the feature point groups 403 and 407 are each expanded as follows before determining the outside of the document range.

[0068] The dotted line 451 is a straight line extending from the line segment formed by the feature point group 403. The control unit 75 obtains a dotted line 455, which is a straight line parallel to the dotted line 451 and is a distance d away from the dotted line 451, toward the rear in the conveying direction. The dotted line 455 is also called a first virtual straight line. The distance d is, for example, about 1 to 10 mm, and preferably about 5 mm. The control unit 75 determines the feature points between the dotted line 451 and the dotted line 455 as part of the feature point group 403. In this way, the feature point group 403 is expanded by determining the feature points extending from the end points 403R and 403L on the right and left sides in the conveying direction of the feature point group 403 as part of the feature point group 403. The expanded feature point group 403 has a shape in which an arc bending toward the rear in the conveying direction is connected to both end points 403R and 403L of the line segment formed by the original feature point group, and has a roughly arched shape. Similarly, the control unit 75 obtains a dotted line 461 obtained by extending the line segment formed by the feature point group 407, and a dotted line 457 (second virtual straight line) that is a distance d away from the dotted line 461 forward in the conveying direction, and determines the feature points between the dotted line 461 and the dotted line 457 to be part of the feature point group 407, thereby expanding the feature point group 407. The expanded feature point group 407 has a shape in which an arc that curves forward in the conveying direction is connected to both end points 407R, 407L of the line segment corresponding to the original feature point group 407, and has a roughly arched shape.

[0069] The control unit 75 determines a straight line connecting end points 403R-E, 407R-E on the right side in the transport direction of the expanded feature points groups 403, 407, and a rectangular area on the right side of this line in the transport direction in the original image 400, as outside the original range 471. The control unit 75 determines the expanded feature points 403 and an area in the original image 400 that is forward of the expanded feature points 403 in the transport direction, as outside the original range 473. The control unit 75 determines a straight line connecting end points 403L-E, 407L-E on the left side in the transport direction of the expanded feature points groups 403, 407, and an area on the left side of this line in the transport direction in the original image 400, as outside the original range 475. The control unit 75 determines the expanded feature point group 407 and the area in the original image 400 that is behind the expanded feature point group 407 in the transport direction as being outside the document range 477 .

[0070] Fig. 19(B) is a diagram for explaining a state in which removal of unnecessary edge information according to the second embodiment has been performed on the original image 400 in Fig. 19(A). The control unit 75 removes feature points outside the document range 471, 473, 475, and 477. According to the second embodiment, the feature point group 445 corresponding to noise is removed, but since the line segment formed by the feature point groups 403 and 407 is expanded to obtain the outside document range 471, 473, 475, and 477, the objects 425 and 435 are determined to be within the document range.

[0071] [3. Third embodiment] A third embodiment will be described. In the first embodiment, unnecessary edge information is removed after performing edge inclination correction. In contrast, in the third embodiment, unnecessary edge information is removed without performing edge inclination correction. Note that only the parts of the configuration and processing that are different from the first embodiment will be mainly described.

[0072] First, the method of the first embodiment, that is, the method of removing unnecessary edge information after performing edge inclination correction, will be described. FIG. 20(A) shows an example of an original image 500 in which inclination detection is performed in step S15 after steps S1 to S13 in FIG. 9, and a group of feature points extracted from the original image 500. As shown in the figure, in the original image 500, an original image 500b of a business card is imaged in a state in which it is arranged diagonally upward to the right on a white background 500a. In step S15, the angle at which the original is inclined is calculated. For example, the control unit 75 calculates an inclination angle θ between the line segment formed by the group of feature points 503 in a direction approximately perpendicular to the conveying direction, i.e., approximately along the main scanning direction, and the main scanning direction.

[0073] FIG. 20B shows a state where the inclination of edge information is corrected using the inclination angle θ calculated in step S15 for the original image 500 and the feature point group 503 in FIG. 20A. As shown in the figure, as a result of the correction of the inclination angle θ, the line segment formed by the feature point group 503 is along a direction perpendicular to the conveying direction (main scanning direction). Here, the main scanning direction is the Y-axis direction, and the sub-scanning direction is the X-axis direction. FIG. 20C is an example of a histogram of vertical edges generated based on the feature point group 503 in FIG. 20B after the inclination of the edge information is corrected. When generating a histogram of edge information in step S31 in FIG. 10, the control unit 75 calculates a straight line parallel to the Y-axis at each point on the X-axis in FIG. 20B, and counts the pixels (feature points) on the straight line to generate bins and generate a histogram.

[0074] In the first embodiment, it is necessary to perform rotation calculations to correct the inclination angle θ for all feature points of the original image 500. This results in a large amount of calculations.

[0075] Next, the method of the third embodiment will be described. In the third embodiment, step S17 in FIG. 9 is not performed. Instead, when generating a histogram of vertical edges in step S19, a group of straight lines 505 having the inclination angle θ obtained in step S15 is obtained from the original image 500 in FIG. 21(A). The straight lines of the group of straight lines 505 are arranged so as to match the group of feature points. The group of straight lines 505 is illustrated in FIG. 21(B). For each straight line of the group of straight lines 505, bins are generated by counting the feature points on that straight line to generate a histogram. As shown in FIG. 21(C), the third embodiment also obtains a histogram similar to that of the first embodiment shown in FIG. 20(C). At this time, since there is no need to perform a rotation calculation on the feature points as in the first embodiment, the amount of calculation can be reduced.

[0076] [4. Modifications] The present disclosure is not limited to the above-described embodiments and modifications, and various modifications are possible. In other words, the technical scope of the present disclosure also includes embodiments obtained by combining technical means that are appropriately modified within the scope of the gist of the present disclosure.

[0077] The programs that run on each device in the embodiments are programs that control the CPU and the like (programs that make a computer function) so as to realize the functions of the above-described embodiments. Information handled by these devices is temporarily stored in a temporary storage device (e.g., RAM) during processing, and is then stored in various storage devices such as ROMs (Read Only Memories) and HDDs, and is read, modified, and written by the CPU as necessary.

[0078] Here, the recording medium for storing the program may be any of semiconductor media (e.g., ROM, non-volatile memory cards, etc.), optical recording media, magneto-optical recording media (e.g., DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray (registered trademark) Disc), etc.), magnetic recording media (e.g., magnetic tape, flexible disk, etc.), etc. Furthermore, not only are the functions of the above-mentioned embodiments realized by executing the loaded program, but the functions of the present disclosure may also be realized by processing in cooperation with an operating system or other application programs, etc., based on instructions from the program.

[0079] In addition, when distributing the program on the market, the program can be stored in a portable recording medium and distributed, or transferred to a server computer connected via a network such as the Internet. In this case, the storage device of the server computer is of course included in the present disclosure. [Explanation of symbols]

[0080] 1, 80, 200 manuscripts Sides 3, 5, 7, 9, 23, 25, 27, 29, 141, 143, 145, 147 11, 31, 51 tables 13, 15, 17, 33, 35, 37, 40b, 53, 55, 57 Stains 20, 40, 100, 300, 400, 500 Original image 20a gray background 40a, 100a, 300a, 400a, 500a White background 21, 41, 100b, 300b, 400b, 500b Original image 43, 45, 47, 49, 51a, 51b pixel groups 60 Image processing device 61 Display section 63 Operation section 65 Image input section 65a manuscript stand 65b Manuscript cover (background) 65c Document transport section 67 Image forming section 69 Communications Department 71 Connection 73 Memory section 75 Control Unit 81, 83, 85, 87, 101, 103, 105, 107 Edge 89 Longitudinal 91 Short side 111, 113, 115, 117, 119, 301, 303, 305, 307, 345, 401, 403, 405, 407, 445, 503 Feature points 113R, 113L, 117R, 117L, 203R, 203L, 207R, 207L, 403R, 403L, 407R, 407L, 403R-E, 403L-E, 407R-E, 407L-E End points 131, 133, 135, 137 Outside manuscript range 201, 203, 205, 207, 403, 407 lines 211, 213, 215, 217 Rounded corners 221, 231 dotted line 223, 233, 323, 333 area 225, 235, 243, 425, 443, 435 Objects 241 Rectangular area 505 straight line group

Claims

1. A control unit is provided, The control unit is The image obtained by scanning the document is acquired, Extracting feature points from the image; Identifying a plurality of feature points aligned in a predetermined direction in the image as feature points corresponding to edges that define the extent of the document; setting an outside of the document range based on the identified feature points; removing the feature points outside the set document range; Image processing device.

2. the image is generated using a line image sensor in which a plurality of image pickup elements are arranged along a first straight line, and a group of light source elements arranged along a second straight line perpendicular to the first straight line; The predetermined direction is parallel to the second straight line. The image processing device according to claim 1 .

3. The image processing apparatus according to claim 2 , wherein the feature points corresponding to the edges include feature points corresponding to two sides of the document that are substantially parallel to the first straight line.

4. Two sides of the document that are substantially parallel to the first straight line are called first and second sides, When straight lines parallel to the first and second sides in the image and separated by a predetermined distance from the feature points corresponding to the first and second sides are called first and second virtual straight lines, respectively, The feature points corresponding to the part of the edge further include a feature point between a straight line extending from the first side and the first virtual straight line, and a feature point between a straight line extending from the second side and the second virtual straight line. The image processing device according to claim 3 .

5. Further comprising an image input unit, the image input unit includes a background portion that becomes a background of the document when the image input unit captures an image of the document, the image includes a background image corresponding to the background portion and an original image corresponding to the original; The image processing device according to claim 1 .

6. The image processing device according to claim 5 , wherein the background is white.

7. The control unit is Calculating the coordinates of the feature points extracted from the image; extracting feature points corresponding to portions of edges defining the document based on the coordinates; The image processing device according to claim 1 .

8. The control unit is Identifying pixels of the feature points extracted from the image; extracting feature points based on the pixels, the feature points corresponding to portions of edges defining the document; The image processing device according to claim 1 .

9. 1. A method for controlling an image processing device, comprising: Acquiring the image obtained by reading the image; Extracting feature points from the image; Identifying a plurality of feature points aligned in a predetermined direction in the image as feature points corresponding to edges that define the extent of the document; setting an outside of the document range based on the identified feature points; removing the feature points outside the set document range; A method for controlling an image processing device.

Images