Reading device, image processing device, and feature quantity detection method
By employing a dual-light illumination system and imaging unit for capturing visible and invisible light images, the method stabilizes edge detection between documents and backgrounds, addressing color variation issues in conventional techniques.
Patent Information
- Application Number
- JP2020038264
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-03-05
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-03-05
AI Technical Summary
Conventional image processing techniques using infrared low reflection sections for edge detection between documents and backgrounds fail to account for document color variations, leading to unstable edge detection.
An illumination system that uses both visible and invisible light, combined with an imaging unit capable of capturing both visible and invisible light images, allows for the detection of spectral reflection property differences to stabilize edge detection between documents and backgrounds.
Enables stable edge detection between documents and backgrounds regardless of document type, improving accuracy and reliability in image processing.
Smart Images

Figure 0007592974000001 
Figure 0007592974000002 
Figure 0007592974000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a reading device, an image processing device, and a feature detection method. [Background technology]
[0002] Conventionally, there is already known an image processing technique that detects edges between an original and a background from an image and corrects the inclination and position of the original based on the detected edges between the original and the background.
[0003] Patent Document 1 discloses a technique for extracting edges between an original and a background by providing an infrared light low reflection part in the background and detecting the edges between the original and the background based on an acquired infrared image. Summary of the Invention [Problem to be solved by the invention]
[0004] However, according to the conventional technology, a low infrared reflectance portion is provided in the background and edges between the document and the background are detected based on the acquired infrared image, but there is a problem that edge detection is not possible depending on the document color.
[0005] The present invention has been made in view of the above, and has an object to enable stable edge detection between a document and its background, regardless of the type of document. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems and achieve the object, the present invention provides an illumination unit that irradiates a subject with visible light and invisible light, an imaging unit that receives the visible light and invisible light reflected by the subject and captures a visible image and an invisible image, a background unit provided in an imaging range of the imaging unit, and a display unit that displays the visible image and the invisible image in accordance with the spectral reflection characteristics of the subject. and the invisible image, based on which of the image having a larger spectral reflectance characteristic difference between the subject and the background portion of the invisible image, and an image processing unit for detecting a feature amount of the subject or the background. Effect of the Invention
[0007] According to the present invention, an effect is achieved in that stable edge detection can be performed between an original and its background regardless of the type of original. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an image forming apparatus according to a first embodiment. [Diagram 2] FIG. 2 is a cross-sectional view illustrating an example of the structure of an image reading device. [Diagram 3] FIG. 3 is a block diagram showing electrical connections between the various components constituting the image reading device. [Figure 4] FIG. 4 is a block diagram showing the functional configuration of the image processing unit. [Diagram 5] FIG. 5 is a diagram showing differences in spectral reflectance characteristics due to media involved in detecting the feature amount of a subject. [Figure 6] FIG. 6 is a diagram showing the difference in the spectral reflectance characteristics between a visible image and an invisible image depending on the type of paper. [Figure 7] FIG. 7 is a diagram showing an example of selection of visible components from which feature amounts are extracted. [Figure 8] FIG. 8 is a diagram showing an example of the spectral reflectance characteristics when the background portion is a low invisible light reflectance portion. [Figure 9] FIG. 9 is a diagram illustrating an example of an invisible light low reflection portion. [Figure 10] FIG. 10 is a diagram showing information obtained from the edge of an object. [Figure 11] FIG. 11 is a diagram showing an example of an edge detection technique. [Figure 12] FIG. 12 is a diagram showing an example of a feature amount using an edge. [Figure 13] FIG. 13 is a diagram showing the selection of a linear equation in the regression line equation. [Figure 14] FIG. 14 is a diagram showing an example of size detection (horizontal direction). [Figure 15] FIG. 15 is a block diagram of a functional configuration of an image processing unit according to the second embodiment. As shown in FIG. [Figure 16]FIG. 16 is a diagram for explaining the OR process of edges. [Figure 17] FIG. 17 is a diagram for explaining how edges appear in a visible image and an invisible image. [Figure 18] FIG. 18 is a diagram showing an example of determination of normal detection of an edge. [Figure 19] FIG. 19 is a diagram showing an example of a failure in the OR process of edges. [Figure 20] FIG. 20 is a diagram showing an example of a subject having a mixture of a plurality of characteristics. [Figure 21] FIG. 21 is a block diagram of a functional configuration of an image processing unit according to the third embodiment. As shown in FIG. [Figure 22] FIG. 22 is a flowchart showing the flow of processing in the image processing unit. [Diagram 23] FIG. 23 is a diagram showing an example of correction of the inclination and position of a subject. [Figure 24] FIG. 24 is a diagram showing applications of invisible images. [Diagram 25] FIG. 25 is a diagram showing an example of correction of the inclination and position of a subject and extraction. [Figure 26] FIG. 26 is a diagram illustrating an example of tilt correction. [Figure 27] FIG. 27 is a diagram showing a search for the right end edge point. [Figure 28] FIG. 28 is a diagram showing a modified example of the image processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a reading device, an image processing device, and a feature detection method will be described in detail with reference to the accompanying drawings.
[0010] (First embodiment) Fig. 1 is a diagram showing an example of a configuration of an image forming apparatus 100 according to a first embodiment. In Fig. 1, the image forming apparatus 100, which is an image processing apparatus, is generally called a multifunction machine having at least two functions of a copy function, a printer function, a scanner function, and a facsimile function.
[0011] The image forming apparatus 100 has an image reading device 101, which is a reading device, and an ADF (Automatic Document Feeder) 102, and below that has an image forming unit 103. In order to explain the internal configuration of the image forming unit 103, the external cover is removed to show the internal configuration.
[0012] The ADF 102 is an original support unit that positions an original to be read at a reading position. The ADF 102 automatically transports an original placed on a placement table to the reading position. The image reading device 101 reads the original transported by the ADF 102 at a predetermined reading position. The image reading device 101 also has a contact glass on its upper surface that is an original support unit on which an original is placed, and reads the original on the contact glass that is the reading position. Specifically, the image reading device 101 is a scanner that has a light source, an optical system, and a solid-state imaging element such as a CMOS image sensor inside, and reads the reflected light of an original illuminated by a light source through the optical system with the solid-state imaging element.
[0013] The image forming unit 103 has a manual feed roller 104 for manually feeding recording paper, and a recording paper supply unit 107 for supplying recording paper. The recording paper supply unit 107 has a mechanism for feeding recording paper from a multi-stage recording paper feed cassette 107a. The supplied recording paper is sent to a secondary transfer belt 112 via a registration roller 108.
[0014] A toner image on the intermediate transfer belt 113 is transferred to the recording paper conveyed on the secondary transfer belt 112 in a transfer section 114 .
[0015] The image forming unit 103 also includes an optical writing device 109, tandem imaging units (Y, M, C, K) 105, an intermediate transfer belt 113, and the secondary transfer belt 112. Through an image creation process by the imaging units 105, the image written by the optical writing device 109 is formed as a toner image on the intermediate transfer belt 113.
[0016] Specifically, the imaging unit (Y, M, C, K) 105 has four rotatable photoconductor drums (Y, M, C, K), and is provided with imaging elements 106 including a charging roller, a developing unit, a primary transfer roller, a cleaner unit, and a static eliminator around each photoconductor drum. The imaging elements 106 function for each photoconductor drum, and the image on the photoconductor drum is transferred onto the intermediate transfer belt 113 by each primary transfer roller.
[0017] The intermediate transfer belt 113 is stretched across a drive roller and a driven roller in the nip between each photosensitive drum and each primary transfer roller. The toner image primarily transferred onto the intermediate transfer belt 113 is secondarily transferred to the recording paper on the secondary transfer belt 112 by the secondary transfer device as the intermediate transfer belt 113 travels. The recording paper is transported to the fixing device 110 as the secondary transfer belt 112 travels, and the toner image is fixed onto the recording paper as a color image. The recording paper is then discharged to a paper discharge tray outside the machine. In the case of double-sided printing, the front and back of the recording paper are inverted by the inversion mechanism 111, and the inverted recording paper is sent onto the secondary transfer belt 112.
[0018] The image forming unit 103 is not limited to one that forms images by the electrophotographic method as described above, but may be one that forms images by an inkjet method.
[0019] Next, the image reading device 101 will be described.
[0020] Fig. 2 is a cross-sectional view showing an example of the structure of the image reading device 101. As shown in Fig. 2, the image reading device 101 has a sensor board 10 equipped with an imaging unit 22 which is a solid-state imaging element, a lens unit 8, a first carriage 6, and a second carriage 7 in a main body 11. The first carriage 6 has a light source 2 which is an LED (Light Emitting Diode) and a mirror 3. The second carriage 7 has mirrors 4 and 5. In addition, the image reading device 101 has a contact glass 1 on the upper surface.
[0021] The light source 2 is configured for visible and invisible light. Here, invisible light refers to light with a wavelength of 380 nm or less or 750 nm or more. In other words, the light source 2 is an illumination unit that irradiates the subject and background 13 with visible light and invisible light (for example, near-infrared (NIR) light).
[0022] Furthermore, the image reading device 101 has a background unit 13, which is a reference white board, on the upper surface. More specifically, the background unit 13 is provided on the opposite side of the subject from the light source 2, which is an illumination unit, within the imaging range of the imaging unit 22.
[0023] In a reading operation, the image reading device 101 irradiates light upward from the light source 2 while moving the first carriage 6 and the second carriage 7 from a standby position (home position) in the sub-scanning direction (direction A). Then, the first carriage 6 and the second carriage 7 form an image on the imaging unit 22 via the lens unit 8, using the light reflected from the original 12, which is the subject.
[0024] Furthermore, when the power is turned on, the image reading device 101 sets a reference by reading the reflected light from the reference white plate 13. That is, the image reading device 101 moves the first carriage 6 to directly below the reference white plate 13, turns on the light source 2, and forms an image of the reflected light from the reference white plate 13 on the imaging unit 22, thereby performing gain adjustment.
[0025] The imaging unit 22 can capture images in the visible and invisible wavelength ranges. The imaging unit 22 has pixels arranged therein that convert the amount of incident light into an electrical signal. The pixels are arranged in a matrix, and the electrical signals obtained from each pixel are transferred to the downstream signal processing unit 222 (see FIG. 3) in a predetermined order at regular intervals (pixel readout signal). A color filter that transmits only light of a specific wavelength is arranged on each pixel. In the imaging unit 22 of this embodiment, each signal obtained from a pixel group in which the same color filter is arranged is called a channel. In this embodiment, an image captured by the imaging unit 22 by irradiating it with visible light is called a visible image, and an image captured by the imaging unit 22 by irradiating it with invisible light such as near-infrared light is called an invisible image.
[0026] Although an image reading device of a reduction optical system is applied as the image reading device 101 of this embodiment, the present invention is not limited to this, and may be an equal magnification optical system (contact optical system: CIS type).
[0027] Fig. 3 is a block diagram showing electrical connections of the components constituting the image reading device 101. As shown in Fig. 3, the image reading device 101 includes an image processing unit 20, a control unit 23, and a light source driving unit 24 in addition to the above-mentioned imaging unit 22 and light source 2. The control unit 23 controls the imaging unit 22, the image processing unit 20, and the light source driving unit 24. The light source driving unit 24 drives the light source 2 under the control of the control unit 23.
[0028] The imaging unit 22 is a sensor for a reduction optical system, such as a CMOS image sensor, etc. The imaging unit 22 includes a pixel unit 221, a signal processing unit 222, etc.
[0029] In this embodiment, the imaging unit 22 is described as having a four-line configuration, but is not limited to four lines. Also, the circuit configuration subsequent to the pixel unit 221 is not limited to the configuration shown in the figure.
[0030] The pixel unit 221 has four lines of pixel groups in which a plurality of pixel circuits constituting pixels are arranged in a matrix. The signal processing unit 222 processes signals output from the pixel unit 221 as necessary and transfers the signals to the image processing unit 20 arranged in the downstream.
[0031] The image processing unit 20 executes various types of image processing in accordance with the intended use of the image data.
[0032] 4 is a block diagram showing the functional configuration of the image processing unit 20. As shown in FIG.
[0033] The feature amount detection unit 201 detects the feature amount of the original 12 , which is the subject, from the visible or invisible image obtained by the imaging unit 22 .
[0034] FIG. 5 is a diagram showing the difference in spectral reflection characteristics due to the medium involved in detecting the feature amount of the subject. When the imaging unit 22 reads the reflected light from the document 12, which is the subject, the spectral reflection characteristics are generally different between the background 13 and the document 12, which is the subject. In the example shown in FIG. 5, the background 13 slopes downward to the right, and the document 12, which is the subject, slopes upward to the right. That is, images with different characteristics are obtained with visible light and invisible light. Therefore, the feature amount detection unit 201 sets the image to be detected in advance as either visible or invisible, depending on the type of the document 12, which is the subject, and the type of the background 13. This makes it easier for the feature amount detection unit 201 to obtain the target feature amount.
[0035] Here, Fig. 6 is a diagram showing differences in the spectral reflectance characteristics between visible and invisible images depending on the paper type. For example, according to the example shown in Fig. 6, when the visible and invisible images of paper type A are compared, it is found that the invisible image has a larger difference in the spectral reflectance characteristics from the background portion 13. Therefore, in the case of paper type A, the feature amount detection unit 201 can set the feature amount detection target to be the invisible image. On the other hand, when the visible and invisible images of paper type B are compared, it is found that the visible image has a larger difference in the spectral reflectance characteristics from the background portion 13. Therefore, in the case of paper type B, the feature amount detection unit 201 can set the feature amount detection target to be the visible image.
[0036] Here, the selection of visible components from which feature amounts are extracted will be described.
[0037] Fig. 7 is a diagram showing an example of selection of visible components from which feature quantities are extracted. Although the light actually irradiated has a broad wavelength range, for simplicity, the representative wavelength of each component is shown by a dotted line in Fig. 7. Also, as an example, near-infrared light is used as invisible light.
[0038] 7, in the near-infrared light components, the component with the greatest difference in reflectance (arrow X) between the subject document 12 and the background portion 13 is the B component. Therefore, by using this B component, the feature amount detection unit 201 can make a difference in the feature amounts between the subject document 12 and the background portion 13.
[0039] That is, the feature amount detection unit 201 compares the difference in the spectral reflectance characteristics of the background 13 and the subject document 12 for invisible light and visible light, and the feature amount of the visible image includes the component of visible light that has the largest difference from invisible light. Generally, the feature amount of the G component, which has a wide wavelength range, is often used from a visible image. However, in the example shown in Fig. 7, when the feature amounts of the visible range and the infrared range are used, the difference in the spectral reflectance characteristics of the document and the background board becomes large for the B component and the infrared component, making it easier to detect edges.
[0040] The feature amount detection unit 201 is not limited to extracting only the B component as the feature amount to be used for the visible light components, but may also extract a part of the RGB components, such as the maximum component value, for example.
[0041] In addition, when there is variation in the spectral reflectance characteristics of the subject, i.e., the original 12, the feature detection unit 201 may determine the visible components to be selected for feature extraction based on measurements taken from a representative subject, i.e., the original 12, or on the average of the measurement results.
[0042] Here, a case where the background portion 13 is a low invisible light reflection portion will be described.
[0043] Fig. 8 is a diagram showing an example of the spectral reflectance characteristics when the background portion 13 is a low invisible light reflectance portion. As shown in Fig. 8, the background portion 13 may be a low invisible light reflectance portion that diffusely reflects visible light and reflects invisible light at a lower reflectance than visible light. This creates a significant difference in the spectral reflectance of the background portion 13 between the visible image and the invisible image, and as a result, there is also a difference in the difference in the spectral reflectance between the subject document 12 and the background portion 13, making it easier for the feature amount detection unit 201 to extract the targeted feature amount.
[0044] 9 is a diagram showing an example of the low invisible light reflection portion. The low invisible light reflection portion may be provided as the whole of the background portion 13, or as a part or pattern of the background portion 13 as shown in FIG.
[0045] In this way, since the background portion 13 has an invisible light low-reflecting portion that diffusely reflects visible light and reflects invisible light at a lower reflectance than visible light, a more significant difference can be produced in the background readings between the visible image and the invisible image, enabling robust edge detection.
[0046] Next, a case where the feature amount detection unit 201 extracts an edge of the document 12, which is a subject, as a feature amount will be described.
[0047] Here, Fig. 10 is a diagram showing information obtained from the edge of a subject. As shown in Fig. 10, an edge refers to the boundary between a subject, ie, an original 12, and a background portion 13. By detecting such an edge, as shown in Fig. 10, it is possible to recognize the position, inclination, size, etc., of the subject, ie, original 12. From the position and inclination of the subject, ie, original 12, it is also possible to perform image correction according to the position and inclination of the subject, ie, original 12, in a later process.
[0048] FIG. 11 is a diagram showing an example of an edge detection method. As shown in FIG. 11(a), an edge detection method may include a method of applying a first-order differential filter to the entire image and binarizing each pixel based on whether it exceeds a predetermined threshold value. In this case, depending on the threshold value, a horizontal edge may appear as several consecutive vertical pixels (and vice versa). This is mainly because the edge is blurred due to the MTF characteristics of the optical system. Therefore, as shown in FIG. 11(b), a method is available in which, for example, the center of consecutive pixels is selected to obtain a representative edge pixel for the calculation of a regression line equation or size detection (part a in FIG. 11(b)).
[0049] FIG. 12 is a diagram showing an example of a feature quantity using edges. The feature quantity may be not only the edge itself extracted from the image, but also something that uses the edge. Examples include a regression line equation calculated from the extracted edge point group using the least squares method or an area (a set of positions) inside the edge, as shown in FIG. 12. Regarding the regression line equation, there is a method of calculating one linear equation from all edge information for each side, but there is also a method of dividing into multiple areas, calculating linear equations, and selecting or integrating a representative one. In that case, a method of deriving the final linear equation includes a straight line whose slope is the median, or a method of obtaining the average value of each linear equation.
[0050] Fig. 13 is a diagram showing the selection of a linear equation in the regression line equation. By dividing into a plurality of regions, calculating linear equations, and selecting or integrating representative ones, as shown in Fig. 13, it is possible to correctly recognize the inclination of the subject, ie, the original 12, even if the subject, ie, the original 12, is damaged, such as having a missing edge.
[0051] As in the above process, the feature amount detection unit 201 extracts the edges of the document 12, which is the subject, as feature amounts, and thus can detect the area of the document 12, which is the subject.
[0052] Next, detection of the size of the document 12, which is the subject, will be described.
[0053] Here, Fig. 14 is a diagram showing an example of size detection (horizontal direction). As shown in Fig. 14, for a representative position in the vertical direction of an image, the distance between the left edge and the right edge of the subject, the document 12, is obtained, and the horizontal size can be calculated from the median of the distances and a separately calculated inclination angle. Similarly, the vertical size can also be calculated.
[0054] The size information detected in this way can be used for error detection, image correction processing (described later), etc. For example, when scanning with a multifunction printer, if a document size different from the document size set by the user in advance is detected, the printer can notify the user to set a document of the correct size.
[0055] In this manner, according to this embodiment, by detecting the features of the subject, namely the original 12 or the background 13, from at least one of the visible image and the invisible image, it is possible to obtain information from the invisible image that cannot be obtained from the visible image, thereby enabling stable edge detection between the original and the background regardless of the type of original.
[0056] Furthermore, the imaging unit 22 receives visible light and invisible light reflected by the document 12, which is the subject, and captures a visible image and an invisible image, thereby making it possible to perform image reading with a simple configuration.
[0057] Furthermore, since the invisible light and the invisible image are infrared light and an infrared image, the image can be read with a simple configuration.
[0058] (Second embodiment) Next, a second embodiment will be described.
[0059] The second embodiment differs from the first embodiment in that feature amounts are extracted from both visible and invisible images and then automatically selected or integrated. In the following description of the second embodiment, the description of the same parts as in the first embodiment will be omitted, and only the parts that differ from the first embodiment will be described.
[0060] 19 is a block diagram showing the functional configuration of an image processing unit according to the second embodiment. As shown in FIG. 19, the image processing unit 20 includes a feature amount selection / integration unit 202 in addition to a feature amount detection unit 201.
[0061] As described above, the feature detection unit 201 detects features of the subject, that is, the document 12 or the background 13, detected from at least one of the visible image and the invisible image obtained by the imaging unit 22.
[0062] The feature selection and integration unit 202 selects or integrates the features detected from each image based on the features of the subject, namely the document 12 or the background 13, detected from at least one of the visible image and the invisible image by the feature detection unit 201.
[0063] More specifically, the feature selection and integration unit 202 automatically performs the selection process described in Fig. 6. As a result, even for a document 12 that is a subject from which a target feature cannot be extracted using a visible image or an invisible image alone, it is possible to obtain a target feature by combining and using them.
[0064] 16 is a diagram for explaining the OR process of edges in the feature amount selection and integration unit 202. The feature amount selection and integration unit 202 of this embodiment executes OR process of edges when extracting edges of the document 12, which is the subject, as feature amounts.
[0065] By taking the OR of each edge extracted from the invisible image and the visible image, it is possible to complement the parts of one image where the edge cannot be captured with the other image. For example, as shown in Fig. 16, in the case of a document with gradation, it is easy to capture the edges of the black areas of the document in the visible image and difficult to capture the edges of the white areas, but the opposite is true in the invisible image.
[0066] Therefore, the feature selection and integration unit 202 combines the edges of the black areas in the visible image and the edges of the white areas in the invisible image to extract edges of the entire document that could not be obtained from either image alone.
[0067] In this way, the feature selection and integration unit 202 integrates the edges of the invisible image and the visible image by ORing the edges, so that there are locations where edges can be detected in either the visible image or the invisible image, making it possible to detect edges between the subject, i.e., the original 12, and the background 13, in more locations.
[0068] Next, the priority given to edges of invisible images will be described.
[0069] Here, Fig. 17 is a diagram for explaining how edges appear in a visible image and an invisible image. As shown in Fig. 17(a), the shadow of the subject, the original 12, may appear in the background of the visible image, and depending on the shape of the shadow, the edge may not be extracted as a straight line, which may affect the accuracy of detecting the inclination of the subject, the original 12. On the other hand, in areas of the visible image where there is no shadow, if the subject, the original 12, is white, there is a high possibility that edge extraction itself may not be possible.
[0070] As shown in FIG. 17(b), in the invisible image, the edge between the white subject document 12 and the background 13 is easily extracted, especially when the background 13 is an invisible light low reflection part. The shadow of the subject document 12 may occur in the invisible image, but since the shadow is darker than the background, for example, if the edge is detected using a first-order differential filter that takes "dark part → light part", the edge between the shadow and the document can be extracted, not the edge between the shadow and the background. Alternatively, even if two types of first-order differential filters are used to take both the edges "dark part → light part" and "light part → dark part", if the luminance is close to that of the background 13, the edge between the shadow and the background 13 does not need to be extracted. Therefore, the invisible image is more likely to allow for more accurate edge detection between the subject document 12 and the background 13 than the visible image, and it is advisable to use the visible image only when the edge cannot be detected normally in the invisible image.
[0071] Here, Fig. 18 is a diagram showing an example of judging whether an edge is normally detected. As a criterion for judging whether an edge is "normally detected", for example, when the obtained edge point group is regressed with a straight line, the error by the least squares method is within a threshold value, the inclination angle of the straight line is within a threshold value, etc. are included. In addition, when selecting and integrating from the above-mentioned multiple straight line expressions, the number of straight line expressions judged to be normal is equal to or greater than a threshold value.
[0072] In this way, when the feature selection and integration unit 202 selects edges, if the edges of the invisible image can be detected normally, it selects the edges of the invisible image, and if the edges of the invisible image cannot be detected normally, it selects the edges of the visible image. In this case, it is highly likely that edges are easier to detect in the invisible image, and the detection accuracy can be improved.
[0073] Next, a case where edges cannot be detected normally in both the visible image and the invisible image will be described.
[0074] FIG. 19 is a diagram showing an example of a failed OR process of edges. When OR process is performed on a visible image and an invisible image, there is a possibility that an unintended edge may be extracted. For example, as shown in FIG. 19, if an edge is extracted between the shadow of the subject, document 12, in the visible image and the background portion 13, the shadow of the subject, document 12, remains when OR process is performed, which affects the calculation of the inclination of the document. However, since OR process has the advantage of increasing the number of edge detection points as described above, the feature detection unit 201 performs OR process only when the edge cannot be detected normally in either the visible image or the invisible image.
[0075] In this way, the feature selection and integration unit 202 performs OR processing of the invisible image edge and the visible image edge when neither the invisible image edge nor the visible image edge can be detected normally. The appearance of edges may differ between the visible image and the invisible image due to the shadow of the document, etc. Therefore, OR processing is performed only when detection cannot be performed normally in each image.
[0076] Here, Fig. 20 is a diagram showing an example of a subject having a mixture of multiple characteristics. As shown in Fig. 20, for example, in the case of a document 12 which is a subject having a mixture of multiple characteristics, it is possible to extract the lower part of the document 12 which is the subject from the visible image, and to extract the upper part of the document 12 which is the subject from the invisible image.
[0077] In this manner, according to this embodiment, the feature amounts of the subject, namely the document 12 or the background 13, are detected from at least one of the visible image and the invisible image, and the feature amounts detected from each image are selected or integrated. This makes it possible to automatically select feature amounts from either the visible image or the invisible image, or to integrate feature amounts from both.
[0078] (Third embodiment) Next, a third embodiment will be described.
[0079] The third embodiment differs from the first and second embodiments in that an image correction unit that performs image correction of a subject is provided. In the following description of the third embodiment, the description of the same parts as the first and second embodiments will be omitted, and only the parts that differ from the first and second embodiments will be described.
[0080] Here, Fig. 21 is a block diagram showing the functional configuration of an image processing unit according to the third embodiment, and Fig. 22 is a flowchart showing the flow of processing in the image processing unit. As shown in Fig. 21, the image processing unit 20 includes an image correction unit 203 in addition to a feature detection unit 201 and a feature selection / integration unit 202.
[0081] As shown in FIG. 22, the feature detection unit 201 detects features of the subject, namely, the document 12 or the background 13, detected from at least one of the visible image and the invisible image obtained by the imaging unit 22 (step S1).
[0082] As shown in FIG. 22, the feature selection and integration unit 202 selects or integrates features detected from each image based on features of the subject, namely, the document 12 or the background 13, detected from at least one of the visible image and the invisible image by the feature detection unit 201 (step S2).
[0083] 22, the image correction unit 203 performs image correction on each of the visible image and the invisible image using the integration result from the feature amount selection and integration unit 202 (step S3). An example of the image correction will be described later.
[0084] 23 is a diagram showing an example of correcting the inclination and position of a subject. In the example shown in Fig. 23, the image correcting unit 203 corrects the inclination and position of the subject, that is, the document 12, using the feature amounts detected by the feature amount selecting and integrating unit 202.
[0085] To correct the inclination of the subject, i.e., the original 12, the image correction unit 203 uses a method in which, as described above, it calculates the inclination when the edge points extracted from each side of the subject, i.e., the original 12, are regressed in a straight line, and the entire image is rotated based on this inclination.
[0086] To correct the position of the subject, original 12, image correction unit 203 uses a method of finding the intersection position of the regression line of the edge point group on the upper side and left side of subject, original 12, and moving that point to the origin.
[0087] FIG. 24 is a diagram showing the use of invisible images. By correcting an image based on the integration result in the feature selection and integration unit 202, effects such as improved visibility of pictures and characters in the document area can be obtained. In addition, the reflectance of invisible components differs greatly from that of visible components depending on the coloring material, and as a result, it is possible to cause color loss as shown in FIG. 24. For this reason, it is conceivable to use the loss of color to perform OCR processing or the like in a later stage. Therefore, correcting not only visible images but also invisible images has the advantage of contributing to improved OCR accuracy.
[0088] In this way, by correcting the inclination and position of the subject, ie, the document 12, based on the edge detection results, it is possible to correct the subject, ie, the document 12, so that it is easier to see. In addition, there is a possibility that the OCR accuracy can be improved.
[0089] Here, Fig. 25 is a diagram showing an example of correction of the inclination and position of the subject and cut-out. Image correction unit 203 cuts out the area of document 12, which is the subject, at just the right size by combining the above-mentioned correction of inclination and position, as shown in Fig. 25. Even if the feature amount cannot be detected well and the inclination and position cannot be corrected, the cut-out itself is possible, although it will not be at the right size.
[0090] FIG. 26 is a diagram showing an example of tilt correction. As an example where tilt correction is not possible, when image processing is performed by hardware, it is necessary to replace consecutive pixels in an image in a group of more than a minimum width for the sake of processing speed, as shown in FIG. 26. However, there is a problem that correction becomes difficult if the tilt is too large. In that case, it may be desirable to delete as much of the background portion 13 as possible, even though tilt correction is not possible.
[0091] In this case, the image correction unit 203 may perform processing such that, for example, if the rightmost edge point is identified, the area to the right of the edge point is outside the area of the document 12, which is the subject, and therefore the area to the right of the edge point is removed from the image.
[0092] Fig. 27 is a diagram showing a search for the right edge point. Even if edges can only be extracted from a partial region of an image due to memory capacity limitations, vertical size information can be known by other means such as sensor information during document transport. In this case, as shown in Fig. 27, image correction unit 203 predicts the right edge point using inclination information from edge pixels in that region. Note that this is not limited to the right edge, but also applies to the top, left and bottom edges.
[0093] The image correction unit 203 can remove unnecessary areas of the background portion 13 by cutting out the area of the subject, the original 12. As a result, in the image forming apparatus 100 such as a multifunction peripheral, for example, it is possible to obtain effects such as reducing the user's effort in inputting the size of the subject, the original 12, improving the appearance of the image, reducing the storage area for saving the image, reducing the size of the recording paper when copying the image, and reducing the amount of ink and toner consumed.
[0094] In this way, by automatically detecting and cutting out the size of the subject document 12 from the image based on the edges, the user can save the trouble of inputting the document size (particularly non-standard sizes). In addition, it has the effect of improving the appearance of the image, reducing the storage area of the image storage destination, reducing the size of the recording paper when copying the image, and reducing the amount of ink and toner consumed.
[0095] As described above, according to this embodiment, the image correction unit 203 corrects at least one of the visible image and the invisible image, thereby making it possible to improve the visibility of the image.
[0096] In the above embodiment, the image processing device of the present invention is described as being applied to a multifunction device having at least two of the functions of a copy function, a printer function, a scanner function, and a facsimile function, but the image processing device can be applied to any image forming device such as a copier, printer, scanner device, or facsimile device.
[0097] In the above-described embodiments, the image reading device 101 of the image forming device 100 is applied as the image processing device, but the present invention is not limited to this. The definition of an image processing device is that it is sufficient if it is a device that can obtain a reading level, such as a line sensor of a life-size optical system (contact optical system: CIS method) shown in Fig. 28(a), even if it does not read an image. The device shown in Fig. 28(a) moves the line sensor or the document to read information on multiple lines.
[0098] Furthermore, the image processing device can also be applied to a bill transport device shown in FIG. 28(b) and a white line detection device for an automated guided vehicle (AGV) shown in FIG. 28(c).
[0099] The object of the paper money transport device shown in Fig. 28(b) is a paper money. The feature amount detected by the paper money transport device is used for correction processing of the image itself. That is, the paper money transport device shown in Fig. 28(b) recognizes the inclination of the paper money by edge detection, and performs skew correction using the recognized inclination.
[0100] The subject of the white line detection device for an automated guided vehicle shown in FIG. 28(c) is a white line. The feature value output by the white line detection device for an automated guided vehicle can be used to determine the moving direction of the automated guided vehicle. That is, the white line detection device for an automated guided vehicle recognizes the inclination of the white line area by edge detection, and determines the moving direction of the automated guided vehicle using the recognized inclination. In addition, the white line detection device for an automated guided vehicle can also perform a post-processing correction of the moving direction according to the position and orientation of the automated guided vehicle. For example, in the case of an automated guided vehicle, it is also possible to perform a process such as stopping the drive when a thickness different from a known thickness of a white line is detected. [Explanation of symbols]
[0101] 2. Lighting section 3. Imaging unit 12 Subject 13 Background section 20 Image processing section 100 Image processing device 101 Reading device 103 Image forming section [Prior art documents] [Patent documents]
[0102] [Patent Document 1] JP 2014-053739 A
Claims
1. an illumination unit that irradiates a subject with visible light and invisible light; an imaging unit that receives visible light and invisible light reflected by the subject and captures a visible image and an invisible image; A background portion provided in an imaging range of the imaging portion; an image processing unit that detects a feature amount of the subject or the background portion based on an image in which a difference in spectral reflectance characteristics between the subject and the background portion of the visible image and a difference in spectral reflectance characteristics between the subject and the background portion of the invisible image are larger in accordance with the spectral reflectance characteristics of the subject; A reading device comprising:
2. The illumination unit irradiates infrared light as the invisible light, The imaging unit captures an infrared image as the invisible image.
2. The reading device according to claim 1, wherein the reading device is a reading device for reading a signal.
3. the image processing unit detects features of the subject or the background from at least one of the visible image and the invisible image, and selects or integrates the features detected from each image; 3. The reading device according to claim 1 or 2.
4. the image processing unit compares a difference between the spectral reflectance characteristics of the background and the subject for invisible light and visible light as the feature amount of the visible image, and determines a component of visible light that has a largest difference from the invisible light; 4. The reading device according to claim 1, wherein the reading device is a reading device for reading a document.
5. The background portion has an invisible light low reflection portion that diffusely reflects visible light and reflects invisible light at a reflectance lower than that of visible light.
5. The reading device according to claim 1, wherein the reading device is a reading device for reading a document.
6. the image processing unit detects an edge of a subject as a feature amount of the visible image or the invisible image.
6. A reading device according to claim 3, wherein the reading device is a reading device for reading a document.
7. the image processing unit performs the edge integration by OR processing of the edges of the invisible image and the edges of the visible image.
7. The reading device according to claim 6,
8. the image processing unit selects the edge as the edge of the invisible image when the edge of the invisible image satisfies a criterion for detecting edge normality, and selects the edge of the visible image when the edge of the invisible image does not satisfy the criterion for detecting edge normality.
7. The reading device according to claim 6,
9. An illumination unit that irradiates a subject with visible light and invisible light; an imaging unit that receives visible light and invisible light reflected by the subject and captures a visible image and an invisible image; A background portion provided in an imaging range of the imaging portion; an image processing unit that detects a feature amount of the subject or the background portion from at least one of the visible image and the invisible image according to a spectral reflectance characteristic of the subject; Equipped with The image processing unit includes: Detecting features of the subject or the background from at least one of the visible image and the invisible image, and selecting or integrating the features detected from each image; detecting an edge of a subject as a feature of the visible image or the invisible image; the edge is selected as an edge of the invisible image if the edge of the invisible image satisfies a criterion for detecting an edge as being normal, and the edge is selected as an edge of the visible image if the edge of the invisible image does not satisfy the criterion for detecting an edge as being normal; A reading device comprising:
10. An illumination unit that irradiates a subject with visible light and invisible light; an imaging unit that receives visible light and invisible light reflected by the subject and captures a visible image and an invisible image; A background portion provided in an imaging range of the imaging portion; an image processing unit that detects a feature amount of the subject or the background portion from at least one of the visible image and the invisible image according to a spectral reflectance characteristic of the subject; Equipped with the background portion has an invisible light low reflection portion that diffusely reflects visible light and reflects invisible light at a reflectance lower than that of visible light, The image processing unit includes: detecting an edge of a subject as a feature of the visible image or the invisible image; the edge is selected as an edge of the invisible image if the edge of the invisible image satisfies a criterion for detecting an edge as being normal, and the edge is selected as an edge of the visible image if the edge of the invisible image does not satisfy the criterion for detecting an edge as being normal; A reading device comprising:
11. the image processing unit performs the edge integration by OR processing of the edge of the invisible image and the edge of the visible image when neither the edge of the invisible image nor the edge of the visible image meets the determination criterion for edge normal detection.
7. The reading device according to claim 6,
12. An illumination unit that irradiates a subject with visible light and invisible light; an imaging unit that receives visible light and invisible light reflected by the subject and captures a visible image and an invisible image; A background portion provided in an imaging range of the imaging portion; an image processing unit that detects a feature amount of the subject or the background portion from at least one of the visible image and the invisible image according to a spectral reflectance characteristic of the subject; Equipped with The image processing unit includes: Detecting features of the subject or the background from at least one of the visible image and the invisible image, and selecting or integrating the features detected from each image; detecting an edge of a subject as a feature of the visible image or the invisible image; When neither the edge of the invisible image nor the edge of the visible image satisfies the judgment criterion for edge normal detection, the edge integration is performed by ORing the edge of the invisible image and the edge of the visible image. A reading device comprising:
13. An illumination unit that irradiates a subject with visible light and invisible light; an imaging unit that receives visible light and invisible light reflected by the subject and captures a visible image and an invisible image; A background portion provided in an imaging range of the imaging portion; an image processing unit that detects a feature amount of the subject or the background portion from at least one of the visible image and the invisible image according to a spectral reflectance characteristic of the subject; Equipped with the background portion has an invisible light low reflection portion that diffusely reflects visible light and reflects invisible light at a reflectance lower than that of visible light, The image processing unit includes: detecting an edge of a subject as a feature of the visible image or the invisible image; When neither the edge of the invisible image nor the edge of the visible image satisfies the judgment criterion for edge normal detection, the edge integration is performed by ORing the edge of the invisible image and the edge of the visible image. A reading device comprising:
14. The image processing unit detects a size of the subject from the edge.
14. A reading device according to any one of claims 6 to 13, characterized in that
15. The image processing unit corrects the inclination and position of the subject based on the edge.
15. A reading device according to any one of claims 6 to 14.
16. The image processing unit extracts an object based on the edge.
16. A reading device according to any one of claims 6 to 15, characterized in that:
17. The image processing unit corrects at least one of the visible image and the invisible image.
17. A reading device according to any one of the preceding claims.
18. A reading device according to any one of claims 1 to 17, An image forming unit; An image processing device comprising:
19. an illumination unit that irradiates a subject with visible light and invisible light; an imaging unit that receives visible light and invisible light reflected by the subject and captures a visible image and an invisible image; A background portion provided in an imaging range of the imaging portion; A feature detection method for a reading device comprising: detecting a feature amount of the subject or the background portion based on an image in which a difference in spectral reflectance characteristics between the subject and the background portion of the visible image and a difference in spectral reflectance characteristics between the subject and the background portion of the invisible image are larger according to the spectral reflectance characteristics of the subject, A feature detection method comprising:
Citation Information
Patent Citations
Paper edge detector
JP2008017067A
Image reader
JP2009164805A
Image reading apparatus, image processing method, and program
JP2012239114A
Image reader and paper sheet processing device
JP2014053739A
