Image processing device, reading device, image forming device, and method

Abstract

To perform highly accurate skew correction by an invisible read image with a simple configuration by using a hardware configuration that corrects skew with image processing of a visible read image.SOLUTION: An image processing apparatus comprises: a first image processing device which processes an invisible image of a subject; a second image processing device which processes a visible image of the subject; and a control unit which controls the first image processing device and the second image processing device. The first image processing device includes a skew detection unit which performs processing of detecting skew of a subject image from the invisible image. The second image processing device includes a skew detection unit which performs processing of detecting skew of the subject image from the visible image, and a correction unit which corrects the skew of the subject image according to the detection result of the skew by the skew detection unit of the first image processing device and the detection result of the skew by the skew detection unit of the second image processing device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing apparatus, a reading apparatus, an image forming apparatus, and How to eat Regarding the law. [Background technology]

[0002] 2. Description of the Related Art Conventionally, an image correction technique is known in which edges of a document are detected from a read image, and the inclination and position of the document are corrected based on the detected edges.

[0003] Patent Document 1 discloses a technique for correcting the skew (also called tilt) of an original image read by a scanner. Patent Document 2 discloses a technique for reading an original image with RGB visible light and correcting the skew using the shadow formed at the boundary between the original surface and the background. Patent Document 3 discloses a method using a light source of a different wavelength (ultraviolet region). Summary of the Invention [Problem to be solved by the invention]

[0004] However, changing the entire image processing device to perform highly accurate skew correction by adding a near-infrared invisible image, compared to a hardware configuration that corrects skew by image processing of a general visible image such as RGB3, poses the problem of requiring a huge amount of labor and cost.

[0005] The present invention has been made in view of the above, and provides an image processing apparatus, a reading apparatus, an image forming apparatus, and a method for performing highly accurate skew correction using an invisible read image with a simple configuration, using a hardware configuration for correcting skew by image processing of a visible read image. How to eat The purpose is to provide the law. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the present invention No.An image processing device according to one embodiment includes a first image processing device that processes an invisible image of a subject, a second image processing device that processes a visible image of the subject, and a control unit that controls the first image processing device and the second image processing device, wherein the first image processing device includes a skew detection unit that performs processing to detect skew of the subject image from the invisible image, and the second image processing device includes a skew detection unit that performs processing to detect skew of the subject image from the visible image, and a correction unit that corrects skew of the subject image in accordance with a result of skew detection by the skew detection unit of the first image processing device or a result of skew detection by the skew detection unit of the second image processing device. the first image processing device has a register unit for setting parameters, and turns on and off a skew detection function of the first image processing device based on the setting of the register unit. It is characterized by: In addition, an image processing device according to a second embodiment of the present invention comprises a first image processing device that processes an invisible image of a subject, a second image processing device that processes a visible image of the subject, and a control unit that controls the first image processing device and the second image processing device, wherein the first image processing device comprises a skew detection unit that performs processing to detect skew of the subject image from the invisible image, and the second image processing device comprises a skew detection unit that performs processing to detect skew of the subject image from the visible image, and a correction unit that corrects the skew of the subject image in accordance with the skew detection result by the skew detection unit of the first image processing device or the skew detection result by the skew detection unit of the second image processing device, and wherein the first image processing device is detachable. An image processing device according to a third embodiment of the present invention includes a first image processing device that processes an invisible image of a subject, a second image processing device that processes a visible image of the subject, and a control unit that controls the first image processing device and the second image processing device, wherein the first image processing device includes an input interface unit that inputs the visible image and the invisible image of a first surface of the subject output by a reading means, a skew detection unit that processes the invisible image input from the input interface unit to detect skew in the subject image, and a second image processing unit that processes the visible image and the invisible image input from the input interface unit to detect skew in the second surface of the subject. the first image processing device has an output interface unit that outputs to an image data path of the first image processing device, and an output image selection unit that selects an image data path from the image data path of the second image processing device to be the output destination for each of the visible image and the invisible image, and the second image processing device has a skew detection unit that performs processing to detect skew in an object image from the visible image input from the image data path, and a correction unit that corrects the skew of the object image in accordance with the skew detection result by the skew detection unit of the first image processing device or the skew detection result by the skew detection unit of the second image processing device. [Effects of the Invention]

[0007] According to the present invention, it is possible to perform highly accurate skew correction using an invisible read image with a simple configuration by using a hardware configuration that corrects skew by image processing of a visible read image. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of hardware blocks of a reading device according to this embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an output data control unit of the first image processing device. [Figure 3] FIG. 3 is an explanatory diagram of a first setting method for the register unit. [Figure 4] FIG. 4 is an explanatory diagram of a second setting method for the register unit. [Figure 5] FIG. 5 is a diagram illustrating an example of the configuration of a hardware block of the NIR skew detection unit. [Figure 6] FIG. 6 is an image diagram showing how the memory read control unit reads image data from the memory while correcting the skew of the image data. [Figure 7] FIG. 7 is a diagram for explaining a method for detecting skew. [Figure 8] FIG. 8 is a diagram illustrating an example of an operation flow of the reading device. [Figure 9] FIG. 9 is a diagram illustrating an example of an operation flow of the second image processing device. [Figure 10] FIG. 10 is a diagram showing an example of a control flow of the CPU when performing skew correction. [Figure 11] FIG. 11 is a diagram illustrating a configuration of an example of an image forming apparatus. [Figure 12] FIG. 12 is a cross-sectional view illustrating an example of the structure of an image reading device. [Figure 13] FIG. 13 is a block diagram showing the electrical connections of the components that make up the image reading device. [Figure 14] FIG. 14 is a block diagram showing the functional configuration of the image processing unit. [Figure 15] FIG. 15 is a diagram showing differences in spectral reflectance characteristics depending on the medium involved in detecting the feature amount of the subject. [Figure 16] FIG. 16 is a diagram showing the difference in spectral reflectance characteristics between visible and invisible images depending on the type of paper. [Figure 17] FIG. 17 is a diagram showing an example of selection of visible components from which feature amounts are extracted. [Figure 18] FIG. 18 is a diagram showing an example of spectral reflectance characteristics when the background portion is a low invisible light reflectance portion. [Figure 19] FIG. 19 is a diagram illustrating an example of an invisible light low reflection portion. [Figure 20] FIG. 20 is a diagram showing information obtained from the edges of an object. [Figure 21] FIG. 21 is a diagram showing an example of an edge detection technique. [Figure 22] FIG. 22 is a diagram showing an example of a feature amount using an edge. [Figure 23] FIG. 23 is a diagram showing the selection of a linear regression formula. [Figure 24]FIG. 24 is a diagram showing an example of size detection (horizontal direction). [Figure 25] FIG. 25 is a block diagram of a functional configuration of an image processing unit according to the second embodiment. [Figure 26] FIG. 26 is a diagram illustrating the OR process of edges. [Figure 27] FIG. 27 is a diagram for explaining how edges appear in a visible image and an invisible image. [Figure 28] FIG. 28 is a diagram showing an example of determination of normal edge detection. [Figure 29] FIG. 29 is a diagram showing an example of a failure in the OR process of edges. [Figure 30] FIG. 30 is a diagram showing an example of a subject having a mixture of multiple characteristics. [Figure 31] FIG. 31 is a block diagram of a functional configuration of an image processing unit according to the third embodiment. [Figure 32] FIG. 32 is a flowchart showing the flow of processing in the image processing unit. [Figure 33] FIG. 33 is a diagram showing an example of correction of the tilt and position of a subject. [Figure 34] FIG. 34 is a diagram showing uses of invisible images. [Figure 35] FIG. 35 is a diagram showing an example of correction of the tilt and position of a subject and extraction. [Figure 36] FIG. 36 is a diagram illustrating an example of tilt correction. [Figure 37] FIG. 37 is a diagram showing a search for the right edge point. [Figure 38] FIG. 38 is a diagram showing a modified example of the image processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an image processing apparatus, a reading apparatus, an image forming apparatus, and a feature amount detection method will be described in detail with reference to the accompanying drawings.

[0010] (First embodiment) FIG. 1 is a diagram showing an example of the configuration of the hardware blocks of a reading device according to this embodiment. FIG. 1 shows the main components of the hardware blocks of the reading device, including a front scanner 310, a back scanner 320, a first image processing device 500, a second image processing device 600, a selector 330, a central processing unit (CPU) 340, a memory 350, a controller 360, a CPU 361, and a memory 362. Among these, the front scanner 310 and the back scanner 320 correspond to a "reading unit." The first image processing device 500, the second image processing device 600, the selector 330, the CPU 340, and the memory 350 correspond to the configuration of an "image processing device." The CPU 340, the memory 350, the controller 360, the CPU 361, and the memory 362 are hardware components typically included in reading devices, and these components can be utilized. The CPU 340 can access both the first image processing device 500 and the second image processing device 600 and control both the first image processing device 500 and the second image processing device 600. The memory 350 is a memory for image processing by the second image processing device 600. The controller 360, the CPU 361, and the memory 362 correspond to a main control unit provided in the reading device.

[0011] The surface scanner 310 is an image scanner that reads visible and invisible images from the surface of a document. Here, the document is an example of an "object." The surface scanner 310 has an imaging sensor that captures light reflected from the surface of the document when irradiated with light. The imaging sensor includes, for example, an RGB reading unit 311 (R (Red), G (Green), and B (Blue)) and an NIR (Near-infrared) reading unit 312. The RGB reading unit 311 reads visible images from the surface of the document. The NIR reading unit 312 reads invisible NIR images from the surface of the document. Note that the RGB and NIR regions of the reading unit of the imaging sensor are examples of visible and invisible reading units, respectively, and the visible and invisible wavelength ranges are not limited to RGB and NIR. The visible wavelength range may be at least one of the RGB wavelength ranges. Furthermore, the invisible wavelength range may be a wavelength range other than the visible range, such as ultraviolet light. In the following, as an example, a configuration using RGB and NIR wavelength regions as visible and invisible will be described.

[0012] The front side scanner 310 irradiates the surface of the document with light to read the image. In this example, the irradiated light is RGB visible light and NIR invisible light. The front side scanner 310 reads the reflected light of the light irradiated onto the surface of the document, and outputs read data for each of the three RGB plates (R, G, B) from the RGB plate reading unit 311, and outputs read data for one NIR plate from the NIR plate reading unit 312. The back side scanner 320 has an image sensor equipped with an RGB reading unit, reads the back side of the document, and outputs read data for each of the three RGB plates (R, G, B).

[0013] The image processing device of the reading device according to this embodiment has a first image processing device 500 and a second image processing device 600 that can communicate with the CPU 340, and detects skew using NIR version read data by the first image processing device 500. Here, skew is information that indicates the inclination of the document.

[0014] The first image processing device 500 has an interface for inputting four-plane (RGB+NIR) read data from the surface scanner 310, and includes an NIR skew detection unit 510 and an output data control unit 520. The NIR skew detection unit 510 detects skew of the original document using the NIR plane read data from the NIR plane reading unit 312, out of the four-plane (RGB+NIR) read data. The output data control unit 520 receives the four-plane (RGB+NIR) read data from the surface scanner 310 and performs output path control to output the image plane data in correspondence with the image data path of the existing image plane of the second image processing device 600.

[0015] The selector 330 is a selective output unit that selects either the output of the back scanner 320 or the output from the first image processing device 500 and outputs it to the second image processing device 600. For example, when skew detection by the first image processing device 500 is enabled, the selector 330 selects the output from the first image processing device 500, and when skew detection by the first image processing device 500 is not enabled, the selector 330 selects the output of the back scanner 320.

[0016] The first image processing device 500 may be configured so that the skew detection function by the NIR skew detection unit 510 can be switched ON and OFF by setting parameters or the like, or the first image processing device 500 may be configured so that it can be attached to and detached from the reading device. In the latter case, when the first image processing device 500 is detached, even if four-plane (RGB+NIR) read data is output from the front side scanner 310, three-plane RGB data is transferred to the second image processing device 600, and three-plane RGB data of the back side scanner 320 selected by the selector 330 is transferred to the second image processing device 600.

[0017] This makes it possible to apply a conventional configuration for detecting and correcting skew from scan data of three RGB colors to the second image processing device 600. When the first image processing device 500 detects skew, the second image processing device 600 corrects the skew from the scan data of three RGB colors based on the detection result of the skew detected by the first image processing device 500. When the first image processing device 500 does not detect skew, the second image processing device 600 detects skew from the shadow of the document from the scan data of the RGB colors and corrects the skew. Correcting the skew of the document corresponds to correcting the "subject image."

[0018] When the first image processing device 500 detects skew using the NIR skew detection unit 510, it notifies the CPU 340 of the detection result by an interrupt. Upon receiving the detection result, the CPU 340 acquires the skew result detected by the NIR skew detection unit 510 of the first image processing device 500.

[0019] In this embodiment, in order to perform highly accurate skew correction, the first image processing device 500 is provided with a function for skew detection using an invisible NIR version image. On the other hand, the second image processing device 600 is configured to perform skew correction using the skew detection result of the first image processing device 500. With this configuration, even when changing to a configuration that performs skew detection using an NIR version, it is possible to utilize the configuration of an existing image processing device that performs skew detection and correction using a visible RGB version image as the second image processing device 600, and therefore it is possible to achieve a change to highly accurate skew correction using an NIR version at low cost.

[0020] Fig. 2 is a diagram showing an example of the configuration of the output data control unit 520 of the first image processing device 500. Fig. 2 also shows part of the configuration of the second image processing device 600 so that the relationship between the interfaces of the first image processing device 500 and the second image processing device 600 can be seen.

[0021] The second image processing device 600 has a general configuration including an interface unit 610 that transfers three scan data sets, one for the front and one for the back of the document, that is, RGB scan data sets. In the interface unit 610, a front transfer path 611 corresponds to a path (image data path) that transfers the RGB scan data sets of the front and back of the document, and a back transfer path 612 corresponds to a path (image data path) that transfers the RGB scan data sets of the back of the document.

[0022] The output data control unit 520 outputs the read data of the three RGB plates on the front and the read data of the one NIR plate to the interface unit 610 and transfers them to the subsequent stage via each path. In this example, the output data control unit 520 inputs four plates, which are the RGB plates plus the NIR plate (X plate shown in the figure), from the input interface unit, and distributes the read data of the four plates to a front transfer path 611 and a back transfer path 612 of the interface unit 610 of the second image processing device 600 for output and transfer.

[0023] Specifically, the output data control unit 520 includes an input interface unit 521, an image processing unit 522, a register unit 530, an output image plane selection unit 540, and an output interface unit 550. The output data control unit 520 inputs four planes (RGB plane + X plane) via the input interface unit 521. The register unit 530 is a setting register in which the CPU 340 sets parameters. The output image plane selection unit 540 determines the image plane patterns to be output to the front transfer path 611 and the back transfer path of the interface unit 610 based on the parameters set in the register unit 530, and selects the output destination of each image plane. Each of the processed data of the four planes (RGB plane + X plane) that has undergone predetermined image processing in the image processing unit 522 is output to the front transfer path 611 and the back transfer path of the second image processing device 600 according to the path selection of the output image plane selection unit 540.

[0024] 2, the three plates of RGB on the front side are output to the front transfer path 611, and the X plate is output to the back transfer path 612, but this is not limiting. By setting the register unit 530, the X plate can be transferred to any RGB path on the back transfer path 612. For example, although an example has been shown in which all X plates are transferred to the RGB paths on the back transfer path 612, it is also possible to transfer three plates, the R plate, the X plate, and the B plate, to the RGB paths on the back transfer path 612. The plates to be output can be switched by the CPU setting the registers using software, so the settings can be changed at any timing.

[0025] The output data control unit 520 also transfers the first NIR read data to the NIR skew detection unit 510. The NIR skew detection unit 510 detects skew using the transferred first NIR read data. The transfer of the first NIR read data by the output data control unit 520 to the NIR skew detection unit 510 may be performed by setting the register unit 530. For example, when the skew detection function is set to ON in the register unit 530, the output data control unit 520 transfers the first NIR read data to the NIR skew detection unit 510. When the skew detection function is set to OFF in the register unit 530, the output data control unit 520 does not transfer the first NIR read data to the NIR skew detection unit 510, that is, does not perform skew detection using the NIR version.

[0026] (Register setting method) Next, a description will be given of a setting method for the register unit 530. First, a setting method (first setting method) for turning skew detection on and off in the first image processing device 500 will be described with reference to Fig. 3, and then a setting method (second setting method) for the output image plane selection unit 540 to select the image plane patterns to be output to the front transfer path 611 and the back transfer path of the interface unit 610 will be described with reference to Fig. 4.

[0027] 3 is an explanatory diagram of a first setting method for the register unit 530. By setting the register unit 530, it is possible to bypass operations such as image processing for skew detection in the first image processing device 500 even when the first image processing device 500 is attached. In the example shown in FIG. 3, when the default value of "through=0" is set, operations such as image processing for skew detection in the first image processing device 500 are performed, but when "through=1" is set, operations such as image processing for skew detection in the first image processing device 500 are bypassed, a path is selected, and the scanned data of each plate is transferred to the second image processing device 600. In addition, thresholds and the like used to determine skew detection in the first image processing device 500 are also set as parameters for skew detection.

[0028] Figure 4 is an explanatory diagram of a second setting method for the register unit 530. This is the setting of the register for selecting the output image plane. data_sel_u shown in Figure 4 is the setting for the front side, and data_sel_d is the setting for the back side. For example, if data_sel_u=0x00 / data_sel_d=0x01, the RGB plane is output to the front transfer path 611, and the X plane is output to the back transfer path.

[0029] (Configuration of NIR skew detection unit 510) 5 is a diagram showing an example of the configuration of hardware blocks of the NIR skew detection unit 510. In the example shown in FIG. 5, the NIR skew detection unit 510 has an edge detection unit 511, a skew registration detection unit 512, and a skew detection result storage unit 513.

[0030] Of the four input image planes, an edge detection unit 511 detects the boundary between the document surface and the background plane from the NIR plane, and a skew registration detection unit 512 identifies the edges of the document to detect skew. When the detection result is stored in a skew detection result storage unit 513, an interrupt signal indicating that skew has been detected is sent to the CPU at the same time. When the interrupt signal is sent, the CPU accesses the skew detection result storage unit 513 and acquires the detection result.

[0031] (Processing of the second image processing device 600) 1, the second image processing device 600 includes a first image processing unit 601, a memory write control unit 602, a memory read control unit 603, and a second image processing unit 604. The memory write control unit 602 and the memory read control unit 603 correspond to the skew detection unit and the correction unit of the second image processing device.

[0032] The first image processing unit 601 performs image processing on the image data read by the front and back scanners. For example, when the first image processing device 500 does not perform skew detection, the first image processing unit 601 detects skew due to the shadow of the document from the read data of three colors of RGB colors.

[0033] The data after image processing is stored in memory 350 via memory write control unit 602. Then, when reading out the stored data, memory read control unit 603 uses the rotary read function to read out image data from any address on memory 350. The CPU 340 controls the read address of memory 350, so that the memory read control unit 603 corrects skew in the image data. When the first image processing device 500 detects skew, the CPU 340 controls the read address of memory 350 based on the skew detection result, so that the memory read control unit 603 corrects skew in the image data.

[0034] The corrected data undergoes image processing in the second image processing unit 604, and is then stored in the memory 362 under the control of the CPU 340.

[0035] 6 is a conceptual diagram of the memory read control unit 603 reading image data from the memory 350 while correcting the skew. The control is performed by providing a point at which the address is switched when reading according to the skew grasped in the image processing unit at the previous stage.

[0036] FIG. 7 illustrates a skew detection method. NIR light, which is invisible light, responds only to specific wavelengths. To take advantage of this characteristic, a material that absorbs NIR light is used for the background plate. The reflected light is then received and captured. As a result, when the first image processing device 500 performs skew detection, the boundary L between the background plate G10 and the original G11 can be reliably and accurately obtained from the NIR image G1, as shown in FIG. 7. Furthermore, when the first image processing device 500 does not perform skew detection, the second image processing device 600 can employ a conventional skew detection method. In this example, skew is detected by detecting the shadow between the original and the background plate based on the images of each of the three RGB plates.

[0037] (Reader operation) (When skew detection is performed by the first image processing device 500) 8 is a diagram showing an example of the operation flow of the reading device. First, before the reading operation by the scanner, the reading device sets parameters for skew detection in the first image processing device 500 (S11). Specifically, the reading device sets parameters such as turning on skew detection by the first image processing device 500 and setting a threshold value used to determine skew detection.

[0038] Next, the reading device starts the scanner and performs the reading operation of the document (S12).

[0039] Next, since the skew detection is set to ON, the reading device starts skew detection processing using the NIR version of the read data in the first image processing device 500 (S13).

[0040] In addition, in parallel with the skew detection process, the reading device outputs the scanned data of the document from the first image processing device 500 to the second image processing device 600 (S14). Note that the order of the skew detection process and the output of the scanned data to the second image processing device 600 may be reversed.

[0041] After starting S13, when the reading device detects the skew of the document, it determines whether the detected skew is equal to or greater than the threshold set in S11 (S15), and if it is equal to or less than the threshold (S15: No), that is, if the inclination of the document is small, the skew correction process ends.

[0042] On the other hand, if the detected skew is equal to or greater than the threshold set in S11 (S15: Yes), that is, if the document is significantly tilted, the reading device stores the skew detection result (S16) and notifies the subsequent stage of a skew detection interrupt (S17).

[0043] In this way, the first image processing device 500 performs the necessary image processing in parallel with the skew detection processing, and outputs the image data to the downstream second image processing device 600. This allows for a smooth transition to the skew correction processing in the second image processing device 600.

[0044] Next, the processing of the second image processing device 600 at the latter stage will be described.

[0045] 9 is a diagram showing an example of the operation flow of the second image processing device 600. This is the operation flow of the second image processing device 600 that receives image data from the first image processing device 500.

[0046] First, the first image processing unit 601 of the second image processing device 600 performs image processing on the image data (S21).

[0047] Next, the memory write control unit 602 of the second image processing device 600 stores the processed data in the memory 350 (S22).

[0048] Next, based on the detection result of the skew detection in the first image processing device 500, it is determined whether the second image processing device 600 should perform skew correction of the image data (S23).

[0049] If skew correction is required (S23: Yes), the memory read control unit 603 of the second image processing device 600 reads the image data stored in the memory 350 so that the skew is corrected (S24).

[0050] If skew correction is not to be performed (S23: No), the memory read control unit 603 reads the image data from the memory 350 as usual (S25).

[0051] Next, the second image processing unit 604 performs image processing on the image data read by the memory read control unit 603 (S26).

[0052] Then, the image data after the image processing is output to the controller 360 (S27).

[0053] 10 is a diagram showing an example of a control flow of the CPU 340 when performing skew correction. First, before starting the scanner operation, the CPU 340 sets parameters for skew detection in the first image processing device 500 (S31). Specifically, the CPU 340 sets skew detection ON and thresholds used to determine skew detection in the register unit 530 of the first image processing device 500.

[0054] Next, the CPU 340 starts scanner read transfer control and waits for an interrupt indicating completion of transfer of image data to the memory 350 (S32).

[0055] Thereafter, the CPU 340 determines whether a skew detection interrupt has been notified (S33). After starting scanner read transfer control, if the first image processing device 500 detects skew, a skew detection interrupt is notified. Therefore, when the CPU 340 detects this notification (S33: Yes), it reads the skew detection result from the skew detection result storage unit 513 of the first image processing device 500 and stores it as information for skew correction (S34).

[0056] After S35, or when no skew detection interrupt notification is detected (S33: No), the CPU 340 similarly determines whether a write completion interrupt to the memory 350 is detected (S35).

[0057] If the CPU 340 does not detect a write completion interrupt (S35: No), it repeats the procedure from S32.

[0058] When the CPU 340 detects a write completion interrupt (S35: Yes), it performs read address control according to the skew detection result (S36), that is, it shifts to skew correction control.

[0059] During correction, the CPU 340 controls the read address from the memory 350 based on the skew detection result acquired from the skew detection result storage unit 513 of the first image processing device 500 .

[0060] Then, the CPU 340 determines whether a read completion interrupt for the memory 350 has occurred (S37), and if the read completion interrupt has not occurred (S37: No), the CPU 340 repeats the process of S36, and if the read completion interrupt has occurred (S37: Yes), the CPU 340 ends the process of S36. The degree of skew correction can be dynamically changed based on the skew detection result.

[0061] (Example of application to image forming devices) Fig. 11 is a diagram showing an example of the configuration of an image forming apparatus 100 according to the first embodiment. In Fig. 11, image forming apparatus 100, which is an image processing apparatus, is generally called a multifunction peripheral that has at least two functions selected from a copy function, a printer function, a scanner function, and a facsimile function.

[0062] The image forming apparatus 100 has an image reading device 101, which is a reading device, and an ADF (Automatic Document Feeder) 102, and an image forming unit 103 below them. These are controlled by an internal controller board (a control board including a controller 360 (see FIG. 1), a CPU 361 (see FIG. 1), and a memory 362 (see FIG. 1)). To explain the internal configuration of the image forming unit 103, the external cover has been removed to show the internal configuration.

[0063] The ADF 102 is an original support unit that positions an original document whose image is to be read at a reading position. The ADF 102 automatically transports an original document placed on a mounting table to the reading position. The image reading device 101 reads the original document transported by the ADF 102 at a predetermined reading position. The image reading device 101 also has a contact glass on its upper surface that serves as an original support unit on which an original document is placed, and reads the original document placed on the contact glass at the reading position. Specifically, the image reading device 101 is a scanner that includes a light source, an optical system, and a solid-state image sensor such as a CMOS image sensor, and reads reflected light from an original document illuminated by the light source via the optical system with the solid-state image sensor.

[0064] The image forming unit 103 has a manual feed roller 104 for manually feeding recording paper, and a recording paper supply unit 107 for supplying recording paper. The recording paper supply unit 107 has a mechanism for feeding recording paper from a multi-stage recording paper feed cassette 107a. The supplied recording paper is sent to a secondary transfer belt 112 via registration rollers 108.

[0065] The toner image on the intermediate transfer belt 113 is transferred to the recording paper conveyed on the secondary transfer belt 112 in the transfer section 114 .

[0066] The image forming unit 103 also includes an optical writing device 109, tandem imaging units (Y, M, C, K) 105, an intermediate transfer belt 113, and the secondary transfer belt 112. Through an image creation process by the imaging units 105, the image written by the optical writing device 109 is formed as a toner image on the intermediate transfer belt 113.

[0067] Specifically, the imaging unit (Y, M, C, K) 105 has four rotatable photosensitive drums (Y, M, C, K), and is provided with imaging elements 106 around each photosensitive drum, each including a charging roller, a developing unit, a primary transfer roller, a cleaner unit, and a static eliminator. The imaging elements 106 function for each photosensitive drum, and the image on the photosensitive drum is transferred onto the intermediate transfer belt 113 by each primary transfer roller.

[0068] Intermediate transfer belt 113 is stretched across a drive roller and a driven roller in the nip between each photosensitive drum and each primary transfer roller. The toner image that has been primarily transferred onto intermediate transfer belt 113 is secondarily transferred onto recording paper on secondary transfer belt 112 by a secondary transfer device as intermediate transfer belt 113 moves. The recording paper is then transported to fixing device 110 as secondary transfer belt 112 moves, where the toner image is fixed onto the recording paper as a color image. The recording paper is then ejected onto an ejection tray outside the machine. In the case of double-sided printing, the recording paper is turned over by a reversing mechanism 111, and the inverted recording paper is sent onto secondary transfer belt 112.

[0069] The image forming unit 103 is not limited to one that forms images by the electrophotographic method as described above, but may also be one that forms images by an inkjet method.

[0070] Next, the image reading device 101 will be described. Fig. 12 is a cross-sectional view showing an example of the structure of an image reading device 101. As shown in Fig. 12, the image reading device 101 has, within a main body 11, a sensor board 10 equipped with an imaging unit 22 that is a solid-state imaging element, a lens unit 8, a first carriage 6, and a second carriage 7. The first carriage 6 has a light source 2 that is an LED (Light Emitting Diode) and a mirror 3. The second carriage 7 has mirrors 4 and 5. The image reading device 101 also has a contact glass 1 on its upper surface.

[0071] The light source 2 is configured with a visible light source and an invisible light source. Here, invisible light refers to light with a wavelength of 380 nm or less or 750 nm or more. In other words, the light source 2 is an illumination unit that irradiates the subject and background 13 with visible light and invisible light (for example, near-infrared (NIR) light).

[0072] Furthermore, the image reading device 101 has a background unit 13, which is a reference white board, on the upper surface. More specifically, the background unit 13 is provided on the opposite side of the subject from the light source 2, which is an illumination unit, within the imaging range of the imaging unit 22.

[0073] In the reading operation, the image reading device 101 irradiates light upward from the light source 2 while moving the first carriage 6 and the second carriage 7 from a standby position (home position) in the sub-scanning direction (direction A). Then, the first carriage 6 and the second carriage 7 form an image on the imaging unit 22 via the lens unit 8, using the light reflected from the original 12, which is the subject.

[0074] Furthermore, when the power is turned on, the image reading device 101 sets a reference by reading the light reflected from the reference white plate 13. That is, the image reading device 101 moves the first carriage 6 to directly below the reference white plate 13, turns on the light source 2, and forms an image of the light reflected from the reference white plate 13 on the imaging unit 22, thereby performing gain adjustment.

[0075] The imaging unit 22 is capable of capturing images in both visible and invisible wavelength ranges. Pixels that convert the amount of incident light into an electrical signal are arranged in the imaging unit 22. The pixels are arranged in a matrix, and the electrical signals obtained from each pixel are transferred (pixel readout signals) to the downstream signal processing unit 222 (see FIG. 13) at regular intervals and in a predetermined order. A color filter that transmits only light of a specific wavelength is arranged on each pixel. In the imaging unit 22 of this embodiment, each signal obtained from a group of pixels having the same color filter arranged thereon is referred to as a channel. Hereinafter, an image captured by the imaging unit 22 by irradiating it with visible light will be referred to as a visible image, and an image captured by the imaging unit 22 by irradiating it with invisible light such as near-infrared light will be referred to as an invisible image.

[0076] Although an image reading device of a reduction optical system is applied as the image reading device 101 of this embodiment, the present invention is not limited to this, and an equal magnification optical system (contact optical system: CIS type) may also be used.

[0077] Fig. 13 is a block diagram showing the electrical connections of the components constituting the image reading device 101. As shown in Fig. 13, the image reading device 101 includes an image processing unit 20, a control unit 23, and a light source driving unit 24 in addition to the imaging unit 22 and light source 2 described above. The control unit 23 controls the imaging unit 22, the image processing unit 20, and the light source driving unit 24. The light source driving unit 24 drives the light source 2 under the control of the control unit 23. Here, the image processing unit 20 mainly corresponds to the first image processing device 500 (see Fig. 1) and the second image processing device 600 (see Fig. 1). The control unit 23 includes a CPU 340 (see Fig. 1) and a memory 350 (see Fig. 1). In the following, for ease of understanding, specific methods of skew detection and skew correction will be described as feature amount detection processing performed by the image processing unit 20 without distinguishing between the first image processing device 500 (see FIG. 1) and the second image processing device 600 (see FIG. 1), but as already explained, it is assumed that the processing is selectively performed by the first image processing device 500 (see FIG. 1) and the second image processing device 600 (see FIG. 1) under the control of the CPU 340 (see FIG. 1). Here, the explanation will be omitted to avoid repetition.

[0078] The imaging unit 22 is a sensor for a reduction optical system, such as a CMOS image sensor, etc. The imaging unit 22 corresponds to a front scanner 310 or a back scanner 320 including a pixel unit 221, a signal processing unit 222, etc.

[0079] In this embodiment, the imaging unit 22 is described using an example of a four-line configuration, but the configuration is not limited to four lines. Furthermore, the circuit configuration subsequent to the pixel unit 221 is not limited to the configuration shown in the figure.

[0080] The pixel unit 221 has four lines of pixel groups in which a plurality of pixel circuits constituting pixels are arranged in a matrix. The signal processing unit 222 processes signals output from the pixel unit 221 as necessary and transfers the signals to the image processing unit 20 arranged downstream.

[0081] The image processing unit 20 executes various image processes according to the intended use of the image data.

[0082] 14 is a block diagram showing the functional configuration of the image processing unit 20. As shown in FIG.

[0083] The feature amount detection unit 201 detects the feature amount of the original 12 as the subject from the visible or invisible image obtained by the imaging unit 22 .

[0084] FIG. 15 is a diagram showing differences in spectral reflectance characteristics depending on the medium used to detect the feature amount of a subject. When the imaging unit 22 reads reflected light from the subject, i.e., the document 12, the background 13 and the subject, i.e., the document 12, generally have different spectral reflectance characteristics. In the example shown in FIG. 15, the background 13 slopes downward to the right, while the subject, i.e., the document 12, slopes upward to the right. In other words, images with different characteristics are obtained using visible light and invisible light. For this reason, the feature amount detection unit 201 pre-sets the image to be detected as either visible or invisible, depending on the type of subject, i.e., the document 12, and the background 13. This makes it easier for the feature amount detection unit 201 to obtain the desired feature amount.

[0085] Here, Fig. 16 is a diagram showing differences in the spectral reflectance characteristics between visible images and invisible images depending on the paper type. For example, according to the example shown in Fig. 16, when the visible image and invisible image of paper type A are compared, it is found that the invisible image has a larger difference in spectral reflectance characteristics from the background portion 13. Therefore, in the case of paper type A, the feature amount detection unit 201 can set the invisible image as the feature amount detection target. On the other hand, when the visible image and invisible image of paper type B are compared, it is found that the visible image has a larger difference in spectral reflectance characteristics from the background portion 13. Therefore, in the case of paper type B, the feature amount detection unit 201 can set the visible image as the feature amount detection target.

[0086] Here, the selection of visible components from which feature amounts are extracted will be described.

[0087] Figure 17 shows an example of selecting visible components from which feature amounts are extracted. The light that is actually irradiated has a wide wavelength range, but for simplicity, the representative wavelength of each component is shown by a dotted line in Figure 17. Also, as an example, near-infrared light is used as invisible light.

[0088] 17, in the near-infrared light component, the component with the greatest difference in reflectance (arrow X) between the subject document 12 and the background portion 13 is the B component. Therefore, by using this B component, the feature amount detection unit 201 can differentiate between the feature amounts of the subject document 12 and the background portion 13.

[0089] That is, the feature detection unit 201 compares the difference in the spectral reflectance characteristics of the background 13 and the subject document 12 for invisible light and visible light, and determines the feature of the visible image to include the component of visible light that differs most from invisible light. Generally, the feature of the G component, which has a wide wavelength range, is used from the visible image. However, in the example shown in Figure 17, when feature amounts in the visible and infrared ranges are used, the difference in the spectral reflectance characteristics of the document and background board becomes large for the B component and the infrared component, making edge detection easier.

[0090] The feature amount detection unit 201 is not limited to extracting only the B component as the feature amount to be used for the visible components, but may also include some of the RGB components, such as the value of the largest component, for example.

[0091] In addition, when there is variation in the spectral reflectance characteristics of the subject document 12, the feature detection unit 201 may determine the visible components to be selected for feature extraction based on measurements taken from a representative subject document 12, or on the average of the measurement results.

[0092] Here, a case where the background portion 13 is a low invisible light reflectance portion will be described.

[0093] 18 is a diagram showing an example of spectral reflectance characteristics when background portion 13 is a low invisible light reflectance portion. As shown in FIG. 18, background portion 13 may be a low invisible light reflectance portion that diffusely reflects visible light and reflects invisible light at a lower reflectance than visible light. This results in a significant difference in the spectral reflectance of background portion 13 between the visible image and the invisible image, which in turn results in a difference in the spectral reflectance between document 12, the subject, and background portion 13, making it easier for feature detection unit 201 to extract targeted features.

[0094] 19 is a diagram showing an example of the low invisible light reflection portion. The low invisible light reflection portion may be provided as the entire background portion 13, or as part or a pattern of the background portion 13, as shown in FIG.

[0095] In this way, the background portion 13 has a low invisible light reflectivity portion that diffusely reflects visible light and reflects invisible light at a lower reflectivity than visible light, which makes it possible to produce a more significant difference in the background readings between the visible image and the invisible image, thereby enabling robust edge detection.

[0096] Next, a case where the feature amount detecting unit 201 extracts the edge of the document 12, which is the subject, as a feature amount will be described.

[0097] Here, FIG. 20 is a diagram showing information obtained from the edges of a subject. As shown in FIG. 20, an edge refers to the boundary between the subject, namely, original document 12, and background portion 13. By detecting such an edge, it is possible to recognize the position, inclination, size, etc. of the subject, namely, original document 12, as shown in FIG. 20. Furthermore, from the position and inclination of the subject, namely, original document 12, it is also possible to perform image correction according to the position and inclination of the subject, namely, original document 12, in subsequent processing.

[0098] FIG. 21 is a diagram showing an example of an edge detection technique. As shown in FIG. 21(a), one example of an edge detection method is to apply a first-order differential filter to the entire image and binarize each pixel based on whether it exceeds a predetermined threshold. In this case, depending on the threshold, a horizontal edge may appear as several consecutive vertical pixels (and vice versa). This is mainly because the edge is blurred due to the MTF characteristics of the optical system. Therefore, as shown in FIG. 21(b), a method is available in which the center of a series of consecutive pixels is selected as a representative edge pixel for purposes such as calculating the regression line equation and detecting size, as described below (part a in FIG. 21(b)).

[0099] FIG. 22 is a diagram showing an example of a feature quantity using an edge. The feature quantity does not have to be the edge itself extracted from the image, but may be something that uses the edge. Examples include a regression line equation calculated from the extracted edge point group using the least squares method or the area inside the edge (a set of positions), as shown in FIG. 22. Regarding the regression line equation, one method is to derive a single linear equation from all edge information for each side, but another method is to divide into multiple areas, calculate linear equations, and select or combine representative ones. In this case, methods for deriving the final linear equation include a line whose slope is the median, or an average value of each linear equation.

[0100] Fig. 23 is a diagram showing the selection of a linear equation in the regression line equation. By dividing into multiple regions, calculating linear equations, and selecting or integrating representative ones, as shown in Fig. 23, it is possible to correctly recognize the inclination of the subject document 12 even if the subject document 12 has damage such as missing edges.

[0101] As in the above process, the feature amount detection unit 201 extracts the edges of the document 12, which is the subject, as feature amounts, and thereby can detect the area of the document 12, which is the subject.

[0102] Next, the size detection of the document 12, which is the subject, will be described. Here, Fig. 24 is a diagram showing an example of size detection (horizontal direction). As shown in Fig. 24, for a representative position in the vertical direction of an image, the distance between the left edge and the right edge of the subject, the document 12, is calculated, and the horizontal size can be calculated from the median of these distances and a separately calculated tilt angle. The vertical size can also be calculated in a similar manner.

[0103] The size information detected in this way can be used for error detection, image correction processing (described later), etc. For example, when scanning with a multifunction device, if a document size different from the one set by the user in advance is detected, the device can notify the user to set a document of the correct size.

[0104] As described above, according to this embodiment, by detecting the feature amounts of the subject, namely the original 12 or the background 13, from at least one of the visible image and the invisible image, it is possible to obtain information from the invisible image that cannot be obtained from the visible image, thereby enabling stable edge detection between the original and the background regardless of the type of original.

[0105] Furthermore, the imaging unit 22 receives visible light and invisible light reflected by the document 12, which is the subject, and captures a visible image and an invisible image, thereby enabling image reading with a simple configuration.

[0106] Furthermore, since the invisible light and the invisible image are infrared light and an infrared image, the image can be read with a simple configuration.

[0107] In addition to the above-mentioned method, the NIR version may also be sent to a second image processing device and transferred directly to memory, so that processing using the NIR version is performed under the control of the CPU.

[0108] The first embodiment has the effect of enabling highly accurate skew correction using an invisible read image with a simple configuration by using a hardware configuration that corrects skew by image processing of a visible read image.

[0109] (Second embodiment) Next, a second embodiment will be described. The second embodiment differs from the first embodiment in that feature amounts are extracted from both visible and invisible images and then automatically selected or integrated. In the following description of the second embodiment, the same parts as in the first embodiment will be omitted, and only the parts that differ from the first embodiment will be described.

[0110] 25 is a block diagram showing the functional configuration of an image processing unit according to the second embodiment. As shown in FIG. 25, the image processing unit 20 includes a feature amount detection unit 201 and a feature amount selection / integration unit 202.

[0111] As described above, the feature detection unit 201 detects the feature of the subject, the original 12 or the background 13, detected from at least one of the visible image and the invisible image obtained by the imaging unit 22.

[0112] The feature selection and integration unit 202 selects or integrates the features detected from each image based on the features of the subject, i.e., the original 12 or the background 13, detected from at least one of the visible image and the invisible image by the feature detection unit 201.

[0113] More specifically, the feature selection and integration unit 202 automatically performs the selection process already described above. As a result, even for a document 12 that is a subject for which the target feature cannot be extracted from a visible image or an invisible image alone, it is possible to obtain the target feature by combining and using them.

[0114] 26 is a diagram illustrating the OR process of edges in the feature amount selection and integration unit 202. The feature amount selection and integration unit 202 of this embodiment performs OR process of edges when extracting edges of the document 12, which is the subject, as feature amounts.

[0115] By taking an OR operation on each edge extracted from the invisible image and the visible image, it is possible to complement areas where edges cannot be extracted from one image with the other image. For example, as shown in Figure 26, in the case of a gradation document, it is easy to extract edges in the black areas of the document in the visible image and difficult to extract edges in the white areas, but the opposite is true in the invisible image.

[0116] Therefore, the feature selection and integration unit 202 combines the edges of the black areas in the visible image with the edges of the white areas in the invisible image to extract the edges of the entire document that could not be obtained from either image alone.

[0117] In this way, the feature selection and integration unit 202 integrates the edges of the invisible image and the visible image by OR processing, and since there are locations where edges can be detected in either the visible image or the invisible image, it is possible to detect edges between the subject, the original 12, and the background 13 in more locations.

[0118] Next, the priority given to edges in invisible images will be described. Here, Figure 27 is a diagram explaining how edges appear in visible and invisible images. As shown in Figure 27(a), the shadow of the subject, original 12, may appear in the background of the visible image, and depending on the shape of the shadow, the edge may not be extracted as a straight line, which may affect the accuracy of detecting the inclination of the subject, original 12. On the other hand, in areas of the visible image where there is no shadow, if the subject, original 12, is white, there is a high possibility that edge extraction itself may not be possible.

[0119] As shown in Figure 27(b), in an invisible image, it is easier to extract the edge between the white subject (document 12) and the background 13, especially when the background 13 is a low-reflectance area for invisible light. Even in an invisible image, a shadow of the subject (document 12) may appear. However, since the shadow is darker than the background, if edge detection is performed using a first-order differential filter that detects edges from "dark to light," for example, it is possible to extract the edge between the shadow and the document, rather than the edge between the shadow and the background. Alternatively, even if two types of first-order differential filters are used to detect edges from "dark to light" and "light to dark," it is not necessary to extract the edge between the shadow and the background 13 if the brightness is close to that of the background 13. Therefore, it is more likely that the edge between the subject (document 12) and the background 13 can be detected more accurately in an invisible image than in a visible image. Therefore, it is recommended to use a visible image only when the edge cannot be detected correctly in an invisible image.

[0120] Here, Fig. 28 is a diagram showing an example of determining whether an edge has been detected normally. As a criterion for determining whether an edge has been "detected normally," for example, when the obtained edge point group is regressed with a straight line, as shown in Fig. 28, there are methods for determining whether the error by the least squares method is within a threshold value, whether the inclination angle of the straight line is within a threshold value, etc. Furthermore, when selecting and integrating from the above-mentioned multiple straight line equations, there are methods for determining whether the number of straight line equations determined to be normal is equal to or greater than a threshold value.

[0121] In this way, when the feature selection and integration unit 202 selects an edge, if the edge of the invisible image can be detected correctly, it is the edge of the invisible image, and if the edge of the invisible image cannot be detected correctly, it is the edge of the visible image. In this case, it is more likely that the edge is easier to detect in the invisible image, and detection accuracy can be improved.

[0122] Next, a case where edges cannot be detected correctly in either the visible image or the invisible image will be described.

[0123] FIG. 29 is a diagram showing an example of a failed OR process for edges. When OR process is performed on a visible image and an invisible image, there is a possibility that an unintended edge may be extracted. For example, as shown in FIG. 29, if an edge is extracted between the shadow of the subject, document 12, and the background 13 in the visible image, the shadow of the subject, document 12, will remain when OR process is performed, which will affect the calculation of the document tilt, etc. However, as mentioned above, OR process has the advantage of increasing the number of edge detection locations, so the feature detection unit 201 performs OR process only when edges cannot be detected correctly in either the visible image or the invisible image.

[0124] In this way, the feature selection and integration unit 202 performs OR processing of the invisible image edge and the visible image edge when it is unable to detect the edge of either the invisible image or the visible image correctly. The edges may appear differently in the visible image and the invisible image due to factors such as the shadow of the document. Therefore, OR processing is performed only when it is unable to detect the edges correctly in each image.

[0125] Here, Fig. 30 is a diagram showing an example of an object having a mixture of multiple characteristics. As shown in Fig. 30, for example, in the case of a document 12 which is an object having a mixture of multiple characteristics, it is possible to extract the lower part of the document 12 which is the object from the visible image, and extract the upper part of the document 12 which is the object from the invisible image.

[0126] As described above, according to this embodiment, the feature amounts of the subject, namely the document 12 or the background 13, are detected from at least one of the visible image and the invisible image, and the feature amounts detected from each image are selected or integrated. This makes it possible to automatically select feature amounts from either the visible image or the invisible image, or to integrate feature amounts from both.

[0127] (Third embodiment) Next, a third embodiment will be described. The third embodiment differs from the first and second embodiments in that it includes an image correction unit that performs image correction of the subject. In the following description of the third embodiment, the description of the same parts as the first and second embodiments will be omitted, and only the parts that differ from the first and second embodiments will be described.

[0128] Here, Fig. 31 is a block diagram showing the functional configuration of an image processing unit according to the third embodiment, and Fig. 32 is a flowchart showing the processing flow in the image processing unit. As shown in Fig. 31, the image processing unit 20 includes an image correction unit 203 in addition to a feature detection unit 201 and a feature selection / integration unit 202.

[0129] As shown in FIG. 32, the feature detection unit 201 detects the feature of the subject, the original 12 or the background 13, detected from at least one of the visible image and the invisible image obtained by the imaging unit 22 (step S1).

[0130] As shown in FIG. 32, the feature selection / integration unit 202 selects or integrates the features detected from each image based on the features of the subject, i.e., the document 12 or the background 13, detected from at least one of the visible image and the invisible image by the feature detection unit 201 (step S2).

[0131] 32, the image correction unit 203 performs image correction on each of the visible image and the invisible image using the integration result from the feature selection / integration unit 202 (step S3). An example of image correction will be described later.

[0132] 33 is a diagram showing an example of correcting the tilt and position of a subject. In the example shown in Fig. 33, the image corrector 203 corrects the tilt and position of the subject, i.e., the original 12, using the feature values detected by the feature selector / integrator 202.

[0133] To correct the inclination of the subject, i.e., the original 12, the image correction unit 203 uses a method in which, as described above, the inclination is calculated when the group of edge points extracted from each side of the subject, i.e., the original 12, is regressed in a straight line, and the entire image is rotated based on this inclination.

[0134] To correct the position of the subject document 12, the image corrector 203 finds the intersection point of the regression lines of the edge point groups on the top and left sides of the subject document 12 and moves that point to the origin.

[0135] FIG. 34 shows the use of invisible images. Correcting an image based on the integration results of the feature selection and integration unit 202 can improve the legibility of pictures and characters in the document area. In addition, the reflectance of invisible components differs significantly from that of visible components depending on the coloring material, and as a result, it is possible to cause color loss as shown in FIG. 34. For this reason, it is conceivable to use this color loss to perform OCR processing or the like in a later stage. Therefore, correcting not only visible images but also invisible images has the advantage of contributing to improved OCR accuracy.

[0136] In this way, by correcting the inclination and position of the subject document 12 based on the edge detection results, the subject document 12 can be corrected to be easier to see. In addition, there is a possibility that OCR accuracy can be improved.

[0137] Here, Fig. 35 is a diagram showing an example of correction of the tilt and position of the subject and cut-out. By combining the above-mentioned tilt and position corrections, the image correction unit 203 cuts out the area of the subject, the original 12, at just the right size, as shown in Fig. 35. Even if the feature amount cannot be detected well and the tilt or position cannot be corrected, the cut-out itself is possible, although it will not be the right size.

[0138] FIG. 36 is a diagram showing an example of tilt correction. When image processing is performed by hardware, tilt correction is not possible. For processing speed reasons, it is necessary to replace consecutive pixels in an image in groups of at least a minimum width, as shown in FIG. 36. However, if the tilt is too large, correction becomes difficult, which is a problem. In such cases, it may be desirable to remove as much of the background portion 13 as possible, even though tilt correction is not possible.

[0139] In this case, the image correction unit 203 may perform processing such as, for example, if the rightmost edge point is identified, removing the area to the right of the edge point from the image since the area is outside the area of the original 12, which is the subject.

[0140] FIG. 37 is a diagram showing how to search for the right edge point. Even if edges can only be extracted from a partial region of the image due to memory limitations, it is sufficient if vertical size information can be obtained by other means, such as sensor information during document transport. In this case, as shown in FIG. 37, the image correction unit 203 predicts the right edge point using inclination information from the edge pixels in that region. Note that this is not limited to the right edge, but also applies to the top, left, and bottom edges.

[0141] The image correction unit 203 can remove unnecessary areas of the background portion 13 by cutting out the area of the subject, which is the original document 12. As a result, in the image forming device 100, such as a multifunction peripheral, it is possible to obtain effects such as reducing the user's effort in inputting the size of the subject, which is the original document 12, improving the appearance of the image, reducing the storage area for saving the image, reducing the size of the recording paper when copying the image, and reducing the amount of ink and toner consumed.

[0142] In this way, by automatically detecting and cutting out the size of the subject document 12 from the image based on the edges, the user can save time by inputting the document size (especially non-standard sizes). This also has the effect of improving the appearance of the image, reducing the storage area required for saving the image, reducing the size of the recording paper when copying the image, and reducing the amount of ink and toner consumed.

[0143] As described above, according to this embodiment, the image correction unit 203 corrects at least one of the visible image and the invisible image, thereby making it possible to improve the visibility of the image.

[0144] In the above embodiment, the image processing device of the present invention is described as being applied to a multifunction device having at least two of the functions of a copy function, a printer function, a scanner function, and a facsimile function, but it can be applied to any image forming device such as a copier, printer, scanner device, or facsimile device.

[0145] In the above embodiments, the image reading device 101 of the image forming device 100 is used as the image processing device, but the present invention is not limited to this. The definition of an image processing device is that it can acquire a reading level, even if it does not read an image, such as a line sensor with a 1x magnification optical system (contact optical system: CIS method) shown in Figure 38(a). The device shown in Figure 38(a) reads information on multiple lines by moving the line sensor or the original.

[0146] Furthermore, the image processing device can also be applied to a banknote transport device shown in FIG. 38(b) and a white line detection device for an automated guided vehicle (AGV) shown in FIG. 38(c).

[0147] The object of the banknote transport device shown in Figure 38(b) is a banknote. The feature values detected by the banknote transport device are used for correction processing of the image itself. That is, the banknote transport device shown in Figure 38(b) recognizes the inclination of the banknote by edge detection and performs skew correction using the recognized inclination.

[0148] The subject of the white line detection device for an automated guided vehicle shown in Figure 38(c) is a white line. The feature values output by the white line detection device for an automated guided vehicle can be used to determine the movement direction of the automated guided vehicle. That is, the white line detection device for an automated guided vehicle recognizes the inclination of the white line area by edge detection and determines the movement direction of the automated guided vehicle using the recognized inclination. Furthermore, the white line detection device for an automated guided vehicle can also perform post-processing to correct the movement direction according to the position and orientation of the automated guided vehicle. For example, in the case of an automated guided vehicle, it is possible to perform processing such as stopping the drive if a white line thickness different from a known thickness is detected.

[0149] Although several embodiments and modifications of the present invention have been described above, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and modifications can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Each of these embodiments and modifications is included in the scope and spirit of the invention, and is included in the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0150] 310 Surface Scanner 320 Backside Scanner 330 Selector 340 CPU 350 memory 500 First Image Processing Device 510 NIR Skew Detector 520 Output data control section 521 Input interface section 522 Image Processing Unit 530 Register Section 540 Output image version selection section 550 Output interface section 600 Second Image Processing Device 601 first image processing unit 602 Memory write control unit 603 Memory read control unit 604 Second image processing unit [Prior art documents] [Patent documents]

[0151] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-176975 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-154305 [Patent Document 3] Japanese Patent Application Publication No. 2020-53931

Claims

1. a first image processing device for processing an invisible image of the object; a second image processing device for processing a visible image of the object; a control unit that controls the first image processing device and the second image processing device; and the first image processing device, a skew detection unit that performs processing to detect skew of a subject image from the invisible image; the second image processing device, a skew detection unit that performs processing to detect skew of an object image from the visible image; a correction unit that corrects the skew of the subject image in accordance with a result of skew detection by the skew detection unit of the first image processing device or a result of skew detection by the skew detection unit of the second image processing device; and the first image processing device, A register unit for setting parameters is provided, turning on and off a skew detection function of the first image processing device based on the setting of the register unit; Image processing device.

2. a first image processing device for processing an invisible image of the object; a second image processing device for processing a visible image of the object; a control unit that controls the first image processing device and the second image processing device; and the first image processing device, a skew detection unit that performs processing to detect skew of a subject image from the invisible image; the second image processing device, a skew detection unit that performs processing to detect skew of an object image from the visible image; a correction unit that corrects the skew of the subject image in accordance with a result of skew detection by the skew detection unit of the first image processing device or a result of skew detection by the skew detection unit of the second image processing device; and the first image processing device is detachable; Image processing device.

3. a first image processing device for processing an invisible image of the object; a second image processing device for processing a visible image of the object; a control unit that controls the first image processing device and the second image processing device; and the first image processing device, an input interface unit for inputting the visible image and the invisible image of the first surface of the subject output by a reading means; a skew detection unit that performs processing to detect skew of a subject image from the invisible image input from the input interface unit; an output interface unit that outputs the visible image and the invisible image input from the input interface unit to an image data path of the second image processing device; an output image selection unit that selects image data paths to which the visible image and the invisible image are to be output from among the image data paths of the second image processing device; and the second image processing device, a skew detection unit that performs processing to detect skew of an object image from the visible image input from the image data path; a correction unit that corrects the skew of the subject image in accordance with a result of skew detection by the skew detection unit of the first image processing device or a result of skew detection by the skew detection unit of the second image processing device; An image processing device having:

4. the output image selection unit selects the image data paths to be output destinations of the visible image and the invisible image from among the image data paths of the second image processing device based on the setting of the register unit; The image processing device according to claim 3 .

5. The input interface unit inputs an invisible image of the surface of the subject output by a surface scanner and a visible image of the surface of the subject, the output image selection unit selects an image data path for a back scanner included in the second image processing device as an output destination of the invisible image of the front side, and selects an image data path for a front scanner included in the second image processing device as an output destination of the visible image of the front side; 5. The image processing device according to claim 3 or 4.

6. A selector for selecting an output to an image data path for a back scanner of the second image processing device, When a skew detection process is performed in the first image processing device, the selector switches from the output of the back scanner to the output of the first image processing device. The image processing device according to claim 5 .

7. The image processing apparatus according to claim 1 , wherein the first image processing device is detachable.

8. the first image processing device, A register unit for setting parameters is provided, turning on and off a skew detection function of the first image processing device based on the setting of the register unit; The image processing device according to any one of claims 2 to 7.

9. a value for turning off the skew detection function is set in the register unit, thereby skipping the skew detection operation inside the first image processing device while the first image processing device is still mounted; 9. The image processing device according to claim 1 or 8.

10. the first image processing device, detecting a skew of the subject image from the invisible image according to a setting for detecting the skew set by the control unit; The image processing device according to any one of claims 1 to 9.

11. the first image processing device, a storage unit for storing the detection result of the skew; The control unit Upon receiving a notification of the detection result from the first image processing device, the skew detection result is acquired from the storage unit; The correction unit of the second image processing device correcting the skew of the subject image in accordance with the skew detection result acquired by the control unit or the skew detection result obtained by the skew detection unit of the second image processing device; The image processing device according to any one of claims 1 to 10.

12. The visible image is a three-plate image of red, green, and blue in the visible wavelength region, and the invisible image is a one-plate image in the invisible wavelength region.

12. An image processing device according to any one of claims 1 to 11.

13. the second image processing device, a memory write control unit that writes the visual image into a memory; a memory read control unit that reads the visual image written in the memory; and when the memory read control unit has acquired the skew detection result from the first image processing device, when reading the visible image written in the memory, the memory read control unit corrects the skew of the subject image based on the skew detection result while reading the visible image.

13. An image processing device according to any one of claims 1 to 12.

14. the memory read control unit performs the reading while correcting a skew of the subject image by a rotational read. The image processing device according to claim 13.

15. If the edges of the invisible image can be detected normally, the skew of the subject image is corrected based on the edges of the invisible image, and if the edges of the invisible image cannot be detected normally, the skew of the subject image is corrected based on the edges of the visible image.

15. An image processing device according to any one of claims 1 to 14.

16. An image processing unit that integrates edges of the subject image by OR processing edges of the invisible image and edges of the visible image.

15. An image processing device according to any one of claims 1 to 14.

17. When neither the edge of the invisible image nor the edge of the visible image can be detected normally, the image processing unit integrates the edges of the invisible image and the visible image by performing an OR process on the edges. The image processing device according to claim 16.

18. the image processing unit detects the size of the subject from the edge; 18. The image processing device according to claim 16 or 17.

19. the image processing unit corrects the tilt and position of the subject based on the edge.

19. An image processing device according to any one of claims 16 to 18.

20. the image processing unit cuts out the subject based on the edge.

20. An image processing device according to any one of claims 16 to 19.

21. the image processing unit corrects at least one of the visible image and the invisible image; 21. An image processing device according to any one of claims 16 to 20.

22. An image processing device according to any one of claims 1 to 21; a background portion that absorbs invisible light; a reading means for irradiating the subject with visible light and the subject with the invisible light to read the visible image and the invisible image of the subject including the background portion; A reading device having the above.

23. An image processing device according to any one of claims 1 to 21; a background portion that absorbs invisible light; a reading means for irradiating the subject with visible light and the subject with the invisible light to read the visible image and the invisible image of the subject including the background portion; an image forming means for forming an image on a medium based on the skew-corrected visible image; An image forming apparatus having the same.

24. A method for correcting skew in an image of a subject by an image processing device, comprising: a step of turning on a skew detection function of a first image processing device by setting a parameter of a register unit of the first image processing device; a step of processing by the first image processing device to detect skew of an image of the object from an invisible image of the object; a step of performing processing by a second image processing device to detect skew of an object image from the visible image of the object; a step of correcting the skew of the subject image in accordance with the detection result of the skew detected by the first image processing device or the detection result of the skew detected by the second image processing device when the control unit acquires the detection result of the skew detected by the first image processing device; A method comprising:

25. A method for correcting skew in an image of a subject by an image processing device, comprising: a step of inputting the visible image and the invisible image output by the reading means into a first image processing device; a step of performing a process of detecting a skew of a subject image from the input invisible image; selecting an image data path for outputting the input visible image and the input invisible image from among the image data paths of a second image processing device; outputting the input visible and invisible images to an image data path of the second image processing device; a step of performing processing to detect skew of an object image from the visible image input from the image data path by the second image processing device; a step of correcting the skew of the subject image in accordance with the detection result of the skew detected by the first image processing device or the detection result of the skew detected by the second image processing device when the control unit acquires the detection result of the skew detected by the first image processing device; A method comprising:

Images