Image processing device, control method and storage medium

Abstract

The present invention relates to an image processing device, a control method and a storage medium. An image processing device having a reading device for reading an image from a sheet includes one or more controllers to perform operations. An image is read from a sheet by using the reading device. An edge detection process is performed on the image. At least one pair of edges is determined from the plurality of edges based on information of the plurality of edges detected in the edge detection process and size information of the sheet.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus that performs processing for adjusting an image forming position relative to a sheet. Background Art

[0002] In a printing device (image processing device) that forms an image on a sheet, a function of adjusting the image forming position relative to the sheet (hereinafter referred to as "print position adjustment") is conventionally used. Japanese Patent Application Laid-Open No. 2016-111628 discusses a technique for acquiring print position adjustment parameters by printing a mark for adjustment on a sheet, reading the printed mark using a reader, and acquiring the positional relationship between the mark and the edge of the sheet. As an example of a reader, an automatic document feeding device for a sheet, which is called an automatic document feeder (ADF), is also discussed.

[0003] The device discussed in Japanese Patent Application Laid-Open No. 2016-111628 has room for improvement in terms of accuracy in detecting sheet edges. This is because, when a sheet is read using a mechanism such as an ADF, linear image noise called reader stripes appears. Such reader stripes are similar in appearance to the shadow of a sheet edge, and thus may be erroneously detected as a sheet edge. If the sheet edge position cannot be accurately acquired, the position relative to the mark cannot be accurately acquired, and thus the print position adjustment parameter cannot be accurately acquired. Summary of the invention

[0004] The present invention relates to an image processing apparatus that can appropriately detect a sheet edge even when image noise is generated, and in particular, to an image processing apparatus that can accurately acquire a printing position adjustment parameter.

[0005] According to one aspect of the present invention, an image processing apparatus having a reading device for reading an image from a sheet includes one or more controllers configured to perform operations including: reading an image from a sheet by using the reading device, performing edge detection processing on the image, and determining at least one pair of edges from among a plurality of edges based on information of a plurality of edges detected in the edge detection processing and size information of the sheet.

[0006] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a control block diagram of a system including an image processing apparatus.

[0008] Figure 2 is a schematic diagram illustrating an image forming apparatus including an image processing apparatus.

[0009] Figure 3 : is a diagram illustrating a sheet library editing screen.

[0010] Figure 4 2 is a diagram illustrating a sheet library editing interface.

[0011] Figure 5 This is a table showing the sheet library.

[0012] Figure 6 is a diagram illustrating a print position adjustment chart.

[0013] Figure 7 is a sequence diagram illustrating a series of processing steps.

[0014] Figure 8 is a flowchart illustrating the operation of a marker relative position measurement process.

[0015] Fig.9A and Fig. 9B are diagrams each illustrating input data and output data for an operation performed by machine learning.

[0016] Fig.10 is a diagram illustrating a reader having an automatic document feeder (ADF) involving paper conveyance.

[0017] Fig.11 is a flowchart illustrating the operation of the vertical sheet edge detection process.

[0018] Fig.12 is a diagram illustrating a positional relationship among an upper region of interest (ROI) image, a lower ROI image, a left ROI image, and a right ROI image.

[0019] Fig.13 : is a flowchart illustrating the operation of the sheet edge candidate extraction process.

[0020] Fig.14 is a diagram illustrating an example of a Hough table.

[0021] Fig.15A is a schematic diagram illustrating an ROI image and a plurality of sheet edge detection results, and Fig. 15B : is a schematic diagram illustrating a binarized image of the upper right ROI image.

[0022] Fig.16 is a flowchart illustrating the operation of the optimum sheet edge pair determination process.

[0023] Fig.17 is a schematic diagram illustrating an example of optimal pair determination processing for left and right sheet edges.

[0024] Fig.18 is a flowchart illustrating the operation of the horizontal sheet edge detection process.

[0025] Fig.19 is a schematic diagram illustrating an example of optimal pair determination processing for upper and lower sheet edges.

[0026] Fig. 20 is a flowchart illustrating the operation of the vertical sheet edge detection process according to the second exemplary embodiment.

[0027] Fig.21 is a flowchart illustrating the operation of the sheet edge detail estimation process.

[0028] Fig. 22 is a flowchart illustrating the operation of a process of measuring relative positions of marks from a plurality of read images.

[0029] Fig.23 is a flowchart illustrating the operation of the paper size estimation process.

[0030] Fig.24 : is a diagram illustrating histogram information for paper size estimation values.

[0031] Fig.25 is a diagram illustrating a visualization example of histogram information for paper size estimation values.

[0032] Fig.26 is a sequence diagram illustrating a series of processing steps according to the third exemplary embodiment. DETAILED DESCRIPTION

[0033] <Control Block Diagram for this Configuration>

[0034] Figure 1 1 is a block diagram schematically illustrating a hardware configuration of a printing system (image processing system) according to an exemplary embodiment. Figure 1 As illustrated, the printing system according to the present exemplary embodiment includes an image forming apparatus 100 and a host computer 101. Furthermore, the image forming apparatus 100 and the host computer 101 are connected to each other via a communication line 102. In the printing system, a plurality of host computers, printing apparatuses, and the like may be connected.

[0035] The host computer 101 can obtain input information from the user via an input device (not shown), create a print job to be sent to the image forming apparatus 100, and send the created print job to the image forming apparatus 100. The controller 110 performs various data processing and controls the operation of the image forming apparatus 100. The console panel 120 adopts a touch screen system and receives various operations from the user. As sheet size information 121, the print sheet size and position adjustment amount are obtained from the sheet library via the console panel 120, as will be described below. The image reading unit 130 is a scanner that scans an original document using an optical sensor and obtains scanned image data. As a print position measurement process 131, the relative coordinates of the read image (image with a prescribed pattern) of the print result of the print position adjustment chart are obtained, as will be described below. The sheet feeding unit 140 is a sheet feeder including a plurality of sheet feeding plates. Various print sheets can be stored in the sheet feeding plates. In each sheet feeding plate, only the topmost sheet among the stored sheets can be separated and conveyed to the image forming unit 150. The image forming unit 150 physically prints image data on a printing sheet (sheet). In the present exemplary embodiment, the image forming unit 150 using an electrophotographic method will be described, but an inkjet method may also be used for the image forming unit 150. The print result 160 is a result of printing by the image forming unit 150.

[0036] Next, the configuration of the controller 110 will be described. An input / output (I / O) control unit 111 controls communication with an external network. A read-only memory (ROM) 112 stores various control programs. A random access memory (RAM) 113 stores control programs read out from the ROM 112. A central processing unit (CPU) 114 executes the control program read into the RAM 113, and comprehensively controls image signals and various devices. A hard disk drive (HDD) 115 is used to temporarily or for a long time hold a large amount of data, such as image data and print data. These modules are connected to each other via a system bus 116. The system bus 116 also connects the controller 110 and each device in the image forming apparatus 100. The RAM 113 is also used as a working memory and a main memory of the CPU 114. In addition to the ROM 112, control programs and operating systems are also stored in the HDD 115. In addition, a non-volatile RAM (NVRAM, not shown) may be included, and the non-volatile RAM may store printing device mode setting information input from the console panel 120.

[0037] <Image Forming Apparatus>

[0038] Next, the structure and operation of the image forming apparatus 100 will be described.

[0039] Figure 21 is a cross-sectional view illustrating the structure of an image forming apparatus 100 according to the present exemplary embodiment. The image forming apparatus 100 includes Figure 2 The illustrated housing 201 contains each mechanism for configuring the engine unit, an engine control unit for controlling each printing process (eg, paper feeding process) through each mechanism, and a control board storage portion for storing a printer controller.

[0040] The image reading unit 130 includes an original document tray 230, a pickup roller 231, a conveying roller 232, a roller 233, a light source 234, a second reading unit 235, a discharge roller 238, and a first reading unit 236. The original documents 237 stacked on the original document tray 230 are sent to the reading path one by one by the pickup roller 231. The original document 237 picked up by the pickup roller 231 is conveyed in the direction of the path via the conveying roller 232. The original document 237 reaches the reading position via the path, and the image information added to the front side of the original document 237 is read by the light source 234 and the first reading unit 236 together with the image information at the end portion of the original document 237. A white member is provided at a position facing the first reading unit 236, and when the original document passes through the position, the first reading unit 236 performs reading. The first reading unit 236 is a reader that will also be used for platen reading. After that, when the original document 237 reaches the reading position of the second reading unit 235, the second reading unit 235 reads the image information on the back side of the original document 237. A white member is provided at a position facing the second reading unit 235, and when the original document 237 passes through the position, the second reading unit 235 performs reading. The second reading unit 235 includes, for example, a contact image sensor (CIS). After that, the original document 237 is discharged by the discharge roller 239.

[0041] By repeating the above-described operation, information of the front side image and the back side image of each original document 237 in the group stacked on the original document tray 230 is read by one transport.

[0042] An example of reading both sides of an original document in one conveyance is described as a desirable example, but is not limited thereto. If a reader including a reverse conveyance mechanism can read both sides of an original document, such a reader can also be used.

[0043] Each mechanism for configuring the engine unit is as follows. The optical processing mechanism forms an electrostatic latent image on the photosensitive drum 205 by laser beam scanning, visualizes the formed electrostatic latent image, performs multiple transfers of the visualized image to the intermediate transfer member 252, and further transfers the color image formed by the multiple transfers to the sheet P as a toner image. The fixing processing mechanism fixes the toner image on the sheet P, the feeding processing mechanism feeds the sheet P, and the conveying processing mechanism conveys the sheet P.

[0044] The optical processing mechanism includes a laser driver at the laser scanner unit 207, which is used to turn on / off the laser beam emitted from the semiconductor laser (not shown) based on the image data supplied from the controller 110. The laser beam emitted from the semiconductor laser is deflected to the scanning direction by the rotating polygon mirror 208. The laser beam deflected to the main scanning direction is guided to the photosensitive drum 205 via the reflecting polygon mirror 209, and the photosensitive drum 205 is exposed to the laser beam in the main scanning direction. At the same time, an electrostatic latent image is formed on the photosensitive drum 205 charged by the primary charger 211 and subjected to the scanning exposure of the laser beam, and the formed electrostatic latent image is visualized as a toner image by the toner supplied by the developing device 212 to be described below. Subsequently, the toner image visualized on the photosensitive drum 205 is transferred (primary transfer) to the intermediate transfer member 252, which is applied with a voltage opposite to the toner image. When forming a color image, images of corresponding colors are sequentially formed on the intermediate transfer member 252 by the yellow (Y) station 220, the magenta (M) station 221, the cyan (C) station 222, and the black (K) station 223. Thus, a full-color visible image is formed on the intermediate transfer member 252.

[0045] Next, the sheet P fed from the storage 210 for storing the transfer material is conveyed, the conveyed sheet P is pressed against the intermediate transfer member 252 by the transfer roller 251, and at the same time, a bias opposite to the toner is applied to the transfer roller 251. The visible image formed on the intermediate transfer member 252 is thereby transferred to the sheet P synchronously conveyed in the conveying direction (sub-scanning direction) of the sheet P by the feeding processing mechanism (secondary transfer).

[0046] The sheet P after the secondary transfer passes through the fixing unit 260, so that the toner transferred to the sheet P is melted by heating, thereby being fixed as an image to the sheet P. In the case of double-sided printing, the sheet P is reversed by the reversing unit 270 after passing therethrough, and is guided to the transfer portion again, so that the back image is transferred to the sheet P. Thereafter, the toner image on the sheet P is fixed by heating while the sheet P passes through the fixing unit 260 in a manner similar to the above, and then the sheet P is discharged to the outside of the printer (image forming apparatus 100), and the printing process is completed.

[0047] <Sheet Library>

[0048] Sheets to be used for printing in the image forming apparatus 100 are managed by the operator using a database called a sheet library. The sheet library is stored in the HDD 115 or the RAM 113 and is read / written by each software module as appropriate. Figure 5 Describe the configuration of the sheet library in detail.

[0049] Figure 3 110 is a diagram schematically illustrating an interface screen for an operator to perform an operation such as editing of a sheet library in the printing system according to the present exemplary embodiment. The sheet library editing screen 300 is an entire interface screen displayed on the console panel 120 by the CPU 114 .

[0050] The sheet list 310 displays a list of sheets stored in the sheet library. In the sheet list 310, sheet properties indicated by columns 311 to 315 are presented to the operator as additional information for each sheet. Column 311 indicates the sheet name of each sheet. The sheet name is a name specified by an operator or the like in order to distinguish sheets from each other. Column 312 and column 313 respectively indicate the sub-scanning direction sheet length and the main scanning direction sheet length of each sheet. Column 314 indicates the basis weight of each sheet. Column 315 indicates the surface characteristics of each sheet. Surface characteristics are properties that indicate the physical characteristics of the front side of a sheet, and examples of surface characteristics include "coating" which means a front side that is coated to increase glossiness and "embossing" which means an uneven front side. The operator can select any sheet by touching the portion of the sheet list 310 where the sheet is displayed on the console panel 120. The selected sheet is highlighted (reverse displayed). In Figure 3 , a state in which "XYZ-manufactured paper color 81" is selected is illustrated as an example. In the case where the number of sheets recorded in the sheet library is greater than the number of sheets that can be displayed at one time in the sheet list 310, a scroll bar 317 is used. The operator can select any sheet by operating the scroll bar 317.

[0051] The Add button 320 is used to add a new sheet to the sheet library. The Edit button 321 is used to edit the sheet properties of the sheet selected in the sheet list 310. When the Add button 320 or the Edit button 321 is pressed, a pop-up window appears. Figure 4 The interface screen is shown in FIG. 3 . The delete button 322 is used to delete the sheet selected in the sheet list 310 from the sheet library. The print position adjustment button 323 is used to adjust the print position relative to the sheet selected in the sheet list 310 .

[0052] <Editing interface of the sheet library>

[0053] Figure 4 400 is a diagram schematically illustrating an interface screen for an operator to edit sheet properties in the printing system according to the present exemplary embodiment. The interface screen 400 is an entire interface screen displayed on the console panel 120 by the CPU 114 .

[0054] The text boxes 401 to 404 are used to input sheet attributes, namely, sheet name, sub-scanning direction sheet length, main scanning direction sheet length, and basis weight, respectively. The sheet attributes can be input into the text boxes through a software keyboard (not shown) or a numeric keypad included in the console panel 120. The combo box 405 is used to specify the surface characteristics of the sheet. In the combo box 405, one surface characteristic can be specified from a list of surface characteristics that are pre-registered and that can be supported by the image forming apparatus 100.

[0055] When the End Edit button 420 is pressed, the sheet attributes input at this time are determined and stored in the sheet library. After that, the interface screen 400 is closed and the sheet library editing screen 300 is displayed again. When the Cancel button 421 is pressed, the sheet attribute editing process is stopped, the interface screen 400 is closed, and the sheet library editing screen 300 is displayed again.

[0056] <Contents of the Sheet Library>

[0057] Figure 5 TABLE 115 is a table representing a sheet library stored in the HDD 115, etc. Although a schematic diagram is used for description, the sheet library is actually stored in the form of digital information such as Extensible Markup Language (XML) or Comma Separated Values ​​(CSV).

[0058] Sheet information 501 to sheet information 508 each indicate a sheet registered in the sheet library.

[0059] Columns 511 to 515 indicate sheet properties specified by the operator for each sheet. Column 511 indicates the sheet name. Columns 512 to 515 respectively indicate sheet properties indicating physical properties of the sheet, namely, sub-scanning direction sheet length, main scanning direction sheet length, basis weight, and surface properties.

[0060] Columns 520 and 521 represent the print position offset on the front side and the print position offset on the back side of each sheet, respectively. The print position offset represents the position offset relative to the ideal print position, and in the present exemplary embodiment is composed of the following items: the leading position, the side position, the main scanning magnification, and the sub-scanning magnification. In actual printing, the image forming apparatus 100 makes adjustments based on these items of the print position offset to achieve printing at the ideal print position (i.e., adjustments to offset the print position offset), and performs printing based on the adjustments. The leading position and the side position represent the print position offset in the sub-scanning direction and the print position offset in the main operation direction relative to the sheet, respectively. The leading position is adjusted by changing the print start position of the image starting from the leading edge of the sheet in the sheet conveying direction, and the side position is adjusted by changing the print start position of the image starting from the left edge of the sheet in the sheet conveying direction. The sub-scanning direction magnification represents the offset of the image length in the sub-scanning direction (the magnification relative to the ideal length). The main scanning direction magnification indicates the deviation of the image length in the main scanning direction (relative to the ideal length). Figure 5 Although not shown in the figure, the skewness indicates the parallelism of either side, the trapezoidality indicates the parallelism between the leading end side and the trailing end side of the image printed on the sheet, and the squareness indicates the rectangularity of the image printed on the sheet.

[0061] Each of these print position shift amounts is calculated by printing an adjustment chart in which predetermined marks are arranged and detecting the positions of the marks on the printed adjustment chart. Figure 6 An example of an adjustment chart is described. The initial value of each of these print position shift amounts is 0, and the initial value is used when new sheet information has just been registered on the sheet library and when print position adjustment has not been performed even if a sheet is registered.

[0062] <Contents of Print Position Adjustment Chart and Measurement>

[0063] Figure 6 114 is a diagram schematically illustrating an example of an adjustment chart to be used in the printing position adjustment. After the printing position adjustment starts, the adjustment chart is printed based on an instruction of the CPU 114.

[0064] The chart original document 601 represents a printed adjustment chart. Marks 602 to 605 are printed at specific positions of the adjustment chart. These marks are printed on the four corners of each of the front and back sides of the adjustment chart, that is, printed at eight places in total, and if the printing position is an ideal position, the image is placed so as to be printed at a position having a fixed distance from the edge of the four sheet corners. The printing position offset is determined by measuring the position relative to each of the edges of the four sheet corners on the adjustment chart.

[0065] In this exemplary embodiment, the measurement is performed by Figure 6 The distances A to H in FIG. 604 each indicate a distance from a corresponding one of the marks 602 to 605 to the nearest edge of the sheet, and a distance of 10 mm is predefined for each of the distances A to H in the present exemplary embodiment.

[0066] <Operation of Print Position Measuring Unit>

[0067] Reference Figure 8 The method of calculating the above distances A to H is described by referring to the flowchart in FIG. For the upper left vertical sheet edge, the upper left horizontal sheet edge, the upper right vertical sheet edge, the upper right horizontal sheet edge, the lower left vertical sheet edge, the lower left horizontal sheet edge, the lower right vertical sheet edge, and the lower right horizontal sheet edge in the flowchart, refer to Figure 6 Schematic diagram in .

[0068] In step S801, the CPU 114 acquires a read image of the adjustment chart via the image reading unit 130. The image reading unit 130 may be an external reader such as an automatic document feeder (ADF), or may be a reader installed in a printer, such as an inline sensor.

[0069] In the present exemplary embodiment, reading by an ADF that conveys a sheet using two rollers on the left and right will be described as an example. Figure 2 When a sheet is conveyed by the rotation of the conveying roller 232 illustrated in FIG. 2 , depending on the sheet size thereof and the torque of the roller, there may be a case where a plurality of rollers are provided on the left and right sides. Fig.10 This example is shown in the figure. Fig.10 is a plan view of the image reading unit 130, and Figure 2 The conveying roller 232 in the cross-sectional view corresponds to Fig.10 1003. Using these rollers, the adjustment chart 1003 is conveyed in the direction indicated by arrow 1004. In this process, it may be desirable to rotate the two rollers 1001 and 1002 at the same speed so as to convey the adjustment chart 1003 in parallel. If the roller diameters are different, a speed difference occurs between the peripheries of the left and right rollers, making parallel conveyance difficult. Fig.10In the case where, for example, the diameter of roller 1001 is greater than the diameter of roller 1002, the speed at which the sheet passes through roller 1001 increases, so that the sheet is conveyed while being bent in the left direction indicated by arrow 1005, and thus, a deformed image is acquired by the second reading unit 235. Image 1006 represents an image acquired in a deformed state. At the start of reading, the upper and lower edges of the sheet remain horizontal, but due to the speed difference, the right side gradually shrinks, and the difference with the left side increases. Therefore, the parallelism between the upper and lower edges is lost. The following processing will be described assuming that the image acquired in this way in which the upper and lower edges are not parallel.

[0070] In step S802, CPU 114 obtains the position of each of the markers 602 to 605 from the read image. The marker position is represented as a two-dimensional coordinate in the horizontal direction (x) and the vertical direction (y) of the scanned image, and the upper left coordinate is represented as the origin (0,0). The marker position can be represented as the centroid coordinate of the marker, or can be calculated with sub-pixel accuracy. In addition, a pattern matching method can be used to perform the process of detecting the marker.

[0071] In step S803, the CPU 114 acquires the sheet size of the adjustment chart from the sheet library editing screen 300. Specifically, the CPU 114 acquires the sub-scanning direction sheet length (mm) in the column 312 and the main scanning direction sheet length (mm) in the column 313.

[0072] In step S804, the CPU 114 detects the upper left vertical sheet edge and the upper right vertical sheet edge. Information of these sheet edges is used to calculate the distance B and the distance D. The details of this processing will be described below.

[0073] In step S805, the CPU 114 detects the lower left vertical sheet edge and the lower right vertical sheet edge. Information of these sheet edges is used to calculate the distance F and the distance H. The details of this processing are similar to those in step S804.

[0074] In step S806, the CPU 114 detects the upper left horizontal sheet edge and the lower left horizontal sheet edge. Information of these sheet edges is used to calculate the distance A and the distance E. The details of this processing will be described below.

[0075] In step S807, the CPU 114 detects the upper right horizontal sheet edge and the lower right horizontal sheet edge. Information of these sheet edges is used to calculate the distance C and the distance G. Details of this processing are similar to those in step S806.

[0076] In step S808, the CPU 114 calculates the mark relative position from the above eight detected sheet edges. In this process, distances A to H are calculated by calculating the normal distance of the linear equation (ρ=xcosθ+ysinθ) from the mark center coordinates to each sheet edge.

[0077] <Vertical Sheet Edge Detection Processing>

[0078] Reference Fig.11 The flowchart in FIG. 1 describes in detail the processing of detecting the vertical sheet edge in step S804 and step S805.

[0079] In step S1101 , the CPU 114 acquires a left ROI (Region of Interest) image and a right ROI image (partial image) based on the marker position acquired in step S802 . Fig.12 is a diagram schematically illustrating the positional relationship between the left ROI image and the right ROI image in the case of detecting the upper left vertical sheet edge and the upper right vertical sheet edge. In the present exemplary embodiment, an area with a width of 0.5 mm and a length of 1.7 mm centered 1.0 mm to the left and 1.0 mm downward of the center coordinates of the upper left mark is extracted as the left ROI image. In addition, an area with a width of 0.5 mm and a length of 1.7 mm centered 1.0 mm to the right and 1.0 mm downward of the center coordinates of the upper right mark is extracted as the right ROI image. The left ROI image and the right ROI image in the case of extracting the lower left vertical sheet edge and the lower right vertical sheet edge are acquired using a similar method.

[0080] In step S1102, the CPU 114 applies an edge detection filter to each of the left ROI image and the right ROI image acquired in step S1101. In the present exemplary embodiment, the following kernel k is used as the edge detection filter.

[0081]

[0082] The edge detection filter is not limited to the kernel k. The edge detection filter may be a horizontal Laplacian filter or a differential filter. In the following description, the image after the edge detection filter is applied will be described as an image Ie(x, y), where x and y are indices of coordinates.

[0083] In step S1103, the CPU 114 detects sets of two sheet edge candidates from the image after the edge detection filter is applied. The details of this processing will be described below.

[0084] In step S1104, the CPU 114 determines the best pair from among the plurality of sheet edge candidates. The details of this processing will be described below.

[0085] <Sheet Edge Candidate Extraction Processing>

[0086] Reference Fig.13 The sheet edge candidate extraction processing performed in step S1103 is described in detail by the flowchart in FIG.

[0087] In step S1301, the CPU 114 applies binarization processing to the image Ie(x, y) calculated in step S1102, thereby acquiring a binarized image Ib(x, y). Specifically, the binarized image Ib(x, y) can be calculated by the following equation.

[0088]

[0089] Where th is the threshold value, and may be a fixed value of 128, for example, or may be determined dynamically from the image using Otsu's method.

[0090] In step S1302, the CPU 114 performs Hough transform on the acquired binary image Ib (x, y). Hough transform is a process of comprehensively detecting a straight line forming a predetermined angle θ from an image by expressing the straight line by the equation (ρ=xicosθ+yisinθ). The operation of Hough transform will be described.

[0091] In the Hough transform, first, all pairs (xi, yi) (0≤i≤M) of coordinates x and y that satisfy Ib(x, y) = 1 are obtained. Next, the ρij value of each pair (xi, yi) is calculated as follows.

[0092] ρ ij =x i cosθ j +y i sinθ j (3)

[0093] θj (0≤j≤N) is a predetermined value, and in the present exemplary embodiment, θ is a value in the range of 88° to 92° in increments of 0.5° (θ0=88, θ1=88.5, ..., θ7=92). The obtained (θj, ρij) represents a straight line passing through the point (xi, yi) at an angle of θj.

[0094] Next, the Hough table H(ρ, θ) is calculated by counting all the obtained pairs (θj, ρij). Specifically, the Hough table H is expressed as follows.

[0095]

[0096] Here, E m,n is a matrix with N rows and (maxρ) columns that satisfies the following equation.

[0097]

[0098] Fig.14 The figure shows an example of a calculated Hough table. The value of the cell (θ, ρ) of the Hough table indicates how many points in the binary image Ib(x, y) are on the straight line ρ = xcosθ + ycosθ. For example, Fig.14 The value of the cell (θ=89°, ρ=2) in is 219, which is a large value. The straight line defined by the cell (θ=89°, ρ=2) can be determined to be within the image. Meanwhile, the value of the cell (θ=92°, ρ=2) is 0, and thus indicates that the straight line defined by the cell (θ=92°, ρ=2) is not within the image.

[0099] In step S1303, the CPU 114 detects a peak position from the Hough table calculated in step S1302. In the present exemplary embodiment, a point satisfying the following conditions is detected as a peak value:

[0100] H(θ,ρ)≥H(θ,ρ+1),

[0101] H(θ,ρ)≥H(θ+1,ρ),

[0102] H(θ,ρ)≥H(θ-1,ρ),

[0103] H(θ,ρ)≥H(θ,ρ-1), and

[0104] H(θ,ρ)>th_hough

[0105] Wherein, th_hough is a threshold value for determining whether the Hough table value is noise, and in the present exemplary embodiment, th_hough is 140. For example, a cell (θ=91°, ρ=5) and a cell (θ=89°, ρ=2) are for Fig.14 Two candidates for the peak detected in the example.

[0106] As described above, a plurality of sheet edge candidates can be extracted from one ROI image in the sheet edge candidate extraction process. Fig.15A and Fig. 15B is a schematic diagram illustrating an example in which a plurality of sheet edge candidates are detected. Fig.15A 2 is a schematic diagram illustrating a scanned image and a right ROI image. The scanned image is a result of reading a sheet surface that is skewed due to a tilt occurring during transport. Fig. 15Bis a schematic diagram illustrating a binarized image Ib(x, y) when the process is applied to the upper right ROI image of the scanned image. A black pixel of the binarized image Ib(x, y) indicates that there is no edge at the pixel position of the pixel, and a white pixel indicates that there is an edge at the pixel position of the pixel. For the scanned image, Fig. 15B As illustrated, two sheet edge candidates (edges) are calculated. Fig. 15B The edge on the left is formed by the shadow of the edge of the sheet and corresponds to Fig.14 The sheet edge candidate at the cell (θ=89°, ρ=2) in the Hough table in . This edge is tilted 1 degree from the vertical due to the skew in the scan. On the other hand, Fig. 15B The sheet edge candidate on the right side of the middle is a sheet edge candidate that is erroneously detected due to reader stripes and corresponds to Fig.14 The cells in the Hough table in (θ=91°, ρ=5). Regardless of the amount of tilt of the original document, the reader stripes are always detected as vertical stripes. Through the following optimal sheet edge pair determination process, the multiple sheet edge candidates detected by the process are narrowed down to one optimal pair (a pair of edges).

[0107] <Optimal Sheet Edge Pair Determination Process>

[0108] Will refer to Fig.16 The flowchart in FIG. 1 describes the optimal sheet edge pair determination processing in step S1104 in detail.

[0109] In step S1601, the CPU 114 determines whether the processing in steps S1602 to S1604 is completed for all sheet edge candidates. If the processing is not completed (No in step S1601), the processing proceeds to step S1602. If the processing is completed (Yes in step S1601), the processing proceeds to step S1605. For example, in the case where three left sheet edge candidates and five right sheet edge candidates are extracted in step S1103, this loop is repeated a total of fifteen times.

[0110] In step S1602, the CPU 114 acquires the first sheet edge candidate ρ L, θ L. For example, the first sheet edge candidate is an upper left vertical sheet edge candidate.

[0111] In step S1603, the CPU 114 acquires the second sheet edge candidate ρ R, θ R. For example, the second sheet edge candidate is an upper right vertical sheet edge candidate.

[0112] In step S1604, the CPU 114 calculates the likelihood in consideration of the parallelism of the sheet edge pair and the paper size. In the present exemplary embodiment, the likelihood of the sheet edge pair is calculated by the following equation. The following equation indicates that the smaller the numerical value, the higher the likelihood of the sheet edge pair.

[0113] l=|psize-(ρ R -ρ L )cosθ L *25.4 / dpi|+γstep(|θ R -θ L |-ε) (6)

[0114] Here, psize represents a standard value of the paper size. The sheet length in the sub-scanning direction is set when a vertical sheet edge is detected, and the sheet length in the main scanning direction is set when a horizontal sheet edge is detected. In addition, dpi is the scanning resolution. In addition, step(x) is a step function that returns 1 when x>0 and returns 0 when x≤0. γ and ε are constants, and γ=1000 and ε=0.5 are used. The term (ρR-ρL)cosθL*25.4 / dpi represents a measured value of the sheet size determined based on a sheet edge candidate pair, and the term |psize-(ρR-ρL)cosθL*25.4 / dpi| represents an error between a standard value and a measured value.

[0115] In addition, γstep(|θR-θL|-ε) is a penalty term based on the parallelism of the sheet. More specifically, when the difference between θR and θL is 0.5° or less, no penalty is given, otherwise a penalty of 1000 is given. In other words, if the absolute value of the difference between the angles is lower than or equal to the threshold, no penalty is given. In the present exemplary embodiment, the likelihood is calculated using the above equation, but the present invention is not limited to this. For example, the minimum value of the numerical value calculated by subtracting the length of the paper size in a predetermined direction from the distance of the sheet edge candidate pair (the distance between the edges, i.e., the difference between the pieces of position information corresponding to the edges) can be used for the most likely sheet edge. In this case, the innermost sheet edge candidate relative to the standard value of the paper size is selected as the most likely sheet edge.

[0116] In step S1605 , the CPU 114 determines the most likely sheet edge pair (minimum value) as the maximum likelihood sheet edge pair.

[0117] The specific operations in this process will refer to Fig.17 The examples in are described. Fig.17The result of the sheet edge candidate extraction process for detecting the upper right sheet edge and the upper left sheet edge of the scanned image is shown in the upper part. For the upper right sheet edge, three candidates are extracted, namely, right candidate A (dirt on the sheet), right candidate B (sheet edge), and right candidate C (reader stripe). On the other hand, for the left sheet edge, only one left candidate (1) (sheet edge) is extracted.

[0118] Fig.17 The table in the lower part indicates the calculated probability results and the sheet edge pairs determined by the optimal sheet edge pair determination process. In the table, the paper standard error represents the numerical value of the term |psize-(ρR-ρL)cosθL*25.4 / dpi| in equation (6), and the value of the parallelism penalty indicator term γstep(|θR-θL|-ε). The probability score indicates the value defined by equation (6). Among the numerical values ​​of the paper standard error, "left candidate (1) × right candidate C" corresponds to the minimum error. However, the right candidate C is a reader stripe and is not parallel to the left candidate (1), and is therefore not the best sheet edge candidate from the perspective of parallelism. The right candidate A and the right candidate B are parallel to the left candidate (1), and therefore the "left candidate (1) × right candidate B" that is closest to the paper standard value is determined as the best sheet edge candidate pair.

[0119] <Horizontal Sheet Edge Detection Processing>

[0120] Next, detection of a horizontal sheet edge, which is a sheet edge in a direction orthogonal to a vertical sheet edge, will be described. Fig.18 The flowchart in describes in detail the processing of detecting the horizontal sheet edge performed in step S806 and step S807.

[0121] In step S1801 , the CPU 114 acquires an upper ROI image and a lower ROI image based on the marker position acquired in step S802 . Fig.12 is a schematic diagram illustrating the positional relationship between the upper ROI image and the lower ROI image in the case of detecting the upper left horizontal sheet edge and the lower left horizontal sheet edge. In the present exemplary embodiment, a region having a width of 1.7 mm and a length of 0.5 mm with a center point located 1.0 mm to the right and 1.0 mm to the top of the center coordinates of the upper left mark is extracted as the upper ROI image. In addition, a region having a width of 1.7 mm and a length of 0.5 mm with a center point located 1.0 mm to the right and 1.0 mm to the bottom of the center coordinates of the lower left mark is extracted as the lower ROI image. The upper ROI image and the lower ROI image in the case of extracting the upper right horizontal sheet edge and the lower right horizontal sheet edge are acquired using a similar method.

[0122] In step S1802, the CPU 114 applies an edge detection filter to each of the upper ROI image and the lower ROI image acquired in step S1801. In the present exemplary embodiment, a differential filter for the top and the bottom is used as the edge detection filter, but the present invention is not limited thereto. The edge detection filter may be another type of filter, such as a vertical Laplacian filter.

[0123] In step S1803, the CPU 114 detects sets of two sheet edge candidates from the image after the edge detection filter is applied. This processing is similar to the processing in step S1103.

[0124] In step S1804, the CPU 114 determines an optimal pair from among the detected plurality of sheet edge candidates.

[0125] The basic flow of this process is also similar to the flow described with reference to step S1104. Fig.16 The process of steps S1601 to S1605 in is performed. However, as described above, due to the roller speed difference, the parallelism between the upper edge and the lower edge of the processing target image is not necessarily guaranteed. In this case, equation (6) based on the parallelism in the above step S1604 does not hold. Therefore, when the possibility of a sheet edge pair is calculated based on the first sheet edge candidate ρT, θT and the second sheet edge candidate ρB, θB, the angle difference Δθ formed by the upper left vertical sheet edge and the upper right vertical sheet edge detected in step S804, and the lower left vertical sheet edge and the lower right vertical sheet edge detected in step S805 is used as a correction value. For example, in the case of ideally acquiring an image with parallel edges, when the angle determined by the upper left and upper right is the upper angle θ1, and the angle determined by the lower left and lower right is the lower angle θ2, the angle difference is 0. However, in the case where there is a difference between these angles, for example, in the case where the upper angle θ1 is 87° and the lower angle θ2 is 90°, the parallelism between the upper and lower edges is lost, and it is estimated from the angle difference Δθ=3° that the upper and lower edges are offset by 3° relative to the horizontal line. Therefore, equation (6) is transformed as follows.

[0126] l=|psize-(ρ T -ρ B )cosθ ave *25.4 / dpi|+γstep(|θ T -θ B +Δθ|-ε) (7) Here, θave is the average value of the angles determined by (θT-θB) / 2.

[0127] The optimum pair of upper and lower edges is determined by adding the angle correction value Δθ to equation (7).

[0128] Reference Fig.19 The examples in describe the specific operations in this process. Fig.19 The result of the sheet edge candidate extraction process for detecting the upper left sheet edge and the lower left sheet edge of the scanned image is shown in the upper part. For the lower left sheet edge, two candidates, lower candidate A (dirt on the sheet) and lower candidate B (sheet edge) are extracted. On the other hand, for the upper left sheet edge, only one upper candidate (1) (sheet edge) is extracted.

[0129] Fig.19 The table in the lower part indicates the calculated likelihood results and the sheet edge pairs determined by the optimal sheet edge pair determination process. In the table, the paper standard error represents the numerical value of the term |psize-(ρR-ρL)cosθave*25.4 / dpi| in equation (7), and the value of the parallelism penalty indicator term γstep(|θT-θB+Δθ|-ε). The likelihood score indicates the value defined by equation (7). In the case where the sheet is fed with the edges parallel as in normal circumstances, "upper candidate (1) × lower candidate A" corresponds to the minimum error. However, due to the correction based on the value of the angle Δθ determined from the left and right edges, the lower candidate A is not parallel to the upper candidate (1) and is therefore not the optimal sheet edge candidate from the point of view of parallelism, so that the lower candidate B is determined as the optimal sheet edge candidate pair.

[0130] <Use Sequence>

[0131] Figure 7 1 is a sequence diagram illustrating a series of steps of processing in the present exemplary embodiment. In the present exemplary embodiment, a main exchange is performed between the operator and the image forming apparatus 100. Here, the sequence starts in a state where the sheet library edit screen 300 is displayed.

[0132] First, in step S701, when the operator presses the print position adjustment button 323, the image forming apparatus 100 determines to start the print position adjustment. In step S702, the image forming apparatus 100 displays a print position adjustment screen for designating a feed cassette via the CPU 114.

[0133] Next, in step S703, the operator designates a feed cassette and issues an instruction to execute a printing position adjustment process.

[0134] Next, in step S704, the image forming apparatus 100 outputs the Figure 6 Adjustment chart shown in the figure.

[0135] Next, in step S705 , the operator places the adjustment chart output in step S704 on the image reading unit 130 .

[0136] Next, in step S706 , the image forming apparatus 100 reads the adjustment chart placed on the image reading unit 130 via the CPU 114 and executes the printing position adjustment process.

[0137] Next, in step S707 , the image forming apparatus 100 executes the printing position adjustment, ie, the printing position adjustment processing in steps S802 to S808 , via the CPU 114 , thereby storing the printing position shift amount of each feed cassette in the sheet library.

[0138] In the present exemplary embodiment, the printing position adjustment is performed in this manner. Then, using the registered printing position shift amount, an image is formed as described below.

[0139] Next, in step S708 , the operator issues an instruction to the host computer 101 to execute the print job.

[0140] Next, in step S709 , the host computer 101 sends the print job to the image forming apparatus 100 .

[0141] Next, in step S710, the image forming apparatus 100 executes a print job. In this process, the print position shift amount registered for the feed cassette is read out from the sheet library and then used for execution of the print job.

[0142] Next, in step S711 , the image forming apparatus 100 provides a print product generated in the execution of the print job.

[0143] <Remarks>

[0144] The above-described processing enables the operator to calculate the printing position shift amount for each of the selected feed cassettes simply by scanning the chart, and to implement the front and back printing position adjustment.

[0145] The method of detecting a plurality of sheet edge candidates and calculating the most likely sheet edge candidate pair based on parallelism and paper size is described above. According to this exemplary embodiment, the relative position of the mark can be robustly detected even when the brightness of the reader background and the brightness of the white of the paper are close and when there is a lot of noise (for example, reader stripes appear). By adjusting the print position to offset the calculated relative position of the mark, the front and back position adjustment can be accurately performed.

[0146] In addition, even when the parallelism between the leading and trailing edges of a sheet is lost due to uneven conveyance of the sheet, the relative positions of the marks can be robustly detected by using the angle difference when obtaining the left and right edges of the sheet as a correction value.

[0147] In this exemplary embodiment, Fig.12 The left and right ROI images and the upper and lower ROI images illustrated in FIG. 1 are described as independent images that do not overlap each other, but the ROI image is not necessarily limited to this type. A wider range (for example, including the four corners on the marked sheet) can be cropped and used for processing as an ROI image common to the vertical sheet edge and the horizontal sheet edge.

[0148] In the first exemplary embodiment, multiple candidates are detected by Hough transform for each of the opposing sheet edges, and the best pair is selected based on the paper size and parallelism. However, the use of Hough transform has such a problem that although multiple candidates can be detected, the accuracy of detecting the sheet edge is not high. Therefore, in the second exemplary embodiment, the best sheet edge pair is detected, and then a more detailed sheet edge estimation is performed, thereby detecting the sheet edge with higher accuracy. The configuration of the printing system in the second exemplary embodiment is basically the same as that in the first exemplary embodiment. Therefore, the same components and steps as those of the first exemplary embodiment are assigned the same reference numerals as those of the first exemplary embodiment, and a detailed description thereof will be omitted. Points different from the first exemplary embodiment will be described.

[0149] <Vertical Sheet Edge Detection Processing>

[0150] Will refer to Fig. 20 The flowchart in describes in detail the processing of detecting the vertical sheet edge in step S804 and step S805.

[0151] In step S2001, the CPU 114 acquires a left ROI image and a right ROI image based on the marker position acquired in step S802. This processing is similar to that of the first exemplary embodiment.

[0152] In step S2002, the CPU 114 applies an edge detection filter to each of the left ROI image and the right ROI image acquired in step S2001. This processing is similar to that of the first exemplary embodiment.

[0153] In step S2003, the CPU 114 detects sets of two sheet edge candidates from the image after the edge detection filter is applied. This processing is similar to that of the first exemplary embodiment.

[0154] In step S2004, the CPU 114 determines an optimal pair from the detected plurality of sheet edge candidates. This processing is similar to that of the first exemplary embodiment.

[0155] In step S2005, the CPU 114 determines a more detailed sheet edge equation based on the first sheet edge equation obtained in step S2004. The details of this processing will be described below.

[0156] In step S2006, the CPU 114 determines a more detailed sheet edge equation based on the second sheet edge equation obtained in step S2004. The details of this processing will be described below.

[0157] <Sheet edge detail estimation processing>

[0158] Reference Fig.21 The flowchart in describes in detail the sheet edge detail estimation processing performed in step S2005 and step S2006.

[0159] In step S2101, the CPU 114 acquires the coefficients (ρ, θ) of the sheet edge equation calculated in step S2004 and the image Ie(x, y) after the edge detection filter is applied calculated in S1102. In the following description, the coefficients are updated so that more detailed coefficients (ρ', θ') are calculated.

[0160] In step S2102, the CPU 114 determines whether the execution of the following processing is completed for all y values ​​of the ROI image. If the processing is not completed (No in step S2102), the processing proceeds to step S2103. If the processing is completed (Yes in step S2102), the processing proceeds to step S2106. For example, in the case where the vertical size of the image Ie (x, y) is 200 pixels, the processing in steps S2103 to S2105 is repeated 200 times.

[0161] In step S2103, the CPU 114 determines the edge position x at the height y by the provisional edge equation. Specifically, the edge position x is calculated as x=(ρ-ysinθ) / cosθ.

[0162] In step S2104, the CPU 114 calculates a position x* at which the edge detection filter response value is maximum within a range of ±4 pixels (neighboring pixels) around the position x. Specifically, the position x* is calculated by the following linear approximation equation.

[0163]

[0164] In step S2105 , the CPU 114 stores the calculated edge position (x*, y) in the RAM 113 .

[0165] In step S2106, the CPU 114 performs least square linear approximation on the calculated plurality of edge position pairs (x*, y). Thus, more accurate coefficients (ρ', θ') than the coefficients (ρ, θ) calculated in step S2004 are calculated.

[0166] <Horizontal Sheet Edge Detection Processing>

[0167] As with the vertical sheet edge detection process described above, the sheet edge detail estimation process is performed after determining a sheet edge candidate pair based on the Hough transform. Specifically, steps S2005 and S2006 are performed after the processes in steps S1801 to S1804 are performed.

[0168] <Remarks>

[0169] As described above, in the present exemplary embodiment, more detailed sheet edge estimation is performed after the optimal sheet edge pair is detected. According to the present exemplary embodiment, it is possible to estimate the sheet edge with high accuracy while taking into account parallelism with the sheet edge and consistency of paper size.

[0170] In the first exemplary embodiment and the second exemplary embodiment, an optimal sheet edge pair is determined from an image based on a standard value of the sheet size. However, due to reasons such as a change in the speed of a conveying roller and a sheet cutting error, there is a case where the paper size on the scanned image is inconsistent with the standard value of the sheet size. In the above exemplary embodiments, the optimal edge pair is detected based on the sheet size, and if there is a difference between the sheet size on the read image and the standard value, there is a case where the registration adjustment cannot be performed normally. Therefore, in the third exemplary embodiment, the sheet edge is detected after printing and scanning a plurality of sheets for registration adjustment, and the possible paper size is estimated. Thus, even if there is a difference between the standard value of the paper size and the paper size of the read image, position correction can be performed robustly. In the following description, only the parts that are different from the first exemplary embodiment will be described.

[0171] <Operation of Print Position Measuring Unit>

[0172] Reference Fig. 22 and Figure 6 The flowchart in describes the operation of the printing position measurement unit.

[0173] In step S2201, the CPU 114 acquires a read image of the adjustment chart via the image reading unit 130. The image reading unit 130 may be an external reader such as an ADF, or may be a reader installed in a printer, including an inline sensor.

[0174] In step S2202, CPU 114 obtains the position of each of the markers 602 to 605 from the read image. The marker position is represented as a two-dimensional coordinate of b in the horizontal direction (x) and the vertical direction (y) of the scanned image, and the upper left coordinate is defined as the origin (0,0). The marker position can be represented as the centroid coordinates of the marker, or can be calculated with sub-pixel accuracy. In addition, a pattern matching method can be used to perform the process of detecting the marker.

[0175] In step S2203, the CPU 114 acquires an ROI image (partial image) based on the acquired marker position. This process is similar to the process in step S1101, and acquires upper ROI images, lower ROI images, left ROI images, and right ROI images of all four corners.

[0176] In step S2204, the CPU 114 applies an edge detection filter to each of the acquired ROI images. This processing is similar to that in step S1102.

[0177] In step S2205, the CPU 114 executes sheet edge extraction processing on the result of step S2204. This processing is similar to the processing in step S1103. In step S2205, a Hough table and a sheet edge candidate are extracted for each of the eight ROI images.

[0178] In step S2206, the CPU 114 determines whether the processing in steps S2201 to S2205 is completed for all original documents. If the processing is completed (Yes in step S2206), the process advances to step S2207. Otherwise (No in step S2206), the process returns to step S2201.

[0179] In step S2207, the CPU 114 estimates the paper size of the original document from the plurality of sheet edge candidates and the Hough table acquired in step S2205. The details of this processing will be described below.

[0180] In step S2208, the CPU 114 detects the upper left vertical sheet edge and the upper right vertical sheet edge. This process is similar to the process in step S804. However, the paper size estimated in step S2207 is used instead of the standard value.

[0181] In step S2209, the CPU 114 detects the lower left vertical sheet edge and the lower right vertical sheet edge. This process is similar to the process in step S805. However, the paper size estimated in step S2207 is used instead of the standard value.

[0182] In step S2210, the CPU 114 detects the upper left horizontal sheet edge and the lower left horizontal sheet edge. This process is similar to the process in step S806. However, the paper size estimated in step S2207 is used instead of the standard value.

[0183] In step S2211, the CPU 114 detects the upper right horizontal sheet edge and the lower right horizontal sheet edge. This process is similar to the process in step S807. However, the paper size estimated in step S2207 is used instead of the standard value.

[0184] In step S2212, the CPU 114 determines whether the processing in steps S2208 to S2211 is completed for all original documents. If the processing is completed (Yes in step S2212), the process proceeds to step S2213. Otherwise (No in step S2212), the process returns to step S2208.

[0185] In step S2213, the CPU 114 calculates the relative position of the mark based on the eight sheet edges detected for the plurality of scanned images. In this process, the normal distance from the mark center coordinates to the linear equation (ρ=xcosθ+ysinθ) of each sheet edge is calculated for each scanned image. Figure 6 The distance A to H in . In addition, the marker relative average position is calculated by determining the average value of the distance A to H calculated for each scan image, and the calculated marker relative average position is used as the correction value.

[0186] <Paper Size Estimation Process>

[0187] Reference Fig.23 The flowchart in describes the paper size estimation processing in step S2207 in detail. Here, the calculation of the left-right paper size will be described as an example, and the top-bottom paper size is also estimated similarly. In this case, the left and right are replaced with the top and bottom to perform the processing.

[0188] In step S2301 , the CPU 114 initializes the value of the histogram value memory to zero. Fig.24 The histogram value memory initialized by this process is shown in the upper part. Fig.24 As illustrated in FIG. 1 , in the present exemplary embodiment, ten arrays are fixed to construct a histogram formed by dividing 209.5 mm to 210.4 mm at intervals of 0.1 mm. In each array, the frequency of occurrence of the estimated value of the corresponding paper size is stored, and 0 is stored by the initialization process.

[0189] In step S2302 , the CPU 114 acquires the right edge candidates ρR, θR acquired in the sheet edge candidate extraction processing in step S2205 .

[0190] In step S2303 , the CPU 114 acquires the left edge candidates ρL, θL acquired in the sheet edge candidate extraction processing in step S2205 .

[0191] In step S2304, the CPU 114 calculates an estimated value (estimation result) of the paper size using the left edge candidate and the right edge candidate (edge ​​information). Specifically, the estimated value is calculated by the following expression.

[0192] (ρ R -ρ L )cosθ L *25.4 / dpi

[0193] In step S2305, the CPU 114 updates the corresponding value of the histogram based on the estimated value. For example, in the case where the estimated value of the paper size calculated in step S2304 is 209.8 mm, Fig.24 The value of the fourth histogram bin from the left in the image is increased by 1. In this process, the value of the histogram can be updated by weighting according to the strength of the edge candidate. For example, by adding the value of the Hough table corresponding to the edge candidate to the corresponding histogram bin, the histogram can reflect the possibility of the paper candidate.

[0194] In step S2306, the CPU 114 determines whether the processing in steps S2302 to S2305 is completed for all left edge candidates. If the processing is completed (Yes in step S2306), the process proceeds to step S2307. Otherwise (No in step S2306), the process returns to step S2303.

[0195] In step S2307, the CPU 114 determines whether the processing in steps S2302 to S2306 is completed for all right edge candidates. If the processing is completed (Yes in step S2307), the processing proceeds to step S2308. Otherwise (No in step S2307), the processing returns to step S2302.

[0196] In step S2308, the CPU 114 determines whether the processing in steps S2302 to S2307 is completed for all original documents. In other words, the CPU 114 determines whether the estimation results of all original documents are aggregated. If the processing is completed (Yes in step S2308), the processing proceeds to step S2309. Otherwise (No in step S2308), the processing returns to step S2302.

[0197] In step S2309, the CPU 114 adopts the paper size corresponding to the mode value of the histogram as an estimated value of the paper size representing the plurality of sheets. Fig.25 It is a graphic Fig.24 Since the mode value of the histogram is 76, the estimated paper size is 209.9 mm.

[0198] <Use Sequence>

[0199] Fig.26 is a sequence diagram illustrating a series of steps of processing according to the present exemplary embodiment. In the present exemplary embodiment, a main exchange is performed between the operator and the image forming apparatus 100. In the sequence diagram, the sequence starts in a state where the sheet library editing screen 300 is displayed. In the present exemplary embodiment, unlike the first exemplary embodiment, a plurality of original documents are read and the print position offset amount is adjusted.

[0200] First, in step S2601, when the operator presses the print position adjustment button 323, the image forming apparatus 100 determines to start the print position adjustment. Subsequently, in step S2602, the image forming apparatus 100 displays a print position adjustment screen via the CPU 114 for designating a feed cassette and the number of print sheets.

[0201] Next, in step S2603, the operator specifies the feed cassette and the number of print sheets, and issues an instruction to execute the print position adjustment process. In this process, the image forming apparatus 100 displays a screen for specifying the number of print sheets of the print adjustment chart on the console panel 120 via the CPU 114. The operator specifies the desired number of sheets on the console panel 120 and issues a print instruction.

[0202] Next, in step S2604, the image forming apparatus 100 outputs the Figure 6 Adjustment chart shown in the figure.

[0203] Next, in step S2605 , the operator places the designated number of adjustment charts output in step S2604 in the image reading unit 130 .

[0204] Next, in step S2606 , the image forming apparatus 100 reads a plurality of adjustment charts placed in the image reading unit 130 via the CPU 114 and executes the sheet edge candidate extraction processing in steps S2201 to 2206 .

[0205] Next, in step S2607 , the image forming apparatus 100 estimates the paper size of each feed cassette via the CPU 114 .

[0206] Next, in step S2608 , the image forming apparatus 100 executes the printing position adjustment processing in steps S2207 to S2211 via the CPU 114 , thereby storing the printing position shift amount of each read image.

[0207] Next, in step S2609 , the image forming apparatus 100 stores, via the CPU 114 , the print correction amount calculated by averaging the print position shift amounts of the respective read images.

[0208] In the present exemplary embodiment, the printing position adjustment is thus performed. Subsequently, using the registered printing position shift amount, an image is formed as described below.

[0209] Next, in step S2610 , the operator issues an instruction to the host computer 101 to execute a print job.

[0210] Next, in step S2611 , the host computer 101 sends the print job to the image forming apparatus 100 .

[0211] Next, in step S2612, the image forming apparatus 100 executes the print job. In this process, the print position shift amount registered for the feed cassette is read out from the sheet library and then used for execution of the print job.

[0212] Next, in step S2613 , the image forming apparatus 100 provides a print product generated by the execution of the print job.

[0213] <Remarks>

[0214] According to the present exemplary embodiment, even when there is a difference between the standard value of the sheet size and the read image size of the original document due to causes such as changes in conveyance speed during image reading, moisture absorption, and sheet cutting errors, a sheet edge pair can be accurately searched.

[0215] In the first exemplary embodiment and the second exemplary embodiment, a method of selecting an optimal sheet edge pair by estimating the angle corresponding to the parallelism lost between the upper edge and the lower edge of the adjustment chart was described. The longer the paper conveying length, the greater this loss of angular parallelism. The loss of parallelism also varies depending on the weight (basis weight) of the sheet. Therefore, in the fourth exemplary embodiment, an example of dynamically changing parameters based on how the parallelism between the upper edge and the lower edge is lost according to information from the sheet library will be described. The configuration of the printing system in the fourth exemplary embodiment is basically the same as that in the first exemplary embodiment. Therefore, the components and steps that are the same as those of the first exemplary embodiment are assigned the same figure numbers as the figure numbers of the first exemplary embodiment, and their detailed description will be omitted. A sheet edge pair determination unit will be described as a point different from the first exemplary embodiment and the second exemplary embodiment.

[0216] First, the deformation coefficient to be used when determining a sheet edge pair will be described. As described above, a list storing various information of sheets to be used for printing is prepared as a sheet library. Figure 3 The sheet list 310 in FIG. 1 shows a list of sheets stored in the sheet library. In the sheet list 310, sheet properties indicated by columns 311 to 315 are presented to the operator as additional information for each sheet. Column 312 and column 313 respectively indicate the sub-scanning direction sheet length and the main scanning direction sheet length of each sheet. Column 314 indicates the basis weight of each sheet.

[0217] Based on the sheet length and basis weight from among the sheet information, the skew amount and deformation amount of the image caused by the conveyance during reading using the ADF are estimated, and the estimated value thus obtained is held as a deformation coefficient. The higher the basis weight and the longer the sheet conveyance length, the larger the deformation coefficient. The estimated deformation amount is calculated for each sheet. In this process, the conveyance length in the ADF is used, and therefore the length in the sub-scanning direction when printing is not necessarily the same as the conveyance length in the ADF. The sheet edge pair is estimated using the deformation coefficient.

[0218] To be in Figure 8 The processing of detecting the vertical sheet edge in step S804 and step S805 performed in step S806 is the same as that according to the first exemplary embodiment. Fig.11 The processing of the flowchart in is similar, and therefore its description will be omitted.

[0219] Next, reference will be made to the description of the first exemplary embodiment. Fig.18 The process of detecting a horizontal sheet edge to be performed in step S806 and step S807 is described with reference to the flowchart in FIG.

[0220] In step S1801, the CPU 114 acquires an upper ROI image and a lower ROI image based on the mark position acquired in step S802. The technique is similar to that described in the first exemplary embodiment. The method of acquiring the upper ROI image and the lower ROI image while extracting the upper right horizontal sheet edge and the lower right horizontal sheet edge is also similar to the method described in the first exemplary embodiment.

[0221] In step S1802, the CPU 114 applies an edge detection filter to each of the upper ROI image and the lower ROI image acquired in step S1801. In the present exemplary embodiment, a differential filter for the top and the bottom is used as the edge detection filter, but the present invention is not limited thereto. The edge detection filter may be another type of filter, such as a vertical Laplacian filter.

[0222] In step S1803, the CPU 114 detects sets of two sheet edge candidates from the image after the edge detection filter is applied. This processing is similar to the processing in step S1103 described in the first exemplary embodiment.

[0223] In step S1804, the CPU 114 determines the best pair from the detected plurality of sheet edge candidates. The basic flow of this process is also similar to the flow described with reference to step S1104 in the first exemplary embodiment. However, a correction value is given based not only on the roller speed difference described in the first exemplary embodiment but also on the sheet deformation coefficient described above.

[0224] This deformation coefficient uses ε in the above equation (6) and equation (7) as a variable depending on the deformation coefficient, rather than as a constant. When the deformation coefficient is d, the equation is transformed as follows.

[0225] l=|psize-(p T -ρ B )cosθ ave *25.4 / dpi|+γstep(|θ T -θ B +Δθ|-ε(d)) (8)

[0226] As described in the first exemplary embodiment, ε is a parameter for a penalty term based on the parallelism of the sheet. More specifically, when the difference between θT and θB is less than or equal to the parameter ε, no penalty is given, and a penalty of 1000 is given in other cases. This threshold for generating a penalty is dynamically changed based on the deformation coefficient. For a sheet with a large deformation coefficient (which tends to be read as a sheet with a large deformation), the value of the parameter ε is large, for example, 1°, so as to have a large allowable amount of angle change. Conversely, for a sheet with a small deformation, the value of the parameter ε is set more strictly, for example, a value of 0.5° is set.

[0227] The optimum combination depending on the four edges of the printed sheet is determined by adding the parameter ε as a variable based on the deformation coefficient to the equation.

[0228] <Remarks>

[0229] According to the present exemplary embodiment, it is possible to more accurately search for sheet edge pairs in combination with the deformation width depending on the characteristics of the sheet. It is also possible to prevent pairs from not being detected for sheets that tend to be read as having large deformations.

[0230] In the present exemplary embodiment, the ADF conveyance length and the basis weight of the sheet are used as information for calculating the deformation coefficient, but it is not limited thereto. If there is some information that will deform the image of the ADF, such information may be added.

[0231] Other Exemplary Embodiments

[0232] In the above exemplary embodiment, the relative positions of the markers are calculated based on Hough transform and edge detection processing, but this can be implemented by machine learning. Fig.9A As shown in FIG, a support vector regression (SVR) that receives the Hough table and the paper size as input and outputs the left and right (ρ, θ) values ​​can be constructed. In this case, the training data is obtained by performing the Hough transform in step S1302 on the pre-acquired read image, and the correct answer data is the manually labeled left and right (ρ, θ) values. In addition, as Fig. 9B As shown in the figure, a model can be constructed that receives image data and paper size as input and outputs the edge position of the sheet using a convolutional neural network (CNN). When training the model, the amount of training data can be increased by image processing (i.e., applying random micro-scaling and micro-angle rotation to the scanned image).

[0233] The present invention can also be implemented by supplying a program for implementing one or more functions in the above exemplary embodiments to a system or device via a network or storage medium and causing one or more processors in a computer of the system or device to read and execute the program. The present invention can also be implemented by a circuit (e.g., an application specific integrated circuit (ASIC)) that implements one or more functions.

[0234] The present invention may be applied to a system including a plurality of devices, or may be applied to an apparatus consisting of one device. For example, a configuration may be adopted in which an apparatus including the image reading unit 130 and an apparatus including the image forming unit 150 are provided as separate apparatuses and are linked by communication. Furthermore, a configuration may be adopted in which an apparatus performing image formation and an apparatus performing image processing are provided as separate apparatuses and are linked by communication.

[0235] The present invention is not limited to each of the above exemplary embodiments, and various modifications (including organic combinations of exemplary embodiments) can be made without departing from the gist of the present invention, and these are not excluded from the scope of the present invention. In other words, the configurations of the above exemplary embodiments and their modified combinations are all included in the present invention.

[0236] Other embodiments

[0237] The embodiment(s) of the present invention may also be implemented by a computer of a system or device that reads and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more completely referred to as a "non-transitory computer-readable storage medium") to perform one or more functions of the above-mentioned embodiment(s) and / or includes one or more circuits (e.g., application-specific integrated circuits (ASICs)) for performing one or more functions of the above-mentioned embodiment(s), and by a computer of the system or device that, for example, reads and executes computer executable instructions from a storage medium to perform one or more functions of the above-mentioned embodiment(s) and / or controls one or more circuits to perform one or more functions of the above-mentioned embodiment(s). The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)) and may include a network of separate computers or separate processors to read and execute computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, one or more of a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage device of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray Disc (BD)TM), a flash memory device, a memory card, etc.

[0238] Other embodiments

[0239] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0240] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image processing device, comprising: Transmission unit; a reading unit for reading an image from a sheet conveyed by the conveying unit, wherein the sheet is an adjustment chart original having a mark printed as a part of the image at a predetermined position and having edges including a left edge, a right edge, an upper edge, and a lower edge; a control unit configured to cause the reading unit to read an image from the sheet to obtain a read image; an acquisition unit, for acquiring a left region of interest ROI image and a right region of interest ROI image from the read image and based on the marks printed at the predetermined positions, each image including at least the shadows of the respective edges of the sheet, and The detection unit is used to perform the following edge detection processing: extracting a plurality of first sheet edge candidates and a plurality of second sheet edge candidates from the left ROI image and the right ROI image, and A sheet edge pair is determined from the plurality of first sheet edge candidates and the plurality of second sheet edge candidates.

2. The image processing apparatus according to claim 1, further comprising An acquisition unit is used to acquire a parameter based on at least the position information of the mark in the image and the position information corresponding to an edge.

3. The image processing device according to claim 2, wherein: The control unit also causes the printer to print the image subject to position adjustment based on at least the one parameter.

4. The image processing apparatus according to claim 2, wherein the edge is a combination of edges whose absolute values ​​of differences between angles among the plurality of second sheet edge candidates are less than or equal to a threshold value and which satisfy a predetermined relationship between a distance between edges and a length of the sheet in a predetermined direction.

5. The image processing apparatus according to claim 1, further comprising: The filter unit is used to perform a filter operation on a partial image of the one image. 6 . The image processing apparatus according to claim 1 , wherein the plurality of second sheet edge candidates are detected using Hough transform.

7. The image processing apparatus according to claim 1, further comprising: an identification unit for identifying a pixel position where a response to detection of the second sheet edge candidate is maximum among neighboring pixels of the edge, and An execution unit is configured to perform a linear approximation on the identified pixel positions.

8. The image processing device according to claim 1, wherein: The edge corresponds to an edge in a conveying direction among edges of the sheet. 9 . The image processing apparatus according to claim 1 , wherein the reading unit includes a contact image sensor (CIS) configured to read an image from the conveyed sheet. 10 . The image processing apparatus according to claim 9 , wherein the reading unit includes a white member facing the CIS.

11. A method for an image processing device, the image processing device comprising: Transmission unit; a reading unit for reading an image from a sheet conveyed by the conveying unit, wherein the sheet is an adjustment chart original having a mark printed as a part of the image at a predetermined position and having edges including a left edge, a right edge, an upper edge, and a lower edge; The method comprises: reading an image from the sheet to obtain a read image; Acquire a left region of interest ROI image and a right region of interest ROI image from the read image and based on the marks printed at the predetermined positions, each image including at least the shadows of the respective edges of the sheet, and Perform edge detection processing as follows: extracting a plurality of first sheet edge candidates and a plurality of second sheet edge candidates from the left ROI image and the right ROI image, and A sheet edge pair is determined from the plurality of first sheet edge candidates and the plurality of second sheet edge candidates.

12. A non-transitory computer-readable storage medium storing computer-executable instructions for causing a computer to execute a method for an image processing apparatus, the image processing apparatus comprising: Transmission unit; a reading unit for reading an image from a sheet conveyed by a conveying unit, wherein the sheet is an adjustment chart original having a mark printed as a part of the image at a predetermined position and having edges including a left edge, a right edge, an upper edge, and a lower edge, the method comprising: reading an image from the sheet to obtain a read image; Acquire a left region of interest ROI image and a right region of interest ROI image from the read image and based on the marks printed at the predetermined positions, each image including at least the shadows of the respective edges of the sheet, and Perform edge detection processing as follows: extracting a plurality of first sheet edge candidates and a plurality of second sheet edge candidates from the left ROI image and the right ROI image, and A sheet edge pair is determined from the plurality of first sheet edge candidates and the plurality of second sheet edge candidates.

Images