Corner bend detection device, corner bend detection system, corner bend detection method, and corner bend detection program

Abstract

The present invention provides a corner fold detection device, a corner fold detection system, a corner fold detection method, and a corner fold detection program that can efficiently and accurately detect corner folds in stacked paper such as printed materials. [Solution] A corner fold detection device for detecting corner folds in laminated paper, comprising: an image acquisition unit that acquires image information of the laminated surface of the laminated paper; and a determination unit that determines whether the corner of the laminated paper is folded. The determination unit determines that a corner fold has occurred in the laminated paper if, in the image information, there exists a blob composed of connected dark pixels whose shape satisfies predetermined conditions.

Description

Technical Field

[0001] The present invention relates to a corner folding detection technique for detecting corner folding of stacked sheets of paper.

Background Art

[0002] In the production of printed materials such as magazines and pamphlets, bookbinding, which involves binding printed sheets such as paper together into one, is carried out as the final process. There are different types of bookbinding depending on the format and binding method. For example, magazines are often produced by saddle stitching. Saddle stitching is a bookbinding method in which printed sheets (printed books) are overlapped and folded in half, and the central fold is fixed with wire or thread. In saddle stitching, "folding" the printed book in the order of pages, "collating" the folded printed books (signatures) in the order of pages, stitching the center of the collated ones with wire, and trimming the three unstitched sides to finish with "three-sided trimming" are performed. Similar operations are often carried out in bookbinding methods other than saddle stitching.

[0003] In bookbinding, due to mechanical factors and external factors such as the environment, corner folding of the printed sheet may occur, where the corners are folded. Corner folding mainly occurs during the folding and collating operations. Printed materials such as magazines that have been three-sided trimmed with corner folding are treated as defective products. Therefore, it is necessary to detect corner folding before shipping the printed materials.

[0004] Detection of corner folding is often carried out by an operator visually checking the side of the three-sided trimmed printed material. The operator visually checks whether there is any corner folding in the printed materials conveyed by a belt conveyor or the like. If corner folding is confirmed, the printed material is extracted as a defective product. Since the printed materials are continuously conveyed and the operator needs to constantly perform visual checks, the operation of detecting corner folding is a burdensome task, and automation is required.

[0005] Corner folding occurs not only during the production of printed materials such as magazines but also when flat-size paper products cut from a roll paper to a specified size are stacked at the time of shipment. Therefore, automation of corner folding detection is also required in such scenarios.

[0006] A technology has been proposed to automatically detect such corner bends. For example, Patent Document 1 proposes a method and apparatus for inspecting whether or not there are folds in the flat sheets of paper in a laminate consisting of many flat sheets. In the method of Patent Document 1, illumination light is shone on the laminated surface of the flat sheets of paper in the laminate, and pairs of bright and dark areas are searched for. If such pairs exist, it is determined that there are folds in those areas. In the method of Patent Document 1, imaging is performed by changing the angle of the illumination light and the angle of the imaging device in order to search for pairs of bright and dark areas.

[0007] Patent Document 2 proposes an apparatus and method for detecting defects such as corner folds and tears in sheets based on a brightness distribution image of the sheet edge surface from the lateral direction of sheets being transported in a scale-overlay state. In the apparatus of Patent Document 2, since the sheets being transported in a scale-overlay state are sparser on the top and denser on the bottom, the brightness distribution image is divided into multiple images in the overlapping direction (up and down direction) where the density of the sheets changes, and the presence or absence of defects is determined by analyzing each divided brightness distribution image. Specifically, the apparatus of Patent Document 2 analyzes the fluctuation state of the number of bright pixels and / or dark pixels included in each divided brightness distribution image, and if the number changes rapidly, it is determined that there are corner folds, tears, etc.

[0008] Patent Document 3 proposes a method and apparatus for determining whether a fold occurs on the fore-edge of a folded signature by measuring the thickness of a saddle-stitched body fixed by a fixing member of the fore-edge trimming section of a three-sided trimming machine, and determining whether the thickness is within the normal range. In the method of Patent Document 3, the detection of a defective book is performed when the saddle-stitched body is fixed by the fixing member of the fore-edge trimming section of the three-sided trimming machine. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2014-38012 [Patent Document 2] Japanese Patent Publication No. 2002-60093 [Patent Document 3] Japanese Patent Publication No. 2014-205208 [Overview of the Initiative] [Problems that the invention aims to solve]

[0010] However, the method described in Patent Document 1 has an unknown accuracy in detecting folds (corner folds), requires adjustment of the angle of the illumination light and the angle of the imaging device to search for pairs of bright and dark areas, and requires imaging to be performed twice. The apparatus described in Patent Document 2 targets sheets in a scale-overlay state and detects corner folds by utilizing the characteristics of that state. Therefore, it is presumed that it would be difficult to detect corner folds in printed materials that are stacked vertically. The method described in Patent Document 3 detects folds (corner folds) by measuring the thickness of the saddle stitching, but the accuracy of this detection is unknown, and a means for measuring the thickness of the saddle stitching is needed.

[0011] The present invention has been made in accordance with the circumstances described above, and the object of the present invention is to provide a corner fold detection device, a corner fold detection system, a corner fold detection method, and a corner fold detection program that can efficiently and accurately detect corner folds in stacked paper such as printed materials. [Means for solving the problem]

[0012] The inventors of this invention have conducted extensive research to solve the above problems and have found that the following invention is suitable for the above purpose, leading to the present invention.

[0013] In other words, the present invention relates to the following invention. <1> A corner fold detection device for detecting corner folds in laminated paper, which is a stacked sheet of paper, A corner fold detection device comprising: an image acquisition unit that acquires image information of the laminated surface of the laminated paper; and a determination unit that determines whether the corner of the laminated paper is folded, wherein the determination unit determines that a corner fold has occurred in the laminated paper if, in the image information, there exists a blob composed of connected dark pixels whose shape satisfies predetermined conditions. <2> The determination unit comprises an image processing unit that performs binarization on the image information and outputs processed image information, and a corner bend determination unit that performs the determination using the processed image information. <1> The corner bending detection device described above. <3> The determination unit performs the determination using the predetermined conditions, which are set for the height and length of the blob and the area information calculated based on the area of ​​the blob. <1> or <2> The corner bending detection device described above. <4> The determination unit uses as area information at least one of the following pieces of information: information indicating the degree to which the blob occupies the frame surrounding the blob, and information indicating the degree to which the blob changes in the longitudinal direction. <3> The corner bending detection device described above. <5> The determination unit performs the determination on adjacent blobs. <1> or <2> The corner bending detection device described above. <6> The determination unit performs the determination using the predetermined conditions, which are set for area information calculated based on the height and length of each adjacent blob, the distance between the adjacent blobs, and the area of ​​the longer of the adjacent blobs. <5> The corner bending detection device described above. <7> The determination unit performs the determination using the conditions set using a learning model that has been learned using information about the corner fold locations in the image information of the laminated paper where the corner fold has occurred, as the predetermined conditions. <1> or <2> The corner bending detection device described above. <8> The laminated paper is a printed material that has been trimmed on three sides during bookbinding, and the image acquisition unit acquires image information of the fore-edge of the printed material. <1> ~ <7> An angle bending detection device as described in any of the following. <9> <1> ~ <8> A corner fold detection system comprising a corner fold detection device and an imaging device as described in any of the above, wherein the imaging device images the laminated surface of the laminated paper and outputs it as image information, and the image acquisition unit acquires the image information output from the imaging device. <10> The transport device further comprises a transport device for transporting the laminated paper, the transport device having a normal path for transporting laminated paper that does not have corner folds and a discharge path for transporting laminated paper that has corner folds, and the transport device transports laminated paper that the determination unit has determined to have corner folds via the discharge path. <9> The corner bend detection system described above. <11> The imaging device further comprises a pressing means for pressing the laminated paper in the stacking direction, and the imaging device captures the stacked surface of the laminated paper pressed by the pressing means. <9> or <10> The corner bend detection system described above. <12> A corner fold detection method for detecting corner folds in laminated paper, comprising: an image acquisition step of acquiring image information of the laminated surface of the laminated paper; and a determination step of determining whether a corner fold has occurred in the laminated paper, wherein in the determination step, if there is a blob in the image information that is composed of connected dark pixels and whose shape satisfies predetermined conditions, it is determined that a corner fold has occurred in the laminated paper. <13> On the computer, <12> A corner bend detection program for executing the corner bend detection method described above. [Effects of the Invention]

[0014] According to the corner fold detection device, corner fold detection system, corner fold detection method, and corner fold detection program of the present invention, corner folds of stacked paper are detected based on blobs composed of connected dark pixels, thereby enabling efficient and accurate detection of corner folds. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows an overview of an example configuration of a corner bend detection system equipped with a corner bend detection device according to the present invention. [Figure 2]It is a block diagram showing a configuration example of a corner break detection device (first embodiment). [Figure 3] It is a block diagram showing a configuration example of a determination unit (first embodiment). [Figure 4] It is a diagram showing an example of drawing binary image information. [Figure 5] It is a schematic diagram for explaining the incidence of illumination light on a corner break location. [Figure 6] It is a diagram showing an example of drawing binary image information at a corner break location. [Figure 7] It is a diagram for explaining parameters used in the first shape condition. [Figure 8] It is a flowchart showing an operation example of a corner break detection system (first embodiment). [Figure 9] It is a block diagram showing a configuration example of a determination unit (second embodiment). [Figure 10] It is a schematic diagram for explaining the incidence of illumination light on a corner break location. [Figure 11] It is a diagram showing an example of drawing binary image information at a corner break location. [Figure 12] It is a diagram for explaining parameters used in the second shape condition. [Figure 13] It is a block diagram showing a configuration example of a determination unit (third embodiment).

Embodiments for Carrying Out the Invention

[0016] In this invention, corner folds that occur in laminated paper such as magazines and brochures, where the corners of the printed sheets constituting the printed material are bent, are detected using image information captured from the surface (laminated surface) that unfolds in the direction of lamination (laminated direction) of the laminated paper. Specifically, in this invention, if there are blobs in the image information whose shape satisfies predetermined conditions, it is determined that a corner fold has occurred. A blob means a lump, and in image processing, blob analysis is sometimes performed in conjunction with binarization as a process targeting blobs. Binarization is a process that converts an image into two values ​​(usually 0 and 1) or grayscale (usually black and white), and blob analysis is a process that detects a set of adjacent pixels as a blob (lump) in a binarized image and analyzes various feature quantities. In this invention, corner folds are detected based on blobs, but since regions where dark pixels are adjacent are detected as blobs, binarization is not necessarily required if dark pixels can be extracted from the image.

[0017] When the laminated surface of laminated paper is imaged, areas without corner folds are brightly imaged because the light illuminating the laminated surface is reflected. However, in areas where corner folds occur, gaps are created due to the folds, and light enters these gaps, resulting in darker images. Even in areas without corner folds, gaps may occur on the laminated surface depending on the condition of the printed sheet, etc., but gaps caused by corner folds have characteristics that stem from the folds. In this invention, this phenomenon and characteristic are utilized to detect regions where dark pixels are connected as blobs, and if a blob whose shape satisfies predetermined conditions exists, it is determined that a corner fold has occurred.

[0018] Thus, in this invention, corner folds are detected using image information captured from the laminated surface. Therefore, corner folds can be efficiently detected with only an imaging device and a corner fold detection device (and a device to irradiate the laminated surface with light as needed). Furthermore, in this invention, corner folds are detected by setting conditions based on a blob, so corner folds can be detected with high accuracy.

[0019] In the present invention, it is possible to use conditions set on information calculated based on the height of the blob, the length of the blob, and the area of ​​the blob (area information) as predetermined conditions (hereinafter referred to as "first shape conditions"). As area information, for example, it is possible to set a frame that circumsects the blob and use the proportion that the blob occupies within that frame (hereinafter referred to as "luminance density"), or information indicating the degree to which the blob changes in the length direction of the blob (hereinafter referred to as "change information").

[0020] In this invention, in corner bend detection based on blobs, it is also possible to perform detection on adjacent blobs. In an image of the stacked surface, where a corner bend occurs, a longer blob and a shorter blob will appear adjacent to each other, based on their characteristics. Therefore, if there are two adjacent blobs whose shape and positional relationship satisfy predetermined conditions (hereinafter referred to as "second shape conditions"), it is determined that a corner bend has occurred. In this invention, as second shape conditions, it is possible to use conditions set for, for example, the height and length of each adjacent blob, the distance between adjacent blobs, the area information of the longer blob, etc.

[0021] In this invention, corner fold detection based on blobs can also be performed using a learned model. For example, corner folds can be detected using conditions set using a learned model that has been trained using image information of laminated paper in which corner folds have occurred (hereinafter referred to as "learned model conditions").

[0022] The present invention may be implemented as a corner bend detection device, or as a corner bend detection system that includes an imaging device for imaging the laminated surface of laminated paper in addition to the corner bend detection device, or as a corner bend detection method for performing the above-mentioned processing, or as a corner bend detection program for executing the corner bend detection method. The corner fold detection system further includes a transport device having a normal path for transporting laminated paper without corner folds and a discharge path for transporting laminated paper with corner folds, and it is also possible to transport laminated paper determined to have corner folds via the discharge path. Furthermore, the corner fold detection system further includes a pressing means for pressing the laminated paper in the stacking direction, and it is also possible for an imaging device to image the stacked surface of the laminated paper pressed by the pressing means.

[0023] Embodiments of the present invention will be described below with reference to the drawings. In each drawing, the same components are denoted by the same reference numerals, and their descriptions may be omitted. Furthermore, the configurations and other aspects described below are illustrative, and the present invention is not limited to these. In addition, the following description focuses on the main configurations and operations for carrying out the present invention, and general processes necessary for carrying out the present invention, such as data input / output processes, may be described or omitted.

[0024] Figure 1 shows an overview of a corner fold detection system (first embodiment) equipped with a corner fold detection device according to the present invention. The corner fold detection system 1 is a system for detecting corner folds in printed materials such as magazines after three-sided trimming during binding, and comprises a corner fold detection device 10, an imaging device 20, an illumination device 30, and a transport device 40. Figure 1 is a diagram combining a perspective view of the imaging device 20 and the illumination device 30 when detecting corner folds in printed materials 50 transported by the transport device 40, and a schematic diagram showing the connection state of each device. In this embodiment, the corner fold detection system 1 detects corner folds in printed materials, but it is also possible to detect corner folds in laminated flat sheets, etc.

[0025] In three-sided trimming of bookbinding, the three sides (top, bottom, and fore-edge) of the printed material 50 that are not bound with wire or the like are trimmed by a three-sided trimming machine or the like to remove excess material and shape it. The top is the part located at the top when the printed material 50 is facing forward, the bottom is the part located at the bottom, and the fore-edge is the part located on the side opposite the unbound edge. The three-sided trimmed printed material 50 is transported by a conveying device 40 such as a belt conveyor and stops for a certain period of time (for example, 3 seconds) in front of the imaging device 20 and the illumination device 30. The imaging device 20 images the fore-edge of the printed material 50, which is illuminated by the illumination device 30, as the stacked surface. The imaging device 20 moves together with the illumination device 30 to take an image of the entire fore-edge of the printed material 50, and the captured image information is sent to the corner fold detection device 10. The corner fold detection device 10 uses the received image information to detect corner folds in the printed material 50. Printed materials 50 in which a corner fold is detected are treated as defective. For example, the transport device 40 has a path for transporting printed materials without corner folds (normal path) and a path for transporting printed materials with corner folds (discharge path). Normally, printed materials are transported via the normal path, but if the corner fold detection device 10 detects a corner fold, the path is switched and the printed materials with corner folds are transported via the discharge path.

[0026] The devices included in the corner bend detection system 1 will now be described. The imaging device 20 images the edge of the printed material 50 as it is transported by the transport device 40. The printed material 50 is placed on the transport belt 41, such as a conveyor belt, of the transport device 40, with its edge parallel to the transport direction. Therefore, the imaging device 20 is installed on the side opposite the edge of the printed material 50, in a direction perpendicular to the horizontal plane to the direction of travel of the transport belt 41 (the transport direction of the printed material 50). Furthermore, in order to image the edge of the printed material 50 from the front, the imaging device 20 is installed at a position that is at the same height as the edge when the printed material 50 is placed on the transport belt 41.

[0027] As described above, the imaging device 20 images the entire edge of the printed material 50, so for example, a line scan camera is used as the imaging device 20. A line scan camera is a camera that images an object in a line, and it images the object or camera in a linear fashion while moving it, and then stitches together the images captured in the line to generate a single image. In the corner fold detection system 1, the imaging device 20 is moved horizontally at a certain distance from the edge of the printed material 50 to perform imaging. For example, a rail 21 is installed parallel to the conveyor belt 41, and the imaging device 20 and lighting device 30 are installed below it in an integrated and movable manner. When imaging the edge of the printed material 50, the imaging device 20 and lighting device 30 move along the rail 21 to perform imaging. The image captured by the imaging device 20 is output to the corner fold detection device 10 as image information Iim. Image information Iim is data (grayscale image) represented by numerical values ​​from 0 to 255 (256 gradations) for each pixel. Furthermore, if it is possible to image the entire edge of the printed material 50, an area scan camera or the like, which images the object as a whole, may be used as the imaging device 20. When using an area scan camera, the illumination device 30 illuminates the entire edge of the printed material 50 with illumination light.

[0028] The illumination device 30 illuminates the edge of the printed material 50 with illumination light so that the imaging device 20 can clearly image the edge of the printed material 50. The illumination device 30 is installed together with the imaging device 20, but is positioned above the imaging device 20, taking into consideration the imaging direction of the imaging device 20, and illuminates the edge of the printed material 50 with illumination light from diagonally above. The illumination device 30 uses a light source capable of emitting directional light, such as an LED (Light Emitting Diode). The illumination device 30 may continuously emit illumination light, or it may only emit illumination when the imaging device 20 is imaging the edge of the printed material 50. In the latter case, the timing of illumination light emission may be controlled by the corner bend detection device 10 or by the imaging device 20.

[0029] The corner bend detection device 10 uses image information Iim output from the imaging device 20 to detect corner bends in the printed material 50. The corner bend detection device 10 uses general-purpose information equipment such as a personal computer, and is equipped with a CPU (including MPUs and MCUs), RAM, ROM, and other memory. Corner bend detection is performed using the memory and a program running on the CPU. The corner bend detection device 10 may be a dedicated device rather than a general-purpose device, and some or all of the corner bend detection process may be performed in hardware.

[0030] Figure 2 shows an example of the configuration of the corner bend detection device 10. The corner bend detection device 10 includes an image acquisition unit 11 and a determination unit 12.

[0031] The image acquisition unit 11 receives image information Iim output from the imaging device 20 and outputs image information IimA. At this time, pre-processing such as image centering and cropping to remove unnecessary parts may be performed using known methods.

[0032] The determination unit 12 uses the image information IimA to determine whether or not a corner fold has occurred in the printed material 50 corresponding to the image information IimA (hereinafter referred to as "corner fold determination"). The determination unit 12 performs the corner fold determination based on blobs detected from the image information IimA. An example of the configuration of the determination unit 12 is shown in Figure 3. The determination unit 12 comprises a binarization processing unit 121, a labeling processing unit 122, and a corner fold determination unit 123, and performs binarization processing and labeling processing on the image information IimA to perform corner fold determination. The binarization processing unit 121, the labeling processing unit 122, and the corner fold determination unit 123 perform binarization processing, labeling processing, and corner fold determination, respectively. The binarization processing unit 121 and the labeling processing unit 122 constitute the image processing unit.

[0033] The binarization processing unit 121 performs binarization on the image information IimA and outputs it as binary image information Ibi. For example, if the image information IimA is a 256-level grayscale image, a predetermined value (e.g., 170) is set as a threshold in advance, and bright pixels (hereinafter referred to as "bright pixels") whose value is greater than or equal to the threshold are replaced with white (1), and dark pixels (hereinafter referred to as "dark pixels") whose value is less than the threshold are replaced with black (0). Alternatively, the binarization process may be performed by inversion, where bright pixels are replaced with black (0) and dark pixels with white (1). The values ​​to be replaced are not limited to 0 and 1, but may also be other values ​​such as 0 and 255. If the image information IimA is a color image, it may be converted to grayscale before binarization, or it may be directly binarized based on certain rules. The threshold may be set as an empirical value, or it may be set using known methods such as the mode method, P-tile method, or discriminant analysis method.

[0034] Figure 4 shows an example of the rendering of binary image information Ibi. Figure 4 is a rendering of a portion of the binary image information Ibi of the entire edge of the printed material 50, with the vertical direction in Figure 4 being the stacking direction of the printed material 50. The binary image information Ibi shown in Figure 4 has been binarized by inversion processing, with black areas corresponding to bright pixels and white areas corresponding to dark pixels. No corner folding has occurred in the area shown in Figure 4, but gaps have occurred at the edge of the printed material 50 due to the condition of the printing sheet, etc., and these areas are represented by white, which are dark pixels.

[0035] The labeling processing unit 122 performs labeling on the binary image information Ibi and outputs it as processed image information Ipd. Labeling is the process of assigning a unique number (label) to each set of consecutive identical values ​​in the binarized image. The determination unit 12 detects areas where dark pixels are connected as blobs, and since the sets to which unique numbers are assigned in the labeling process correspond to blobs, it detects sets of consecutive values ​​(black (0), or white (1) in the case of inversion) assigned to dark pixels in the binary image information Ibi and assigns a number to each set. In the case of the binary image information Ibi shown in Figure 4, numbers are assigned to the white areas. There are two types of labeling: 4-neighborhood (4-connected), which assigns the same number to consecutive sets in the vertical and horizontal directions of the binarized image, and 8-neighborhood (8-connected), which assigns the same number to consecutive sets in the vertical, horizontal, and diagonal directions. The labeling processing unit 122 may use either of these methods for labeling.

[0036] The corner bend detection unit 123 detects blobs from the processed image information Ipd and performs corner bend detection using a first shape condition that has been set in advance for the shape of the blob.

[0037] Here, we will explain the characteristics of a blob when a corner bend occurs. Figure 5 is a diagram illustrating the incidence of illumination light to the corner-folded area (corner-folded area) when illumination light from the illumination device 30 is shone on the edge of the printed material 50 where the corner fold has occurred. Figure 5 is a plan view of the printed sheet 51 with a corner fold, seen from above. The lower side of the printed sheet 51 in Figure 5 is the edge side of the printed material 50, and the corner fold has occurred at the right corner of the printed sheet 51 toward the edge of the printed material 50. Illumination light IL from the illumination device 30, traveling in the direction indicated by the thick arrow, is shone onto the corner-folded area.

[0038] At the corner bend, a gap is created between the non-corner-bent printed sheet 51 located above and below the corner-bent printed sheet 51. The non-corner-bent areas are brightly imaged because the illumination light IL from the illumination device 30 is reflected off the edge, but the corner bend areas are darkly imaged because the illumination light IL enters the gap created there. Also, at the corner bend, when viewed from a position facing the edge, the gap width (length in the stacking direction of the printed material 50) is usually larger at the area closer to the edge (indicated by arrow P1 in Figure 5) than at the area further away (indicated by arrow P2 in Figure 5). The darkly imaged areas are processed as dark pixels in the binarization process by the binarization processing unit 121, and the region where dark pixels are connected becomes a blob. Therefore, the blob corresponding to the darkly imaged corner bend reflects the characteristics of the gap described above. The corner bend determination unit 123 uses this characteristic to set a first shape condition, and if a blob that satisfies the first shape condition exists, it determines that a corner bend has occurred. In this embodiment, the corner bend determination unit 123 uses conditions set for the height, length, and area information of the blob, as well as luminance density and change information, as first shape conditions.

[0039] The corner bend detection unit 123 sequentially detects sets of dark pixels assigned the same number as blobs from the processed image information Ipd. If the detected blobs satisfy the following first shape condition, it is determined that a corner bend has occurred. (1) The height of the blob (length of the printed material 50 in the stacking direction) is within a specified range (for example, 0.3 mm or more and less than 0.5 mm). (2) The length of the blob (length in the direction perpendicular to the lamination direction on the fore-edge of the printed material 50) is within a predetermined range (for example, 3 mm or more and less than 5 mm). (3) The luminance density, which is the proportion of the blob that occupies the frame surrounding the blob (hereinafter referred to as the "blob frame"), for example, the ratio of the area of ​​the blob to the area of ​​the blob frame, is within a predetermined range (for example, 50% or more and less than 75%). (4) Change information, which is information indicating the degree to which the blob changes in the length direction of the blob, for example, the magnitude (absolute value) of the difference in the ratio of the area of ​​the blob to the area of ​​the divided region, calculated in each divided region when the blob frame is divided in the length direction of the blob, is greater than or equal to a predetermined value (for example, 20%).

[0040] Since the shape of the blob may be affected by the thickness and quality of the printed sheet 51, it is preferable to determine the predetermined range and value in the first shape condition described above by taking into consideration the thickness and quality of the printed sheet 51.

[0041] Figure 6 shows an example of plotting binary image information Ibi at a corner bend. The image in Figure 6 corresponds to the corner bend shown in Figure 5, where the corner bend is plotted as a blob Bl. The height of the area indicated by arrow P1, which is close to the edge in Figure 5, is greater than the height of the area indicated by arrow P2, which is farther from the edge in Figure 5.

[0042] Figure 7 is a diagram illustrating the parameters used in the first shape condition described above for the blob Bl shown in Figure 6. Specifically, as shown in Figure 7(A), a blob frame Fr is set that circumscribes the blob Bl, with the vertical length of the blob frame Fr (length in the vertical direction in Figure 7(A)) H being the height of the blob Bl, and the horizontal length of the blob frame Fr (length in the horizontal direction in Figure 7(A)) L being the length of the blob Bl. The luminance density of the blob Bl is the value (percentage) obtained by dividing the area of ​​the blob Bl by the area of ​​the blob frame Fr. The change information of the blob Bl is the magnitude of the difference between the value (percentage) obtained by dividing the area occupied by the blob Bl in one region R1 by the area of ​​region R1 and the value (percentage) obtained by dividing the area occupied by the blob Bl in the other region R2 by the area of ​​region R2, as shown in Figure 7(B). In addition, since blob Bl corresponds to the corner fold on the right side of the printed material 50 towards the fore-edge, the portion of the blob in region R1 on the left side is larger than the portion of the blob in region R2. However, in the case of a blob corresponding to a corner fold on the left side, the portion of the blob in the region on the right side will be larger. Therefore, by identifying the location (right or left) where the corner fold occurs and determining the order of subtraction to find the difference according to the identified location, the condition can be judged using the value of the difference rather than the size of the difference. The corner fold determination unit 123 may also use the ratio of the difference, rather than the size or value of the difference, as change information, and may calculate the change information by dividing the blob frame into thirds, quarters, etc., instead of bisecting it. The corner fold determination unit 123 may select from the above four conditions and use them as the first shape condition, or it may use conditions other than those above as the first shape condition, as long as they are related to the shape of the blob. For example, a condition using the ratio of the height of the blob in region R1 to the height of the blob in region R2 may be used as the first shape condition.

[0043] If the corner bend detection unit 123 determines that a corner bend has occurred in the printed material 50, it outputs a signal (hereinafter referred to as the "detection signal") Sdt indicating that a corner bend has been detected. The detection signal Sdt is input to the transport device 40.

[0044] Upon receiving the detection signal Sdt, the transport device 40 switches the transport path for the printed material and transports the printed material 50 that was the target of the detection signal Sdt via the discharge path. After that, it switches the path again and the transport device 40 transports the printed material via the normal path.

[0045] In this configuration of the corner bend detection system 1, an example of its operation will be explained with reference to the flowchart in Figure 8. It should be assumed that before a corner bend is detected, the printed material 50 is being transported along the normal path of the transport device 40.

[0046] The imaging device 20 captures an image of the edge of the printed material 50 that has been transported by the transport device 40 and outputs image information Iim (step S10). The image information Iim is input to the corner bend detection device 10.

[0047] Image information Iim is input to the image acquisition unit 11, and the image acquisition unit 11 outputs image information Iim as image information IimA to the determination unit 12 (step S20).

[0048] In the determination unit 12, the binarization processing unit 121 performs binarization on the image information IimA and outputs the binary image information Ibi to the labeling processing unit 122 (step S30). The labeling processing unit 122 performs labeling on the binary image information Ibi and outputs the processed image information Ipd to the corner bend determination unit 123 (step S40).

[0049] The corner bend detection unit 123 detects a blob from the processed image information Ipd (step S50). If the detected blob satisfies the first shape condition (step S60), the corner bend detection unit 123 outputs a detection signal Sdt to the transport device 40 (step S70). Upon receiving the detection signal Sdt, the transport device 40 switches the transport path for the printed material to the discharge path (step S80), and the corner bend detection device 10 terminates corner bend detection for the image information Iim. If the detected blob does not satisfy the first shape condition (step S60), the corner bend detection unit 123 detects the next blob from the processed image information Ipd.

[0050] If the corner bend determination unit 123 detects all blobs from the processed image information Ipd without detecting any blobs that satisfy the first shape condition (step S90), the corner bend detection device 10 terminates corner bend detection for the image information Iim.

[0051] The corner bend detection system 1 repeatedly performs the above operations each time it images the edge of the printed material 50.

[0052] The following modifications are possible to this embodiment.

[0053] When imaging the edge of the printed material 50 with the imaging device 20, the printed material 50 may be pressed in the stacking direction while imaging. Printed materials such as magazines manufactured by saddle stitching may have gaps between the printed sheets even when stacked, and these gaps may affect the accuracy of corner fold detection. By pressing the printed material in the stacking direction, the gaps between the printed sheets are reduced, and this effect is suppressed. Even when corner fold detection of printed materials is performed visually by an operator, the operator may hold the printed material by hand to check, so this process is automated. As a means of pressing, for example, a device that drives a flat plate placed on the top surface of the printed material 50 up and down by a motor may be used.

[0054] The corner fold detection system 1 uses the imaging device 20 to image the fore-edge of the printed material 50. However, in addition to the fore-edge, or instead of the fore-edge, the top and bottom of the printed material 50 may also be imaged as stacked surfaces, and corner folds may be detected using the image information from these surfaces. In this case, multiple imaging devices may be used for imaging, or a single imaging device may be moved to each side for imaging.

[0055] The corner-fold detection system 1 removes printed materials 50 that it has determined to have folded corners by transporting them through the discharge path of the transport device 40. However, it is also possible to notify the operator of the presence of folded corners so that the operator can remove the printed materials. In this case, the discharge path would not be necessary. For example, the corner-fold detection device 10 is equipped with a rotating warning light, and when it determines that a folded corner has occurred, it activates the rotating warning light. This allows the operator to know that a folded corner has occurred in the printed materials being transported. Instead of a rotating warning light, the operator may be notified by an alarm or by displaying a message on a display means provided by the corner-fold detection device 10.

[0056] The corner fold detection unit 123 only performs a corner fold detection to determine whether or not a corner fold has occurred in the printed material 50. However, it may also identify the location of the corner fold in the printed material 50. That is, since the corner fold detection unit 123 determines that a corner fold has occurred if there is a blob that satisfies the first shape condition, it may also output the location of the blob that satisfies the first shape condition as information. In this case, since corner folds can occur in multiple locations on the printed material 50, the corner fold detection unit will detect all blobs that satisfy the first shape condition. The location of the corner fold may be displayed in the drawn image by, for example, drawing binary image information Ibi using a display means provided by the corner fold detection device 10.

[0057] Other embodiments of the present invention will be described.

[0058] First, a second embodiment of the present invention will be described. In the corner bend detection device 10 of the first embodiment, the determination unit 12 performs corner bend detection using a first shape condition set based on the shape of one blob, but it is also possible to perform corner bend detection using a second shape condition set based on the shape and positional relationship of adjacent blobs.

[0059] Figure 9 shows an example of the configuration of the determination unit that performs corner bend detection using the second shape condition. Compared to the determination unit 12 in the first embodiment shown in Figure 3, in the determination unit 62 of the second embodiment, the corner bend detection unit 123 has been replaced with a corner bend detection unit 623.

[0060] The corner bend detection unit 623 detects two adjacent blobs from the processed image information Ipd and performs corner bend detection using a pre-set second shape condition.

[0061] Here, we will describe the characteristics of adjacent blobs when a corner bend occurs. Figure 10 is the same as Figure 5, and Figure 11 is a diagram drawn with the left side extended compared to the example of drawing the binary image information Ibi at the corner bend shown in Figure 6. In Figure 11, blob Bl1 is the same as blob Bl shown in Figure 6.

[0062] At the corner bend, as described above, a gap (hereinafter referred to as the "first gap") is created from near the location indicated by arrow P1 to near the location indicated by arrow P2. However, due to the thickness of the printed sheet 51 at the corner bend, another gap (hereinafter referred to as the "second gap") is created at the location to the left of the first gap, indicated by arrow P3. However, since the fore-edge portion of the folded printed sheet 51 remains unfolded in the second gap, the second gap is smaller in length and width than the first gap.

[0063] When the binary image information Ibi is drawn, as shown in Figure 11, the first gap is drawn as blob Bl1 and the second gap is drawn as blob Bl2. Blob Bl2 reflects the characteristics of the second gap described above, so its height and length are smaller than blob Bl1. Also, the end of the corner bend located between the first and second gaps (indicated by arrow P4) is brightly imaged because the printed sheet is densely packed there, causing the illumination light IL from the illumination device 30 to reflect. The corner bend determination unit 623 sets a second shape condition using the characteristics of two adjacent blobs at these corner bend locations, and determines that a corner bend has occurred if there is a blob that satisfies the second shape condition. Note that blobs Bl1 and Bl2, like blob Bl, correspond to the corner bend on the right side toward the edge of the printed material 50, so blob Bl2 is located to the left of blob Bl1. However, in the case of a blob corresponding to a corner bend on the left side, blob Bl2 will be located to the right of blob Bl1. In the following, the longer blob, such as blob Bl1, will be referred to as the primary blob, and the shorter blob, such as blob Bl2, will be referred to as the secondary blob.

[0064] The corner bend detection unit 623, similar to the corner bend detection unit 123, sequentially detects sets of dark pixels assigned the same number as blobs from the processed image information Ipd. It then determines that a corner bend has occurred if two adjacent blobs satisfy the following second shape condition. (1) The height of the main blob is within a predetermined first range (e.g., 0.3 mm or more and less than 0.5 mm), and the height of the secondary blob is within a predetermined second range that is smaller than the first range (e.g., 0.05 mm or more and less than 0.3 mm). (2) The length of the main blob is within a predetermined third range (e.g., 3 mm or more and less than 5 mm), and the length of the secondary blob is within a predetermined fourth range that is smaller than the third range (e.g., 1 mm or more and less than 3 mm). (3) The brightness density of the main blob and the sub-blob is within a predetermined range (for example, 50% or more and less than 75%). (4) The change information of the main blob is greater than or equal to a predetermined value (e.g., 20%). (5) The distance between the main blob and the secondary blob (hereinafter referred to as the "blob distance") is less than a predetermined value (e.g., 1 mm). Furthermore, there may be overlapping areas between the first and second ranges, and there may also be overlapping areas between the third and fourth ranges.

[0065] Figure 12 is a diagram illustrating the parameters used in the second shape conditions described above for blobs Bl1 and Bl2 shown in Figure 11. Specifically, as shown in Figure 12, blob frames Fr1 and Fr2 are set to circumscribe blobs Bl1 and Bl2, respectively. The vertical length H1 and horizontal length L1 of blob frame Fr1 become the height and length of the main blob, respectively, and the vertical length H2 and horizontal length L2 of blob frame Fr2 become the height and length of the sub-blob, respectively. The luminance density of the main blob and sub-blob is calculated based on blob frames Fr1 and Fr2, respectively, and the change information of the main blob is calculated based on blob frame Fr1. The distance Gp between blob frames Fr1 and Fr2 in the lateral direction (left-right direction in Figure 12) becomes the distance between the blobs. Note that the corner bend detection unit 623 may use conditions other than those described above as second shape conditions.

[0066] If the corner bend detection unit 623 determines that a corner bend has occurred in the printed material 50, it outputs a detection signal Sdt indicating that a corner bend has been detected, similar to the corner bend detection unit 123, and the detection signal Sdt is input to the transport device 40.

[0067] An example of operation of a corner bend detection system including a corner bend detection device equipped with a determination unit 62 is the same as that of the corner bend detection system 1, except that the operation of the corner bend determination unit is different as described above.

[0068] A third embodiment of the present invention will now be described. Corner bend detection can be performed not only using conditions with defined ranges and thresholds, such as the first and second shape conditions, but also using learned model conditions set using a learned model trained with image information of the corner bend location.

[0069] Figure 13 shows an example configuration of a determination unit that performs corner bending detection using learned model conditions. Compared to the determination unit 12 in the first embodiment shown in Figure 3, the determination unit 72 in the third embodiment has an added learned model storage unit 724, the labeling processing unit 122 is absent, the corner bending detection unit 123 is replaced by the corner bending detection unit 723, and the binary image information Ibi output from the binarization processing unit 121 is input to the corner bending detection unit 723. In this embodiment, the binarization processing unit 121 alone constitutes the image processing unit.

[0070] The learning model storage unit 724 stores a learning model Lmd that has been learned using binary image information (hereinafter referred to as "learning image information") generated by performing the same binarization process as the binarization process in the binarization processing unit 121 on the image information of the corner fold in the image information of the edge of a printed material in which a corner fold has occurred.

[0071] The learning model Lmd is a model (hereinafter referred to as the "detection model") used in object detection algorithms based on machine learning, such as deep learning or convolutional neural networks. Such object detection algorithms require training with images of the objects to be detected, so the above-mentioned training image information is collected and used for training. Training may be performed using an existing training system or a training system created in-house.

[0072] The corner bend detection unit 723 performs corner bend detection based on the binary image information Ibi, using the learning model conditions set using the learning model Lmd stored in the learning model storage unit 724. When a detection model is used as the learning model Lmd, the corner bend detection unit 723 performs corner bend detection based on the object detection algorithm. The object detection algorithm performs a detection process to find a region where an object exists and an identification process to determine the object present within the found region. Therefore, the corner bend detection unit 723 performs the detection and identification processes on the binary image information Ibi to determine whether a blob corresponding to the corner bend exists. For example, the object detection algorithm outputs the location where a corner bend is presumed to have occurred in the binary image information Ibi and the probability (score) that it is a corner bend. The corner bend detection unit 723 performs corner bend detection using the learning model conditions that indicate a corner bend has occurred if the score is greater than or equal to a predetermined value. Note that there are various object detection algorithms, such as algorithms that perform the detection and identification processes consecutively and algorithms that perform both processes simultaneously, but the corner bend detection unit 723 may use any algorithm.

[0073] If the corner bend detection unit 723 determines that a corner bend has occurred in the printed material 50, it outputs a detection signal Sdt indicating that a corner bend has been detected, similar to the corner bend detection unit 123, and the detection signal Sdt is input to the transport device 40.

[0074] An example of operation of a corner bend detection system including a corner bend detection device equipped with a determination unit 72 is the same as that of the corner bend detection system 1, except that there is no operation by the labeling processing unit 122 and the operation of the corner bend determination unit is different as described above.

[0075] The learning model storage unit 724 stores a detection model as the learning model Lmd, and the corner bend determination unit 723 performs corner bend determination based on an object detection algorithm, but other forms of learning models may also be used. For example, the determination unit 72 includes a labeling processing unit 122, similar to the determination units 12 and 62 in the first and second embodiments, and the corner bend determination unit 723 detects blobs from the processed image information Ipd output from the labeling processing unit 122. The learning model storage unit 724 stores a model learned using image information of blobs corresponding to corner bend locations, and the corner bend determination unit 723 uses that learning model to determine whether the detected blob corresponds to a corner bend location.

[0076] The above-described embodiment can be implemented using a computer and memory configuration, by using the learning model storage unit as memory and implementing the processing of the other components as a program as described above. Each component can also be implemented using hardware such as a dedicated IC (Integrated Circuit) or FPGA (Field Programmable Gate Array). Furthermore, although the above-described embodiment is described in the form of an apparatus or system, the present invention can also take the form of a method or program.

[0077] It should be noted that the present invention is not limited to the above embodiments, and various modifications are possible without departing from the spirit of the invention. Furthermore, matters not explicitly disclosed in the above embodiments do not deviate from what is normally practiced by those skilled in the art, and values ​​that can be easily anticipated by those skilled in the art may be adopted. [Explanation of Symbols]

[0078] 1. Corner Bend Detection System 10-corner bend detection device 11 Image acquisition unit 12, 62, 72 Judgment section 20 Imaging device 21 rails 30 Lighting devices 40 Conveying device 41 Conveyor belt 50 printed matter 51 Print Sheets 121 Binarization Processing Unit 122 Labeling Processing Unit 123, 623, 723 Corner bend detection section 724 Learning Model Memory Unit

Claims

1. A corner fold detection device for detecting corner folds in laminated paper, which is a stacked sheet of paper, An image acquisition unit that acquires image information of the laminated surface of the aforementioned laminated paper, The system includes a determination unit for determining whether the corners of the laminated paper are folded, Corner fold detection device characterized in that the determination unit determines that a corner fold has occurred in the laminated paper when, among the blobs composed of connected dark pixels in the image information, there is a blob whose shape satisfies predetermined conditions.

2. The determination unit, An image processing unit that performs a binarization process on the aforementioned image information and outputs the processed image information, The corner bend detection device according to claim 1, further comprising a corner bend determination unit that performs the determination using the processed image information.

3. The corner bend detection device according to claim 1, wherein the determination unit performs the determination using conditions set as predetermined conditions for the height and length of the blob and area information calculated based on the area of ​​the blob.

4. The corner bend detection device according to claim 3, wherein the determination unit uses at least one of the following pieces of information as area information: information indicating the degree to which the blob occupies a frame circumscribing the blob, and information indicating the degree to which the blob changes in the longitudinal direction.

5. The corner bend detection device according to claim 1, wherein the determination unit performs the determination on adjacent blobs.

6. The corner bend detection device according to claim 5, wherein the determination unit performs the determination using conditions set as predetermined conditions for area information calculated based on the height and length of each of the adjacent blobs, the distance between the adjacent blobs, and the area of ​​the longer of the adjacent blobs.

7. The corner fold detection device according to claim 1, wherein the determination unit performs the determination using conditions set using a learning model learned using information on the corner fold location in the image information of the laminated paper in which the corner fold has occurred, as the predetermined conditions.

8. The aforementioned laminated paper is a printed material that has been trimmed on three sides during bookbinding. The corner bend detection device according to claim 1, wherein the image acquisition unit acquires image information of the edge of the printed material.

9. An angle-bending detection system comprising an angle-bending detection device and an imaging device according to any one of claims 1 to 8, The imaging device captures an image of the laminated surface of the laminated paper and outputs it as image information. An angle bend detection system characterized in that the image acquisition unit acquires image information output from the imaging device.

10. The device further comprises a conveying device for conveying the aforementioned laminated paper, The aforementioned transport device It has a normal path for transporting laminated paper that has not had its corners folded and a discharge path for transporting laminated paper that has had its corners folded. The corner bend detection system according to claim 9, wherein the stacked paper that has been determined to have a corner bend by the determination unit is transported through the discharge path.

11. The system further comprises a pressing means for pressing the stacked paper in the stacking direction, The corner fold detection system according to claim 9, wherein the imaging device images the laminated surface of the laminated paper pressed by the pressing means.

12. A method for detecting corner folds in laminated paper, which is a stacked sheet of paper, An image acquisition step of acquiring image information obtained by capturing an image of the laminated surface of the laminated paper, The system includes a determination step for determining whether the corners of the laminated paper are folded, The method for detecting corner folds, characterized in that, in the determination step, if there is a blob in the image information that is composed of connected dark pixels and whose shape satisfies predetermined conditions, it is determined that a corner fold has occurred in the laminated paper.

13. A corner bend detection program for causing a computer to perform the corner bend detection method described in claim 12.

Images