Inspection system, inspection method, and inspection program

Abstract

To enable inspection of a bundle comprising a plurality of sheets of recording medium.SOLUTION: An inspection system of the present invention comprises an acquisition unit for acquiring information on the shape of a bundle comprising a plurality of sheets of processed recording medium, and a determination unit configured to inspect the bundle on the basis of the information acquired by the acquisition unit.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an inspection system, an inspection method, and an inspection program. [Background technology]

[0002] Usually, post-processing such as booklet binding is performed after printing. Conventionally, there are known techniques for inspecting printed matter produced by printing. For example, Patent Document 1 describes "a system that uses a sensor that detects the surface height of printed paper to inspect both paper curl and shrinkage defects." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-88322 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the past, there was no technology for inspecting a bundle including a plurality of recording media, and therefore inspection of booklets and the like produced by post-processing had to be done manually.

[0005] An object of the present invention is to provide an inspection system, an inspection method, and an inspection program capable of inspecting a bundle including a plurality of recording media. [Means for solving the problem]

[0006] The inspection system of the present invention includes an acquisition unit that acquires information regarding the shape of a stack including a plurality of processed recording media, and a judgment unit that inspects the stack based on the information acquired by the acquisition unit.

[0007] The inspection method according to the present invention obtains information about the shape of a bundle including a plurality of processed recording media, and inspects the bundle based on the obtained information.

[0008] The inspection program of the present invention is an inspection program for causing a computer to execute an acquisition process for acquiring information regarding the shape of a bundle including a plurality of processed recording media, and a judgment process for inspecting the bundle based on the information acquired by the acquisition process. Effect of the Invention

[0009] According to the present invention, a stack including a plurality of recording media can be inspected. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of an inspection system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of saddle stitching. [Diagram 3] FIG. 3 is a diagram illustrating an example of a bookbinding without stitching. [Figure 4] FIG. 4 is a functional block diagram showing a partial configuration of an inspection system according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing an example of a monochrome heat map corresponding to a non-defective paper stack. [Figure 6] FIG. 6 is a diagram showing an example of a monochrome heat map corresponding to a defective paper stack. [Figure 7] FIG. 7 is a diagram showing an example of two-dimensional contour lines corresponding to a non-defective paper stack. [Figure 8] FIG. 8 is a diagram showing an example of two-dimensional contour lines corresponding to defective paper stacks. [Figure 9] FIG. 9 is a schematic diagram for explaining the flow of non-defective product learning. [Figure 10] FIG. 10 is a schematic diagram illustrating the flow of inference using a learning model. [Figure 11]FIG. 11 is a diagram illustrating a case where there is no difference between the original image and the generated image. [Figure 12] FIG. 12 is a diagram for explaining a case where there is a difference between an original image and a generated image. [Figure 13] FIG. 13 is a diagram for explaining an example of a paper stack having a defective portion. [Figure 14] FIG. 14 is a flowchart showing an example of a method for constructing a learning model. [Figure 15] FIG. 15 is a diagram for explaining normalization performed in the surface direction of a paper stack. [Figure 16] FIG. 16 is a diagram for explaining normalization performed in the height direction of a paper stack. [Figure 17] FIG. 17 is a diagram for explaining an example in which the shape of a paper stack in the height direction changes depending on the basis weight of the cover. [Figure 18] FIG. 18 is a diagram for explaining an example in which the shape of a paper bundle in the height direction changes depending on the number of sheets. [Figure 19] FIG. 19 is a flowchart showing an example of a method for inspecting a paper stack using the inspection system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this specification and the drawings, elements having substantially the same functions or configurations are given the same reference numerals, and duplicated explanations will be omitted. The present invention relates to a technique for inspecting a bundle including a plurality of processed recording media. In the following embodiment, a case where the recording media is paper will be described as an example. However, the recording media is not limited to paper.

[0012] FIG. 1 is a schematic diagram showing an example of the configuration of an inspection system according to an embodiment of the present invention. 1, the inspection system 10 includes a paper feeder 11, a printer 12, an image inspection machine 13, a post-processing machine 14, a bundle conveyor 15, a discharge mechanism 16, an acquisition unit 17, and an inspection computer 18. In the figure, the left-facing arrow A indicates the paper feed direction.

[0013] The paper feeder 11 supplies paper. The paper feeder 11 supplies paper stored in a tray or the like, one sheet at a time. The printer 12 prints an image on one side or both sides of the paper supplied by the paper feeder 11. The printer 12 corresponds to a printing unit.

[0014] The image inspection machine 13 inspects the image printed on the paper by the printer 12. The image inspection machine 13 has a first image inspection sensor 13a that inspects the image printed on the first side of the paper, and a second image inspection sensor 13b that forms the image printed on the second side of the paper. From the paper feeder 11 to the post-processing machine 14, the paper is transported along a transport path 19 by a paper transport mechanism (not shown). In FIG. 1, the transport path for double-sided printing is omitted.

[0015] A media sensor 20 is disposed on the conveying path 19 from the paper feeder 11 to the printing press 12. The media sensor 20 detects information related to the paper. The information detected by the media sensor 20 includes the basis weight of the paper. The printing press 12 is also provided with an operation unit 26 and a system control unit 27.

[0016] The operation unit 26 functions as a user interface. Although not shown, the operation unit 26 includes an input unit and a display unit. The input unit accepts various information input from the user. The display unit displays various information to the user. The information accepted by the input unit includes information related to the print job. The information related to the print job includes the number of sheets of paper that make up a bundle containing multiple sheets of paper (hereinafter also referred to as a "paper bundle"). The paper bundle is generated by processing by the post-processing machine 14. Note that the basis weight of the paper may be obtained using the media sensor 20 or may be input by the user via the operation unit 26, for example.

[0017] The system control unit 27 comprehensively controls the operation of the entire inspection system 10. The objects controlled by the system control unit 27 include the paper feeder 11, the printing machine 12, the image inspection machine 13, the post-processing machine 14, the bundle conveyor 15, the discharge mechanism 16, the acquisition unit 17, and the like.

[0018] The post-processing machine 14 generates a bundle including a plurality of sheets of paper, that is, a paper bundle 21, by processing. The post-processing machine 14 corresponds to a processing section. The post-processing machine 14 is disposed downstream of the printing machine 12 in the paper passing direction A. Therefore, the plurality of sheets of paper on which images are printed by the printing machine 12 are processed by the post-processing machine 14. The post-processing machine 14 is also disposed downstream of the image inspection machine 13 in the paper passing direction A. Therefore, the plurality of sheets of paper on which images are inspected by the image inspection machine 13 are processed by the post-processing machine 14. The post-processing machine 14 generates a paper bundle 21 including the plurality of sheets of paper by processing, using the plurality of sheets of paper on which images are printed by the printing machine 12 and inspected by the image inspection machine 13. Note that the sheets of paper determined to have a printing defect by the image inspection machine 13 may be sent to a paper discharge section (not shown) disposed upstream of the processing section of the post-processing machine 14 and removed from the processing target. By removing them before processing, at least papers with printing defects will not be included in the papers to be processed. If papers with printing defects are not removed before processing, the inspection results by the image inspection machine 13 are stored in a memory unit, and after processing, the paper stack 21 containing the defective papers is sorted as defective products and distinguished from non-defective products. Processing is also called post-processing, since it is performed after printing.

[0019] The paper bundle includes at least one of the following (A) to (C). (A) A stack of multiple sheets of paper stacked together without folding (B) A stack of multiple sheets of paper that have been folded and stacked together (C) A stack of folded sheets of paper

[0020] The post-processing machine 14 processes a plurality of sheets of paper based on the conditions specified in the print job. In general, there are many types of processing. For example, processing includes stapling, punching, case binding, folding, saddle stitching, non-binding, and perfect binding. Non-binding is also called scrum binding or air-stitching. Booklet binding is one form of processing. A booklet is obtained by binding the above-mentioned (B) bundle or the above-mentioned (C) bundle into a booklet. The bound booklet is broadly divided into booklets that require binding and booklets that do not require binding. Booklets that require binding include saddle stitched booklets. Booklets that do not require binding include non-binding booklets. In addition, an example of a paper bundle that is not a booklet is a paper bundle obtained by stapling or punching the above-mentioned (A) bundle.

[0021] FIG. 2 is a diagram illustrating an example of saddle stitching. In saddle stitching, first, multiple sheets of paper 1 are stacked together and multiple staples 2 are driven into the center line of the sheets of paper 1. Next, the multiple sheets of paper 1 are folded along the center line. This results in a saddle stitched booklet, i.e., a saddle stitched booklet 3.

[0022] FIG. 3 is a diagram illustrating an example of a bookbinding without stitching. In non-binding, multiple sheets of paper 1 folded in half are stacked in page order to form a booklet. This results in a non-binding bound booklet 4. Non-binding bound booklets are also called air-bound booklets because they are not bound with staples or thread.

[0023] In the unstitched booklet 4, defects such as folding or tearing of the paper 1 may occur. In the saddle-stitched booklet 3, in addition to defects such as folding or tearing of the paper 1, defects related to the stapling process may occur. Defects related to the stapling process include, for example, blank driving of the staple 2 and buckling of the staple 2. The position at which the staple 2 is driven is specified as one of the conditions of the above-mentioned processing. Therefore, it is possible to identify the location where the defect related to the stapling process occurs. In contrast, the location where the defect such as folding or tearing of the paper 1 occurs is often not determined. Also, the shape in which the paper 1 will be folded or torn is often not determined. Therefore, it is difficult to identify the location where the defect such as folding or tearing of the paper 1 occurs.

[0024] The bundle conveyor 15 conveys a bundle including a plurality of sheets of paper that have been processed by the post-processing machine 14, that is, a paper bundle 21. The bundle conveyor 15 is constituted by, for example, a plurality of belt conveyors.

[0025] The discharge mechanism 16 switches the destination of the paper stack depending on the inspection result by the judgment unit 32 (Figure 2). The discharge mechanism 16 corresponds to the discharge unit. The judgment unit 32 judges whether the paper stack to be inspected is a non-defective product. Details of the judgment unit 32 will be described later. The discharge mechanism 16 has a posture conversion function. The posture conversion function is a function that converts the posture of the discharge mechanism 16 between a first posture and a second posture. The first posture is the posture shown by the dotted line in Figure 1. The second posture is the posture shown by the solid line in Figure 1.

[0026] When the discharge mechanism 16 is in the first position, the destination of the paper stack is switched to a non-defective product collection section (not shown). When the discharge mechanism 16 is in the second position, the destination of the paper stack is switched to a defective product collection section (not shown). When the determination section 32 determines that the paper stack to be inspected is a non-defective product, the discharge mechanism 16 is maintained in the first position so that the destination of the non-defective paper stack is the non-defective product collection section. On the other hand, when the determination section 32 determines that the paper stack to be inspected is not a non-defective product, that is, a defective product, the discharge mechanism 16 changes from the first position to the second position so that the destination of the defective paper stack is the defective product collection section.

[0027] The acquisition unit 17 acquires information about the shape of a stack including multiple processed sheets of paper. The information acquired by the acquisition unit 17 is preferably three-dimensional information. The three-dimensional information includes information about the surface shape of the paper stack and information about the height shape of the paper stack. The surface shape of the paper stack is the shape of the paper stack when viewed in a plane, in other words, the shape that defines the size of the paper stack. The height direction of the paper stack can be rephrased as the thickness direction of the paper stack. The three-dimensional information can also be rephrased as information that indicates the three-dimensional shape of the paper stack.

[0028] The acquisition unit 17 has a first acquisition unit 17a and a second acquisition unit 17b. The first acquisition unit 17a acquires information on the shape of the paper bundle from the front side of the paper bundle. The second acquisition unit 17b acquires information on the shape of the paper bundle from the back side of the paper bundle. The first acquisition unit 17a is arranged above the bundle conveyor 15. When the first acquisition unit 17a is arranged in this manner, the front side of the paper bundle corresponds to the upper side of the paper bundle 21 conveyed by the bundle conveyor 15. The second acquisition unit 17b is arranged below the bundle conveyor 15. When the second acquisition unit 17b is arranged in this manner, the back side of the paper bundle corresponds to the lower side of the paper bundle 21 conveyed by the bundle conveyor 15. In the following description, when it is not necessary to distinguish between the first acquisition unit 17a and the second acquisition unit 17b, they are simply referred to as the acquisition unit 17.

[0029] The acquisition unit 17 acquires information about the shape of the paper stack 21 by using light emitted toward the paper stack 21. The acquisition unit 17 is configured, for example, by a light-cutting type laser displacement meter. The acquisition unit 17, which is configured by a light-cutting type laser displacement meter, acquires three-dimensional information (three-dimensional data) about the shape of the paper stack 21 by irradiating a band-shaped laser light onto the surface of the paper stack 21 and receiving the reflected light from the paper stack 21. When the acquisition unit 17 is configured by a light-cutting type laser displacement meter, the above three-dimensional information can be acquired with high accuracy without distorting the shape of the paper stack 21.

[0030] In the present embodiment, a laser displacement meter of a light cutting type is given as an example of the acquisition unit 17 that acquires the above three-dimensional information using light, but the present invention is not limited thereto. For example, the acquisition unit 17 may be configured to acquire the above three-dimensional information using a ToF (Time of Flight) sensor, a multi-lens camera, a pattern illumination method, a structured illumination method, or the like. The acquisition unit 17 may also be configured to acquire information about the shape of the paper stack 21 using a sound emitted to the paper stack 21. Specifically, the acquisition unit 17 acquires three-dimensional information about the shape of the paper stack 21 by transmitting a sound wave toward the surface of the paper stack 21 and receiving the sound wave reflected by the surface of the paper stack 21. Since the above-mentioned methods using light or sound are both non-contact methods, the above three-dimensional information can be acquired without destroying the shape of the paper stack 21. The acquisition unit 17 may also be configured to acquire information about the shape of the paper stack 21 using a contact member that contacts the paper stack 21. The contact member is, for example, a probe. When a contact member is used, the contact member is brought into contact with the paper stack 21 with a contact pressure that does not cause the shape of the paper stack 21 to be distorted.

[0031] The inspection computer 18 is a computer used to inspect paper bundles. The inspection computer 18 includes processors such as a GPU (Graphics Processing Unit) and a CPU (Central Processing Unit), and storage units such as a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), and an SSD (Solid State Drive). Each function realized by the inspection computer 18 will be described later.

[0032] FIG. 4 is a functional block diagram showing a partial configuration of an inspection system according to an embodiment of the present invention. As shown in Fig. 4, the inspection system 10 includes the above-mentioned acquisition unit 17, inspection computer 18, media sensor 20, and operation unit 26. The inspection computer 18 includes a preprocessing unit 31, a determination unit 32, and a recording unit 33. The preprocessing unit 31 takes in and preprocesses the three-dimensional information acquired by the acquisition unit 17. The preprocessing unit 31 sends information (data) generated by the preprocessing to the determination unit 32. The preprocessing unit 31 includes a normalization processing unit 35 and an imaging processing unit 36. The information preprocessed by the preprocessing unit 31 is input to a learning model 37.

[0033] The normalization processing unit 35 normalizes the three-dimensional information acquired by the acquisition unit 17 according to the thickness and size of the paper stack. Normalization is performed as one of the pre-processing processes. Normalization refers to a process of converting a value to be normalized into a certain range based on a certain rule. As described above, the three-dimensional information acquired by the acquisition unit 17 includes information on the shape of the paper stack in the surface direction and information on the shape of the paper stack in the height direction, as well as information on the shape of the paper stack in the surface direction. Furthermore, the information on the shape of the paper stack in the surface direction includes a numerical value representing the length and width of the paper stack. The length and width of the paper stack can be rephrased as the lengths of the long and short sides of the paper stack. On the other hand, the information on the shape of the paper stack in the height direction includes a numerical value representing the thickness of the paper stack. The thickness of the paper stack can be rephrased as the height of the paper stack.

[0034] The imaging processing unit 36 ​​images the three-dimensional information acquired by the acquisition unit 17. The imaging is performed as one of the pre-processing processes. The imaging is a process for enabling the three-dimensional information acquired by the acquisition unit 17 to be handled as an image. For this reason, the imaging is also a process for converting three-dimensional information into two-dimensional information, that is, two-dimensionalization. However, the image generated by imaging the three-dimensional information includes information about the shape of the paper stack in the height direction. Specific examples of images generated by imaging include a two-dimensional heat map or a two-dimensional contour line. In other words, the above three-dimensional information is represented by a two-dimensional heat map or a two-dimensional contour line.

[0035] Two-dimensional heat maps are divided into two types. One is an image that expresses the difference in height with a difference in color. In this case, the two-dimensional heat map is a heat map image having a three-layer structure of R (red), G (green), and B (blue). In this embodiment, as an example, the three-dimensional information imaged by the imaging processing unit 36 ​​is represented by a heat map image having the above-mentioned three-layer structure. The other is an image that expresses the difference in height with a difference in light and dark, that is, a monochromatic gradation. In this case, the two-dimensional heat map is a heat map image with a one-layer structure. The two-dimensional heat map represented by a monochromatic gradation is, for example, a monochrome heat map. FIG. 5 is a diagram showing an example of a monochrome heat map corresponding to a non-defective paper stack. FIG. 6 is a diagram showing an example of a monochrome heat map corresponding to a defective paper stack.

[0036] A two-dimensional contour line is an image that expresses differences in height as differences in the density of the contour lines. The density of the contour lines increases as the intervals between the contour lines become narrower. FIG. 7 is a diagram showing an example of two-dimensional contour lines corresponding to a non-defective paper stack. FIG. 8 is a diagram showing an example of two-dimensional contour lines corresponding to a defective paper stack. The image generated by imaging the three-dimensional information may be an image that combines a two-dimensional heat map and two-dimensional contour lines.

[0037] The determination unit 32 inspects the paper stack based on the three-dimensional information acquired by the acquisition unit 17. The acquisition unit 17 is disposed downstream of the post-processing machine 14 in the paper feed direction A. Therefore, the determination unit 32 inspects the paper stack processed by the post-processing machine 14. The determination unit 32 also inspects the paper stack using the learning model 37. In the paper stack inspection performed by the determination unit 32, it is determined whether the paper stack to be inspected is a non-defective product. The determination unit 32 determines whether the paper stack is good or bad by inference using the learning model 37. When the acquisition unit 17 has the first acquisition unit 17a and the second acquisition unit 17b as described above, the determination unit 32 can inspect both the front side and the back side of the paper stack. The determination method by the determination unit 32 will be described later.

[0038] The learning model 37 is a deep learning model (DLM; Deep Learning Module). The deep learning model is a learning model created by deep learning. By using the deep learning model to inspect paper stacks, it becomes possible to detect defective paper stacks like a human being. The deep learning model uses data normalized by the normalization processing unit 35. Also, the deep learning model uses data imaged by the image processing unit 36.

[0039] In this embodiment, an unsupervised learning model is used as the deep learning model. When using an unsupervised learning model, it is not necessary to define defective products when inspecting a paper stack. Therefore, it is possible to detect defective paper stacks that are not anticipated in advance.

[0040] In this embodiment, an autoencoder is used as the deep learning model. An autoencoder belongs to a model of an unsupervised learning method. An autoencoder is a model that is trained so that input data and output data are the same. In this embodiment, a convolutional autoencoder is used as the deep learning model. A convolutional autoencoder is an autoencoder that uses a convolutional neural network. A convolutional autoencoder is an autoencoder that is suitable for learning images.

[0041] When the judgment unit 32 judges the paper stack to be inspected as defective, the recording unit 33 records information about the defective product. The information about the defective product includes, for example, information about the time when the defective product occurred and the defective part of the defective product. The recording unit 33 accumulates information (data) about the defective product every time the judgment unit 32 detects a defective product. The data accumulated in the recording unit 33 can be used for machine learning, for example.

[0042] Next, a method for constructing the learning model 37 used for inspecting a stack of paper will be described. The learning model 37 may be constructed using the inspection system 10, or may be constructed using a system other than the inspection system 10. In this embodiment, as a preferred example, the learning model 37 is constructed using the inspection system 10.

[0043] First, the worker prepares the number of paper bundles 21 required for deep learning. The prepared paper bundles 21 are all non-defective paper bundles 21a. In this embodiment, a learning model 37 is constructed by unsupervised learning using only non-defective paper bundles 21a, i.e., non-defective learning.

[0044] FIG. 9 is a schematic diagram for explaining the flow of non-defective product learning. 9, the non-defective paper stacks 21a are placed on the stack conveyor 15 at intervals in the paper feed direction A. The non-defective paper stacks 21a are conveyed in the paper feed direction A by the drive of the stack conveyor 15. At this time, the acquisition unit 17 sequentially acquires three-dimensional information (three-dimensional data) relating to the shape of each of the paper stacks 21a.

[0045] The three-dimensional information acquired by the acquisition unit 17 is imaged (two-dimensionalized) by the imaging processing unit 36, and converted into a two-dimensional heat map 40a. The converted two-dimensional heat map 40a is input to the AI ​​model 37a. The AI ​​model 37a is a model that functions as an image AI (Artificial Intelligence). The AI ​​model 37a is also a model before learning or during learning. The AI ​​model 37a is trained to restore an image identical to the input image.

[0046] In this embodiment, a convolutional autoencoder is used as the AI ​​model 37a. The convolutional encoder includes an encoder that converts input data into low-dimensional data by dimensional compression, and a decoder that restores an image from the low-dimensional data. As a result, the AI ​​model 37a is trained to restore an image identical to the original image with a small amount of information. Therefore, the image output by the AI ​​model 37a using the convolutional encoder is an image in which only the main features of the original image are restored. The trained AI model 37a is used as the training model 37 to inspect a stack of paper.

[0047] FIG. 10 is a schematic diagram for explaining the flow of inference using the learning model 37. As shown in Fig. 10, the multiple paper bundles 21 conveyed by the bundle conveyor 15 may include defective paper bundles 21b in addition to non-defective paper bundles 21a. In this case, when the three-dimensional information of each of the paper bundles 21a and 21b is acquired in sequence by the acquisition unit 17 and the acquired three-dimensional information is imaged (two-dimensionalized) by the imaging processing unit 36, different two-dimensional heat maps 40a and 40b are obtained. The two-dimensional heat map 40a is a heat map obtained by imaging the three-dimensional information of the non-defective paper bundle 21a. The two-dimensional heat map 40b is a heat map obtained by imaging the three-dimensional information of the defective paper bundle 21b.

[0048] When the two-dimensional heat map 40a is input to the learning model 37, the learning model 37 generates an image 40c by inference based on the above-mentioned non-defective product learning. In non-defective product learning, learning is performed using only non-defective paper stacks 21a. Therefore, the image 40c generated by the learning model 37 is substantially the same as the original image, the two-dimensional heat map 40a. Therefore, as shown in FIG. 11, the difference between the original image 40a, which is the image input to the learning model 37, and the generated image 40c, which is the image generated by the learning model 37, is almost zero. In this way, when there is no difference between the original image 40a and the generated image 40c, the judgment unit 32 judges the paper stack 21a to be inspected to be a non-defective product.

[0049] In contrast, when the two-dimensional heat map 40b is input to the learning model 37, the learning model 37 generates an image 40d by inference based on the above-mentioned good product learning. At this time, the learning model 37 attempts to restore the original image according to the features extracted by the good product learning. However, in the good product learning, information on defective products is not learned. For this reason, the learning model 37 cannot restore defective parts present in the defective paper stack 21b. Therefore, the image 40d generated by the learning model 37 is an image that is substantially different from the two-dimensional heat map 40b, which is the original image. Therefore, as shown in FIG. 12, the difference between the original image 40b and the generated image 40d is not zero. When there is such a difference between the original image 40b and the generated image 40d, the judgment unit 32 judges the paper stack 21b to be inspected to be defective.

[0050] Here, as shown in FIG. 13, a case is considered in which four defective parts P1, P2, P3, and P4 exist in the paper stack 21b. In the defective part P1, the staple 2 is not driven in normally and buckles, and a part of the staple 2 protrudes from the edge of the paper stack 21b. In the defective part P2, the paper stack 21b is folded. When the paper stack 21b is folded, the height of the folded part is lower than that of the non-folded non-defective paper stack 21a. In both the defective part P3 and the defective part P4, the edges of the paper stack 21b overlap irregularly. In this case, the learning model 37 cannot restore the respective defective parts P1, P2, P3, and P4. Therefore, as shown in FIG. 12, abnormal characteristic parts 3a, 3b, 3c, and 3d remain in the difference between the original image 40b and the generated image 40d. The abnormal characteristic part 3a appears at the position of the defective part P1. Abnormal feature portion 3b appears at the position of defect portion P2. Abnormal feature portion 3c appears at the position of defect portion P3. Abnormal feature 3d appears at the position of defect portion P4. This enables judgment unit 32 to detect defective products based on the presence or absence of the difference, and to identify the defective portion of the detected defective product.

[0051] FIG. 14 is a flowchart showing an example of a method for constructing the learning model 37. First, the inspection computer 18 acquires the basis weight of the cover of the paper bundle and the number of sheets constituting the paper bundle as information about the paper bundle used for the above-mentioned good quality learning (step S1). The cover of the paper bundle is made of paper, but the cover of a bundle including multiple recording media does not necessarily have to be made of paper. The information on the basis weight and number of sheets may be input by the user via the operation unit 26. In this embodiment, the number of paper bundles used for good quality learning is M. The above basis weight and number of sheets are common to the M paper bundles used for good quality learning.

[0052] Next, the acquisition unit 17 acquires three-dimensional information of one of the M paper bundles used for the non-defective learning (step S2). The number of paper bundles used for the non-defective learning is determined according to the number of paper bundles required for learning the model by deep learning.

[0053] Next, the normalization processing unit 35 normalizes the three-dimensional information acquired by the acquisition unit 17 in step S2 (step S3). Here, a specific example of normalization will be described. The three-dimensional information to be normalized is information about the shape of the paper stack. This three-dimensional information includes information about the shape of the paper stack in the surface direction and information about the shape of the paper stack in the height direction. Therefore, normalization of the three-dimensional information is performed in both the surface direction and the height direction of the paper stack.

[0054] Fig. 15 is a diagram for explaining normalization performed in the surface direction of a paper stack. For convenience of explanation, in Fig. 15, the shape of the paper stack in the surface direction is shown as a two-dimensional image. As shown in FIG. 15, the size of the paper stack before normalization (Ha×Wa) is converted to a predetermined range of size (Hb×Wb) by normalization in the plane direction. Then, during learning shown in FIG. 9, the two-dimensional heat map 40a converted to a certain range of size (Hb×Wb) by normalization in the plane direction is input to the AI ​​model 37a. Also, during inference shown in FIG. 10, the two-dimensional heat maps 40a, 40b converted to a certain range of size (Hb×Wb) by normalization in the plane direction are input to the learning model 37. This allows the size of the two-dimensional heat map to be consistent between learning and inference.

[0055] Fig. 16 is a diagram for explaining normalization performed in the height direction of a paper stack. For convenience of explanation, Fig. 16 shows the shape of the paper stack in the height direction as a two-dimensional image. As shown in FIG. 16, the height dimension (thickness dimension) Ta of the paper stack before normalization is converted into a height dimension Tb within a predetermined fixed range by normalization in the height direction. Then, during learning shown in FIG. 9, the two-dimensional heat map 40a converted into a height dimension Tb within a fixed range by normalization in the height direction is input to the AI ​​model 37a. Also, during inference shown in FIG. 10, the two-dimensional heat maps 40a, 40b converted into a height dimension Tb within a fixed range by normalization in the height direction are input to the learning model 37. This makes it possible to align the height (thickness) of the two-dimensional heat map during learning and inference.

[0056] By normalizing the three-dimensional information in this way, it is possible to inspect paper stacks of different sizes or thicknesses using the same learning model. When normalizing in the height direction of a paper stack, the entire paper stack may be normalized at a uniform magnification, or the high and low points of the paper stack may be normalized at different magnifications.

[0057] Returning to FIG. 14 for further explanation. Next, the imaging processor 36 images the three-dimensional information normalized in step S3 (step S4). In step S4, the three-dimensional information is imaged, whereby the three-dimensional information is converted into a two-dimensional bitmap.

[0058] Next, the inspection computer 18 judges whether M two-dimensional bitmaps, which are images for learning, have been prepared (step S5). If the inspection computer 18 judges negative in step S5, the process returns from step S5 to step S2. If the inspection computer 18 judges negative in step S5, in other words, the preparation of images for learning has not been completed. If the inspection computer 18 judges positive in step S5, the process proceeds from step S5 to step S6. If the inspection computer 18 judges positive in step S5, in other words, the preparation of images for learning has been completed.

[0059] In step S6, the inspection computer 18 uses M images (two-dimensional heat maps) to train the AI ​​model 37a shown in Fig. 9 on non-defective products. Specifically, the M two-dimensional heat maps are input to the AI ​​model 37a as input images, and the AI ​​model 37a is trained to output the same image as the input image for all two-dimensional heat maps.

[0060] Next, the inspection computer 18 stores the trained AI model 37a in the training database 42 (see FIG. 1) as the training model 37 (step S7). As a result, a model trained on the combination of the above-mentioned basis weight and the above-mentioned number of sheets is stored in the training database 42. The training database 42 is one of the databases included in the inspection computer 18.

[0061] The processing flow shown in Fig. 14 is performed for each combination of the basis weight and the number of sheets. As a result, a model trained for each combination of the basis weight and the number of sheets is stored in the learning database 42 as shown in Fig. 1.

[0062] The reason why models trained for each combination of the basis weight of the cover of the paper bundle and the number of sheets constituting the paper bundle are stored in the training database 42 in this manner is as follows. First, consider a paper bundle such as a booklet in which multiple sheets of paper are folded and stacked. The height-wise shape of this paper bundle is greatly affected by the basis weight of the cover of the paper bundle. For this reason, as shown in FIG. 17, the shape of the paper bundle in the height direction is different between a paper bundle 21 in which the basis weight of the cover and the basis weight of the body are both small and a paper bundle 21 in which the basis weight of the cover is large and the basis weight of the body is small. Also, the acquisition unit 17 observes the height of the paper bundle in the height direction of the recording medium arranged on the outermost side of the paper bundle 21, that is, the cover. Also, the shape of the paper bundle in the height direction is greatly affected by the number of sheets of paper that constitute the paper bundle. For this reason, as shown in FIG. 18, the shape of the paper bundle in the height direction is different between a paper bundle 21 in which the number of sheets of paper is large and a paper bundle 21 in which the number of sheets of paper is small.

[0063] For the above reasons, when the judgment unit 32 inspects a paper bundle using the learning model 37, it is preferable to refer to a model corresponding to a combination of the basis weight of the cover of the paper bundle and the number of sheets of paper that make up the paper bundle. Therefore, in this embodiment, the model learned for each combination of the basis weight of the cover of the paper bundle and the number of sheets of paper that make up the paper bundle is stored in the learning database 42, so that the judgment unit 32 can read out the above-mentioned reference model from the learning database 42.

[0064] Fig. 19 is a flowchart showing an example of a method for inspecting a paper bundle using the inspection system according to an embodiment of the present invention. The process of the flowchart shown in Fig. 19 is performed for each paper bundle.

[0065] First, the inspection computer 18 acquires the basis weight of the cover of the paper bundle and the number of sheets constituting the paper bundle as information on the paper bundle to be inspected (step S11). The basis weight of the cover of the paper bundle can be acquired from the media sensor 20. The number of sheets constituting the paper bundle can be acquired from the operation unit 26.

[0066] Next, the acquisition unit 17 acquires three-dimensional information of the paper bundle to be inspected (step S12). At this time, the paper bundle from which the acquisition unit 17 acquires three-dimensional information is a paper bundle that has been processed by the post-processing machine .

[0067] Next, the normalization processing unit 35 normalizes the three-dimensional information acquired by the acquisition unit 17 in the above step S12 (step S13). The normalization of the three-dimensional information is as described above with reference to Figs. 15 and 16.

[0068] Next, the imaging processor 36 images the three-dimensional information normalized in step S13 (step S14). In step S14, the three-dimensional information is imaged, whereby the three-dimensional information is converted into a two-dimensional bitmap.

[0069] Next, the judgment unit 32 inspects the paper stack using the learning model 37 (step S15). When inspecting the paper stack, the judgment unit 32 first reads out the learning model 37 corresponding to the combination of the basis weight and the number of sheets acquired in step S11 from the learning database 42. Next, the judgment unit 32 inputs a two-dimensional heat map, which is an image for inspection, into the learning model 37. As a result, the learning model 37 generates an image by inference based on the non-defective learning. Next, the judgment unit 32 obtains a difference between the original image and the generated image. Then, if there is no difference (see FIG. 11), the judgment unit 32 judges the paper stack to be inspected as a non-defective product. Also, if there is a difference (see FIG. 12), the judgment unit 32 judges the paper stack to be inspected as a defective product. Also, if at least one defective part of the paper stack judged as a defective product is located at or near the position where the staple 2 is driven in, the judgment unit 32 judges the defective part as a defect related to the stapling process.

[0070] Next, the system control unit 27 judges whether the paper bundle to be inspected is defective or not based on the judgment result of the judgment unit 32 in the above step S15 (step S16). If the system control unit 27 judges in step S16 that the paper bundle is defective, the process proceeds to step S17.

[0071] In step S17, the system control unit 27 judges whether or not there was a defect related to the stapling process based on the judgment result of the judgment unit 32 in the above step S15. Specifically, if the judgment unit 32 detects a defect related to the stapling process in the above step S15, the system control unit 27 stops the operation of the system (step S18). This stops the operation of the paper feeder 11, the printing machine 12, the image inspection machine 13, the post-processing machine 14, the bundle conveyor 15, and the like provided in the inspection system 10. Once a defect related to the stapling process occurs, it tends to occur repeatedly. For this reason, if the judgment unit 32 detects a defect related to the stapling process, it is preferable to stop the system operation for maintenance of the post-processing machine 14.

[0072] Furthermore, if the determination unit 32 does not detect any defects related to the stapling process in step S15, the discharge mechanism 16 receives a control command from the system control unit 27 and switches the destination of the paper stack (step S19). Specifically, the discharge mechanism 16 switches the destination of the paper stack from the non-defective product collection unit to the defective product collection unit by converting from the first position to the second position. As a result, as shown in FIG. 1, the defective paper stack 21 is collected in the defective product collection unit by falling under its own weight or the like. Therefore, even if the determination unit 32 detects a defective paper stack, the inspection system 10 can continue production of booklets, etc., without stopping the operation of the system.

[0073] Moreover, when the process of step S18 is completed, or when the process of step S19 is completed, or when the system control unit 27 makes a negative determination in step S16, the series of processes ends at that point.

[0074] Although not shown in FIG. 1, the inspection system 10 may be configured to include a marking unit instead of or in addition to the discharge mechanism 16. The marking unit applies marking to the paper stack according to the inspection result by the judgment unit 32. Specifically, the marking unit applies marking to the paper stack that the judgment unit 32 judges to be defective. The marking method may be any method that allows a person to visually distinguish whether the paper stack is good or defective. Specific examples of the marking method include printing a mark on the paper stack, attaching a sticky note to the paper stack, and the like. By including the marking unit in the inspection system 10, even if the judgment unit 32 detects a defective paper stack, the inspection system 10 can continue production of booklets, etc. without stopping the operation of the system.

[0075] As described above, the inspection system 10 and the inspection method according to the embodiment of the present invention can automatically inspect a paper stack without relying on manual inspection of the paper stack. This makes it possible to automate the booklet binding production process.

[0076] In addition, since the acquisition unit 17 acquires three-dimensional information about the paper stack, information about the shape of the paper stack in a direction other than the plane direction, i.e., the height direction, can be obtained. This allows the judgment unit 32 to detect defects that cannot be detected in a photograph taken from directly above the paper stack. Defects that cannot be detected in the above photograph include defects such as folded or torn paper.

[0077] In addition, inspection of booklet binding includes deformation of the outer shape such as folding or tearing of the booklet, and blank staples and buckling. Conventional techniques such as pattern matching require manual definition of the defect patterns to be detected. For this reason, it is difficult for techniques such as pattern matching to deal with unknown defect patterns that have not been defined. Furthermore, conventional techniques that compare the image to be inspected with the correct image have a narrow tolerance for ambiguity in the input data, making it difficult to set the defect threshold.

[0078] In contrast, in the inspection system 10 according to the embodiment of the present invention, the judgment unit 32 inspects the paper stack using the learning model 37, which is a deep learning model. This allows for a quality judgment that is close to that of a human, including ambiguity. In particular, in digital printing, since it is possible to produce booklets one at a time, it is virtually impossible to define defective patterns each time, as in the above-mentioned pattern matching. Therefore, it is very effective to inspect the stack using a deep learning model.

[0079] Furthermore, the acquisition unit 17 acquires three-dimensional information of the paper stack 21 downstream of the post-processing machine 14, and the judgment unit 32 inspects the paper stack 21 that has been processed by the post-processing machine 14. This allows the processing and inspection of the paper stack 21 to be performed inline. This allows the paper stack 21 to be inspected without human intervention.

[0080] Furthermore, the post-processing machine 14 processes a plurality of sheets of paper on which images have been printed by the printing machine 12 into a paper bundle. As a result, in addition to the above-mentioned processing and inspection of the paper bundle 21, the printing of images is also performed in-line. This makes it possible to reduce mistakes and waste caused by the movement of the paper from the printing machine 12 to the post-processing machine 14. Therefore, the inspection system 10 can increase the reliability of the system, and can also achieve automation and labor savings.

[0081] Moreover, the post-processing machine 14 is disposed downstream of the image inspection machine 13. Therefore, the image of each sheet constituting the paper bundle is inspected by the image inspection machine 13, and then the post-processing machine 14 performs processing and the paper bundle is inspected by the judgment unit 32. This makes it possible to inspect the entire paper bundle, including the image of each sheet.

[0082] <Modifications, etc.> The technical scope of the present invention is not limited to the above-described embodiment, and includes various modifications and improvements within the scope of the specific effects obtained by the constituent elements of the invention and their combinations.

[0083] For example, in the above embodiment, three-dimensional information has been described as an example of information acquired by the acquisition unit 17, but the present invention is not limited thereto. The information acquired by the acquisition unit 17 may be two-dimensional information. Examples of the acquisition unit 17 that acquires two-dimensional information include a camera and a scanner. In addition, when the information acquired by the acquisition unit 17 is two-dimensional information, the number of defective items of the paper stack that can be detected by the judgment unit 32 can be increased by devising the arrangement of the camera that photographs the paper stack. Specifically, by photographing the paper stack from an oblique direction with a camera, the number of defective items of the paper stack that can be detected by the judgment unit 32 can be increased. In addition, by photographing the paper stack from vertically above and horizontally with a camera, the number of defective items of the paper stack that can be detected by the judgment unit 32 can be increased.

[0084] In the above embodiment, when inspecting a paper bundle using the learning model 37, which is a deep learning model, the determination unit 32 refers to a model corresponding to a combination of the basis weight of the cover of the paper bundle and the number of sheets of paper constituting the paper bundle, but the present invention is not limited to this. For example, the determination unit 32 may refer to a model corresponding to the basis weight of the cover of the paper bundle. In this case, as a deep learning model to be used for inspecting the paper bundle, a model learned for each basis weight of the cover of the paper bundle may be stored in the learning database 42. In addition, the determination unit 32 may refer to a model corresponding to the number of sheets of paper constituting the paper bundle. In this case, as a deep learning model to be used for inspecting the paper bundle, a model learned for each number of sheets of paper constituting the paper bundle may be stored in the learning database 42.

[0085] Furthermore, some of the functions realized by the inspection computer 18 may be provided in an external computer that can communicate with the inspection computer 18 via a network. For example, the learning database 42 that stores the learning model 37 may be provided in the external computer. The judgment unit 32 may inspect the paper stack by exchanging data such as images between the inspection computer 18 and the external computer.

[0086] The present invention is not limited to an inspection system and an inspection method, but can also be provided as an inspection program or a computer-readable storage medium storing an inspection program. The inspection program is a program for causing a computer to execute an acquisition step of acquiring information about the shape of a bundle including a plurality of processed recording media, and a determination step of inspecting the bundle based on the information acquired by the acquisition step. The computer-readable storage medium is, for example, a CD-ROM. [Explanation of symbols]

[0087] 1...Paper (recording medium) 10…Inspection system 12...Printing machine (printing department) 13...Image inspection machine (Image inspection department) 14...Post-processing machine (processing section) 16...Discharge mechanism (discharge part) 17…Acquisition part 17a…1st acquisition part 17b…Second acquisition part 18...Inspection computer 21…Paper bundle (bundle) 27...System control unit 31...Pretreatment section 32…Judgment section 33…Recording section 35...Normalization processing section 36...Image processing unit 37…Learning model (deep learning model) 42…Learning database

Claims

1. an acquisition unit that acquires information regarding the shape of a bundle including a plurality of processed recording media; a determination unit that inspects the bundle based on the information acquired by the acquisition unit; An inspection system equipped with:

2. The determination unit inspects the bundle using a deep learning model. The inspection system according to claim 1 .

3. As the deep learning model, an unsupervised learning model is used. The inspection system according to claim 2 .

4. An autoencoder is used as the deep learning model. The inspection system according to claim 2 .

5. The deep learning model is a model trained for each basis weight of the cover of the bundle, When inspecting the bundle using the deep learning model, the determination unit refers to a model corresponding to the basis weight of the cover of the bundle. The inspection system according to claim 2 .

6. The deep learning model is a model trained for each number of recording media constituting the bundle, When inspecting the bundle using the deep learning model, the determination unit refers to a model corresponding to the number of recording media constituting the bundle. The inspection system according to claim 2 .

7. The deep learning model is a model trained for each combination of the basis weight of the cover of the bundle and the number of recording media constituting the bundle, The determination unit refers to a model corresponding to the combination when inspecting the bundle using the deep learning model. The inspection system according to claim 2 .

8. The information acquired by the acquisition unit is three-dimensional information. The inspection system according to claim 1 .

9. The information acquired by the acquisition unit is three-dimensional information, a normalization processing unit that normalizes the three-dimensional information according to a thickness and a size of the bundle, The deep learning model uses data normalized by the normalization processing unit. The inspection system according to claim 2 .

10. The information acquired by the acquisition unit is three-dimensional information, an imaging processing unit that visualizes the three-dimensional information; The deep learning model uses data visualized by the image processing unit. The inspection system according to claim 2 .

11. The three-dimensional information is represented as a two-dimensional heat map or a two-dimensional contour line. The inspection system according to claim 8 or 9.

12. The information acquired by the acquisition unit is two-dimensional information. The inspection system according to claim 1 .

13. A processing unit that processes a bundle of recording media, The judgment unit inspects the bundle processed by the processing unit. The inspection system according to claim 1 .

14. A printing unit that prints an image on a recording medium, The plurality of recording media on which the images are printed by the printing unit are processed by the processing unit. The inspection system according to claim 13.

15. an image inspection unit that inspects an image printed on a recording medium by the printing unit, The plurality of recording media, the images of which have been inspected by the image inspection unit, are processed by the processing unit. The inspection system according to claim 14.

16. The acquisition unit acquires the information by utilizing light emitted to the bundle. The inspection system according to claim 1 .

17. The acquisition unit includes a first acquisition unit that acquires information about the shape of the bundle on the front side of the bundle, and a second acquisition unit that acquires information about the shape of the bundle on the back side of the bundle. The inspection system according to claim 1 .

18. A discharge unit that switches the destination of the bundle depending on the inspection result by the determination unit is provided. The inspection system according to claim 1 .

19. A marking unit is provided for marking the bundle in accordance with the inspection result by the judgment unit. The inspection system according to claim 1 .

20. The processing includes stapling, a system control unit that stops the operation of the system when the determination unit determines that there is a defect related to the staple processing The inspection system according to claim 1 .

21. A recording unit is provided for recording information about the defective product when the determining unit determines that the bundle to be inspected is defective. The inspection system according to claim 1 .

22. Acquire information regarding the shape of a bundle including a plurality of processed recording media; Inspecting the bundle based on the acquired information Inspection method.

23. An acquisition step of acquiring information regarding the shape of a bundle including a plurality of processed recording media; a determination step of inspecting the bundle based on the information acquired by the acquisition step; An inspection program for causing a computer to execute the above.

Images