Information processing systems, programs, and information processing methods

Abstract

This generates a machine learning model used to generate shape information for a recording medium based on the attribute information of that recording medium. [Solution] The inspection device 100, which constitutes the information processing system, includes a learning shape information generation model generation unit 113 that generates a learning shape information generation model in which the explanatory variables include the attribute information related to the recording medium bundle and the objective variable is the shape information of the recording medium bundle that has been processed correctly, using learning data that includes attribute information related to the recording medium bundle and shape information of the recording medium bundle that has been processed correctly.

Description

Technical Field

[0001] The present invention relates to an information processing system, a program, and an information processing method for inspecting a bundle of recorded media that have been post-processed after printing.

Background Art

[0002] Techniques for inspecting printed matter (recorded media) generated by printing have been put into practical use. However, there is no technique for inspecting / quality-checking three-dimensional printed matter (bundles of recorded media) such as booklets that have been post-processed (processed). For this reason, the inspection work of post-processed printed matter has to rely on manual labor. Note that post-processing includes, for example, saddle-stitched booklets and perfect binding.

[0003] Regarding the inspection of the shape of post-processed printed matter, a technique of acquiring three-dimensional shape (shape information) and determining good / bad products using a machine learning model can be considered. In order to generate a machine learning model, it is necessary to prepare a large amount of shape information as learning data. Since it is difficult or costly to collect a large amount of shape information, it is desirable to generate shape information.

[0004] As a technology for generating three-dimensional shape information, there is a generation system described in Patent Document 1. The generation system includes an acquisition unit that acquires three-dimensional information of a target shape. The generation system also includes an image processing application unit that generates image information of a target image having image features corresponding to the three-dimensional features of the target shape by converting the three-dimensional features of the target shape into image features in a two-dimensional image based on the three-dimensional information of the target shape. Furthermore, the generation system includes a learning unit that generates a trained image generator that generates variation images based on the features of the target image by training a learning model on the features of the target image. The generation system also includes an image generation unit that generates the variation images using the trained image generator. In addition, the generation system includes a three-dimensional pattern generation unit that generates three-dimensional information of a variation shape having three-dimensional features corresponding to the image features of the variation image by converting the image features of the variation image into three-dimensional features based on the image information of the variation image. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2024-013009 [Overview of the project] [Problems that the invention aims to solve]

[0006] The generation system described in Patent Document 1 generates variations of the target shape. The shape of a post-processed printed material (recording medium) changes significantly depending on attribute information such as paper size, number of sheets, basis weight, and type of post-processing. Therefore, it is necessary to prepare target shapes according to combinations of paper size, number of sheets, basis weight, and type of post-processing, which incurs considerable costs.

[0007] This invention has been made in view of the above background, and aims to provide an information processing system, program, and information processing method for generating a machine learning model used to generate shape information of a recording medium according to the attribute information of the recording medium. [Means for solving the problem]

[0008] (1) An information processing system comprising a learning shape information generation model generation unit that generates a learning shape information generation model in which the explanatory variables include attribute information relating to the recording medium bundle and the objective variable is the shape information of the recording medium bundle that has been properly processed, using learning data that includes attribute information relating to the recording medium bundle and shape information of the recording medium bundle that has been properly processed.

[0009] (2) An information processing system according to (1), comprising: an inspection shape information generation model generation unit that generates a learning shape information generation model for each of the target attribute information which is a specific attribute information, using the learning shape information generation model; performs unsupervised learning using the learning shape information to generate an inspection shape information generation model corresponding to the target attribute information, in which the shape information of the recording medium bundle to be inspected is used as an explanatory variable and the inspection shape information is used as an objective variable.

[0010] (3) The information processing system according to (2), comprising an inspection unit that generates inspection shape information based on the shape information of the recording medium bundle to be inspected using an inspection shape information generation model corresponding to the attribute information of the recording medium bundle to be inspected, determines that the recording medium to be inspected is a defective product if the difference between the shape information of the recording medium bundle to be inspected and the inspection shape information exceeds a predetermined value, and determines that the recording medium to be inspected is a good product if the difference is less than or equal to the predetermined value.

[0011] (4) The information processing system according to (3), further comprising: a processing unit that processes the recording medium bundle by performing the processing on the recording medium on which an image has been formed; and a shape acquisition sensor that acquires shape information of the recording medium bundle processed by the processing unit, wherein the inspection unit determines whether the recording medium bundle is a good product or a defective product based on the shape information acquired by the shape acquisition sensor.

[0012] (5) The information processing system according to (4), further comprising an discharge unit that switches the destination of the recording medium bundle, wherein the discharge unit switches the destination of the recording medium according to the good / defective determination of the inspection unit.

[0013] (6) The information processing system according to (4), comprising an image inspection unit for inspecting an image formed on the recording medium, wherein the processing unit performs processing on the recording medium that has passed inspection by the image inspection unit.

[0014] (7) The information processing system according to (4), further comprising a printing unit that forms an image on the recording medium, wherein the processing unit performs the processing on the recording medium on which the printing unit has formed an image.

[0015] (8) The information processing system according to (7), wherein if the inspection unit determines that the recording medium is defective, the printing unit forms an image on the recording medium again, and the processing unit performs the processing on the recording medium again to process the bundle of recording media.

[0016] (9) The information processing system according to (2), wherein the inspection shape information generation model generation unit generates the inspection shape information generation model in addition to the shape information of the recording medium bundle that has been processed normally, using the learning shape information generation model.

[0017] (10) The information processing system according to (4), wherein the shape acquisition sensor acquires the shape information by utilizing the light emitted from the processed recording medium.

[0018] (11) The attribute information in the information processing system according to (1) includes the size, basis weight, number of media, and type of processing of the recording media included in the recording media bundle.

[0019] (12) The attribute information in the information processing system according to (1) is acquired from an image forming system that performs processing after forming an image on the recording media.

[0020] (13) The attribute information in the information processing system according to (1) is information input by a user who uses an image forming system that performs processing after forming an image on the recording media.

[0021] (14) The processing in the information processing system according to (1) is any one of saddle stitching, perfect binding, and die cutting.

[0022] (15) The learning shape information generation model in the information processing system according to (1) is a deep learning model.

[0023] (16) The inspection shape information generation model in the information processing system according to (2) is a deep learning model.

[0024] (17) The inspection shape information generation model in the information processing system according to (2) is an autoencoder.

[0025] (18) The inspection unit in the information processing system according to (3) records information related to the recording media when the recording media is determined to be a defective product.

[0026] (19) A program that causes a computer to function as an information processing system including a learning shape information generation model generation unit that generates a learning shape information generation model using learning data including attribute information related to a recording media bundle including recording media and shape information of the recording media bundle that has been normally processed, where the explanatory variable includes the attribute information related to the recording media bundle and the objective variable is the shape information of the recording media bundle that has been normally processed.

[0027] An information processing method in which an information processing system executes a step of generating a learning shape information generation model that includes, as explanatory variables, attribute information related to a bundle of recording media, and uses, as the objective variable, shape information of the bundle of recording media that has been normally processed, the learning data including the attribute information related to the bundle of recording media and the shape information of the bundle of recording media that has been normally processed.

Effect of the Invention

[0028] According to the present invention, it is possible to provide an information processing system, a program, and an information processing method for generating a machine learning model used for generating shape information of a recording medium according to attribute information of the recording medium.

Brief Description of the Drawings

[0029] [Figure 1] It is a diagram for explaining the outline of processing of the information processing system according to the present embodiment. [Figure 2] It is an example of shape information of a bundle of recording media that are good products according to the present embodiment. [Figure 3] It is an example of shape information of a bundle of recording media that are defective products according to the present embodiment. [Figure 4] It is a diagram for explaining the height of a bundle of recording media when the basis weight of the cover is small according to the present embodiment. [Figure 5] It is a diagram for explaining the height of a bundle of recording media when the basis weight of the cover is large according to the present embodiment. [Figure 6] It is an overall configuration diagram of the information processing system according to the present embodiment. [Figure 7] It is a functional block diagram of the inspection device according to the present embodiment. [Figure 8] It is a data configuration diagram of the acquired shape information database according to the present embodiment. [Figure 9] It is a diagram for explaining the normalization in the horizontal plane direction according to the present embodiment. [Figure 10] It is a diagram for explaining the normalization in the height direction according to the present embodiment. [Figure 11]This is a diagram illustrating the inspection process according to this embodiment. [Figure 12] This is a diagram illustrating the inspection process according to this embodiment. [Figure 13] This is a flowchart of the learning shape information generation model generation process according to this embodiment. [Figure 14] This is a flowchart of the process for generating a shape information generation model for inspection according to this embodiment. [Figure 15] This is a flowchart of the inspection process according to this embodiment. [Modes for carrying out the invention]

[0030] ≪Overview of the Information Processing System≫ The following describes an overview of an information processing system in an embodiment for carrying out the present invention. The information processing system performs inspection of a bundle of recording media (a bundle of paper, a stack, a booklet) that has been processed (generated) from one or more recording media (paper) on which an image has been printed.

[0031] Figure 1 is a diagram illustrating the processing overview of the information processing system according to this embodiment. In the learning shape information generation model generation process, the information processing system generates a learning shape information generation model 140 using the shape information and attribute information 401 of actual good products as learning data (first learning data). Shape information refers to information about the three-dimensional shape, including the height, when the post-processed recording medium bundle (booklet) of good products is placed on a horizontal plane (see Figure 2 below). Attribute information includes the size, number of sheets, basis weight, and type of post-processing (post-processing type) of the recording medium (paper) included in the recording medium bundle.

[0032] In the subsequent process of generating a shape information generation model for inspection, the information processing system uses the learning shape information generation model 140 to generate learning shape information 402, which is the shape information of the recording medium bundle. The learning shape information 402 is generated for each attribute information of the recording medium bundle. Next, the information processing system uses the learning shape information 402 as training data (second training data) for unsupervised learning to generate a shape information generation model 161 for inspection. The shape information generation model 161 is, for example, an autoencode generator and is a machine learning model whose target variable (output) is the shape information of a good product that is similar to the shape information of the explanatory variable (input).

[0033] In the inspection process, the information processing system uses the inspection shape information generation model 161 to generate inspection shape information 404 based on the shape information 403 of the recording medium bundle to be inspected. The inspection shape information 404 is the shape information of a good product that is similar to the shape information 403. Next, the information processing system calculates the difference 405 between the shape information 403 and the inspection shape information 404. The information processing system determines that the product is good if the difference 405 is less than or equal to a predetermined value, and defective if it exceeds the predetermined value.

[0034] According to this information processing system, training data (training shape information 402) used to generate the inspection shape information generation model 161 can be generated using the training shape information generation model 140. By using the inspection shape information generation model 161 corresponding to various attribute information, inspection shape information 404 corresponding to the attribute information of the object to be inspected can be generated, enabling highly accurate inspection.

[0035] ≪Shape information≫ The shape information of the recording medium bundle is described below. Figure 2 is an example of the shape information 410 of a good recording medium bundle according to this embodiment. A good recording medium bundle is a recording medium bundle that has been processed normally. The shape information includes information on the height when the post-processed recording medium bundle is placed on a horizontal plane. The recording medium bundle in Figure 2 is the shape information 410 of a recording medium bundle that has been saddle-stitched as a post-processing step. The left side is saddle-stitched and is higher than the right side, and the higher the side, the darker the hatching.

[0036] Figure 3 shows an example of shape information 420 of a defective recording medium bundle according to this embodiment. Compared to the shape information 410 of a good product shown in Figure 2, the area 421 in the upper and lower center on the left side is concave and lower. Defects in perfect binding include folding or tearing of the recording medium (paper). Defects in saddle stitching include folding or tearing of the recording medium, as well as buckling or misfiring of the staples used for binding.

[0037] Shape information varies depending on the size and number of recording media (paper) included in the recording media bundle, as well as the basis weight and type of post-processing. Even if the basis weight of the main body is the same, if the basis weight of the cover is different, the shape information will be different. Figure 4 is a diagram illustrating the height of the recording media bundle 431 when the basis weight of the cover is small, according to this embodiment. Figure 5 is a diagram illustrating the height of the recording media bundle 432 when the basis weight of the cover is large, according to this embodiment. Figures 4 and 5 are views of the recording media bundle bound on the left side from the horizontal direction (bottom side in Figure 2, see arrow 416). The basis weight of the main body of the recording media bundles 431 and 432 is the same. The basis weight of the cover is larger for the recording media bundle 432, and the central part is higher compared to the recording media bundle 431.

[0038] Thus, even for a bundle of good quality recording media, the shape information differs depending on the attribute information. Therefore, when generating shape information, it is necessary to generate it according to the attribute information. In the following, the shape information of a bundle of good quality recording media may also be simply referred to as the shape information of good quality products.

[0039] <<Configuration of the Information Processing System>> Figure 6 is an overall configuration diagram of the information processing system 10 according to this embodiment. The information processing system 10 comprises a paper feeder 210, a printing device 220, an image inspection device 230, a processing device 240, an operation unit 250, a transport path 260, and a system control unit 280. The information processing system 10 further comprises an inspection device 100, a transport path 270, a discharge mechanism 275, and shape acquisition sensors 271 and 272.

[0040] Information Processing Systems: Paper Feeding Devices and Printing Devices The paper feeder 210 supplies recording media (paper) stored in a tray or the like to the printing device 220 one sheet at a time. The printing device 220 (printing unit) forms an image on one or both sides of the supplied recording media and prints it.

[0041] A recording medium sensor 211 is positioned along the transport path 260 from the paper feed device 210 to the printing device 220. The recording medium sensor 211 detects information about the recording medium, such as thickness, surface properties, basis weight, moisture content, stiffness, and electrical resistance. The recording medium is transported along the transport path 260 from the paper feed device 210 to the processing device 240 (described later) by a transport mechanism (not shown), in the direction from right to left in Figure 6.

[0042] ≪Information Processing System: Image Inspection Equipment≫ The image inspection device 230 (image inspection unit) inspects the image printed on the recording medium by the printing device 220. The image inspection device 230 includes an image inspection sensor 231 that inspects the image printed on one side of the recording medium and an image inspection sensor 232 that inspects the image printed on the other side.

[0043] ≪Information Processing System: Processing Equipment≫ The processing device 240 (processing unit) processes the recording media bundles 451 to 454 by performing processing (post-processing, post-processing) on ​​one recording media from which an image has been printed by the printing device 220 and the image has been inspected. Types of processing (types of post-processing) include saddle stitching, perfect binding, die-cutting, etc.

[0044] Recording media determined to have printing defects by the image inspection device 230 may be sent to a paper discharge unit (not shown) and not transported to the processing device 240. In other words, the processing device 240 may perform processing only on recording media that have passed the image inspection by the image inspection device 230. By removing recording media with printing defects before processing, recording media with printing defects will not be included in the processing target.

[0045] If recording media with printing defects are not removed before processing, the information processing system 10 records the inspection results from the image inspection device 230. The information processing system 10 may also sort the bundle of recording media containing the recording media with printing defects as defective products, separating them from good products after processing. In the following, a bundle containing multiple recording media may be simply referred to as a recording media.

[0046] ≪Information Processing System: Operation Unit≫ The operation unit 250 functions as a user interface. The operation unit 250 includes an input unit (not shown) and a display unit. The input unit receives various types of information from the user. The display unit displays various types of information to the user. The information received by the input unit includes information related to print jobs. The information related to print jobs includes the size and number of recording media that make up the recording media bundle.

[0047] Information Processing System: Discharge Mechanism The recording media bundles 451-454 processed by the processing process are transported along the transport path 270. The transport path 270 is composed of, for example, multiple belt conveyors. The discharge mechanism 275 (discharge unit) changes its orientation according to the inspection result (good product / defective product) by the inspection device 100 described later, and switches the destination of the recording medium bundle. If the inspection result is a defective product, the discharge mechanism 275 assumes the orientation shown by the solid line, and the destination of the defective recording medium bundle 453 is switched to a defective product recovery unit (not shown). If the inspection result is a good product, the discharge mechanism 275 assumes the orientation shown by the dotted line, and the destination of the good recording medium bundle 454 is switched to a good product recovery unit (not shown).

[0048] ≪Information Processing System: Shape Acquisition Sensor≫ Shape acquisition sensors 271 and 272 acquire information (shape information) regarding the three-dimensional shape of the recording medium bundle processed by the processing device 240. The shape information is three-dimensional information that includes shape information in the horizontal plane and shape information in the height direction of the recording medium bundle. The shape information in the horizontal plane is the shape when the recording medium bundle is viewed from above, and indicates the shape and size of the recording medium bundle. The shape information in the height direction indicates the thickness of the recording medium bundle.

[0049] The shape acquisition sensor 272 is positioned above the transport path 270 and acquires shape information of the recording medium bundle 451 as seen from the front side. The shape acquisition sensor 271 is positioned below the transport path 270 and acquires shape information of the recording medium bundle 451 as seen from the back side.

[0050] The shape acquisition sensors 271 and 272 acquire shape information by utilizing light emitted towards the recording medium bundle 451. The shape acquisition sensors 271 and 272 are, for example, composed of light section type laser displacement meters. The shape acquisition sensors 271 and 272 may also be configured to acquire shape information using ToF (Time of Flight) sensors, multi-lens cameras, pattern illumination methods, structured illumination methods, etc. Alternatively, the shape acquisition sensors 271 and 272 may be configured to acquire shape information by utilizing sound emitted towards the recording medium bundle 451. In addition, the shape acquisition sensors 271 and 272 may be configured to acquire shape information by utilizing contact members that come into contact with the recording medium bundle 451.

[0051] Information Processing System: System Control Unit The system control unit 280 comprehensively controls the operation of the entire information processing system 10. The objects controlled by the system control unit 280 include the paper feeder 210, the printing device 220, the image inspection device 230, and the processing device 240. The system that processes (generates) the recording medium bundle, including the paper feeder 210, the printing device 220, the image inspection device 230, and the processing device 240, is also referred to as the image forming system 20.

[0052] <<Configuration of the inspection device>> Figure 7 is a functional block diagram of the inspection device 100 according to this embodiment. The inspection device 100 is a computer and comprises a control unit 110, a storage unit 120, and a communication unit 180. The communication unit 180 is equipped with a communication device and is capable of sending and receiving data with the system control unit 280 and the shape acquisition sensors 271 and 272.

[0053] ≪Inspection device: Memory unit≫ The memory unit 120 is comprised of memory devices such as ROM (Read Only Memory), RAM (Random Access Memory), and SSD (Solid State Drive). The memory unit 120 stores the acquired shape information database 130, the learning shape information generation model 140, the learning shape information database 150, the inspection shape information generation model database 160, and the program 128. The program 128 contains descriptions of the processes to be executed by the functional units provided in the control unit 110, which will be described later.

[0054] ≪Memory Unit: Acquired Shape Information Database≫ Figure 8 is a data structure diagram of the acquired shape information database 130 according to this embodiment. The acquired shape information database 130 stores training data (first training data) used to train the learning shape information generation model 140. The acquired shape information database 130 is, for example, tabular data. One row (record) of the acquired shape information database 130 stores information relating to a bundle of recording media that is a good product. Whether a product is good or not is determined, for example, by a human. The record of the acquired shape information database 130 includes columns (attributes) for identification information, paper size, number of sheets, basis weight, type of post-processing, and shape acquisition. The attribute information of the bundle of recording media includes the size (paper size), number of sheets (number of media), basis weight, and type of post-processing of the recording media included.

[0055] The identification information (labeled "ID" in Figure 8) is the identification information of the recording medium bundle. Paper size, number of sheets, and basis weight refer to the size, number of sheets, and basis weight of the recording media included in the recording medium bundle. Post-processing type refers to the type of processing, for example, "saddle stitching". Shape acquisition refers to the shape information of the recording medium bundle acquired by shape acquisition sensors 271 and 272.

[0056] The records in the acquired shape information database 130 may include attributes indicating the composition of the recording medium bundle if it contains multiple types of recording media (paper). For example, if the basis weight of the cover and the main body are different, attributes including the basis weight of each may be included. Furthermore, the record may include attributes such as thickness, surface properties, moisture content, stiffness, and electrical resistance obtained by the recording medium sensor 211 (see Figure 6) as information about the recording medium. The attribute information of the recording medium bundle may include the thickness, surface properties, moisture content, stiffness, and electrical resistance of the recording media it contains.

[0057] ≪Memory Unit: Shape Information Generation Model for Learning≫ Returning to Figure 7, let's continue the explanation of the memory unit 120. The learning shape information generation model 140 is a machine learning model that generates the learning data (second learning data) used to generate the inspection shape information generation model 161 (see Figure 1), which will be described later.

[0058] The training shape information generation model 140 is, for example, a deep learning model. The explanatory variables of the training shape information generation model 140 include attribute information of the recording medium bundle. The target variable of the training shape information generation model 140 is the shape information of a good recording medium bundle corresponding to the attribute information. The learning shape information generation model 140 may also be, for example, a generator of a conditional adversarial generative network that uses attribute information as a condition (label). Alternatively, the learning shape information generation model 140 may also be a conditional variational autoencoder that uses attribute information as a condition.

[0059] The learning shape information generation model 140 is generated by training it with information on good quality recording medium bundles in the acquired shape information database 130 as learning data (first learning data). The explanatory variables of the learning data are the size, number of sheets, basis weight, and type of post-processing of the recording medium. The explanatory variables may also include thickness, surface properties, moisture content, stiffness, electrical resistance, etc. The target variable of the learning data is the shape information normalized by the pre-processing unit 112 described later.

[0060] ≪Memory Unit: Learning Shape Information Database≫ The learning shape information database 150 stores learning data (second learning data) used to generate the inspection shape information generation model 161 (see Figure 1). The learning data is shape information of good quality recording media bundles that have the same or similar (close) attribute information. Having the same / similar attribute information means that the size and post-processing type are identical, and the number of sheets and basis weight are within a predetermined range. For example, attribute information with size A4, number of sheets 30-34, basis weight 62-66 g / m2, and post-processing type saddle stitching is considered to have the same / similar attribute information.

[0061] ≪Memory Unit: Database for Generating Shape Information for Inspection≫ The inspection shape information generation model database 160 stores the inspection shape information generation model 161. The inspection shape information generation model 161 is an unsupervised learning machine learning model, such as a deep learning model. The inspection shape information generation model 161 may also be an autoencoder. The inspection shape information generation model 161 is generated for each attribute information that can be considered the same / similar and stored in the inspection shape information generation model database 160.

[0062] The explanatory variables (inputs) and objective variables (outputs) of the inspection shape information generation model 161 are shape information. Since the inspection shape information generation model 161 has learned the shape information of good products, it outputs shape information that has the characteristics of good product shape information. If the input is the shape information of a good product, it outputs shape information that is similar to that shape information. Therefore, the difference between the input and output is small. If the input is the shape information of a defective product, it outputs shape information that has the characteristics of a good product that is similar to that shape information. Therefore, the difference between the input and output is larger than when a good product is input.

[0063] ≪Inspection device: Control unit≫ The control unit 110 includes a CPU (Central Processing Unit). The control unit 110 comprises a shape information acquisition unit 111, a preprocessing unit 112, a learning shape information generation model generation unit 113, an inspection shape information generation model generation unit 114, and an inspection unit 115. The control unit 110 may also include a GPU (Graphics Processing Unit) or an NPU (Neural (network) Processing Unit). Furthermore, the control unit 110 may also include an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc.

[0064] ≪Control Unit: Shape Information Acquisition Unit≫ The shape information acquisition unit 111 stores the shape information of the recording medium bundle acquired by the shape acquisition sensors 271 and 272 (see Figure 6) in the acquired shape information database 130. The shape information acquisition unit 111 also acquires attribute information of the recording medium bundle and stores it in association with the shape information. The attribute information may be information acquired by the recording medium sensor 211, or information entered by the user from the operation unit 250. The shape information acquisition unit 111 may also acquire attribute information of the recording medium bundle included in the print job from the system control unit 280 (image forming system 20).

[0065] <Control Unit: Pre-processing Unit> The preprocessing unit 112 normalizes the shape information. The normalization includes normalization in the horizontal direction and normalization in the height direction. Figure 9 is a diagram illustrating the normalization in the horizontal direction according to this embodiment. The preprocessing unit 112 normalizes the shape information of the recording medium bundle in the horizontal direction so that the aspect ratio is 1 or the difference from 1 is within a predetermined value (also referred to as approximately 1). For vertically elongated shape information 461, the preprocessing unit 112 normalizes it to shape information 464 with an aspect ratio of approximately 1 by shrinking it in the vertical direction. Similarly, for horizontally elongated shape information 462, the preprocessing unit 112 normalizes it to shape information 464 with an aspect ratio of approximately 1 by shrinking it in the horizontal direction.

[0066] If the rectangular shape information is slanted in the horizontal plane, the preprocessor 112 rotates the shape information so that the sides are straight both vertically and horizontally, normalizing it into shape information 464. The preprocessor 112 may also rotate the recording medium bundle so that the taller part is oriented in a predetermined direction. As shown in Figures 2 and 3, the preprocessor 112 may rotate and normalize the shape information so that the left side is taller.

[0067] Figure 10 is a diagram illustrating the height normalization according to this embodiment. Figure 10 is a view of a stack of recording media bound on the left side from the horizontal direction (bottom side in Figure 2, see arrow 416). The preprocessing unit 112 normalizes the height of the shape information so that the maximum height of the shape information becomes a predetermined value H. If the height is represented by the intensity of hatching as shown in Figure 2, then the darkest parts in the normalized shape information will all be the same intensity.

[0068] ≪Control Unit: Model Generation Unit for Generating Shape Information for Learning≫ Returning to Figure 7, let's continue the explanation of the control unit 110. The learning shape information generation model generation unit 113 trains and generates the learning shape information generation model 140. The training data (first training data) used for training is data in the acquired shape information database 130. More specifically, the explanatory variables of the training data include the size, number of sheets, basis weight, and type of post-processing of the recording medium. The explanatory variables may also include thickness, surface properties, moisture content, stiffness, electrical resistance, etc. The target variable of the training data includes shape information of good products normalized by the pre-processing unit 112. The learning shape information generation model generation unit 113 may generate a generator and a discriminator of a conditional generative adversarial network, and the generator may be set as the learning shape information generation model 140.

[0069] ≪Control Unit: Model Generation Unit for Generating Shape Information for Inspection≫ The inspection shape information generation model generation unit 114 trains and generates an inspection shape information generation model 161 for each attribute information that can be considered the same / similar, and stores it in the inspection shape information generation model database 160. The inspection shape information generation model generation unit 114 trains and generates the inspection shape information generation model 161 using an unsupervised learning method. The training data (second training data) used for training is the normalized shape information of good products in the training shape information database 150.

[0070] ≪Control Unit: Inspection Department≫ The inspection unit 115 inspects the recording medium bundle to be inspected using the inspection shape information generation model 161. The inspection unit 115 acquires attribute information of the recording medium bundle to be inspected and inspects it using the inspection shape information generation model 161 that corresponds to attribute information that can be considered the same as or similar to the said attribute information. More specifically, the inspection unit 115 acquires the shape information of the recording medium bundle to be inspected acquired by the shape acquisition sensors 271 and 272. Next, the inspection unit 115 inputs this shape information into the inspection shape information generation model 161 and performs inspection by calculating the difference between it and the output shape information (see inspection shape information 404 shown in Figure 1).

[0071] Figure 11 is a diagram illustrating the inspection process according to this embodiment. Shape information 471 is the shape information of a good recording medium bundle. Shape information 472 is the shape information (inspection shape information) output when shape information 471 is input to the inspection shape information generation model 161. The difference 473 is the difference in the height direction between shape information 471 and 472. Since shape information 471 is the shape information of a good recording medium bundle, the output shape information 472 is the same as shape information 471, and the difference 473 is approximately 0.

[0072] Figure 12 is a diagram illustrating the inspection process according to this embodiment. Shape information 475 is the shape information of the defective recording medium bundle shown in Figure 3, where the upper and lower central part on the left side is concave. Shape information 476 is the shape information (inspection shape information) output when shape information 475 is input to the inspection shape information generation model 161. Difference 477 is the difference in height between shape information 475 and 476. Shape information 475 is the shape information of the defective recording medium bundle, but the output shape information 476 has the characteristics of good product shape information and does not have a concave part in the upper and lower central part on the left side. Therefore, a difference in height appears in the upper and lower central part on the left side in difference 473.

[0073] The difference is the sum of the height differences at each point on the horizontal plane. The difference may also be the maximum value of the height differences. The inspection process in the inspection unit 115 will be explained again using a flowchart (see Figure 15 below).

[0074] <<Process for generating learning shape information model>> Figure 13 is a flowchart of the learning shape information generation model generation process according to this embodiment. At the start of the learning shape information generation process, the acquired shape information database 130 (see Figure 8) already stores attribute information and shape information of good quality recording medium bundles.

[0075] In step S11, the learning shape information generation model generation unit 113 starts the process of repeating steps S12 and S13 for each record in the acquired shape information database 130. In step S12, the learning shape information generation model generation unit 113 performs preprocessing (normalization) on the shape information of the record.

[0076] In step S13, the learning shape information generation model generation unit 113 generates learning data including the paper size, number of sheets, basis weight, type of post-processing, and shape information after pre-processing of the record. In step S14, the learning shape information generation model generation unit 113 trains and generates a learning shape information generation model 140 using the learning data (first learning data) generated in step S13.

[0077] ≪Process for generating shape information models for inspection≫ Figure 14 is a flowchart of the inspection shape information generation model generation process according to this embodiment. At the start of the inspection shape information generation model generation process, the learning shape information generation model 140 has already been generated.

[0078] In step S21, the inspection shape information generation model generation unit 114 determines the target attribute information, which is the attribute information (specific attribute information) of the inspection shape information generation model 161 to be generated. For example, the target attribute is the attribute information of a frequently generated bundle of recording media. For each of the size, number, basis weight, and post-processing type of recording media used as the target of processing, all combinations of these may be used as the target attribute. Among these target attributes, those that can be considered identical / similar may be used as a single target attribute information.

[0079] In step S22, the inspection shape information generation model generation unit 114 starts the process of repeating steps S23 and S24 for each of the target attribute information to be generated determined in step S21. In step S23, the inspection shape information generation model generation unit 114 uses the learning shape information generation model 140 to generate a predetermined number of learning shape information, which is shape information, based on the attribute information to be generated, and stores them in the learning shape information database 150.

[0080] In step S24, the inspection shape information generation model generation unit 114 generates an inspection shape information generation model 161 by performing unsupervised learning using the learning shape information (second learning data) generated in step S23. Next, the inspection shape information generation model generation unit 114 stores the generated inspection shape information generation model 161 in the inspection shape information generation model database 160, associating it with the attribute information to be generated.

[0081] ≪Inspection Process≫ Figure 15 is a flowchart of the inspection process according to this embodiment. At the start of the inspection process, the inspection shape information generation model 161 corresponding to the attribute information of the recording medium bundle to be inspected is already stored in the inspection shape information generation model database 160.

[0082] In step S31, the inspection unit 115 acquires attribute information and shape information of the recording medium bundle to be inspected. The inspection unit 115 acquires attribute information from the system control unit 280 (image forming system 20) and shape information from shape acquisition sensors 271 and 272. In step S32, the preprocessing unit 112 performs preprocessing (normalization) on the shape information acquired in step S31.

[0083] In step S33, the inspection unit 115 uses the inspection shape information generation model 161 to generate inspection shape information 404 (see Figure 1) based on the shape information after preprocessing. Here, the inspection shape information generation model 161 is the inspection shape information generation model 161 that corresponds to the attribute information acquired in step S31.

[0084] In step S34, the inspection unit 115 calculates the difference between the shape information after pre-processing and the shape information for inspection. In step S35, if the difference calculated in step S34 is less than or equal to a predetermined threshold (step S35 → YES), the inspection unit 115 proceeds to step S36. If the difference exceeds the threshold (step S35 → NO), the inspection unit 115 proceeds to step S37.

[0085] In step S36, the inspection unit 115 determines that the recording medium bundle to be inspected is a good product and notifies the discharge mechanism 275. Upon receiving the notification, the discharge mechanism 275 changes its orientation so that the recording medium bundle is transported to the good product collection unit. In step S37, the inspection unit 115 determines that the recording medium bundle to be inspected is defective and notifies the discharge mechanism 275. Upon receiving the notification, the discharge mechanism 275 changes its orientation so that the recording medium bundle is transported to the defective product collection unit.

[0086] ≪Features of the Information Processing System≫ According to the information processing system 10 described above, a learning shape information generation model 140 can be generated to generate learning data (learning shape information 402 shown in Figure 1) used to generate the inspection shape information generation model 161. By generating learning data (second learning data) using the learning shape information generation model 140, it becomes possible to generate inspection shape information generation models 161 according to various attribute information. Consequently, inspection shape information 404 according to the attribute information of the object to be inspected can be generated, enabling high-precision inspection of recording media bundles.

[0087] ≪Training Data≫ In the embodiment described above, the training data (second training data) for the inspection shape information generation model 161 is generated using the training shape information generation model 140 and stored in the training shape information database 150. The training data may further include, for example, shape information of actual recording media bundles that have been manually confirmed as good products. The training data may also include shape information of actual recording media bundles that the inspection unit 115 has determined to be good products. The training data may also be, for example, shape information of actual recording media bundles that are good products, stored in the acquired shape information database 130. The training data may include at least one piece of shape information of actual good recording media bundles. By including shape information of actual good products in addition to the shape information generated by the training shape information generation model 140, it is expected that the quality of the shape information generated by the inspection shape information generation model 161 will be improved.

[0088] ≪Torture: Shape Information≫ In the embodiment described above, one of the shape information is height information. This height information may be considered as the brightness information of each pixel in a grayscale image. The height information may also be considered as the color information (RGB value) of each pixel in a color image. By treating the shape information as image information, training data (training shape information 402 shown in Figure 1), a training shape information generation model 140, and an inspection shape information generation model 161 can be generated using image-related machine learning techniques and tools.

[0089] ≪Variant: Inspection Department≫ The inspection unit 115 may record information related to the judgment result (good product / defective product). For example, the inspection unit 115 may acquire the judgment result, judgment time, name / number of the job processing the recording medium bundle, shape information and record the identification information of the shape acquisition sensors 271, 272, shape information, etc. The records may be stored in the storage unit 120 of the inspection device 100, or in the storage device provided in the image forming system 20.

[0090] ≪Variation: Dealing with defects≫ If the image inspection device 230 detects a defect in the image printed on the recording medium, the recording medium may be discarded, and the printing device 220 may print the image onto the recording medium again. If the inspection unit 115 determines that the bundle of recording media is defective, the printing device 220 may reprint all the recording media, and the processing device 240 may perform processing again. These processes may be performed by the system control unit 280.

[0091] <<Other variations>> Although several embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and furthermore, various modifications such as omissions and substitutions can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention as described herein and elsewhere, as well as in the scope of the invention and its equivalents as described in the claims. [Explanation of Symbols]

[0092] 10. Information Processing Systems 20 Image Forming Systems 100 Inspection device 111 Shape information acquisition unit 112 Pre-processing 113 Learning Shape Information Generation Model Generation Unit 114 Inspection Shape Information Generation Model Generation Unit 115 Inspection Department 130 Acquired Shape Information Database 140 Shape information generation model for learning 150 Learning Shape Information Database (Learning Shape Information) 160 Database of Shape Information Generation Models for Inspection 161 Model for generating shape information for inspection 210 Paper feeder 211 Recording medium sensor 220 Printing device (printing department) 230 Image Inspection Equipment (Image Inspection Department) 240 Processing equipment (processing section) 250 Operation section 260 transport paths 270 Conveyor paths 271,272 Shape acquisition sensors 275 Ejection mechanism (ejection part) 280 System Control Unit 402 Shape information for learning 404 Shape information for inspection

Claims

1. Using training data that includes attribute information relating to a bundle of recording media including the recording media, and shape information of the bundle of recording media that has been properly processed, The system includes a learning shape information generation model generation unit that generates a learning shape information generation model in which the explanatory variables include attribute information relating to the recording medium bundle and the target variable is the shape information of the recording medium bundle after it has been properly processed. Information processing system.

2. Using the aforementioned learning shape information generation model, learning shape information, which is the shape information of the recording medium bundle, is generated for each specific attribute information, which is the target attribute information to be generated. The system includes an inspection shape information generation model generation unit that performs unsupervised learning using the learning shape information to generate an inspection shape information generation model corresponding to the generated target attribute information, with the shape information of the recording medium bundle to be inspected as the explanatory variable and the inspection shape information as the objective variable. The information processing system according to claim 1.

3. Using a shape information generation model for inspection that corresponds to the attribute information of the recording medium bundle to be inspected, the shape information for inspection is generated based on the shape information of the recording medium bundle to be inspected. If the difference between the shape information of the recording medium bundle to be inspected and the shape information for inspection exceeds a predetermined value, the recording medium to be inspected is determined to be a defective product. The inspection unit determines that the recording medium to be inspected is a good product if the difference is less than or equal to the predetermined value. The information processing system according to claim 2.

4. A processing unit that processes the recording medium bundle by performing the processing on the recording medium on which an image has been formed, The processing unit further comprises a shape acquisition sensor that acquires shape information of the recording medium bundle processed by the processing unit, The aforementioned inspection unit, Based on the shape information acquired by the shape acquisition sensor, the recording medium bundle is determined to be either good or defective. The information processing system according to claim 3.

5. The system further includes an discharge unit that switches the destination of the aforementioned recording medium bundle, The aforementioned discharge section is The destination for transporting the recording medium is switched according to the inspection unit's determination of whether the product is good or defective. The information processing system according to claim 4.

6. The system includes an image inspection unit for inspecting an image formed on the recording medium, The aforementioned processing section is Processing is performed on recording media that have passed inspection by the aforementioned image inspection unit. The information processing system according to claim 4.

7. The recording medium further comprises a printing unit that forms an image, The aforementioned processing section is The printing unit performs the processing on the recording medium on which the image has been formed. The information processing system according to claim 4.

8. If the inspection unit determines that the recording medium is defective, The aforementioned printing unit is An image is formed on the recording medium again. The aforementioned processing section is The processing is performed again on the recording medium to process the bundle of recording media. The information processing system according to claim 7.

9. The aforementioned inspection shape information generation model generation unit is: As the aforementioned learning shape information, In addition to the shape information of the actual recording medium bundle that has been processed correctly, the shape information generation model for inspection is generated using the learning shape information generation model described above. The information processing system according to claim 2.

10. The aforementioned shape acquisition sensor is The shape information is acquired by utilizing the light emitted from the processed recording medium. The information processing system according to claim 4.

11. The aforementioned attribute information is, The recording media bundle includes the size, basis weight, number of media, and type of processing applied to the recording media bundle. The information processing system according to claim 1.

12. The aforementioned attribute information is, Acquired from an image forming system that processes an image after forming it on the aforementioned recording medium. The information processing system according to claim 1.

13. The aforementioned attribute information is, This is information input by a user of an image forming system that processes an image after it has been formed on the aforementioned recording medium. The information processing system according to claim 1.

14. The aforementioned processing is It is either saddle-stitched, perfect-bound, or die-cut. The information processing system according to claim 1.

15. The aforementioned learning shape information generation model is Deep learning model The information processing system according to claim 1.

16. The aforementioned shape information generation model for inspection is: Deep learning model The information processing system according to claim 2.

17. The aforementioned shape information generation model for inspection is: It is an autoencoder. The information processing system according to claim 2.

18. The aforementioned inspection unit, If the aforementioned recording medium is determined to be defective, the information relating to the recording medium is recorded. The information processing system according to claim 3.

19. Computers, Using training data that includes attribute information relating to a bundle of recording media including the recording media, and shape information of the bundle of recording media that has been properly processed, This information processing system includes a learning shape information generation model generation unit that generates a learning shape information generation model in which the explanatory variables include attribute information relating to the recording medium bundle and the target variable is the shape information of the recording medium bundle after proper processing. program.

20. Information processing system, Using training data that includes attribute information relating to a bundle of recording media including the recording media, and shape information of the bundle of recording media that has been properly processed, The step of generating a learning shape information generation model is performed, in which the explanatory variables include attribute information relating to the recording medium bundle and the target variable is the shape information of the recording medium bundle that has been properly processed. Information processing methods.

Images