Information processing device, control method for information processing device, and program

Abstract

The goal is to enable inspection of printed materials, which will ultimately become three-dimensional objects, taking into account their appearance in three dimensions. [Solution] The system identifies a pair of regions in the scanned image obtained by optically reading a printed material that will come into contact with each other when the printed material transforms into a three-dimensional object. The identified pair of regions is then displayed on a display means in an identifiable manner.

Description

Technical Field

[0001] The present disclosure relates to a technique for inspecting printed matter.

Background Art

[0002] Conventionally, inspection devices for automatically inspecting the quality of printed matter are known. In such an inspection device, an image serving as a standard (referred to as a "reference image" or a "correct image") when inspecting printed matter is compared with a read image (referred to as an "inspection image") obtained by reading the printed matter, thereby inspecting whether the printed matter is a good product or a defective product. Patent Document 1 discloses a technique for printing a pattern (commonly called a "dragonfly") for position deviation inspection at a peripheral position of a pattern arranged on the front and back of a sheet, and correcting and inspecting the position deviation between the front and the back.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Incidentally, a method is known for producing three-dimensional products by printing a flat pattern of a product package, cutting out unnecessary parts from the resulting printout, and assembling the pieces. Since it is more efficient to inspect such printed product packages before post-printing processes such as cutting and assembly are carried out, inspection images obtained by reading the printed material in its flat state are often used. In this case, printing defects that were not noticeable in the printed material may become noticeable in the three-dimensional product, particularly in areas that come into contact with each other. However, conventional inspection techniques, including the one described in Patent Document 1, have not been able to address the need to inspect printing defects that may occur in the three-dimensional product at the printing stage. This disclosure aims to realize inspection of printed materials that will ultimately become three-dimensional products, taking into account their state when they are formed into three dimensions. [Means for solving the problem]

[0005] The information processing device for inspecting printed materials according to this disclosure is characterized by comprising: acquisition means for acquiring a read image obtained by optically reading the printed material; identification means for identifying a pair of regions consisting of two regions that are to be inspected in the read image and that will come into contact when the printed material is transformed into a three-dimensional object; and display control means for displaying the identified pair of regions in an identifiable manner on a display means. [Effects of the Invention]

[0006] According to this disclosure, it is possible to perform inspections on printed materials that will ultimately become three-dimensional products, taking into account their state when they are formed into a three-dimensional object. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing the overall configuration of the print inspection system. [Figure 2] A block diagram showing the overall configuration of a print inspection system that performs inspection processing via a cloud server. [Figure 3] A functional block diagram showing the software configuration of an information processing device. [Figure 4] A flowchart showing the general flow of the inspection process. [Figure 5A] A diagram showing an example of a GUI related to the inspection process. [Figure 5B] A diagram showing an example of a GUI related to the inspection process. [Figure 5C] A diagram showing an example of a GUI related to the inspection process. [Figure 6] A flowchart illustrating the details of the region pair identification process. [Figure 7] A diagram showing an example of a GUI related to the inspection process. [Figure 8] A flowchart showing the details of the inspection process. [Figure 9] (a) is a diagram illustrating absolute positional displacement, and (b) is a diagram illustrating relative positional displacement. [Figure 10] (a) and (b) are diagrams illustrating variations in positional misalignment. [Figure 11] (a) and (b) are diagrams illustrating variations in positional misalignment. [Figure 12] A flowchart showing the details of the shielding area identification process. [Figure 13] A diagram showing an example of the results of the shielding area identification process. [Figure 14] (a) is a diagram showing an example of printed material with an inspection mark, and (b) and (c) are enlarged views of the inspection mark. [Modes for carrying out the invention]

[0008] The embodiments described below will be explained with reference to the drawings. Note that the following embodiments are not limiting to the present invention, and not all combinations of features described in each embodiment are essential to the solution of the present invention. Furthermore, various other forms that do not depart from the spirit of the present invention are also included, and parts of the following embodiments can be combined as appropriate.

[0009] [Embodiment 1] In the present embodiment, for a printed matter that finally becomes a three-dimensional product, regions that are separated in the planar state but come into contact with each other in the three-dimensional state (hereinafter referred to as "region pairs") are displayed in a state recognizable by the user, and an aspect that enables appropriate inspection of the printed matter will be described.

[0010] [Confirmation of Problems] Inspection of the printed matter of a box-shaped product package such as a caramel type or a pillow type created using one or more printed matters is more efficient if it is performed before operations such as cutting and assembly. When reading and inspecting a printed matter in a planar state, it is possible that a printing defect that was not noticeable in the printed matter at the time of the planar state becomes noticeable in the three-dimensional product for parts that come into contact with each other when it becomes a three-dimensional product. Specific examples of such printing defects include misalignment of patterns or shapes that are completed after assembly, and differences in color or gloss on adjacent surfaces of the three-dimensional product. In order to eliminate design constraints due to the inability to sufficiently inspect the product package, which is the face of the product, in the planar state and ensure design freedom, it is very important to perform appropriate inspection in the planar state. Therefore, in the following embodiments, processing is performed to realize inspection considering the state when the printed matter finally becomes a three-dimensional product.

[0011] [Configuration of Printing Inspection System] FIG. 1 is a diagram showing the overall configuration of a printing inspection system that performs output and inspection of a printed matter, including the information processing apparatus according to the present embodiment. The printing inspection system shown in FIG. 1 includes an information processing apparatus 100 that performs inspection of a printed matter and post-processing such as stapling, a printing server 180 that generates and outputs a print job, and a printing apparatus 190 that performs printing processing based on the print job.

[0012] The printing device 190 forms an image on a printing recording medium such as paper or a plastic sheet (hereinafter referred to as "paper") based on a printing job input from the printing server 180. The printing device 190 has a paper feeding unit 191, and the user sets the paper in the paper feeding unit 191 in advance. When a printing job is input, the printing device 190 conveys the paper set in the paper feeding unit 191 along the conveyance path 192, forms an image on one or both sides of the paper, and sends it to the information processing device 100. The printing method of the printing device 190 is an electrophotographic method. Other printing methods such as an inkjet method may be used, and the printing method is not particularly limited.

[0013] The information processing device 100 inspects defects in the paper on which printing has been performed (hereinafter referred to as "printed matter"). The information processing device 100 has a CPU 101, a RAM 102, and a ROM 103, and the CPU 101 realizes the functions of each part in the device by executing a program stored in the RAM 102 or the ROM 103. The information processing device 100 optically reads the printed matter conveyed through the conveyance path 192 of the printing device 190 with an image reading device 104 such as a scanner or a camera to obtain a read image for inspection (hereinafter referred to as "inspection image"). Then, the obtained inspection image is compared with an image serving as an inspection standard (hereinafter referred to as "reference image") to inspect the printed matter while it is in a flat state. The information processing device 100 also has various interfaces such as a network I / F 105, a printing device I / F 106, and a general-purpose I / F 107, and a UI (user interface) panel 108 for the user to perform operation inputs, confirmation of inspection results, etc. Each module in the information processing device 100 is interconnected via a main bus 109.

[0014] The information processing device 100 has a transport path 110 connected to the transport path 192 of the printing device 190, and the image reading device 104 scans and reads one or both sides of the printed material sent from the printing device 190 on the transport path 110, acquiring it as inspection image data. The printing device interface 106 is connected to the printing device 190, and the information processing device 100 can communicate with the printing device 190 through the printing device interface 106. For example, the printing device 190 and the information processing device 100 can be synchronized via the printing device interface 106, and their operating status can be notified to each other. The UI panel 108 is a display device such as a liquid crystal display and functions as a user interface that provides the user with the current status or setting information of the information processing device 100. The UI panel 108 may also be equipped with input devices such as a touch panel or buttons, and can accept instructions from the user.

[0015] The information processing device 100 inspects the printed material output from the printing device 190 for printing defects using an inspection image and a corresponding reference image while the material is moving along the transport path 110. If the inspection results in a pass, the printed material is transported to the output tray 111. If the inspection results in a fail, the printed material is transported to the output tray 112. Through this paper output operation, only printed materials that have been determined to be free of defects are output to the output tray 111. Note that the information processing device 100 may be implemented by multiple information processing devices.

[0016] Figure 2 shows an example of a system configuration in which a cloud server is added to the system configuration shown in Figure 1, and inspection is performed via the cloud server. The cloud server 200 is a server device that provides cloud services over the internet. The cloud server 200 receives inspection image data acquired by the image reading device 104 of the information processing device 100, performs inspection of the printed material, and transmits the inspection results to the information processing device 100 as needed. The cloud server 200 has a CPU 201, RAM 202, ROM 203, storage device 204, and network I / F 205, and these components are interconnected by a system bus 207. The cloud server 200 may be composed of one server device or multiple server devices. Furthermore, the functions of multiple server devices may be realized in a single server device using virtualization software. In addition, the cloud server 200 may be interconnected not only with the information processing device 100 but also with the printing device 190 to manage print jobs and inspection results.

[0017] <Functions of the information processing device> Next, the software configuration (logical configuration) of the information processing device 100 will be explained. Figure 3 is a functional block diagram showing the software configuration of the information processing device 100. The information processing device 100 includes a reference image acquisition unit 301, an inspection image acquisition unit 302, an operation input reception unit 303, an area identification unit 304, an inspection parameter setting unit 305, an inspection unit 306, a display control unit 307, and a print control unit 308.

[0018] The reference image acquisition unit 301 acquires a reference image for comparison with the inspection image. The reference image is obtained by scanning a printed document that the user has visually confirmed to be free of printing defects. Alternatively, a printed image obtained by RIP processing of PDL data included in the print job may be used as the reference image.

[0019] The inspection image acquisition unit 302 acquires an inspection image of the printed material to be inspected. The inspection image is acquired by reading the printed material output from the printing device 190 with the image reading device 104.

[0020] The operation input reception unit 303 receives operation input from the user, for example, the specification of pairs of regions that touch in 3D on the inspection image, and the input of inspection parameters.

[0021] The region identification unit 304 identifies a pair of regions (region pairs) that will come into contact with each other when a three-dimensional output is created by assembling flat printed materials. This identification may be performed automatically or manually, as described later.

[0022] The inspection parameter setting unit 305 sets inspection parameters based on user input and other factors. The inspection parameters are threshold values ​​for pass / fail determination corresponding to the inspection items. In this embodiment, inspection parameters are set for the aforementioned region pairs and for regions other than the region pairs. Inspection items include misalignment of patterns and shapes that appear after assembly, and the color and gloss of adjacent surfaces of the three-dimensional product. The inspection unit 306 performs an inspection to determine whether the inspection image contains printing defects based on the set inspection parameters.

[0023] The display control unit 307 controls the display of the GUI (Graphical User Interface) using the UI panel 108. For example, it displays each region pair on the inspection image in a way that the user can identify, or displays the inspection results.

[0024] The print control unit 308 controls the printing device 190. For example, depending on the content of the print image, it adds alignment marks and inspection marks corresponding to the aforementioned area pairs to the print image and causes the printing device 190 to perform the printing process.

[0025] <Operation Flow of Information Processing Device> Figure 4 is a flowchart showing the general flow of the inspection process in the information processing device 100. The series of processes shown in the flowchart of Figure 4 are realized by the CPU 101 reading a control program stored in the ROM 103 or other storage device, loading it into the RAM 102, and executing it. The data used in the series of processes is stored in the ROM 103, RAM 102, or a separately prepared storage device, and is read out as needed for processing. The following explanation will follow the flowchart of Figure 4. Note that the symbol "S" means step.

[0026] In S401, the reference image acquisition unit 301 acquires a reference image corresponding to the printed material related to the page to be inspected, which is output from the printing device 190. In the following S402, the inspection image acquisition unit 302 acquires an inspection image of the printed material related to the page to be inspected.

[0027] In S403, the reference image acquired in S401 and the inspection image acquired in S402 are aligned. For example, the four corners of the printed material, or separately added alignment marks, are used to align each position in the reference image with each position in the inspection image.

[0028] In S404, the region identification unit 304 performs a process to identify region pairs that are separated on the printed material but will come into contact with each other when the printed material becomes a three-dimensional product. Region pairs are identified automatically, for example, by acquiring three-dimensional CAD data of the printed material related to the inspection image and analyzing the 3D image representing the shape when the printed material is transformed into a three-dimensional object using AI (Artificial Intelligence) technology. Alternatively, the region pairs may be identified by the user directly specifying them via a GUI while viewing the inspection image. Alternatively, region pairs may be identified by acquiring and referring to a table prepared in advance by the user that associates regions in the planar state of the printed material with edges when it becomes a three-dimensional product. Details of the region pair identification process according to this embodiment will be described later.

[0029] In S405, the display control unit 307 displays the region pairs identified in S404 on the GUI of the UI panel 108 in a state that can be identified by the user. Figure 5A is an example of a GUI related to the inspection process according to this embodiment. The UI screen 500 shown in Figure 5A consists of a first pane 510 that displays the inspection image, a second pane 520 that displays a 3D image corresponding to the inspection image, and a third pane for setting inspection parameters. The first pane 510 displays an inspection image obtained by scanning a printed copy of a box-shaped product package. Different pattern designs are superimposed on the inspection image for each of the three region pairs 511 to 513 identified in S404, and each region pair is highlighted so that it can be identified. Here, region pair 511 consists of region 511a and region 511b, region pair 512 consists of region 512a and region 5121b, and region pair 513 consists of region 513a and region 513b. Other region pairs exist, but for the sake of explanation, three region pairs are shown here. All region pairs identified in S404 may be displayed, or only some region pairs, such as prominent parts of a product package, may be displayed. The second pane 520 displays a 3D image corresponding to the inspection image, and the three sides 521-523 corresponding to the region pairs 511-513 highlighted in the first pane 510 are similarly highlighted. Note that the highlighting of region pairs may be performed on only one of either the inspection image in the first pane or the 3D image in the second pane. In the example above, the pattern is different for each region pair, but for example, the same pattern may be used but with different colors, or a unique string such as "Pair 1" or "Pair 2" may be attached. In addition, instead of the static representation described above, the highlighting may be performed using dynamic representations, such as different blinking timings or sequential switching. Furthermore, the inspection image or 3D image may be made transparent so that only the region pairs are visible.

[0030] In S406, the inspection parameter setting unit 305 sets the inspection parameters based on the user's operation input received by the operation input reception unit 303. For example, in the case of the UI screen 500 shown in Figure 5A above, the user sets the inspection parameters for the region pair and the inspection parameters for regions other than the region pair using the input fields 531 and 532 in the third pane. At this time, the user may select a region pair of interest from the region pairs highlighted in the first pane 510 and set the desired inspection parameters for each region pair. Note that the UI screen 500 is an example where the inspection item is setting the tolerance value for "positional misalignment," and it is possible to set different tolerance values ​​(inspection thresholds) for each inspection item, for example. At this time, for example, the tolerance value for the region pair may be set automatically so that it is smaller than the tolerance value for regions other than the region pair (i.e., the inspection level becomes stricter). Alternatively, multiple tolerance values ​​may be prepared and set automatically according to the feature quantity (complexity) of the image. It is also possible to set different inspection parameters for each region pair.

[0031] In S407, the inspection unit 306 compares the reference image acquired in S401 with the inspection image acquired in S402 based on the inspection parameters set in S406, and performs an inspection to determine whether a printing defect exists based on the difference between the two images. Details of the inspection process will be described later.

[0032] In S408, the display control unit 307 displays the results of the inspection in S407 on the GUI. In the following S409, it is determined whether the inspection of all pages of the submitted print job has been completed. If there are any unprocessed pages, the process returns to S401 and continues with processing the next page. On the other hand, if the inspection of all pages has been completed, this process is terminated. The above is a general overview of the inspection process in the information processing device 100.

[0033] <Details of region pair identification process> Next, we will explain in detail the region pair identification process in S404. Figure 6 is a flowchart detailing the region pair identification process. The following explanation will follow the flow shown in Figure 6. In the following explanation, the symbol [S] represents a step.

[0034] In S601, a 3D image corresponding to the inspection image acquired in S402 is obtained. Here, the 3D image can be obtained, for example, by storing three-dimensional CAD data for all pages fed into the information processing device 100 along with the print job in a storage device (not shown) and reading the data for the page to be inspected.

[0035] In S602, the next process to be executed is determined by how the region pair is identified. If it is identified automatically, S603 is executed; if it is identified manually, S605 is executed.

[0036] In S603, the 3D image acquired in S601 is input into a pre-trained learning model, and regions that touch each other when the printed material related to the inspection image is transformed into a three-dimensional object are extracted. In the following S604, each region pair on the inspection image acquired in S402 is identified based on the touching regions extracted in S603. Note that the method for automatically identifying region pairs is not limited to the example above; for example, region pairs within the inspection image may be identified by estimating the three-dimensional state from the inspection image using AI technology.

[0037] In S605, the inspection image acquired in S402 and the 3D image acquired in S601 are displayed in the first and second panes of the GUI, respectively. Figures 5B and 5C are examples of GUIs when the user manually specifies a region pair; Figure 5B is an example of a GUI when specifying a region pair on an inspection image, and Figure 5C is an example of a GUI when specifying a region pair on a 3D image. In the case of GUI 500 shown in Figure 5B, a message prompting the user to specify a region pair is displayed in the first pane 510, and the user can specify a pair of regions that make up the region pair using an input device such as a mouse. In the case of GUI 500 shown in Figure 5C, a message prompting the user to specify a region pair is displayed in the first pane 520, and the user can specify a pair of regions that make up the region pair using an input device such as a mouse. At this time, when a region pair is specified in one pane, the content indicating the specified region pair may be reflected in the other pane.

[0038] In S606, the system receives an input from the user specifying a pair of regions that constitute a region pair on the inspection image or 3D image (two regions that are separate in a planar state but touch when viewed in 3D). In the subsequent S607, the pair of regions related to the user input received in S606 is identified as a region pair. The user performs steps S606 and S607 for each desired region pair. Once the region pair identification is complete, either automatically or manually, the system returns to the main flow shown in Figure 4, and the identified one or more region pairs are highlighted in S405. At this time, as mentioned above, each region pair is displayed in an identifiable manner, but supplementary information on how each region pair touches (e.g., 90-degree rotation, left-right flip, upside down, etc.) may be displayed on the GUI, as shown in Figure 7. This makes it easier for the user to understand the difference in positional relationships between the inspection state and the 3D state. The above describes the region pair identification process.

[0039] <Details of the inspection process> Next, we will explain the inspection process in S407 in detail. Figure 8 is a flowchart detailing the inspection process. The inspection process is performed in predetermined region units within the inspection image, depending on the presence or absence of region pairs and the inspection items. Below, we will explain the inspection of positional misalignment when two separated regions that connect when the image is made into a 3D object are set as region pairs, following the flow in Figure 8. In the following explanation, the symbol [S] means step.

[0040] In S801, the next process to be executed is determined based on whether the target area of ​​the inspection image acquired in S402 is part of a region pair. If the target area is part of a region pair, S802 is executed next; if it is not part of a region pair, S806 is executed next.

[0041] In S802, the absolute difference value relative to the reference image is calculated for one of the two regions constituting the region pair of areas of interest (hereinafter referred to as the "first target region"). In the subsequent S803, the absolute difference value relative to the reference image is calculated for the other region of the two regions constituting the region pair of areas of interest (hereinafter referred to as the "second target region"). Figure 9(a) is a diagram illustrating the absolute positional shift relative to the reference image for the first and second target regions on an inspection image relating to a certain region pair. In Figure 9(a), the × mark 903 indicates the ideal position of a pixel within the first target region 901 in the inspection image, and the black circle 904 indicates the actual pixel position. Also, the × mark 905 indicates the ideal position of a pixel within the second target region 902 in the inspection image, and the black circle 906 indicates the actual pixel position. Currently, the first target area 901 is shifted by Δx1 in the x direction and Δy1 in the y direction, while the second target area 902 is shifted by Δx2 in the x direction and Δy2 in the y direction. When the inspection item is the amount of print position misalignment, the absolute difference value is calculated in this way.

[0042] In S804, the relative difference between the absolute difference value of the first target region calculated in S802 relative to the reference image and the absolute difference value of the second target region calculated in S803 relative to the reference image is calculated. Figure 9(b) is a diagram illustrating how the positional displacement amount as a relative difference value is determined in the specific example shown in Figure 9(a) above. As shown in Figure 9(b), the × marks 903 and 905, which indicate the ideal pixel positions, coincide, while the black circles 904 and 906, which indicate the actual pixel positions, are shifted by "Δx2-Δx1" in the x direction and "Δy2-Δy1" in the y direction. As a result, when made into a 3D object, the pattern will be shifted vertically and horizontally between the first target region and the second target region related to the region pair by the amount of the relative positional displacement mentioned above. Figures 10 and 11 are diagrams illustrating variations in positional displacement that may be subject to inspection. Figures 10(a) and (b) show specific examples where the entire shape (edge) is printed correctly, but the internal pattern is not. Figure 10(a) shows a case where the left edge of pattern 1001 is distorted diagonally downward to the left, and Figure 10(b) shows a case where the left edge of pattern 1002 is cut off. Figures 11(a) and (b) also show specific examples where the shape is not printed correctly; (a) shows a case where the circled area 1101 is distorted diagonally downward to the left, and (b) shows a case where the circled area 1102 is compressed horizontally. Although not shown as specific examples, it is also possible that both the shape and the pattern may be distorted during printing. In all of these cases, by determining the absolute positional deviation amount from the reference image for each of the first and second target areas on the inspection image related to the region pair, and then determining the relative positional deviation amount based on that result, it is possible to detect positional deviations that are unacceptable in 3D.

[0043] In S805, a pass / fail judgment is made according to the inspection parameters set for the region pair in S406. In this way, for example, it becomes possible to detect positional misalignment in two separated regions on an inspection image that constitute a certain pattern on a product package, where the misalignment is within the acceptable range for each region individually, but becomes outside the acceptable range when assembled into a three-dimensional object. In cases such as package printing or label printing, if the assembly error in the three-dimensional object is known in advance, the relative difference may be calculated by adding a margin to the tolerance value by the amount of that assembly error.

[0044] In S806, absolute difference values ​​are calculated for the regions other than the region pair that are of interest as the target region, relative to the reference image. In the following S807, a pass / fail judgment is made according to the inspection parameters set for the region in S406.

[0045] The above describes the inspection process. While we have explained the inspection of discrepancies in patterns and shapes within region pairs, color differences (color misalignment) within region pairs can also be inspected in a similar manner. In the case of color misalignment inspection, region pairs that overlap with the same color when the object is made three-dimensional are identified, an acceptable value for the color difference ΔE in the identified region pair is set, and the color misalignment inspection is performed based on the set acceptable value. In this case, the acceptable value for color misalignment may be set automatically according to the color characteristics (hue, area, color change). For example, if the color that overlaps the identified region pair is a color with high contrast, such as black, or if the identified region pair is a prominent part of the product package and a single color is widely used, the color change is more noticeable, so the acceptable value may be set more strictly. In addition to color misalignment, various image quality items such as gloss, granularity, and sharpness can also be inspected in a similar manner.

[0046] <Variation> In the above embodiment, the region pairs were displayed identifiable on the GUI before setting the inspection parameters, but they may also be printed, for example, on the inspection report showing the inspection results. Alternatively, they may be printed on the parts of the printed material that are unnecessary in the three-dimensional output (so-called waste in the waste removal process).

[0047] As described above, according to this embodiment, it is possible to identify pairs of regions that are separate on a printed surface but will come into contact when transformed into a three-dimensional product, and to inspect for printing defects in those regions while they remain in a planar state.

[0048] [Embodiment 2] For example, in the case of package printing, there are areas within the printed material that become invisible when the material is made three-dimensional because they overlap with other areas (overlap areas). Also, in the case of label printing or sticker printing, there are areas within the printed material that become invisible or difficult to see depending on the shape of the product when wrapped around or attached to the product. For such areas that are not visible or difficult to see on the final three-dimensional output (hereinafter referred to as "obscured areas"), the need for inspection is low. Therefore, in Embodiment 2, we will describe an embodiment in which the inspection level is set low or obscured areas within the printed material are excluded from inspection. Note that the basic configuration of the print inspection system is the same as in Embodiment 1, so below we will describe the differences, namely the operation flow of the information processing device according to this embodiment.

[0049] <Operation Flow of Information Processing Device> The difference between the inspection process according to this embodiment and the inspection process according to Embodiment 1 is evident in S404 and S405 of the series of processes shown in the flowchart of Figure 4. Specifically, in this embodiment, in S404, the "area pair identification process" is replaced with the "shielded area identification process," and in S405, the "display of area pairs" is replaced with the "display of shielded areas."

[0050] Figure 12 is a flowchart detailing the shielding area identification process according to this embodiment. The following explanation will follow the flow shown in Figure 12. In the following explanation, the symbol [S] represents a step.

[0051] In S1201, a 3D image corresponding to the inspection image acquired in S402 is obtained. Here, the 3D image can be obtained, for example, by storing three-dimensional CAD data for all pages fed into the information processing device 100 along with the print job in a storage device (not shown) and reading the data for the page to be inspected.

[0052] In S1202, the next process to be executed is determined by how the shielded area is identified. If it is identified automatically, S1203 is executed; if it is identified manually, S1204 is executed.

[0053] In S1203, the occluded areas within the inspection image are identified by inputting the inspection image acquired in S402 and the 3D image acquired in S1201 into a pre-trained learning model. Figure 13 shows the results of the occluded area identification process according to this embodiment applied to an inspection image obtained by scanning a printed copy of a box-shaped product package. In Figure 13, the areas that are filled in black indicate the identified occluded areas, and the other white areas indicate areas other than the occluded areas.

[0054] In S1204, the inspection image acquired in S402 and the 3D image acquired in S1201 are displayed on the GUI. The user operates an input device such as a mouse according to messages prompting them to specify the occluded area displayed in the GUI, for example, by specifying the occluded area present in the inspection image. Note that since the occluded area is an area that does not appear in the 3D image, it is not necessary to display the 3D image on the GUI.

[0055] In S1205, the system receives input from the user to specify an occlusion area on the examination image. In the subsequent S1206, the area related to the user input received in S1205 is identified as the occlusion area. The user may specify all of the occlusion areas on the examination image, or only a portion of them.

[0056] Once the overlap areas have been identified automatically or manually, the process returns to the main flow, and the identified one or more shielding areas are highlighted in S405. This concludes the description of the shielding area identification process.

[0057] Subsequently, in S406 of the main flow, the inspection parameters for the shielded area and the inspection parameters for the area outside the shielded area are set based on the user's operation input received by the operation input reception unit 303. At this time, as mentioned above, the user sets the tolerance value for the shielded area to be larger than the tolerance value for the area outside the shielded area (so that the inspection level is lenient), or to exclude the shielded area from inspection. In this case, if a tolerance value for the area outside the shielded area is entered, a larger tolerance value may be automatically entered. Furthermore, the relationship between the tolerance values ​​of the shielded area and the area outside the shielded area, and whether or not to exclude the area outside the shielded area from inspection, may be determined depending on the type of paper.

[0058] As described above, according to this embodiment, when inspecting printed materials, areas that are not visible to the human eye or are difficult to see at the stage of the three-dimensional output are either given a lower inspection level or excluded from inspection. This limits the inspection of printed materials to the necessary areas, thereby suppressing a decrease in the productivity of the print inspection system.

[0059] [Embodiment 3] Next, Embodiment 3 describes a method in which inspection marks (hereinafter referred to as "inspection marks") corresponding to each region pair are printed on the margins of the paper and used for inspection. Inspection marks are also called inspection patterns. Note that the parts common to Embodiment 1 will be omitted from the explanation, and the following explanation will focus on the parts specific to this embodiment, referring to Figures 14(a) to (c).

[0060] Figure 14(a) is an example of a printed document with inspection marks added, output from the printing device 190 in this embodiment. In the example in Figure 14(a), a first inspection mark consisting of a pair of marks 1401 and 1402 and a second inspection mark consisting of a pair of marks 1411 and 1412 are printed for each of the two region pairs 1400 / 1410. Figures 14(b) and (c) are enlarged views of the first and second inspection marks in Figure 14(a), with (b) showing the case of inspection failure and (c) showing the case of inspection success.

[0061] In Figure 14(b), marks 1401 and 1402, which constitute the first inspection mark, are striped patterns in a direction that is sensitive to color misalignment, suitable for evaluating the positional misalignment (color plate misalignment) for each color in each of the two regions related to region pair 1400. The two regions related to region pair 1400 touch each other in a 90-degree inversion relative to their planar state when they are in three dimensions. Therefore, mark 1401 is a striped pattern suitable for evaluation in the horizontal direction, and mark 1402 is a striped pattern suitable for evaluation in the vertical direction. Both marks 1401 and 1402 are high-frequency striped patterns composed of straight lines corresponding to each color of the printing pigment (e.g., CMYK), and they show a state where the straight lines of each color are slightly misaligned. If there is no positional misalignment between color plates, the pair of marks 1401 and 1402 will be printed so that the straight lines corresponding to each color overlap at the same position, and therefore will not form a striped pattern as shown in Figure 14(c).

[0062] Similarly, in Figure 14(b), marks 1411 and 1412, which constitute the second inspection mark, are striped patterns consisting of straight lines in a direction that makes it easy to detect whether there is a step in the pattern at the boundary of the region pair 1410 when the horizontally extending gray pattern 1420 becomes three-dimensional. Both marks 1411 and 1412 are high-frequency striped patterns consisting of straight lines corresponding to each color of the printing pigment (e.g., CMYK), and they show a state in which the straight lines of each color are slightly misaligned. If there is no misalignment between the color plates, the pair of marks 1411 and 1412 will be printed so that the straight lines corresponding to each color overlap at the same position, and therefore will not form a striped pattern as shown in Figure 14(c).

[0063] The inspection marks, consisting of a pair of marks as described above, are printed by the print control unit 308 at positions on the paper that will be discarded during cutting, for example, and in close proximity to each pair of regions, according to the content of the printed image. During the inspection process, the pixel values ​​of the marked areas included in the inspection image are analyzed to determine whether a color misalignment exceeding the tolerance limit has occurred. Alternatively, an inspector may directly visually check the inspection marks formed on the printed material to confirm whether a color misalignment has occurred.

[0064] In the example described above, a printed image with inspection marks is generated and printed, and the inspection is performed based on the inspection marks contained in that image. However, the method is not limited to this. For example, inspection marks may be added to a printed material obtained by printing without inspection marks, and the inspection may be performed based on the inspection marks contained in that image. Furthermore, the inspection marks are variable depending on the purpose of the inspection. For example, if the purpose is to inspect for color misalignment, at least one dominant color patch in the two regions constituting the region pair may be assigned as the inspection mark. This allows for automatic analysis or visual confirmation by an inspector of color misalignment between regions related to the region pair. In addition to color misalignment, this method can also be applied to various image quality items such as glossiness, sharpness, and graininess.

[0065] As described above, according to this embodiment, inspection marks corresponding to the inspection purpose are printed separately at adjacent positions to the area pair and the inspection is performed. This improves inspection accuracy and makes visual inspection by inspectors easier.

[0066] <Other Embodiments> The embodiments described above primarily illustrate the inspection of printed unfolded diagrams of box-shaped product packages that fit on a single sheet of paper, but are not limited to this. In other words, each embodiment can be broadly applied to printed materials that are flat during printing and inspection, but become three-dimensional products after processing such as blanking (cutting) and assembly. Specifically, it can be applied to cases where three-dimensional packages or paper crafts are created from parts printed on multiple sheets of paper. It can also be applied to label printing (wrap printing) where the labels are wrapped around three-dimensional products and the ends meet. Furthermore, it can be applied to book band printing where the book's image and the band's image are continuous to form a single image. It can also be applied to stickers that are overlaid on a background image so that the image and color blend seamlessly at the boundary.

[0067] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0068] Furthermore, this disclosure includes the following configurations and methods.

[0069] [Configuration 1] An information processing device for inspecting printed materials, Acquisition means for acquiring a read image obtained by optically reading the aforementioned printed material, A means for identifying a pair of regions in the read image that are to be inspected and which consist of two regions that come into contact when the printed material transforms into a three-dimensional object, A display control means that causes the identified region pair to be displayed on the display means in an identifiable manner, An information processing device characterized by having the following features.

[0070] [Configuration 2] The information processing apparatus according to configuration 1, characterized in that the identifying means identifies the region pair based on an image corresponding to the printed material.

[0071] [Configuration 3] The information processing apparatus according to configuration 2, characterized in that the identifying means analyzes an image corresponding to the printed material to identify the region pair.

[0072] [Structure 4] The display control means causes the display means to display an image corresponding to the printed material. The information processing apparatus according to configuration 2, characterized in that the identifying means identifies the region pair based on user operation input for an image corresponding to the printed material displayed on the display means.

[0073] [Composition 5] The information processing apparatus according to any one of configurations 2 to 4, characterized in that the image corresponding to the printed material is the read image or a 3D image representing the shape when the printed material is transformed into a three-dimensional object.

[0074] [Composition 6] The aforementioned display means is a graphical user interface, The information processing apparatus according to configuration 5, characterized in that the display control means causes the graphical user interface to display the region pair.

[0075] [Composition 7] The display control means is The graphical user interface is used to display the read image. The region pair is displayed superimposed on the read image within the graphical user interface. The information processing device according to configuration 6, characterized by the features described therein.

[0076] [Structure 8] The display control means is The 3D image is displayed on the graphical user interface. The region pair is displayed superimposed on the 3D image in the graphical user interface. The information processing device according to configuration 6, characterized by the features described therein.

[0077] [Composition 9] The information processing apparatus according to any one of configurations 6 to 8, characterized in that the display control means causes a plurality of the region pairs to be displayed on the graphical user interface, and the display mode of each region pair is different.

[0078] [Configuration 10] The information processing apparatus according to configuration 9, characterized in that the display control means makes the display modes of a plurality of region pairs different by static representation.

[0079] [Composition 11] The information processing apparatus according to configuration 10, characterized in that the display control means makes the pattern different, the color different, or the string of characters added different for a plurality of the region pairs.

[0080] [Composition 12] The information processing apparatus according to configuration 9, characterized in that the display control means dynamically differentiates the display modes of a plurality of the area pairs.

[0081] [Composition 13] The information processing apparatus according to configuration 12, characterized in that the display control means causes the blinking timing of a plurality of region pairs to be different, or sequentially switches between a plurality of region pairs.

[0082] [Composition 14] The information processing device according to configuration 6, characterized in that the display control means further displays supplementary information on how the two regions constituting the region pair touch each other in three dimensions.

[0083] [Composition 15] The system further includes setting means for setting parameters for the aforementioned inspection, The setting means sets separately the parameters for the region relating to the region pair in the read image and the parameters for the region other than the region relating to the region pair in the read image. An information processing device according to any one of configurations 1 to 14, characterized by the above.

[0084] [Composition 16] The information processing apparatus according to configuration 15, characterized in that the setting means automatically sets the parameters such that the inspection level for the region related to the region pair in the read image is stricter than the inspection level for the region other than the region related to the region pair in the read image.

[0085] [Composition 17] The information processing apparatus according to configuration 15, characterized in that the setting means automatically sets the parameters according to the characteristic quantities of the images contained in the printed material.

[0086] [Composition 18] The information processing apparatus according to any one of claims 1 to 17, characterized in that the printed material is a printed material to which inspection marks corresponding to the region pair are attached.

[0087] [Composition 19] The information processing apparatus according to configuration 18, further comprising a printing control means for causing a printing device to print a printed document bearing the aforementioned inspection mark.

[0088] [Configuration 20] The information processing apparatus according to configuration 19, characterized in that the printing control means causes the inspection marks to be printed in the margin of the printed material at positions adjacent to each of the two regions constituting the region pair.

[0089] [Composition 21] The information processing apparatus according to any one of configurations 18 to 20, characterized in that the inspection mark is a straight line extending in a direction that is sensitive to positional misalignment.

[0090] [Composition 22] The information processing apparatus according to any one of claims 18 to 20, characterized in that the inspection mark is a color patch of at least one dominant color in the two regions constituting the region pair.

[0091] [Composition 23] The system further includes an inspection means that performs the inspection by comparing the read image with a reference image corresponding to the read image, The inspection means, in inspecting the two regions constituting the region pair, The absolute difference between each of the two aforementioned regions and the reference image is calculated. The relative difference of the absolute difference of each of the two calculated regions relative to the reference image is calculated, It determines whether the calculated relative difference exceeds the acceptable limit. An information processing device according to any one of configurations 1 to 22, characterized by the features described herein.

[0092] [Composition 24] An information processing device for inspecting printed materials, Acquisition means for acquiring a read image obtained by optically reading the aforementioned printed material, A means for identifying an obscured area in the image to be inspected that becomes difficult to see when the printed material changes into a three-dimensional form, A display control means that causes the identified shielding area to be displayed on the display means in an identifiable manner, An information processing device characterized by having the following features.

[0093] [Composition 25] The information processing apparatus according to configuration 24, characterized in that the identifying means identifies the occluded area based on an image corresponding to the printed material.

[0094] [Composition 26] The information processing apparatus according to configuration 25, characterized in that the identifying means analyzes an image corresponding to the printed material to identify the occluded area.

[0095] [Composition 27] The display control means causes the display means to display an image corresponding to the printed material. The information processing apparatus according to configuration 25, characterized in that the identifying means identifies the occluded area based on user operation input for an image corresponding to the printed material displayed on the display means.

[0096] [Composition 28] The information processing apparatus according to any one of configurations 25 to 27, characterized in that the image corresponding to the printed material is the read image or a 3D image representing the shape when the printed material is transformed into a three-dimensional object.

[0097] [Composition 29] The aforementioned display means is a graphical user interface, The information processing apparatus according to configuration 28, characterized in that the display control means causes the occluded area to be displayed on the graphical user interface.

[0098] [Composition 30] The display control means is The graphical user interface is used to display the read image. The occluded area is displayed by superimposing it on the read image within the graphical user interface. The information processing apparatus according to configuration 29, characterized by the features described herein.

[0099] [Composition 31] The system further includes setting means for setting parameters for the aforementioned inspection, The setting means sets separately the parameters for the occluded region of the read image and the parameters for the region of the read image other than the occluded region. An information processing apparatus according to any one of the configurations 24 to 30, characterized by the features described herein.

[0100] [Composition 32] The information processing apparatus according to configuration 31, characterized in that the setting means automatically sets the parameters such that the inspection level for the occluded area in the read image is stricter than the inspection level for areas other than the occluded area in the read image.

[0101] [Configuration 33] The information processing apparatus according to any one of configurations 24 to 32, characterized in that the shielding region is an overlap region within the printed material that becomes invisible when the printed material changes into a three-dimensional shape and overlaps with other regions.

[0102] [Method 1] A control method for an information processing device that performs inspection of printed materials, The acquisition step involves acquiring a read image obtained by optically reading the aforementioned printed material, A step of identifying a pair of regions in the read image that are to be inspected and which consist of two regions that will come into contact when the printed material is transformed into a three-dimensional object, A display control step that causes the identified pair of regions to be displayed on the display means in an identifiable manner, A control method characterized by including

[0103] [Method 2] A control method for an information processing device that performs inspection of printed materials, The acquisition step involves acquiring a read image obtained by optically reading the aforementioned printed material, A step of identifying an obscured area in the image to be inspected that becomes difficult to see when the printed material changes into a three-dimensional shape, A display control step that causes the identified shielding area to be displayed on the display means in an identifiable manner, A control method characterized by including

[0104] [Composition 34] A program for causing a computer to function as an information processing device as described in any one of configurations 1 to 33.

Claims

1. An information processing device for inspecting printed materials, Acquisition means for acquiring a read image obtained by optically reading the aforementioned printed material, A means for identifying a region pair consisting of two regions that will come into contact when the printed material transforms into a three-dimensional object, which are regions to be inspected in the read image, A display control means that causes the identified region pair to be displayed on the display means in an identifiable manner, An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, characterized in that the identifying means identifies the region pair based on an image corresponding to the printed material.

3. The information processing apparatus according to claim 2, characterized in that the identifying means analyzes an image corresponding to the printed material to identify the region pair.

4. The display control means causes the display means to display an image corresponding to the printed material. The information processing apparatus according to claim 2, characterized in that the identifying means identifies the region pair based on user operation input for an image corresponding to the printed material displayed on the display means.

5. The information processing apparatus according to claim 2, characterized in that the image corresponding to the printed material is the read image or a 3D image representing the shape when the printed material is transformed into a three-dimensional object.

6. The aforementioned display means is a graphical user interface, The information processing apparatus according to claim 5, characterized in that the display control means causes the graphical user interface to display the region pair.

7. The display control means is The graphical user interface is used to display the read image. The region pair is displayed superimposed on the read image within the graphical user interface. The information processing apparatus according to feature 6.

8. The display control means is The 3D image is displayed on the graphical user interface. The region pair is displayed superimposed on the 3D image in the graphical user interface. The information processing apparatus according to feature 6.

9. The information processing apparatus according to any one of claims 6 to 8, characterized in that the display control means causes a plurality of the region pairs to be displayed on the graphical user interface, and the display mode of each region pair is different.

10. The information processing apparatus according to claim 9, characterized in that the display control means causes the display modes of a plurality of region pairs to differ by static representation.

11. The information processing apparatus according to claim 10, characterized in that the display control means makes the pattern different, the color different, or the string of characters added different for a plurality of the region pairs.

12. The information processing apparatus according to claim 9, characterized in that the display control means dynamically differentiates the display modes of a plurality of the region pairs.

13. The information processing apparatus according to claim 12, characterized in that the display control means causes the blinking timing of a plurality of region pairs to be different, or sequentially switches between a plurality of region pairs.

14. The information processing apparatus according to claim 6, characterized in that the display control means further displays supplementary information on how the two regions constituting the region pair touch each other in three dimensions.

15. The system further includes setting means for setting parameters for the aforementioned inspection, The setting means sets separately the parameters for the region relating to the region pair in the read image and the parameters for the region other than the region relating to the region pair in the read image. The information processing apparatus according to feature 1.

16. The information processing apparatus according to claim 15, characterized in that the setting means automatically sets the parameters such that the inspection level for the region related to the region pair in the read image is stricter than the inspection level for the region other than the region related to the region pair in the read image.

17. The information processing apparatus according to claim 15, characterized in that the setting means automatically sets the parameters according to the characteristic quantities of the patterns contained in the printed material.

18. The information processing apparatus according to claim 1, characterized in that the printed material is a printed material to which inspection marks corresponding to the region pair are attached.

19. The information processing apparatus according to claim 18, further comprising a printing control means for causing a printing device to print a printed document bearing the aforementioned inspection mark.

20. The information processing apparatus according to claim 19, characterized in that the printing control means causes the inspection marks to be printed in the margin of the printed material at positions adjacent to each of the two regions constituting the region pair.

21. The information processing apparatus according to claim 18, characterized in that the inspection mark is a straight line extending in a direction that is sensitive to positional misalignment.

22. The information processing apparatus according to claim 18, characterized in that the inspection mark is a color patch of at least one dominant color in the two regions constituting the region pair.

23. The system further includes an inspection means that performs the inspection by comparing the read image with a reference image corresponding to the read image, The inspection means, in inspecting the two regions constituting the region pair, The absolute difference between each of the two aforementioned regions and the reference image is calculated, The relative difference of the absolute difference of each of the two calculated regions relative to the reference image is calculated, It determines whether the calculated relative difference exceeds the acceptable limit. The information processing apparatus according to feature 1.

24. An information processing device for inspecting printed materials, Acquisition means for acquiring a read image obtained by optically reading the aforementioned printed material, A means for identifying an obscured area in the image to be inspected that becomes difficult to see when the printed material changes into a three-dimensional shape, A display control means that causes the identified shielding area to be displayed on the display means in an identifiable manner, An information processing device characterized by having the following features.

25. The information processing apparatus according to claim 24, characterized in that the identifying means identifies the occluded area based on an image corresponding to the printed material.

26. The information processing apparatus according to claim 25, characterized in that the identifying means identifies the occluded area by analyzing an image corresponding to the printed material.

27. The display control means causes the display means to display an image corresponding to the printed material. The information processing apparatus according to claim 25, characterized in that the identifying means identifies the occluded area based on user operation input for an image corresponding to the printed material displayed on the display means.

28. The information processing apparatus according to claim 25, characterized in that the image corresponding to the printed material is the read image or a 3D image representing the shape when the printed material is transformed into a three-dimensional object.

29. The aforementioned display means is a graphical user interface, The information processing apparatus according to claim 28, characterized in that the display control means causes the occluded area to be displayed on the graphical user interface.

30. The display control means is The graphical user interface is used to display the read image. The occluded area is displayed by superimposing it on the read image within the graphical user interface. The information processing apparatus according to feature 29.

31. The system further includes setting means for setting parameters for the aforementioned inspection, The setting means sets separately the parameters for the occluded region of the read image and the parameters for the region of the read image other than the occluded region. The information processing apparatus according to feature 24.

32. The information processing apparatus according to claim 31, characterized in that the setting means automatically sets the parameters such that the inspection level for the occluded area in the read image is stricter than the inspection level for the area other than the occluded area in the read image.

33. The information processing apparatus according to claim 24, characterized in that the shielding region is an overlap region within the printed material that becomes invisible when the printed material changes into a three-dimensional shape and overlaps with other regions.

34. A control method for an information processing device that performs inspection of printed materials, The acquisition step involves acquiring a read image obtained by optically reading the aforementioned printed material, A step of identifying a region pair consisting of two regions that will come into contact when the printed material is transformed into a three-dimensional object, in the region to be inspected in the read image, A display control step that causes the identified pair of regions to be displayed on the display means in an identifiable manner, A control method characterized by including

35. A control method for an information processing device that performs inspection of printed materials, The acquisition step involves acquiring a read image obtained by optically reading the aforementioned printed material, A step of identifying an obscured area in the image to be inspected that becomes difficult to see when the printed material changes into a three-dimensional shape, A display control step that causes the identified shielding area to be displayed on the display means in an identifiable manner, A control method characterized by including

36. A program for causing a computer to perform the control method described in claim 34 or 35.

Images