Reference image selection device, incorrect collating inspection machine, reference image selection method, and reference image selection program

Abstract

To provide a technology that can automatically select a reference image low in similarity to other reference image.SOLUTION: A reference image selection device comprises: an information acquisition unit 201 that, by a printed matter respectively picked up from a plurality of stockers being imaged a plurality of times such that at least a partial imaging range is made different from each other, acquires an obtained image group of a printed matter for each of a plurality of stockers; an amount-of-characteristic calculation unit 202 that calculates a characteristic amount of each image included in the image group for each of the plurality of stockers, using an amount-of-characteristic calculation method; and a comparison selection unit 204 that selects a reference image of one stocker of the image group belonging to the one stocker, and performs the selection relative to the plurality of stockers. The comparison selection unit 204 is configured to select the reference image so as to be different for each of the plurality of stockers on the basis of a learning model for obtaining similarity among a plurality of images from respective characteristic amounts of the plurality of images, and a calculated characteristic amount of each image for each of the plurality of stockers.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for inspecting whether or not there is misprinting in a bookbinding machine that pulls out printed materials such as signatures one by one and binds them into a book. [Background technology]

[0002] Conventionally, in a collating machine, multiple stockers (frames) are installed in a row in which multiple folded sheets of paper, which are printed materials with multiple pages printed on them, are stored in a stack. A folded sheet is pulled out one by one from each stocker by a caliper, overlapped with another folded sheet transported from upstream, and then transported to the next stage, and this operation is repeated.

[0003] Such bookbinding machines are equipped with a misprint inspection device that, as folded sections are pulled out one by one from the stocker, captures a specific portion of the pulled out folded section as an inspection image and checks for misprinting by comparing the obtained folded section image with a reference image registered in advance (see Patent Document 1).

[0004] In the case of the above-mentioned misprint inspection machine, if the reference images of the signatures in adjacent stockers are similar, for example, if the types of signatures stored in each stocker are mistaken and the same type of signatures are stored in adjacent stockers, there is a risk that the signatures drawn from these stockers will be determined to be genuine. Therefore, conventionally, an operator checks whether the reference images between the adjacent stockers are similar, and if the reference images are similar, an image of a signature that is less similar, preferably one that has no similarity, is selected as the reference image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 11-53552 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide a technique capable of automatically selecting a reference image that has low similarity to other reference images. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, one aspect of the present invention includes an acquisition unit that acquires a group of images of printed materials for each of a plurality of stockers owned by a collation machine, the group of images being obtained by imaging printed materials extracted from each of the plurality of stockers multiple times so that at least a portion of the imaging range is different from one another; a calculation unit that calculates a feature amount of each image included in the image group for each of the plurality of stockers using a predetermined feature amount calculation method; and a selection unit that selects a reference image for a stocker from a group of images belonging to the stocker and performs the selection for the plurality of stockers in order to select a reference image for misprinting inspection for each of the plurality of stockers, wherein the selection unit selects the reference image to be different for each of the plurality of stockers based on a learning model for determining the similarity between the plurality of images from the feature amounts of each of the plurality of images and the calculated feature amounts of each image in the image group for each of the plurality of stockers. Effect of the Invention

[0008] According to the embodiment of the present invention, it is possible to automatically select a reference image that has low similarity to other reference images. Note that other effects of the embodiment of the present invention will also be described in the following section on preferred embodiments of the present invention. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a schematic diagram showing a configuration of a part of a collating machine according to an embodiment. [Diagram 2] FIG. 13 is a diagram for explaining an imaging range of a signature. [Diagram 3] 1 is a block diagram showing a hardware configuration of a reference image selection device according to an embodiment. [Figure 4]1 is a block diagram showing a functional configuration of a reference image selection device according to an embodiment; [Diagram 5] 11A and 11B are diagrams for explaining calculation of a feature amount using a phase-only correlation method according to an embodiment. [Figure 6] FIG. 2 is a diagram for explaining a data set according to the embodiment. [Figure 7] 4 is a flowchart showing the operation of the misprint inspection machine and the reference image selection device according to the embodiment. [Figure 8] 10 is a flowchart showing a selection process according to the embodiment. [Figure 9] 11 is a flowchart showing a comparison and change process according to the embodiment. [Figure 10] FIG. 13 is a diagram for explaining a data set when a reference image is changed. [Figure 11] 10 is a flowchart showing an adjustment process according to the embodiment. [Figure 12] 13 is a diagram for explaining a data set in which the selection of the reference image has been adjusted by the adjustment process. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and duplicated explanations will be omitted. In addition, in this embodiment, a misprint inspection machine incorporating an image selection device according to this embodiment will be described as an example.

[0011] Fig. 1 is a schematic diagram showing a part of the configuration of a collating machine according to this embodiment. As shown in Fig. 1, the collating machine 1 according to this embodiment includes a plurality of stockers (pieces) 12 that store a plurality of folded signatures W in a stacked state, a plurality of calipers 14 that are individually arranged in each of the plurality of stockers 12 and pull out one signature W from the plurality of folded signatures W stored in the stockers 12, a camera 16 that captures an image of the folded signatures W, a pagination inspection machine 18 that inspects for pagination, and a reference image selection device 20. Note that, for the sake of explanation, only one each of the stockers 12 and the calipers 14 is shown in Fig. 1.

[0012] The multiple stockers 12 are arranged in a row along a conveying device (not shown) that extends in a substantially horizontal direction from upstream to downstream. The caliper 14 reciprocates, for example, every time the rotating shaft 141 rotates once, to grip the signature W in the bottom layer of the stockers 12 and pull it out. The direction in which the signature W is pulled out, in other words, the direction in which the caliper 14 swings, is along the longitudinal direction of the signature W. The signature W pulled out from the most upstream stocker 12 is conveyed downstream by the conveying device, and the signatures W pulled out from the multiple stockers 12 located downstream are sequentially overlapped to form a booklet before wire or perfect binding.

[0013] Camera 16 captures images multiple times each time one signature W is pulled out by caliper 14. The imaging range of camera 16 for one imaging includes the entire widthwise direction of signature W and a part of the widthwise direction. An image is captured each time a certain length of signature W is pulled out by caliper 14 so that this imaging range is imaged with a relative positional shift of a certain distance along the lengthwise direction of signature W. For example, a rotary encoder that generates a position detection pulse in response to the rotation of rotation shaft 141 of caliper 14 may be used for such imaging.

[0014] The rotary encoder generates k position detection pulses every time the rotating shaft 141 rotates once. In other words, one position detection pulse is generated every time the rotating shaft 141 rotates 360 / k degrees, that is, every time the caliper 14 pulls out the signature W by a certain length. The camera 16 captures an image in response to this position detection pulse. Such control may be performed by a control device of the collating machine 1 or the misprinting inspection machine 18 (not shown). In this embodiment, eight position detection pulses are generated and eight images are captured by the camera 16, thereby obtaining eight signature images (a group of images belonging to a certain stocker 12). Note that the position of the top or bottom end of the signature W may be read by a sensor, and the camera 16 may capture an image every time it is detected that the signature W is pulled out by a certain length.

[0015] Fig. 2 is a diagram for explaining the imaging range of a signature. T1 to T8 shown in Fig. 2 indicate misaligned imaging ranges (trigger points). For ease of understanding, imaging ranges T1 to T8 in Fig. 2 are indicated by solid lines for odd-numbered imaging times and dotted lines for even-numbered imaging times. Camera 16 is connected to misprinting inspection machine 18 by wire or wirelessly so as to be able to transmit data, and misprinting inspection machine 18 can obtain eight signature images for each stocker 12.

[0016] The misprinting inspection machine 18 performs a misprinting inspection for each stocker 12. In the misprinting inspection here, one of eight signature images obtained by imaging one signature W in one inspection target stocker 12 by the camera 16 is selected and compared with a reference image recorded in advance corresponding to the inspection target stocker 12. The selected signature image corresponds to the reference image, and if the reference image is an image in the imaging range T3, the selected signature image is also an image in the imaging range T3. The misprinting inspection machine 18 compares the selected signature image with the reference image, and if they match, it evaluates that no misprinting has occurred, and if they differ, it evaluates that misprinting has occurred. If it is evaluated that misprinting has occurred, it notifies the operator which stocker has misprinting through a notification means such as a display or speaker (not shown).

[0017] The reference image selection device 20 is connected to the misprinting inspection machine 18 via wire or wireless connection so as to be able to perform data communication with the misprinting inspection machine 18 in order to select the above-mentioned reference image from a plurality of quire images prior to the misprinting inspection. Specifically, the reference image selection device 20 acquires quire images from the misprinting inspection machine 18, selects an image that is preferable as a reference image from among the quire images, and performs this individually for each stocker 12. The reference image selection device 20 transmits a reference image corresponding to each selected stocker 12 to the misprinting inspection machine 18. The misprinting inspection machine 18, having acquired the reference image, registers the acquired reference image in a storage device and performs the misprinting inspection. Such selection of a reference image is performed every time the contents of the quire W are changed.

[0018] The reference image selection device 20 will be described in detail below. Fig. 3 is a block diagram showing the hardware configuration of the reference image selection device according to this embodiment. As shown in Fig. 3, the reference image selection device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a communication I / F (Interface) 24, a user I / F 25, and a storage device 26.

[0019] The CPU 21 executes the BIOS (Basic Input / Output System) read from the ROM 22, which is a non-volatile storage area, and executes the OS (Operating System), general-purpose applications, and various programs read from the storage device 26 by expanding them on the RAM 23, which is a volatile storage area. Examples of the programs include AI (Artificial Intelligence) and machine learning, which will be described later. The communication I / F 24 is used for data communication with the misprinting inspection machine 18, mainly for sending and receiving image data. The reference image selection device 20 may be connected to the misprinting inspection machine 18 from a remote location via a network or the like so as to be able to communicate with the misprinting inspection machine 18.

[0020] The user I / F 25 includes a display, input devices such as a keyboard and a mouse, a speaker, etc., and notifies the user of various information and receives input from the user. The storage device 26 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and stores various information used in the selection process executed by the reference image selection device 20. An example of the stored information is a learning model 261. The selection process and the learning model 261 will be described in detail later.

[0021] Examples of such a reference image selection device 20 include an information processing device such as a PC (Personal Computer) and a tablet terminal such as a smartphone.

[0022] Fig. 4 is a block diagram showing the functional configuration of the reference image selection device according to this embodiment. As shown in Fig. 4, the reference image selection device 20 includes, as functions, an information acquisition unit 201 for acquiring a quire image and the like, a feature amount calculation unit 202 for calculating a feature amount from the quire image, a generation unit 203 for generating a data set based on the feature amount, a comparison and selection unit 204 for selecting a reference image using the data set, a judgment processing unit 205 for making various judgments in the selection process, a transmission unit 206 for transmitting the selected reference image to the misprinting inspection machine 18, and a learning unit 207 for generating and updating a learning model for evaluating the similarity between images from the feature amount. These functions are realized by cooperation of the hardware of the reference image selection device 20 described above.

[0023] The selection process according to this embodiment is a process for selecting reference images so that the reference images of not only adjacent stockers 12 but all stockers 12 are not similar to each other or have little similarity to each other. To enable such selection, the reference image selection device 20 generates a data set by calculating feature amounts for each of the eight signature images for each stocker 12 using a phase-only correlation method, and compares the feature amounts of each stocker 12 in a round-robin manner using the generated data set.

[0024] Phase-only correlation is one of the image processing techniques that is resistant to disturbances and has few major errors, and is used for image matching. Simply put, phase-only correlation is a process that mathematically processes a digitalized image using Fourier transform, etc., and replaces the amplitude component with a fixed value from the amplitude component (shade data) and phase component (contour data) obtained as the processing result, in other words, it is a process that can find the strongest correlation using the phase component.

[0025] 5 is a diagram for explaining calculation of feature quantities using the phase-only correlation method according to this embodiment, and shows the processing results of the phase-only correlation method on two-dimensional image data in 3D. Z in FIG. 4 indicates a two-dimensional correlation peak value (hereinafter referred to as Z value). For example, if the sections W stored in the two stockers 12 are identical due to mismatching, the eight section images of these sections W will have the same Z value and the same X and Y values, which are the coordinate positions, in the corresponding imaging ranges T1 to T8. On the other hand, if the two sections W differ in design even partially, the two section images in the same imaging range including the different sections will differ from each other in at least one of the X, Y, and Z values.

[0026] The reference image selection device 20 uses such X, Y, and Z values ​​as features and evaluates the similarity between images by comparing the features, thereby making it possible to select reference images in which all the reference images have no or low similarity to each other.

[0027] FIG. 6 is a diagram for explaining a data set according to this embodiment. The reference image selection device 20 calculates and stores X, Y, and Z values ​​for each of eight signature images in one stocker 12 using a phase-only correlation method to generate a data set. In the example shown in FIG. 5, the data set is generated as a table in which trigger numbers uniquely indicating signature images in the imaging ranges T1 to T8 are assigned to the vertical columns, and XYZ values ​​are assigned to the horizontal columns. This data set is associated with a stocker number that identifies each of the multiple stockers 12, and therefore a data set is generated for each of the multiple stockers 12. Note that this table is merely for illustrative purposes, and the data set does not necessarily have to be generated in the form of a table.

[0028] The reference image selection device 20 uses this data set and the above-mentioned learning model 261 to select quire images that are not similar to each other or have low similarity between the stockers 12. The learning model 261 is generated as a result of machine learning such as deep learning and supervised learning. In this embodiment, the learning model 261 is generated by preparing a plurality of teacher data in which a combination of two images with low similarity, preferably two data sets corresponding to two quire images, is a correct answer, and a combination of two data sets corresponding to two images with low similarity is an incorrect answer, and providing the teacher data to the learning unit 206, and the learning unit 206 repeats learning. Note that although the data set as the teacher data includes a feature amount, it is not necessary to include a stocker number and a trigger number. In addition, a similarity indicating the degree of similarity judged by a person looking at the quire image is attached to this data set. By providing two data sets, the generated learning model 261 is capable of outputting the feature amount, that is, the similarity of X, Y, and Z values, more specifically, the similarity indicating the degree of similarity, between the two data sets. In addition, the similarity may be output as either 0 or 1, i.e., there is similarity or there is no similarity. In that case, it is not necessary to include the similarity in the teacher data. In addition, it is preferable that the learning model 261 is evaluated using a confusion matrix, that is, learning is repeated until the output result has a sufficiently high matching rate.

[0029] (device operation) The operation of the reference image selection device 20 will be described in detail together with the operation of the misprinting inspection machine 18. FIG. 7 is a flow chart showing the operation of the misprinting inspection machine and the reference image selection device according to this embodiment. When the collation machine 1 starts operation in response to a start operation by an operator, the camera 16 collects quire images of each of the imaging ranges T1 to T8 of the quire W in each stocker 12, and transmits the quire images to the misprinting inspection machine 18 in order to register a reference image from the collected quire images. This flow shows the operation of the misprinting inspection machine 18 and the reference image selection device 20 after the quire images are transmitted. The misprinting inspection machine 18 may collect the quire images by the camera 16. The flow on the left side of FIG. 7 shows the operation of the misprinting inspection machine 18, and the flow on the right side shows the operation of the reference image selection device 20 (reference image selection method).

[0030] 7, after the camera 16 captures an image of the signature W, the misprinting inspection machine 18 acquires signature images of the signature W in each stocker 12 (S101). In this embodiment, 30 stockers 12 are provided, and therefore a total of 30×8, or 240, signature images are acquired. After acquisition, the misprinting inspection machine 18 automatically selects a reference image for each stocker 12 from the eight signature images for each stocker 12 and provisionally registers the image (S102). In this automatic selection, the similarity between the reference images for each stocker 12 is not taken into consideration, and the image with the largest amount of information (for example, average amount of information) is selected as the reference image.

[0031] After the provisional registration, the misprinting inspection machine 18 transmits the acquired fold section images, i.e., 240 fold section images, and registration information indicating the provisionally registered fold section images to the reference image selection device 20 (S103). The fold section images transmitted at this time are associated with address information indicating which stocker 12 they belong to and which imaging range they are in.

[0032] After the transmission, the information acquisition unit 201 of the reference image selection device 20 receives the quire image and the registration information (S104), and a selection process is executed to select a new reference image (S105). The selection process will be described in detail later. After the selection process, the transmission unit 206 of the reference image selection device 20 transmits reference image information related to the reference image to the misprinting inspection machine 18 (S106). If there is a change in the provisionally registered reference image, the reference image information includes the changed address information, and if there is no change in the provisionally registered reference image, the reference image information includes information indicating that there is no change. Note that if there is a change in the provisionally registered reference image, the changed reference image and address information may be transmitted instead of the reference image information.

[0033] After transmitting the reference image information, the misprint inspection machine 18 acquires the reference image information (S107) and judges whether or not there is a change in the provisionally registered reference image based on the information (S108). If it is judged that there is a change in the reference image (S108, YES), the misprint inspection machine 18 updates the provisionally registered reference image to be changed to the changed reference image (S109) and executes the misprint inspection described above (S110). On the other hand, if it is judged that there is no change in the reference image (S108, NO), the provisionally registered reference image is used as is and the misprint inspection of step S110 is executed.

[0034] (Selection process) Next, the above-mentioned selection process will be described in detail. FIG. 8 is a flowchart showing the selection process according to this embodiment. As shown in FIG. 8, the feature amount calculation unit 202 of the reference image selection device 20 selects eight quire images of unselected pieces (stocker 12) (S201), and calculates the feature amounts using a predetermined feature amount calculation method (S202). The feature amount calculation method here is the above-mentioned phase-only correlation method, and the calculated feature amounts are the above-mentioned X, Y, and Z values. After the calculation, the generation unit 203 generates a data set as shown in FIG. 6 based on the calculated feature amounts (S203).

[0035] After the generation, the judgment processing unit 205 judges whether or not the data sets of all the frames have been generated (S204). If it is judged that the data sets of all the frames have not been generated (S204, NO), the process returns to step S201 to select the signature images of the unselected frames. On the other hand, if it is judged that the data sets of all the frames have been generated (S204, YES), the comparison and change process is executed (S205), and the selection process ends.

[0036] (Compare and change processing) The selection process will be described in detail. Fig. 9 is a flowchart showing the comparison and change process according to this embodiment. As shown in Fig. 9, the comparison and selection unit 204 selects a pair of unselected data sets (S301), and compares the similarity of the quire images based on the feature amounts of the selected pair of data sets (S302). Specifically, the comparison and selection unit 204 inputs the pair of data sets to the learning model 261, specifically, inputs the feature amounts linked to each trigger number in one data set and the feature amounts linked to each trigger number in the other data set, performs this for all trigger numbers, and outputs the similarity for each combination of trigger numbers.

[0037] After comparing the similarities, the determination processing unit 205 determines whether the similarity output by inputting a feature amount linked to a trigger number corresponding to a provisionally registered reference image in one data set and a feature amount linked to a trigger number having the same number as the trigger number in the other data set exceeds a predetermined similarity threshold, in other words, whether the similarity correlation between images based on the two feature amounts is strong or the same (S303). The reason for determining the similarity of features between the same trigger numbers is that when similar images or the like are printed in the same imaging range in two signatures W, it is highly likely that the same signatures W are stored in two stockers 12, and therefore it is possible to easily determine whether misprinting has occurred by this determination.

[0038] When it is determined that the similarity exceeds a predetermined similarity threshold (S303, YES), the comparison and selection unit 204 changes the provisionally registered reference image in the other data set (S304). Specifically, the comparison and selection unit 204 changes the reference image in the other data set to a reference image corresponding to a trigger number associated with a feature whose similarity is equal to or less than a predetermined similarity threshold, with respect to the feature associated with the trigger number corresponding to the provisionally registered reference image in one data set. Note that, when there are a plurality of combinations of feature whose similarity is equal to or less than a predetermined similarity threshold in the other data set, a trigger number may be selected randomly from among them, or a trigger number with the lowest similarity may be selected.

[0039] Fig. 10 is a diagram for explaining a data set in the case where the reference image is changed. In Fig. 10, the data set of stocker number 1 shown on the left side of the figure shows that the provisionally registered reference image shown by hatching is trigger number 2, and in the data set of stocker number 30 shown on the right side of the figure, the provisionally registered reference image shown by hatching is also trigger number 2. When the learning model 261 outputs that the similarity of the reference image shown by trigger number 2 is high, the process proceeds to step S304, and the reference image is changed. In the example shown in Fig. 10, as a result of the change, the signature image of trigger number 4 is changed to the reference image in the data set of stocker number 30.

[0040] Returning to Fig. 9, after changing the reference image, the determination processing unit 205 determines whether there is an unselected pair of datasets, that is, whether there is a pair of datasets for which a similarity comparison has not been performed (S305). If it is determined that there is an unselected pair of datasets (S305, YES), the process proceeds to step S301 again to select an unselected pair of datasets. On the other hand, if it is determined that there is no unselected pair of datasets (S305, NO), it is determined that all datasets have been selected once, and the determination processing unit 205 determines whether the selection process end condition is satisfied (S306).

[0041] The selection process end condition here is that there is no data set in which the reference image has been changed multiple times, or that there is a data set in which the reference image has been changed multiple times, and the re-similarity comparison of the set of data sets including the data set has been completed. In this embodiment, not only the comparison of the similarity of the reference images between adjacent stockers 12, but also the comparison of the similarity of the reference images in all stockers 12, in other words, a round-robin comparison (all frame comparison) is performed. Therefore, when the reference image is changed in a data set in one stocker, it affects the similarity of the reference images in the other stockers 12. Therefore, as a selection process end condition, in a data set in which the reference image has been changed, it is necessary to compare that data set with all other data sets. Note that after the re-similarity comparison, the reference image may be changed in some of the other data sets. In that case, it is determined that there is a data set in which the reference image has been changed multiple times, so the selection process end condition is not satisfied.

[0042] If it is determined that the selection process end condition is not satisfied (S306, NO), that is, if it is determined that there is a data set in which the reference image has been changed multiple times, an adjustment process is executed (S307), and this comparison and change process is terminated. Also, if it is determined in step S303 that the similarity does not exceed a predetermined similarity threshold (S303, NO), it is determined that there is no need to change the reference image, and the process proceeds to the next process without performing the process of step S304.

[0043] (Adjustment process) The adjustment process will be described in detail. Fig. 11 is a flowchart showing the adjustment process according to this embodiment. As shown in Fig. 11, the comparison and selection unit 204 selects an unselected data set (hereinafter referred to as a target data set) in which the reference image has been changed multiple times (S401). After the selection, the determination processing unit 205 determines whether or not there is a reference image in all other data sets whose similarity to the reference image of the selected target data set is higher than a predetermined similarity threshold (S402).

[0044] If it is determined that there is a reference image whose similarity is higher than a predetermined similarity threshold (S402, YES), it is determined that the change of the reference image in the target data set has affected the reference images of the other data sets, that is, the similarity between the reference image in the target data set and the reference images of the other data sets is high, and thus it is possible that misprints cannot be found. Therefore, the comparison and selection unit 204 then changes the reference image whose similarity is higher than the predetermined similarity threshold to a reference image whose similarity to the reference images of all data sets including the target data set is low, preferably equal to or lower than the predetermined similarity threshold, and more preferably the lowest (S403).

[0045] Fig. 12 is a diagram for explaining a data set in which the selection of the reference image has been adjusted by the adjustment process. While the reference image of the data set in stocker number 30 shown in Fig. 10 was trigger number 4, in the example shown in Fig. 12, it can be seen that the reference image of the data set in stocker number 30 has been changed to trigger number 6. The reference image corresponding to trigger number 6 has a low similarity to the other reference images.

[0046] 11, after changing the reference image, the determination processing unit 205 determines whether or not there is an unselected dataset, specifically, whether or not there is a dataset that has not been selected in step S401 (S404). If it is determined that there is an unselected dataset (S404, YES), the process returns to S401 to select an unselected dataset. On the other hand, if it is determined that there is no unselected dataset (S404, NO), the adjustment process ends.

[0047] Furthermore, if a reference image with a similarity higher than a predetermined similarity threshold has not been set in any of the other datasets (S402, NO), it is determined that the change in the reference image in the target dataset has not affected the reference images in all of the other datasets, that is, the similarities between the reference image in the target dataset and the reference images in all of the other datasets are still low, and the process proceeds to step S404.

[0048] According to the present embodiment described above, a reference image is selected using a learning model that can evaluate the similarity of images based on the feature amount obtained by the phase-only correlation method, so that it is possible to automatically select reference images that are low in similarity to each other. Also, because the similarity of the reference images is compared on a round-robin basis for all frames, it is possible to reliably prevent the occurrence of misprinting not only in adjacent stockers 12 but in all stockers 12. Also, because the reference image is selected without human intervention, there is no need for a person to visually check the reference image and make changes based on intuition, so that it is possible to significantly reduce the work time and, in turn, reduce costs.

[0049] In this embodiment, the pagination inspection machine 18 and the reference image selection device 20 are described as separate devices, but this is not limited to this. The pagination inspection machine 18 and the reference image selection device 20 may be combined into a single device, for example, the functions of the reference image selection device 20 may be incorporated into the pagination inspection machine 18.

[0050] Also, although it has been described that the signature W is the subject of processing such as registration of a reference image and inspection for misprinting, it goes without saying that any printed matter in sheet form can also be the subject of processing.

[0051] Also, in the learning model 261, it has been explained that the similarity of images is compared based on the feature amounts linked to each trigger number in the two data sets. However, the learning model 261 may be constructed so that all data sets are input at once, and a combination with the lowest similarity of the reference images is output as an output. In that case, in the comparison process of step S205, the comparison and selection unit 204 simply inputs the data sets to the learning model 261, and the combination of reference images with the lowest similarity is output, and these combinations can be selected as the reference images.

[0052] In the above-described embodiment, a program (reference image selection program) for realizing various functions of the reference image selection device 20 is pre-installed in the reference image selection device 20. However, the program may be provided by recording the program in a computer-readable recording medium in an installable or executable format file or the like. The storage medium includes all media that can be read and executed by a computer as the above-described reference image selection device 20, such as a magnetic tape, a magnetic disk (hard disk drive, etc.), an optical disk (CD-ROM, DVD disk, etc.), a magneto-optical disk (MO, etc.), a flash memory, etc., which are removable from the reference image selection device 20, and a medium that can be transmitted via a network. Note that, as a medium that can be transmitted via a network, for example, a medium stored on a computer such as an external server connected to a network such as the Internet can be cited as an example, and this is provided via the network. Note that at least a part of the functions of the reference image selection device 20 (for example, the learning unit 207, etc.) may be constructed by an external device of the collation machine 1, such as a server or a tablet terminal, and data (for example, the learning model 261) may be provided to the reference image selection device 20 from the external device by communication via the network.

[0053] The present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiments are merely illustrative in all respects and should not be interpreted as being limiting. The scope of the present invention is defined by the claims and is not limited to the text of the specification. Furthermore, all modifications, various improvements, substitutions and alterations within the scope of the claims are within the scope of the present invention. [Explanation of symbols]

[0054] 18 Misprinting inspection machine 20 Reference image selection device 201 Information Acquisition Department (Acquisition Department) 202 Feature calculation unit (calculation unit) 204 Comparison and Selection Section (Selection Section) 205 Judgment processing unit (judgment unit) W Signature (printed material)

Claims

1. an acquisition unit that acquires a group of images of the printed matter extracted from each of the multiple stockers of the collating machine by capturing images of the printed matter multiple times such that at least a portion of the captured image range is different from each other; a calculation unit that calculates a feature amount of each image included in the image group for each of the plurality of stockers by using a predetermined feature amount calculation method; a selection unit that selects a reference image for a stocker from an image group belonging to the stocker in order to select a reference image for a misprint inspection for each of the plurality of stockers, and performs the selection for the plurality of stockers; Equipped with The selection unit selects the reference image so that the reference image differs for each of the plurality of stockers, based on a learning model for determining similarity between the plurality of images from the feature amounts of the respective images and the calculated feature amounts of each image in the image group for each of the plurality of stockers. A reference image selection device comprising:

2. the feature amount calculation method is a phase-only correlation method, The feature amount is a correlation peak value and its coordinates in the image.

2. The reference image selection device according to claim 1.

3. The learning model outputs a similarity between at least two images.

3. The reference image selection device according to claim 2.

4. The selection unit selects an image having a similarity between the at least two images equal to or less than a predetermined threshold as a reference image.

4. The reference image selection device according to claim 3.

5. A determination unit that determines whether or not to change the reference image based on the similarity output from the learning model, the selection unit selects a pair of two stockers from the plurality of stockers, and outputs a similarity between a provisional reference image preset corresponding to one stocker and an image of the other stocker having an imaging range substantially the same as an imaging range of the provisional reference image by inputting respective feature amounts into the learning model; the determination unit determines whether or not to change a provisional reference image preset in the other stocker based on the similarity output from the learning model; When the determining unit determines that the image should be changed, the selecting unit selects an image from the image group in the one stocker, the image having a similarity to a provisional reference image preset in correspondence with the other stocker equal to or less than the threshold value, and sets the image as the reference image.

5. The reference image selection device according to claim 4.

6. The reference image selection device 2. The misprint inspection machine according to claim 1, further comprising:

7. A reference image selection device acquiring a group of images of the printed matter extracted from each of the plurality of stockers of the collating machine, the group of images being obtained by imaging the printed matter multiple times such that at least a portion of the imaging range is different from each other; calculating a feature amount of each image included in the image group for each of the plurality of stockers using a predetermined feature amount calculation method; selecting a reference image for a stocker from an image group belonging to the stocker in order to select a reference image for a misprint inspection for each of the plurality of stockers, and performing the selection for the plurality of stockers; In the selection of the reference image, the reference image is selected so as to be different for each of the plurality of stockers, based on a learning model for determining similarity between the plurality of images from the feature amounts of the respective images and the calculated feature amounts of each image in the image group for each of the plurality of stockers. A reference image selection method comprising:

8. Computer, an acquisition unit that acquires a group of images of the printed matter extracted from each of the multiple stockers of the collating machine by capturing images of the printed matter multiple times such that at least a portion of the captured image range is different from each other; a calculation unit that calculates a feature amount of each image included in the image group for each of the plurality of stockers by using a predetermined feature amount calculation method; a selection unit that selects a reference image for a stocker from a group of images belonging to the stocker in order to select a reference image for a misprint inspection for each of the plurality of stockers, and performs the selection for the plurality of stockers; Functioning as a The selection unit selects the reference image so that the reference image differs for each of the plurality of stockers, based on a learning model for determining similarity between the plurality of images from the feature amounts of the respective images and the calculated feature amounts of each image in the image group for each of the plurality of stockers. A reference image selection program comprising:

Images