Printed image acquisition system and printed image acquisition program

Abstract

This technology contributes to improving the accuracy of estimating the print state by capturing high-quality images of the printed area printed by a marking device such as an inkjet printer. [Solution] A print image acquisition system in which a calculation device assists in photographing print while viewing a real-time image output by an imaging device on a display device, wherein the calculation device generates a reference pattern of the print based on a first image photographed of a first sample print, overlays the reference pattern on the real-time image of a second sample print output by the imaging device and displays it on the display device, calculates an evaluation index for the imaging state of the second sample print in the real-time image based on the real-time image and the reference pattern, and acquires a second image photographed of the second sample print by the imaging device if the evaluation index satisfies a predetermined condition.

Description

[Technical Field]

[0001] The present invention relates to a print image acquisition system and a print image acquisition program for acquiring images for inspecting the state of prints printed by a marking device such as an inkjet printer. [Background technology]

[0002] For example, industrial inkjet printers are installed on production lines to print characters such as product lot numbers and manufacturing dates on products. These inkjet printers can sometimes produce printing defects depending on their operating conditions and installation environment. Therefore, a print inspection device is sometimes installed on the production line alongside the inkjet printer to inspect whether the characters, symbols, etc. printed by the inkjet printer are printed correctly.

[0003] One known example of this type of technology is a pattern recognition device that includes a memory in which models for matching with dot patterns on an image are registered, and that recognizes dot patterns on an input image by capturing an image of the dot pattern to be recognized and matching the image with each model in the memory, the pattern recognition device comprising: a deformation pattern creation means that creates a deformation pattern by shifting the position of a predetermined number of dots that make up a reference dot pattern to be registered in the memory; and a registration processing means that registers the deformation pattern in the memory together with the reference dot pattern (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-187187 Summary of the Invention [Problem to be solved by the invention]

[0005] Print inspection requires not only detecting print defects but also identifying the causes of the print defects. However, there are many different causes of print defects, making it difficult for manufacturing operators to identify the causes and take appropriate measures. While the device described in Patent Document 1 has the advantage of being able to inspect prints with deformed dot patterns, it is unable to identify the causes of print defects. In reality, operators may take a photo of the print defect with a camera, such as a smart device, and present it to an expert to determine the cause of the print defect and the appropriate remedial measures based on the print image. However, when an operator photographs the print defect, the positional relationship between the subject and the camera is not constant. As a result, the print defect may tilt in the pitch, yaw, or roll directions in the captured image, or the angle of view may change, making it difficult to accurately determine the cause of the print defect. To accurately determine the cause of the print defect, it is necessary to obtain a good captured image.

[0006] The object of the present invention is to provide a print image acquisition system and a print image acquisition program that can acquire good images of printed areas printed by a marking device such as an inkjet printer, thereby contributing to improving the accuracy of estimating the print state, etc. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the present invention provides a print image acquisition system that uses a calculation device to assist in photographing print while viewing a real-time image output by an imaging device on a display device, wherein the calculation device generates a reference pattern of the print based on a first image photographed of a first sample print, overlays the reference pattern on the real-time image of a second sample print output by the imaging device and displays it on the display device, calculates an evaluation index for the imaging state of the second sample print in the real-time image based on the real-time image and the reference pattern, and acquires a second image photographed of the second sample print by the imaging device if the evaluation index satisfies a predetermined condition. [Effects of the Invention]

[0008] According to the present invention, an object of the present invention is to obtain a good photographed image of a printed portion printed by a marking device such as an inkjet printer, thereby contributing to improving the accuracy of estimating the print state. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of an inkjet printer. [Figure 2] FIG. 2 is a schematic diagram showing an example of a dot pattern. [Figure 3] FIG. 1 is a schematic diagram showing the printing and inspection process on a production line. [Figure 4A] FIG. 10 is a diagram showing an example of printout determined to be correctly printed. [Figure 4B] FIG. 10 is a diagram showing an example of printout determined to be incorrectly printed. [Figure 5] 1 is a block diagram showing a processing sequence of a print image acquisition system according to a first embodiment of the present invention. [Figure 6] 5 is a flowchart showing an example of a processing procedure performed by the print image acquisition system according to the first embodiment of the present invention. [Figure 7A] FIG. 10 is a diagram showing a dot pattern as an example of a reference pattern. [Figure 7B] FIG. 10 is a diagram showing a grid pattern as an example of a reference pattern. [Figure 7C] FIG. 10 is a diagram showing a character pattern as an example of a reference pattern. [Figure 8] 10A and 10B are diagrams illustrating the amount and angle of deviation in the transport direction of the print target. [Figure 9] FIG. 10 is a diagram illustrating the dot arrangement period (vertical direction). [Figure 10] FIG. 10 is a diagram illustrating the dot arrangement period (horizontal direction). [Figure 11] FIG. 10 is a diagram illustrating dot intervals. [Figure 12] FIG. 2 is a diagram illustrating a printing area. [Figure 13]10 is a flowchart of a print information acquisition process. [Figure 14A] FIG. 10 is a diagram showing an example of a superimposed display of a reference pattern. [Figure 14B] FIG. 10 is a diagram showing an example of a superimposed display of a reference pattern. [Figure 15A] 5A and 5B are schematic diagrams illustrating the matching rate between a reference pattern and printed dots. [Figure 15B] 5A and 5B are schematic diagrams illustrating the matching rate between a reference pattern and printed dots. [Figure 16A] 10A and 10B are schematic diagrams illustrating a state in which the display of print dots is shifted from the reference pattern due to the X and Y directions of the imaging device for the second sample. [Figure 16B] 10A and 10B are schematic diagrams illustrating a state in which the display of print dots is shifted from the reference pattern due to the X and Y directions of the imaging device for the second sample. [Figure 17A] 10 is a schematic diagram illustrating a state in which the display of print dots is shifted from the reference pattern due to the rotation of the imaging device in the roll direction relative to the second sample. FIG. [Figure 17B] 10 is a schematic diagram illustrating a state in which the display of print dots is shifted from the reference pattern due to the rotation of the imaging device in the roll direction relative to the second sample. FIG. [Figure 18] 10 is a schematic diagram illustrating a state in which the display of print dots is shifted from the reference pattern due to a difference between the distance of the imaging device to the second sample and the appropriate distance. FIG. [Figure 19] 10 is a flowchart showing another example of the processing procedure performed by the print image acquisition system according to the first embodiment of the present invention. [Figure 20] 10 is a schematic diagram illustrating a state in which the display of print dots is shifted from the reference pattern due to the inclination of the imaging device relative to the print surface of the second sample. FIG. [Figure 21] 10 is a flowchart showing yet another example of the processing procedure performed by the print image acquisition system according to the first embodiment of the present invention. [Figure 22A] 10A and 10B are diagrams illustrating an example of a shooting guide generated based on the deviation between a reference pattern and print dots. [Figure 22B]10A and 10B are diagrams illustrating an example of a shooting guide generated based on the deviation between a reference pattern and print dots. [Figure 23] FIG. 7 is a diagram showing an example of the estimation result of the print state displayed in step S109 of FIG. 6 etc. [Figure 24] FIG. 10 is a block diagram showing a processing sequence of a print image acquisition system according to a second embodiment of the present invention. [Figure 25] FIG. 10 is a block diagram showing a processing sequence of a print image acquisition system according to a third embodiment of the present invention. [Figure 26] 10 is a flowchart showing an example of a processing procedure performed by a print image acquisition system according to a third embodiment of the present invention. [Figure 27] FIG. 10 is a schematic diagram for explaining a method for selecting the best image. [Figure 28] 10A and 10B are schematic diagrams illustrating variations of evaluation items for evaluating the photographed state of a photographed image of print. [Figure 29] FIG. 10 is a block diagram showing a processing sequence of a print image acquisition system according to a fourth embodiment of the present invention. [Figure 30] 10 is a flowchart showing an example of a processing procedure performed by a print image acquisition system according to a fourth embodiment of the present invention. [Figure 31] FIG. 11 is a block diagram showing a processing sequence of a print image acquisition system according to a fifth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] First Embodiment -Inkjet printer- Figure 1 is a schematic diagram showing an example of the configuration of an inkjet printer. The inkjet printer 10 shown in Figure 1 is for industrial use and is installed, for example, on a product production line to print information such as the product's lot number and manufacturing date on a product T, which is an object to be printed. The inkjet printer 10 includes an ink bottle 11, a supply pump 12, an excitation source 13, a nozzle 14, a charging electrode 15, a deflection electrode 16, a gutter 17, and a recovery pump 18.

[0012] Ink stored in ink bottle 11 is pumped up by supply pump 12, separated by vibrations generated by excitation source 13, and ejected as ink droplets a from nozzle 14. Some of the ink droplets a ejected from nozzle 14 are charged by charging electrode 15 to a voltage corresponding to the position of the desired dot pattern, and as they pass deflection electrode 16, they are deflected according to the amount of charge, and land at the position of the desired dot pattern on the surface of product T. Ink droplets a not used for printing enter gutter 17 and are recovered into ink bottle 11 by recovery pump 18.

[0013] -Dot pattern- Fig. 2 is a schematic diagram showing an example of a dot pattern. In this embodiment, the left-right direction of the printing surface shown in Fig. 2 is referred to as the horizontal direction, and the up-down direction is referred to as the vertical direction.

[0014] Inkjet printer 10 prints characters and symbols using dots. As shown in Figure 2, inkjet printer 10 prints characters and symbols by selectively depositing ink droplets a at each dot position in a grid-arranged dot pattern in which dots are periodically arranged vertically and horizontally, forming characters and symbols. The black circles in Figure 2 indicate printed areas where ink droplets a have landed, and ideally, the positions and sizes of these dots match those of the target dots. The dashed circles in Figure 2 indicate dots (non-printed areas) where ink droplets a have not landed. The figure shows an example of printing "A."

[0015] Figure 3 is a schematic diagram showing the printing and inspection processes on a production line. In the part of the production line shown in Figure 3, an inkjet printer 10 and a print inspection device 20 are arranged along a conveyor 30. In the direction in which the conveyor 30 transports the product T, the print inspection device 20 is located downstream of the inkjet printer 10.

[0016] In FIG. 3, product T, which is the printing target, is transported in the direction of the arrow by conveyor 30. During this transport process, after product T is printed by inkjet printer 10, an image of the printed portion of product T is captured by imaging device 21 of print inspection device 20, and the product T is inspected for correct printing. For example, print inspection device 20 determines that the printed dots are correct if the deviations from a predetermined standard in both size and position are within tolerances, and determines that the printed dots are incorrect if either deviation is outside the tolerances. Product T1 determined to be correctly printed is transported directly to a downstream process by conveyor 30. On the other hand, product T2 determined to be incorrectly printed is pushed off conveyor 30 by ejector 40 and transferred to conveyor 31, which branches off from conveyor 30. FIG. 4A shows an example of print determined to be correctly printed by print inspection device 20. FIG. 4B shows an example of print determined to be incorrect by print inspection device 20. In the example of FIG. 4B, some dots are misaligned from the positions where they should be printed, as indicated by the dashed lines.

[0017] -Outline of the print image acquisition system- FIG. 5 is a block diagram showing a processing sequence of the print image acquisition system according to the first embodiment of the present invention. The print image acquisition system 100 in FIG. 5 is a computer having a storage device 110 and an arithmetic device 120. The computer constituting the print image acquisition system 100 may be a single computer or may be composed of multiple computers. The computer constituting the print image acquisition system 100 may also be a server (including a cloud server), or a smart device such as a smartphone, tablet PC, or notebook PC that is integrated with an imaging device 50 and a display device 51. In this embodiment, the imaging device 50 is a camera mounted on the smart device, and the display device 51 is a monitor mounted on the smart device.

[0018] Print image acquisition system 100 has a function to assist an operator (photographer) in offline photographing of images of printed portions, which are necessary to estimate the printing condition of product T removed from the production line, using imaging device 50. Specifically, print image acquisition system 100 assists in photographing the print (live view photography) while viewing real-time images IR output by imaging device 50 on display device 51, by executing print image acquisition program 150 on computing device 120. Print image acquisition program 150 is stored in storage device 110 and can also be recorded on various media 160, such as CDs, DVDs, and USB memory. For example, when performing offline inspection of the printing condition of product T2 determined to be improperly printed by print inspection device 20 in FIG. 3, print image acquisition system 100 guides the operator in handheld photographing of the print using imaging device 50, as described below.

[0019] The processing executed by the arithmetic unit 120 in the print image acquisition system 100 mainly includes a processing 130 for generating the reference pattern D2 and a processing 140 for acquiring the print image of the inspection target and estimating its state.

[0020] - Overview of the process for generating reference patterns - The arithmetic device 120 generates a reference pattern D2 of the print of the first sample S1 based on a first image I1 obtained by capturing the print of the first sample S1 through a process 130 for generating a reference pattern D2. The first sample S1 is printed by the inkjet printer 10 and determined to be correctly printed by the print inspection device 20, and in the example of FIG. 3, this corresponds to the product T1. Specifically, the process 130 for generating the reference pattern D2 includes a print information extraction process 131 and a reference pattern generation process 132.

[0021] First, the operator removes a first sample S1 from the production line and photographs the printed portion of the first sample S1 with the imaging device 50 to obtain a first image I1 capturing the printed portion of the first sample S1. The first image I1 may be a single image captured from a single first sample S1, but it is preferable that the first image I1 be multiple images captured from multiple first samples S1. In the print information extraction process 131, print information D1 is extracted from the first image I1 by image processing. If multiple first images I1 are captured, print information D1 is acquired from each first image I1. In addition, in the reference pattern generation process 132, a reference pattern D2 is generated based on the print information D1 (described below).

[0022] Examples of print information D1 extracted by print information extraction process 131 will be described later with reference to Figure 7 etc., but include, for example, the spacing between print dots in the first image I1, the dot arrangement period of the print in the first image I1, and the dot size of the print in the first image I1. Other examples of print information D1 that can be extracted by print information extraction process 131 include the amount of dot misalignment in the print feed direction of the print in the first image I1 and the size of the print area in the first image I1. In print information extraction process 131, at least one of these pieces of print information D1 is extracted.

[0023] - Overview of the process for acquiring print images and estimating their status - The processing 140 performed by the calculation device 120 regarding the acquisition of printed images and state estimation includes superposition processing 141, evaluation index calculation processing 142, guide generation processing 143, second image acquisition processing 144, image feature extraction processing 145, and print state estimation processing 146.

[0024] In the superimposition process 141, the arithmetic unit 120 superimposes the reference pattern D2 generated in the reference pattern generation process 132 on a real-time image IR of the print of the second sample S2 output by the imaging device 50 held by the operator, and displays the superimposed image on the display device 51. The second sample S2 was printed by the inkjet printer 10 and determined to be printed incorrectly by the print inspection device 20, and in the example of Figure 3, this corresponds to product T2. The second sample S2 is, for example, a product printed by the same inkjet printer 10 as the first sample S1.

[0025] In the evaluation index calculation process 142, the calculation device 120 calculates an evaluation index D3 for the imaging condition of the print of the second sample S2 in the real-time image IR based on the real-time image IR and the reference pattern D2. The evaluation index D3 may be used to determine whether the imaging condition of the print of the second sample S2 is appropriate as a captured image for evaluating, for example, the position and size deviation of the print of the second sample relative to the dot pattern. One example of the evaluation index D3 is the match rate between the print captured in the real-time image IR and the reference pattern D2 generated by the reference pattern generation process 132 (FIG. 6). The dot size ratio between the print captured in the real-time image IR and the reference pattern D2 can also be used as the evaluation index D3 (FIG. 19). The amount of change in dot size of the print captured in the real-time image IR can also be used as the evaluation index D3 (FIG. 21).

[0026] In the guide generation process 143, the calculation device 120 generates a shooting guide D4 ( FIG. 22 ) that instructs the operator on how to operate the imaging device 50, based on the evaluation index D3 calculated in the evaluation index calculation process 142, and displays the generated shooting guide D4 on the display device 51, superimposed on the real-time image IR. For example, in this guide generation process 143, the calculation device 120 calculates the amount of deviation between the print shown in the real-time image IR and the reference pattern D2, and calculates the shooting guide D4 based on the amount of deviation. The operator performs operations such as changing the position and posture of the imaging device 50 according to the shooting guide D4 that is successively displayed on the display device 51.

[0027] 5 does not mention a shooting guide for capturing the first image I1, but the entire print area or the desired area that forms the basis of the reference pattern D2 must fit within the first image I1. Therefore, the print image acquisition system 100 desirably has a function for displaying a frame on the display device 51 so that the first image I1 is captured with the entire print area or the desired area within that frame. However, since the printing of the first sample S1 is guaranteed to be in good condition, if the print area fits within the frame, it can be corrected to generate the reference pattern D2.

[0028] In the second image acquisition process 144, the calculation device 120 automatically acquires a second image I2 obtained by capturing the print of the second sample S2 using the imaging device 50 when the evaluation index D3 satisfies a predetermined condition, and stores the second image I2 in the storage device 110. If the matching rate exemplified above is used as the evaluation index D3, the predetermined condition determined in the second image acquisition process 144 is, for example, that the matching rate exceeds a predetermined value (FIG. 6). If the dot size ratio is used as the evaluation index D3, the predetermined condition is, for example, that the dot size ratio exceeds a predetermined value (FIG. 19). If the dot size change amount is used as the evaluation index D3, the predetermined condition is, for example, that the dot size change amount is equal to or less than a predetermined value (FIG. 21). It is preferable to set multiple predetermined conditions, and acquire the second image I2 when all of the predetermined conditions are simultaneously satisfied. The calculation device 120 repeats the processes from the superposition process 141 to the guide generation process 143 until the predetermined condition is satisfied.

[0029] In image feature extraction processing 145, the arithmetic unit 120 extracts print image feature D5 from the second image I2. The feature D5 extracted in image feature extraction processing 145 is, for example, the printed characters, character height, dot size, dot spacing, etc., but is not limited to these, and may also include the second image I2 itself, luminance variation and average luminance of the second image, etc.

[0030] In print condition estimation processing 146, arithmetic device 120 estimates the print condition of second sample S2 based on second image I2, and causes display device 51 to display an estimation result D6 of the print condition of second sample S2. For example, arithmetic device 120 causes display device 51 to display at least one of the causes of print quality degradation (ink deterioration, etc.) and an improvement method (ink replacement, etc.) as print condition estimation result D6 based on the print condition of second sample S2. For example, feature quantities such as the height of printed characters exemplified above are stored in storage device 110 as table data associated with estimated print condition estimation results D6, and arithmetic device 120 derives estimation result D6 from the feature quantities by referring to this table data.

[0031] -Example of processing procedure (1)- Fig. 6 is a flowchart showing an example of a processing procedure by the print image acquisition system 100. The processing procedure in Fig. 6 can be executed in parallel with the procedures described later with reference to Figs. 19 and 21 after the reference pattern D2 is generated.

[0032] 6 starts, the print image acquisition system 100 executes a superimposition process 141 to superimpose the reference pattern D2 on the real-time image IR of the second sample S2 and display it on the display device 51 (step S101). After superimposing the reference pattern D2 on the real-time image IR, the print image acquisition system 100 executes an evaluation index calculation process 142 to calculate the match rate of the print dots of the characters printed on the second sample S2 to the reference pattern D2 on the real-time image IR of the second sample S2 (step S102). The print image acquisition system 100 also executes a guide generation process 143 to generate a shooting guide D4 (step S103) and superimpose the shooting guide D4 on the real-time image IR and display it on the display device 51 (step S104). The shooting guide D4 includes the reference pattern D2 and instructions for adjusting the angle of view (adjusting the position and attitude of the imaging device 50) (FIG. 22).

[0033] Next, the printed image acquisition system 100 executes the second image acquisition process 144 and determines whether the matching rate is equal to or greater than a predetermined value (X%) (step S105). If the matching rate is less than X%, the printed image acquisition system 100 calculates the amount of deviation between the print dots of the characters printed on the second sample S2 on the real-time image IR and the reference pattern D2, updates the angle of view adjustment instructions in the imaging guide D4, etc., and displays them on the display device 51 (step S110), and returns to step S101. The printed image acquisition system 100 repeats steps S101 to S105 and S110 until the operator adjusts the angle of view according to the imaging guide D4, etc., and the determination in step S105 is satisfied.

[0034] When the matching rate reaches X% or more, print image acquisition system 100 acquires real-time image IR at that time as second image I2 and saves it in storage device 110 (step S106). After acquiring second image I2, print image acquisition system 100 executes image feature extraction process 145 to extract feature amounts of the print image through image processing (step S107). After extracting the feature amounts, print image acquisition system 100 executes print condition estimation process 146 to estimate the print condition from the extracted feature amounts (step S108), and displays estimation result D6 on display device 51 (step S109).

[0035] -Standard pattern- Here, we will explain the reference pattern D2 displayed on the shooting guide D4. With the inkjet printer 10, dots are selectively formed at each position of the periodic dot pattern shown in Figure 2 above to print characters. In other words, if the printing quality is consistent, the printed dots will always match any dot in the reference pattern D2 in position and size within the tolerance. Here, we will explain a specific example of the reference pattern D2, in which dots or lines are periodically arranged.

[0036] 7A to 7C are diagrams illustrating examples of reference pattern D2. Fig. 7A shows a dot pattern, Fig. 7B shows a grid pattern, and Fig. 7C shows a character pattern. As described above, reference pattern D2 is generated based on print information D1 obtained by acquiring print information D1 from a first image I1 captured by print inspection device 20 of a printed portion determined to be correctly printed.

[0037] In the dot pattern shown in FIG. 7A, dots of size (e.g., diameter) D are arranged at a constant interval Y1 in a direction perpendicular to the belt of the conveyor 30 and at a constant interval X1 in the direction of belt movement (horizontal direction) of the conveyor 30. Note that because the product T is transported by the conveyor 30 while the inkjet printer 10 moves vertically to print dots, the vertical column of dots is tilted by an angle θ with respect to the direction perpendicular to the belt of the conveyor 30. As a result, a horizontal offset of d occurs between the dots at both ends of the vertical column of dots. Furthermore, the interval P between adjacent dots in the vertical direction is slightly wider than the interval Y1. The dot size D, intervals X1, Y1, P, tilt angle θ, and offset d are an example of print information D1 extracted from the first image I1.

[0038] The grid pattern shown in Figure 7B is obtained by replacing the vertical columns of dots and the horizontal columns of dots in the dot pattern shown in Figure 7A with lines of width W. The width W corresponds to the dot size D of the dots. The spacing X1, Y1, P, tilt angle θ, and deviation amount d are common to the dot pattern in Figure 7A. The width W, spacing X1, Y1, P, tilt angle θ, and deviation amount d are also examples of print information D1 extracted from the first image I1.

[0039] The dot patterns and grid patterns shown in Figures 7A and 7B can be used as the reference pattern D2 for general purposes regardless of the type of character to be inspected. However, if the character to be inspected is known in advance, the character pattern of a specific character may be used as the reference pattern D2, as shown in Figure 7C.

[0040] -Example of printed information- The print information D1 acquired from the first image I1 will now be described. As mentioned above, the dot size D is an example of the print information D1. The dots are circular, and the dot size D is information such as the diameter or radius of the circle. In addition to the amount of shift d and dot spacing X1, Y1, P described in FIGS. 7A and 7B, the print area and the like are also exemplified as print information D1. Some examples of this print information D1 will be described below.

[0041] FIG. 8 is a diagram illustrating the printing offset d and tilt angle θ. By deflecting ink droplets a in a direction perpendicular to the belt of the conveyor 30, vertical rows of dots aligned in the up-down direction in the figure are printed, and by transporting the product T on the conveyor 30, horizontal rows of dots aligned in the left-right direction (horizontal direction) in the figure are printed. In this case, depending on the ejection speed of the ink droplets a and the transport speed of the conveyor 30, the vertical rows of dots may be printed at an angle of tilt θ with respect to the direction perpendicular to the belt of the conveyor 30. As printing information D1 obtained from the first image I1, for example, the tilt angle θ and the offset d between dots at both ends of the vertical row are obtained.

[0042] FIG. 9 is a diagram illustrating the dot arrangement period (vertical direction). In the dot pattern, dots are arranged at regular intervals both vertically and horizontally. As shown in FIG. 9, the interval Y1 between horizontal rows of dots L1 and the interval P between dots in the extension direction of vertical rows of dots L2, which are inclined by an inclination angle θ, can also be obtained as print information D1.

[0043] Fig. 10 is a diagram for explaining the dot arrangement period (horizontal direction). As shown in Fig. 10, the interval X1 between the vertical dot columns L2 can also be acquired as print information D1.

[0044] Although FIGS. 8 to 10 show examples of dot intervals, since dots are arranged periodically, a method of acquiring the spatial frequency of the dots as print information D1 may also be employed.

[0045] In addition, the above has described the interval between adjacent dots. However, as indicated by the arrow in FIG. 11, a small interval with respect to the interval between adjacent dots, that is, H times (0 < H < 1) of the interval between adjacent dots, may be obtained as the printing information D1. Also, a large interval with respect to the interval between adjacent dots, that is, K times (K > 1) of the interval between adjacent dots, may be obtained as the printing information D1. Although an example of the printing information D1 is shown by the arrow in FIG. 11, the interval shown by the arrow is not limited thereto.

[0046] FIG. 12 is a diagram for explaining the printing area. Regarding the printing area where single or multiple characters are printed, for example, the vertical size Y2 and the horizontal size X2 of this printing area may be included in the printing information D1 to be obtained.

[0047] -Printing Information Acquisition Flow- FIG. 13 is a diagram for explaining the flow of obtaining the dot interval. The printing image acquisition system 100 executes the process illustrated in FIG. 13 in the printing information extraction process 131 of FIG. 5. When starting the printing information extraction process 131, the printing image acquisition system 100 captures the first sample S1 to obtain the first image I1 (step S201), and obtains the printing information D1 (dot size D, etc.) from the first image I1 (step S202). When obtaining the dot size D, for example, a method such as obtaining the centroid position of each printing dot and matching each dot with the dot centroid as the center of a circle to estimate the radius or diameter of the circle can be applied. However, the method for obtaining the dot size D is not limited to this. The dot size D may be obtained from a single dot, or when there are multiple dots, all dot sizes may be obtained and a statistical value (for example, average value, median value, mode value) may be taken.

[0048] Furthermore, even if the print inspection device 20 determines that the dots are printed correctly, each dot may deviate from its ideal position within a tolerance range. Therefore, although print information D1 may be extracted from a single first image I1, print information D1 that takes tolerance into account can be acquired by extracting print information D1 from each of multiple first images I1 and calculating statistical values ​​(e.g., average, median, mode). Therefore, in the example of FIG. 13, the print image acquisition system 100 determines whether print information D1 has been acquired for N (a predetermined number of) first images I1 (step S203), and repeats the processes of steps S201 and S202 until print information D1 for N images has been acquired. After acquiring print information D1 for N images, the print image acquisition system 100 performs statistical analysis on the print information D1 for the N images, calculates final print information D1 that will serve as the basis for the reference pattern D2 (step 204), and then ends the process of FIG. 13.

[0049] -Example of superimposed reference pattern- FIGS. 14A and 14B illustrate an example of a superimposed display of a reference pattern D2. These figures illustrate an example in which a second image I2 is captured using a smart device 52 equipped with an imaging device 50 and a display device 51. As shown in FIG. 14A, when the operator captures the second image I2, a reference pattern D2 consisting of multiple dots 91, indicated by dashed lines, is displayed on the display device 51 of the smart device 52. The black circles represent the printed dots 92 of the second sample S2 displayed in the real-time image IR. By superimposing the reference pattern D2 on the real-time image IR, the operator can check the deviation of the printed dots 92 from the reference pattern D2 and adjust the angle of view by moving the smart device 52 to minimize the deviation while capturing the image. The operator adjusts the position and orientation of the smart device 52 relative to the printed surface of the second sample S2 so that each printed dot 92 overlaps the dots 91 of the reference pattern D2. In FIG. 14A, the display area R1 of the reference pattern D2 is limited to a predetermined range in the center of the screen of the display device 51. The display area R1 is set in advance according to the size of the printing area of ​​the second sample S2. FIG. 14B illustrates a display area R1 set to the entire screen. In the example of FIG. 14B, in consideration of ease of matching the printing dots 92 with the dots 91 of the reference pattern D2, the spacing between the dots 91 is kept approximately the same as in the example of FIG. 14A, but the number of dots 91 is increased, and the reference pattern D2 is displayed to fill the screen. However, examples of the display area R1 are not limited to the examples of FIGS. 14A and 14B.

[0050] -Example of evaluation index (match rate) judgment- 15A and 15B are schematic diagrams illustrating the match rate between the reference pattern D2 and the printed dots. FIGS. 15A and 15B show an example of a printing defect in which some printed dots of a character are misaligned with the reference pattern D2. In FIG. 15A, the reference pattern D2 (dashed line) and the printed dots (black circles) are shown overlapping. In FIG. 15B, only the overlapping portions of the reference pattern D2 and the printed dots are shown. The total area of ​​the dots photographed for the printed character, i.e., the total area of ​​the 12 black circles shown in FIG. 15A, is defined as A. On the other hand, the total area of ​​the overlapping portions of the reference pattern D2 and the printed dots shown in FIG. 15B is defined as B. In this case, the match rate can be expressed as B / A. The better the printing condition, the higher the match rate. Under ideal printing conditions, A = B. When acquiring the second image I2, a threshold value X is set for the match rate, as previously described in FIG. 6. If the match rate is less than X%, the amount of misalignment is calculated and the image is retried. The threshold value may be adjustable by the operator. In this example, the state in which the display of the printed dots is shifted relative to the reference pattern D2 due to an XY-direction shift (a shift in the horizontal or vertical direction) or a roll direction rotation (a rotation around the optical axis of the imaging device 50) of the smart device 52 relative to the printing of the second sample S2 is explained using Figures 16A, 16B, 17A, and 17B.

[0051] FIGS. 16A and 16B are schematic diagrams illustrating a state in which the display of printed dots is shifted relative to the reference pattern D2 due to an XY shift of the smart device 52 relative to the printing of the second sample S2. Similar to FIGS. 15A and 15B, FIG. 16A displays the reference pattern D2 (dashed line) and the printed dots (black circles) overlapping, while FIG. 16B displays only the overlapping portions of the reference pattern D2 and the printed dots. The examples in FIGS. 16A and 16B show a state in which the printed dots are shifted horizontally relative to the reference pattern D2. In this example, as shown in FIG. 16B, the area of ​​each overlapping portion of the printed dots with respect to the reference pattern D2 also decreases, resulting in a lower match rate. In this case, the print image acquisition system 100 calculates the amount of shift of the printed dots relative to the reference pattern D2 and displays an imaging guide D4 on the display device 51 to prompt the operator to correct the shift. The amount of shift can be calculated, for example, by point cloud matching between the reference pattern D2 and the printed dots. However, the method for calculating the shift deviation amount is not limited to point cloud matching. In addition, since the calculated shift deviation amount includes information on the direction of deviation, it is possible to create an imaging guide D4 based on the calculated deviation amount.

[0052] 17A and 17B are schematic diagrams illustrating a state in which the display of printed dots is misaligned with respect to the reference pattern D2 due to the roll direction rotation of the smart device 52 relative to the print of the second sample S2. Similar to FIGS. 15A and 15B, FIG. 17A displays the reference pattern D2 (dashed line) and the printed dots (black circles) overlapping, while FIG. 17B displays only the overlapping portions of the reference pattern D2 and the printed dots. The examples in FIGS. 17A and 17B show a state in which the printed dots are displayed with rotational misalignment with respect to the reference pattern D2 in the roll direction. In this example, as shown in FIG. 17B, the overlapping area of ​​each printed dot with respect to the reference pattern D2 decreases, resulting in a lower match rate. In this case, the print image acquisition system 100 calculates the amount of rotational misalignment of the printed dots with respect to the reference pattern D2 and displays an imaging guide D4 on the display device 51 to prompt the operator to correct the misalignment. The amount of rotational misalignment can also be calculated, for example, by point cloud matching between the reference pattern D2 and the printed dots. However, the method for calculating the amount of rotational deviation is not limited to point cloud matching. In addition, since the calculated amount of rotational deviation includes information on the rotation direction of the deviation, it is possible to create an imaging guide D4 based on the calculated amount of deviation.

[0053] 15A to 17, an example of using the match rate between the reference pattern D2 and the printed dots has been described as a method for evaluating the imaging condition based on the reference pattern D2 and the captured image. However, a method for evaluating the shortest distance between the dots of the reference pattern D2 and the printed dots on the real-time image IR can also be used. In this case, the distances between the corresponding reference pattern D2 and the printed dots are calculated, and their statistical values ​​(e.g., average values) are calculated. When the reference pattern D2 and the printed dots match, the distance between the dots of the reference pattern D2 and the printed dots on the image is zero. With an appropriate angle of view, the distance between the dots of the reference pattern D2 and the printed dots on the image is shortest. However, if the angle of view deviates from the appropriate state, a distance appears between the dots of the reference pattern D2 and the printed dots on the image.

[0054] -Example of processing procedure (2)- An example of a processing procedure for determining the shooting condition using the matching rate between the reference pattern D2 and the printed dots as an evaluation index has been explained in Figure 6, but there are other factors that can cause the deviation between the dots of the reference pattern D2 and the printed dots besides shift deviation and roll direction rotation of the imaging device 50, and by evaluating other evaluation indexes as well, a better second image I2 can be obtained.

[0055] FIG. 18 is a schematic diagram illustrating a state in which the display of printed dots is misaligned with respect to the dots of the reference pattern D2 due to a discrepancy between the distance of the smart device 52 from the printed surface of the second sample S2 and the appropriate distance. As in FIG. 15A, FIG. 18 displays the reference pattern D2 (dashed line) and the printed dots (black circles) superimposed on each other. In the example of FIG. 18, the distance of the smart device 52 from the printed surface of the second sample S2 is farther than the appropriate distance, and the printed dot size is smaller than the dot size of the reference pattern D2 in the real-time image IR. In this case, the print image acquisition system 100 displays an imaging guide D4 on the display device 51 to prompt the operator to move the imaging device 50 closer to the second sample S2.

[0056] Fig. 19 is a flowchart showing an example of a processing procedure by a print image acquisition system that is useful in a case like that of Fig. 18. The processing of Fig. 19 is a modified example of the processing of Fig. 6, but it is desirable to execute it in parallel with the processing of Fig. 6. The processing of Fig. 19 differs from the processing of Fig. 6 in the procedure for calculating the evaluation index and determining it; other steps that are similar to the processing of Fig. 6 are assigned the same step numbers in Fig. 19 as in Fig. 6, and descriptions thereof will be omitted where appropriate.

[0057] 19 begins, the print image acquisition system 100 superimposes the reference pattern D2 on the real-time image IR (step S101), similar to the process of FIG. 6, and calculates the dot size of the print dots in the real-time image IR of the second sample S2, rather than the match rate (step S102'). Then, while displaying the imaging guide D4 (steps S103 and S104), the print image acquisition system 100 executes the second image acquisition process 144 and determines whether the ratio of the dot size of the print dots to the dot size of the reference pattern D2 is equal to or greater than a predetermined value (Y%) (step S105'). If the dot size is less than Y%, the print image acquisition system 100 calculates the deviation between the dot size of the reference pattern D2 and the dot size of the print dots, and displays the imaging guide D4 on the display device 51 to encourage the imaging device 50 to move closer to the second sample S2 (step S110), and returns to step S101.

[0058] When the dot size reaches Y% or more, the print image acquisition system 100 acquires the real-time image IR at that time as the second image I2 and saves it in the storage device 110 (step S106). At this time, if the processing of FIG. 6 and the processing of FIG. 19 are executed in parallel, the procedure of step S105 of FIG. 6 and the procedure of step S105' of FIG. 19 can be executed together, and if the determinations of both steps S105 and S105' are satisfied simultaneously, the second image I2 can be acquired. After acquiring the second image I2, the processing of step S107 and subsequent steps is executed as in the processing of FIG. 6, and the processing of FIG. 19 ends.

[0059] 19 has been described using an example in which the dot size of the printed dots is smaller than the dot size of the reference pattern D2, but the same procedure can be used when the dot size of the printed dots is larger than the dot size of the reference pattern D2. Specifically, the print image acquisition system 100 determines whether the ratio of the dot size of the reference pattern D2 to the dot size of the printed dots is equal to or greater than a predetermined value using a procedure equivalent to step S105'. If the dot size of the printed dots is larger than the predetermined value, the print image acquisition system 100 displays an imaging guide D4 on the display device 51 to prompt the user to move the imaging device 50 away from the second sample S2, and when the dot size of the printed dots becomes equal to or smaller than the predetermined value, the system proceeds to the process of acquiring the second image I2.

[0060] -Example of processing procedure (3)- The misalignment between the reference pattern D2 and the printed dots can be caused by the tilt of the imaging device 50 in the pitch and yaw directions relative to the printing surface of the second sample S2. The pitch direction here refers to the direction in which the imaging device 50 oscillates around a line extending horizontally on the paper in FIG. 2. The yaw direction refers to the direction in which the imaging device 50 oscillates around a line extending vertically on the paper in FIG. 2.

[0061] FIG. 20 is a schematic diagram illustrating a state in which the display of printed dots is misaligned with respect to the reference pattern D2 due to tilting of the smart device 52 relative to the printed surface of the second sample S2. Similar to FIG. 15A, FIG. 20 displays the reference pattern D2 (dashed line) and printed dots (black circles) superimposed on each other. In the example of FIG. 20, the smart device 52 is tilted in the yaw direction relative to the printed surface of the second sample S2, and the size of the printed dots in the real-time image IR is smaller on the left side than the dots in the reference pattern D2 and increases toward the right. This indicates that the second sample S2 and the image capture device 50 are not directly facing each other, and the distance between the second sample S2 and the image capture device 50 is farther than the appropriate distance to the left of the optical axis and approximately the appropriate distance to the right. In this case, the print image acquisition system 100 displays an imaging guide D4 on the display device 51 to prompt the operator to directly face the image capture device 50 toward the second sample S2.

[0062] Figure 21 is a flowchart showing an example of a processing procedure by a print image acquisition system that is useful in a case like that of Figure 20. The processing of Figure 21, like the processing of Figure 19, is a modified example of the processing of Figure 6, but it is desirable to execute it in parallel with the processing of Figures 6 and 19. The processing of Figure 21 differs from the processing of Figures 6 and 19 in the evaluation index to be calculated and the procedure for determining it, and other steps that are similar to the processing of Figures 6 and 19 are assigned the same step numbers in Figure 21 as in Figures 6 and 19, and explanations thereof will be omitted as appropriate.

[0063] When the processing of Figure 21 begins, the print image acquisition system 100 superimposes the reference pattern D2 on the real-time image IR (step S101), as in the processing of Figures 6 and 19, and calculates the dot size distribution of the print dots in the real-time image IR of the second sample S2 (step S102"). If the imaging device 50 is not directly facing the second sample S2, the measured dot sizes will not be uniform, and the dot size will change depending on the tilt of the imaging device 50. The print image acquisition system 100 calculates the amount of change in these dot sizes (for example, the rate of change between the maximum and minimum dot sizes). Then, while displaying the imaging guide D4 (steps S103, S104), the print image acquisition system 100 executes the second image acquisition process 144 and determines whether the amount of change in dot size of the reference pattern D2 is less than a predetermined value (Z%) (step S105"). If the dot size change due to the tilt of the imaging device 50 exceeds Y%, the print image acquisition system 100 calculates the amount of dot size change of the reference pattern D2, displays a shooting guide D4 on the display device 51 to prompt the imaging device 50 to face the second sample S2 directly (step S110), and returns the procedure to step S101.

[0064] When the amount of dot size change reaches Z % or more, print image acquisition system 100 acquires real-time image IR at that time as second image I2 and saves it in storage device 110 (step S106). At this time, if the processing of FIGS. 6 and 19 and the processing of FIG. 21 are executed in parallel, the procedures of steps S105 and S105' of FIGS. 6 and 18 and the procedure of step S105" of FIG. 21 can be executed together, and if the determinations of steps S105, S105', and S105" are simultaneously satisfied, second image I2 can be acquired. Once second image I2 has been acquired, the processing from step S107 onwards is executed as in the processing of FIGS. 6 and 19, and the processing of FIG. 21 ends.

[0065] 6, 19, and 21 are flowcharts showing the imaging guide D4 for correcting misalignment of the printed dots relative to the reference pattern D2. These flowcharts individually illustrate the following: the attitude correction for the XY-direction shift or roll-direction rotation of the imaging device 50 based on the match rate determination; the working distance correction based on the dot size determination; and the attitude correction for the pitch-direction tilt or yaw-direction tilt based on the amount of change in dot size. However, in reality, for example, even if the determination in step S105 in FIG. 6 is satisfied, the determinations in steps S105' and S105" in FIG. 19 and FIG. 21 may not be satisfied. The second image I2 acquired in such a state cannot be said to be good. Therefore, as described above, it is desirable to simultaneously or sequentially perform the determinations described in FIGS. 6, 19, and 21 to appropriately guide the distance and attitude of the imaging device 50 relative to the second sample S2, thereby acquiring a good second image I2 in a state where all of the determinations in steps S105, S105', and S105" are satisfied.

[0066] -Example of a photography guide- 22A and 22B are diagrams showing an example of a shooting guide D4 generated based on the deviation between the reference pattern D2 and the printed dots. FIGS. 22A and 22B show an example of capturing an image of the print of a second sample S2 using the imaging device 50 and display device 51 of a smart device 52. A real-time image IR of the print dots 92 of the second sample S2 is displayed on the display device 51 of the smart device 52, and the reference pattern D2 is statically displayed at a predetermined position and size. When capturing a second image I2 of the print of the second sample S2, a shooting guide D4 is displayed on the display device 51, which provides text or graphics indicating how far and in what direction the smart device 52 should be moved to align the print dots 92 with the reference pattern D2 in the real-time image IR.

[0067] The example of Figure 22A shows a state in which the print dots 92 are misaligned in the roll direction with respect to the reference pattern D2, similar to the example of Figure 17. The amount of rotational misalignment (match rate or angle) in the roll direction of the print dots 92 with respect to the reference pattern D2 and the rotation direction are calculated by the print image acquisition system 100 while the reference pattern D2 is superimposed on the real-time image IR. Figure 22A shows an example in which a match rate 2201, text 2202 instructing how to move the smart device 52, and an arrow 2203 visually indicating the direction in which to move the smart device 52 are displayed as the imaging guide D4.

[0068] 22B shows an example in which, in addition to text 2202 and arrow 2203, a reference line 2204, which is a common tangent line that touches the lower edge of each dot 91 in the bottom row on the screen of reference pattern D2, and a print line 2205, which is a common tangent line that touches the lower edge of each print dot 92 in the bottom row on the screen, are displayed as shooting guide D4. Because the task of simultaneously aligning a large number of print dots 92 with dots in reference pattern D2 is highly difficult, displaying guide lines such as reference line 2204 and print line 2205 and aligning these guide lines can improve the ease of correcting rotational misalignment in the roll direction.

[0069] Note that the match rate 2201, text 2202, arrow 2203, reference line 2204, and print line 2205 do not all need to be displayed; for example, any one of them may be displayed as the shooting guide D4. In the example of FIGS. 22A and 22B, the arrow 2203 represents the angle adjustment direction of the smart device 52, but it may also represent the angle adjustment direction of the print. The shooting guide D4 shown in FIGS. 22A and 22B is merely an example, and other guidance methods, such as a shooting guide D4 using audio information, may also be employed. Furthermore, using information on brightness variation and average brightness extracted from the first image I1, if the brightness of the real-time image IR is lower than that of the first image I1, the shooting guide D4 may encourage the user to brighten the lighting environment or correct the exposure.

[0070] -Displaying the estimation results- FIG. 23 is a diagram showing an example of the print condition estimation result D6 displayed in step S109 of FIG. 6 etc. The print condition estimation result D6 includes, for example, the print quality level, factors causing the print defect, and countermeasures for improving the print quality. FIG. 23 shows an example in which the estimation result D6 displays "ink condition deterioration" as the factor causing the print defect and an instruction such as "replace the ink" as the countermeasure. The displayed content explains the print condition and is not limited to these examples. For example, it is also conceivable to display a video explaining the countermeasure.

[0071] -effect- According to this embodiment, a reference pattern D2 based on a first image I1 captured of the printing of a first sample S1 is superimposed on a real-time image IR of the printing of a second sample S2, and a second image I2 captured of the printing of the second sample S2 is acquired when an evaluation index of the imaging condition of the printing of the second sample S2 based on the real-time image IR and the reference pattern D2 satisfies a predetermined condition. This ensures a certain level of quality in the captured image of the printing of the second sample S2, allowing for acquisition of a good image of the printed portion printed by the inkjet printer 10 and contributing to improved accuracy in estimating the print condition. This embodiment is also useful for acquiring a good image of the printed portion of a marking device that prints with dots other than the inkjet printer 10.

[0072] In addition, by superimposing a shooting guide D4 that instructs the operation of the imaging device 50 based on the evaluation index on the real-time image IR, the operator's operation of the imaging device 50 can be guided, the operator's workload when acquiring the second image I2 can be reduced, and the second image I2 can be acquired efficiently.

[0073] In addition, by estimating the printing condition of the second sample S2 based on the second image I2 and displaying the estimation result D6 on the display device 51, the operator can be informed of the estimated cause of the printing defect without relying on an expert.

[0074] Furthermore, based on the print state of the second sample S2, a method for improving print quality can be displayed on the display device 51 as the estimated result D6 to prompt the operator to take action, which can contribute to early improvement of print quality.

[0075] Second Embodiment Fig. 24 is a block diagram showing the processing sequence of a print image acquisition system according to a second embodiment of the present invention, and corresponds to Fig. 5 of the first embodiment. Elements in Fig. 24 that are the same as or correspond to those in the first embodiment are given the same reference numerals as those in the previously mentioned drawings, and descriptions thereof will be omitted as appropriate.

[0076] As described above, the print image acquisition system 100 can be configured with a single computer or multiple computers. This embodiment is one example of a configuration of a print image acquisition system 100 configured with multiple computers. The print image acquisition system 100 in FIG. 24 includes a smart device 52 and servers 60 and 70. The smart device 52 is connected to the servers 60 and 70 via a network such as the Internet. At least one of the servers 60 and 70 may be installed in the country in which the smart device 52 is used, or may be installed overseas. Furthermore, the servers 60 and 70 may be different computers or the same computer.

[0077] 24, the smart device 52 is equipped with the imaging device 50 and the display device 51, and therefore also has a function of capturing the first image I1 and the real-time image IR, and a function of displaying the real-time image, etc. In this embodiment, the superimposition process 141, the evaluation index calculation process 142, the guide generation process 143, and the second image acquisition process 144 are executed by the smart device 52. The print information extraction process 131 and the reference pattern generation process 132 are executed by the server 60, and the image feature extraction process 145 and the print state estimation process 146 are executed by the server 70.

[0078] That is, the smart device 52 transmits the first image I1 captured by the imaging device 50 to the server 60, and receives the reference pattern D2 generated by the server 60 from the server 60. Thereafter, the smart device 52 transmits the second image I2 acquired through execution of the guide generation process 143 etc. based on the received reference pattern D2 to the server 70, and receives the print state estimation result D6 from the server 60 and displays it on the display device 51. In this way, the processing of the print image acquisition system 100 can be shared among multiple computers. The functional sharing of which processes are executed by which computers is not limited to the example in FIG. 24 and can be changed as appropriate.

[0079] In this embodiment, the same effects as in the first embodiment can be obtained.

[0080] <Third embodiment> Fig. 25 is a block diagram showing the processing sequence of a print image acquisition system according to a third embodiment of the present invention, and corresponds to Fig. 5 of the first embodiment. Elements in Fig. 25 that are the same as or correspond to those in the first embodiment are given the same reference numerals as those in the previously mentioned drawings, and descriptions thereof will be omitted as appropriate.

[0081] This embodiment differs from the first embodiment in that a best image selection process 147 is executed when the second image I2 is acquired. In the print image acquisition system 100 of this embodiment, the calculation device 120 stores multiple frame images of the real-time image IR of the print of the second sample S2 up until the second image I2 is acquired in the storage device 110 during the second image I2 capture mode (e.g., while the reference pattern D2 is superimposed on the real-time image IR). The stored frame images may be all frames during that period, or one frame per predetermined number of frames. After acquiring the second image I2 in the second image acquisition process 144, the print image acquisition system 100 then selects, in the best image selection process 147, the best image with the best evaluation index as the second image I2 from the multiple frame images stored up to that point and the second image I2 acquired in the second image acquisition process 144. In other words, the second image I2 is updated with the best image. After selecting the second image I2 as the best image, the print image acquisition system 100 proceeds to the image feature extraction process 145 as in the first embodiment, estimates the print state of the second sample based on the second image I2 (best image), and displays the estimated print state result D6 of the second sample S2 on the display device 51.

[0082] Fig. 26 is a flowchart showing an example of a processing procedure by a print image acquisition system according to a second embodiment of the present invention, and corresponds to Fig. 6 of the first embodiment. The processing in Fig. 26 differs from the processing in Fig. 6 in that the procedure related to the above-mentioned best image selection processing 147 is added; other steps similar to the processing in Fig. 6 are assigned the same step numbers in Fig. 26 as in Fig. 6, and descriptions thereof will be omitted where appropriate. Also, although description will be omitted in this embodiment, as in the first embodiment, the processing in Fig. 26 is desirably executed in parallel with the determination of the evaluation indexes in Figs. 19 and 21.

[0083] 26 starts, print image acquisition system 100 executes steps S101 to S104 in the same way as in the first embodiment, and stores multiple frame images of real-time image IR during that time in storage device 110 along with the matching rate calculated in step S102 (step S111). Thereafter, print image acquisition system 100 stores second image I2 when the matching rate reaches X% or higher (step S106), and selects the best image (the image with the best evaluation index) from among the frame images and second image I2 stored in step S111 up to that point as second image I2 (step S112), and executes the processes from step S107 onwards in the same way as in the first embodiment.

[0084] FIG. 27 is a schematic diagram illustrating a method for selecting a best image. FIG. 27 illustrates the provisional second image I2 acquired in step S106 and the n (n=1, 2, 3, . . .) frame images IR1, IR2, . . . IRn stored up to that point. Because the operator sequentially moves the imaging device 50 until the second image I2 is acquired, the print capture conditions (e.g., the value of the evaluation index, such as the match rate) differ between these images. The second image I2 acquired in step S106 is acquired with the evaluation index satisfying a predetermined condition. However, due to, for example, a delay in the image acquisition process after the evaluation index determination, a frame image better than the second image I2 may exist by the time the second image I2 is acquired in step S106. Therefore, in this embodiment, a best image is reselected from the frame images IR1, IR2, . . . IRn and the second image I2 acquired in step S106. In this case, if the shooting condition is evaluated using multiple evaluation indexes such as dot size (FIG. 19) and dot size distribution (FIG. 21) in addition to the match rate calculated in the example of FIG. 26, priorities are assigned to the evaluation items, and the image with the highest evaluation index for the evaluation item with the highest priority is selected as the second image I2. For example, if there are multiple images with the highest evaluation index for the evaluation item with the highest priority, the image with the highest evaluation index for the evaluation item with the next highest priority is selected from those multiple images.

[0085] As described above, the second image I2 is acquired when the evaluation index satisfies a predetermined condition, but there is a possibility that a better angle of view will be obtained in the process of moving the imaging device 50 in accordance with the imaging guide D4 before the second image I2 is acquired. Therefore, according to this embodiment, the frame images up until the second image I2 is acquired are saved, and the best image from the combination of these frame images is selected again as the second image I2, so that a better second image I2 can be acquired.

[0086] In this embodiment, the best image selection process 147 is executed when the evaluation index satisfies a predetermined condition, but it may also be configured so that, for example, once a predetermined amount of frame images (e.g., a predetermined number of frame images, or frame images for a predetermined time) have been saved, the best image from among them is selected as the second image I2. In other words, in the procedure of step S105, instead of determining the evaluation index, it may be configured to determine whether a predetermined amount of frame images have been accumulated.

[0087] -Other evaluation items- Other evaluation items that are useful for evaluating the photographed state of a photographed image of print will now be described.

[0088] FIG. 28 is a schematic diagram illustrating variations in evaluation criteria for evaluating the capture quality of a captured image of a print. This figure illustrates a real-time image IR displayed when the imaging device 50 is not directly facing the printed surface of the second sample S2. If the imaging device 50 were directly facing the printed surface of the second sample S2, a top line 281 connecting the printed dots in the topmost horizontal row of the reference pattern D2 in the vertical direction would be parallel to a bottom line 282 connecting the printed dots in the bottommost horizontal row. Therefore, the parallelism of the top line 281 and the bottom line 282 can be used as an evaluation criterion for evaluating the facing quality (degree of tilt in the yaw direction) between the imaging device 50 and the printed surface. Other lines, such as the vertically center horizontal line 283, can also be used. For example, the facing (degree of tilt in the pitch direction) between the imaging device 50 and the printing surface can be evaluated based on the parallelism between the left edge line 284 and the right edge line 285 of the printing area. Also, the matching rate of the placement frequency of the printing dots on the top line 281 and the bottom line 282 can be used as an evaluation index. Furthermore, it is also conceivable that the periodicity of the dots can be used to calculate the amount of distortion of the printing relative to the reference pattern D2 as an evaluation index, and the image with the least amount of distortion can be selected as the best image.

[0089] <Fourth embodiment> Fig. 29 is a block diagram showing the processing sequence of a print image acquisition system according to a fourth embodiment of the present invention, and corresponds to Fig. 5 of the first embodiment. Elements in Fig. 29 that are the same as or correspond to those in the first embodiment are given the same reference numerals as those in the previously mentioned drawings, and descriptions thereof will be omitted as appropriate.

[0090] This embodiment differs from the first embodiment in that it includes a frontal image generation process 148 that generates a frontal image from multiple frame images. In the print image acquisition system 100 of this embodiment, the arithmetic unit 120 stores multiple frame images of the real-time image IR of the print of the second sample S2 in the storage device 110 up until the second image I2 is acquired in the second image acquisition process 144. The storage of multiple frame images is similar to that of the third embodiment. In the frontal image generation process 148, the print image acquisition system 100 generates a frontal image of the print of the second sample S2 from these multiple frame images through image processing. The frontal image would be obtained if the imaging device 50 captured the print surface of the second sample S2 under conditions where it was facing the print surface of the second sample S2. In this process, the print image acquisition system 100 selects the best image with the best evaluation index from the provisional second image I2 and the frontal image acquired in the second image acquisition process 144 as the second image I2. The method of selecting the best image is similar to that of the third embodiment. Thereafter, the print image acquisition system 100 executes the print condition estimation process 146, estimates the print condition of the second sample S2 based on the second image I2 (best image), and displays the estimation result D6 of the print condition of the second sample S2 on the display device 51.

[0091] Fig. 30 is a flowchart showing an example of a processing procedure by a print image acquisition system according to a fourth embodiment of the present invention, and corresponds to Fig. 26 of the third embodiment. The processing in Fig. 30 differs from the processing in Fig. 26 in that a procedure related to the above-mentioned front-facing image generation processing 148 is added, and other steps similar to the processing in Fig. 26 are assigned the same step numbers in Fig. 30 as in Fig. 26, and explanations thereof will be omitted as appropriate. Furthermore, although explanations will be omitted in this embodiment, as in the first to third embodiments, the processing in Fig. 30 is desirably executed in parallel with the determination of the evaluation indexes in Figs. 19 and 21.

[0092] 30 starts, the print image acquisition system 100 executes the processes of steps S101 to S106 as in the third embodiment and saves the second image I2. Thereafter, the print image acquisition system 100 generates a frontal image from each frame image saved in step S111 (step S113), selects the best image (the image with the best evaluation index) from the second image I2 acquired in step S106 and the generated image as the second image I2 (step S112), and executes the processes from step S107 onwards as in the third embodiment.

[0093] The front-facing image can be generated by, for example, any viewpoint image generation technology using self-position estimation, but the method for generating the front-facing image is not limited. Also, similar to the third embodiment, in the procedure of step S105, instead of determining the evaluation index, it may be configured to determine whether a predetermined number of frame images have been accumulated.

[0094] According to this embodiment, by adding the generated frontal image to the candidates for the second image I2, a better captured image of the printed area can be obtained. Furthermore, if the procedure in step S105 is configured to determine whether a predetermined number of frame images have been accumulated instead of determining the evaluation index, the second image I2 can be acquired once a predetermined number of frame images have been accumulated, even if it is difficult to align the printed dots with the dots of the reference pattern D2. In this case, it is expected that the generated image will be selected as the second image I2, and the generated image is useful for determining the print condition because it can be expected to be of a certain standard.

[0095] Fifth Embodiment Fig. 31 is a block diagram showing the processing sequence of a print image acquisition system according to a fifth embodiment of the present invention, and corresponds to Fig. 5 of the first embodiment. Elements in Fig. 31 that are the same as or correspond to those in the first embodiment are given the same reference numerals as those in the previously mentioned drawings, and descriptions thereof will be omitted as appropriate.

[0096] This embodiment differs from the first embodiment in that the first image I1 and the second image I2 are acquired by the imaging device 21 of the print inspection device 20 on the production line that produced the first sample S1 and the second sample S2. The position of the imaging device 21 of the print inspection device 20 may be shifted due to interference with an operator or obstacles, vibrations of the production line, etc. This embodiment is an example in which the imaging guide D4 of the print image acquisition system 100 is used to adjust the position and attitude of the imaging device 21 of the print inspection device 20 on the production line. Therefore, it is sufficient if the imaging guide D4 is displayed on the display device 22 of the print inspection device 20, for example, and the functions of the image feature extraction process 145 and the print condition estimation process 146 related to estimation of the print condition can be omitted.

[0097] The print inspection device 20 performs print inspection based on captured images of a sample S (product T) on the production line. Therefore, the print inspection device 20 sequentially acquires first images I1 of first samples determined to be properly printed. The print image acquisition system 100 generates a reference pattern D2 based on these first images I1 and stores it in the storage device 110. The process 130 for generating the reference pattern D2 is the same as in the first embodiment.

[0098] Thereafter, if adjustment of the position of the imaging device 21 or the like becomes necessary, a sample S3 determined to be properly printed from among the inspected samples S is placed on the stopped conveyor 30 as a second sample, and while a real-time image IR of the printed portion of sample S3 output by the imaging device 21 is confirmed on the display device 22 together with the reference pattern D2, the position and orientation of the imaging device 21 are adjusted in accordance with the imaging guide D4 displayed on the display device 22. By adjusting the position and orientation of the imaging device 21 and acquiring a second image I2 (or by the evaluation index satisfying a predetermined condition), adjustment of the position of the imaging device 21 or the like is completed. The generation and display processes of the imaging guide D4 are also the same as those in the first embodiment.

[0099] As described above, according to this embodiment, by obtaining a good photographed image of the printed portion printed by a marking device such as an inkjet printer, it is possible to easily adjust the position of the imaging device 21 of the print inspection device 20, which in turn contributes to improving the accuracy of subsequent estimations of the print state, etc.

[0100] The print image acquisition system 100 of the third and fourth embodiments can also be used to adjust the imaging device 21 of the print inspection device 20.

[0101] <Additional remarks> The present invention is not limited to the above-described embodiments and may include various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. For example, it is possible to replace some of the configurations with other configurations. It is also possible to delete some of the configurations of the embodiments or add other configurations. [Explanation of symbols]

[0102] 20...print inspection device, 21...imaging device, 22...display device, 50...imaging device, 51...display device, 52...smart device, 100...print image acquisition system, 120...arithmetic unit, 150...print image acquisition program, 2201...match rate, d...dot shift amount, D...dot size, D1...print information, D2...reference pattern, D3...evaluation index, D4...photography guide, D6...estimation result, I1...first image, I2...second image, IR...real-time image, IRn...frame image, P...dot spacing, S1...first sample, S2...second sample, X1...dot spacing, X2...print area size, Y1...dot spacing, Y2...print area size

Claims

1. A print image acquisition system in which a computing device supports the photographing of prints while viewing real-time images output by an imaging device on a display device, The computing device generating a reference pattern of the print based on a first image obtained by photographing the print of a first sample; superimposing the reference pattern on the real-time image of the print of the second sample output by the imaging device and displaying it on the display device; calculating an evaluation index for an imaging state of the print of the second sample in the real-time image based on the real-time image and the reference pattern; When the evaluation index satisfies a predetermined condition, a second image of the print of the second sample is captured by the imaging device. Print image acquisition system.

2. 2. The print image acquisition system of claim 1, The computing device generating an imaging guide for instructing an operation of the imaging device based on the evaluation index; The imaging guide is superimposed on the real-time image and displayed on the display device. Print image acquisition system.

3. 3. The print image acquisition system of claim 2, The computing device Calculating the amount of deviation between the print shown in the real-time image and the reference pattern; The imaging guide is calculated based on the amount of deviation. Print image acquisition system.

4. 2. The print image acquisition system of claim 1, the evaluation index is a matching rate between the print imaged in the real-time image and the reference pattern, The predetermined condition is that the match rate exceeds a predetermined value. Print image acquisition system.

5. 2. The print image acquisition system of claim 1, the evaluation index is a dot size ratio between the print image shown in the real-time image and the reference pattern, The predetermined condition is that the dot size ratio exceeds a predetermined value. Print image acquisition system.

6. 2. The print image acquisition system of claim 1, the evaluation index is a change in dot size of the print imaged in the real-time image, The predetermined condition is that the amount of change in dot size is equal to or less than a predetermined value. Print image acquisition system.

7. 2. The print image acquisition system of claim 1, The computing device Extracting print information from the first image; The reference pattern is generated based on the print information. Print image acquisition system.

8. The print image acquisition system of claim 7, The print information is the dot spacing of the printing of the first image; a dot arrangement period for printing the first image; a dot size for printing the first image; The dot shift amount in the print feed direction of the printing of the first image, and Print area size in the first image A printing condition estimation processing system including at least one of the above.

9. 2. The print image acquisition system of claim 1, The computing device Estimating a printing state of the second sample based on the second image; The estimated result of the printing state of the second sample is displayed on the display device. Print image acquisition system.

10. 10. The print image acquisition system of claim 9, The calculation device displays at least one of the causes of degradation in print quality and a method for improving print quality on the display device based on the print state of the second sample.

11. 2. The print image acquisition system of claim 1, The computing device storing a plurality of frame images of the real-time image of the print of the second sample in a storage device until the second image is acquired; selecting a best image with the highest evaluation index from the second image and the plurality of frame images; Estimating the printing state of the second sample based on the best image; The estimated result of the printing state of the second sample is displayed on the display device. Print image acquisition system.

12. 2. The print image acquisition system of claim 1, The computing device storing a plurality of frame images of the real-time image of the print of the second sample in a storage device until the second image is acquired; generating a front-facing image of the print of the second sample obtained when the imaging device captures an image of the print surface of the second sample from the plurality of frame images; selecting a best image having the best evaluation index from the second image and the frontal image; Estimating the printing state of the second sample based on the best image; The estimated result of the printing state of the second sample is displayed on the display device. Print image acquisition system.

13. 2. The print image acquisition system of claim 1, The print image acquisition system, wherein the imaging device is a camera mounted on a smart device.

14. 2. The print image acquisition system of claim 1, The print image acquisition system, wherein the imaging device is a camera of a print inspection device on the production line that produced the first sample and the second sample.

15. A print image acquisition program that supports photographing a print while viewing a real-time image output by an imaging device on a display device, generating a reference pattern of the print based on a first image obtained by photographing the print of a first sample; superimposing the reference pattern on the real-time image of the print of the second sample output by the imaging device and displaying it on the display device; calculating an evaluation index for an imaging state of the print of the second sample in the real-time image based on the real-time image and the reference pattern; When the evaluation index satisfies a predetermined condition, a second image of the print of the second sample is captured by the imaging device. A print image acquisition program that causes a computing device to execute the process.

Images