inkjet printing system

Abstract

The present invention provides a novel inkjet printing system capable of improving the inspection performance of a print inspection device using machine learning. The system includes: a random test printing function unit (301) having a function of printing a random dot arrangement of various characters printed on a print area of a print object; and an evaluation function training function unit (302) having a function of optimizing an evaluation function for inspecting a printed image based on an image obtained by photographing the printed result.

Description

Technical Field

[0001] The present invention relates to an inkjet printing system, and more particularly to an inkjet printing system combined with a print inspection device having a machine learning function. Background Art

[0002] Conventional continuous-ejection, electrically controlled inkjet printers used in manufacturing plants and other applications have an ink tank installed in the printer body. An ink supply pump supplies ink from this tank to the printhead. The ink supplied to the printhead is continuously ejected from the ink nozzles as ink droplets.

[0003] Ink droplets used for printing are charged and deflected so as to fly to the printing position of the desired printing object. Ink droplets not used for printing are not charged or deflected, but are collected by an ink tank and returned to the ink container by an ink recovery pump.

[0004] Furthermore, inkjet printers are installed in production lines of factories that manufacture products such as packaging containers for storing food and PET bottles for filling beverages, and information such as expiration dates, manufacturing plants, and manufacturing numbers are printed on the surfaces of the products.

[0005] In addition, in production lines of manufacturing plants, it is sometimes necessary to inspect whether an inkjet printer has correctly performed printing. For this reason, some production lines have introduced an inkjet printing system that incorporates a print inspection device.

[0006] The print inspection device uses a camera to capture the printed area of a product and uses the image to determine whether the print is "good" or "bad." Images of normal and abnormal prints are pre-registered in the print inspection device, and these images are compared with the captured image to determine whether the print is "good" or "bad."

[0007] Furthermore, there is a recent demand for further improvement in the accuracy of printed character inspection. Machine learning is an effective method for improving this inspection accuracy. For example, Japanese Patent Application Laid-Open No. 2007-281723 (Patent Document 1) describes a method using machine learning in a print inspection device for an inkjet printer.

[0008] Patent Document 1 describes a method in which color patches are printed using multiple test ink volume sets, images of these color patches are input using a scanner, and machine learning is performed using a dataset that corresponds the test ink volume sets to a granularity index obtained by analyzing the image data as training signals. Thus, print inspection devices often include machine learning capabilities.

[0009] Prior art literature

[0010] Patent Literature

[0011] Patent Document 1: Japanese Patent Application Laid-Open No. 2007-281723 Summary of the Invention

[0012] Technical problem to be solved by the invention

[0013] As mentioned above, using machine learning to improve inspection accuracy is an effective method. However, with continuous jet type charged control type inkjet printers, a variety of characters and symbols such as Japanese are printed on the printing object (hereinafter referred to as characters as representatives). Therefore, it is difficult to learn all characters individually. In addition, when printing characters such as shelf life and manufacturing numbers that are different for each printing object, the characters of the inspection object change for each printing object. Therefore, the method of learning fixed characters cannot be used. Therefore, an inkjet printing system that can improve inspection accuracy and perform inspection operations efficiently is required.

[0014] An object of the present invention is to provide a novel inkjet printing system that can improve the inspection accuracy of a print inspection device based on machine learning and perform inspection work efficiently.

[0015] Means for solving technical problems

[0016] To solve the above technical problems, for example, the structure described in the claims is adopted. The present invention includes various means for solving the above technical problems. For example, the present invention is characterized in that: a print inspection device includes a machine learning function unit based on a neural network, the machine learning function unit including: a random test printing function unit having a function of printing a dot arrangement pattern including a plurality of random dots based on various characters to be printed in the print area of the print object; and an evaluation function training function unit having a function of optimizing an evaluation function of a neural network used to inspect the print image based on a print image obtained by photographing the printed dot arrangement pattern.

[0017] Effects of the Invention

[0018] In the present invention, a dot arrangement pattern (random dot arrangement pattern) composed of a plurality of random dots based on various characters to be printed is test printed on the printing object, and the evaluation function of the neural network is optimized based on it, thereby improving the inspection accuracy and performing efficient inspection operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram illustrating a printing method of an inkjet recording device.

[0020] Figure 2 This is a structural diagram illustrating the printing principle of an inkjet recording device.

[0021] Figure 3This is a schematic configuration diagram showing an overview of the overall configuration of an inkjet printing system.

[0022] Figure 4 This is a flowchart showing the entire process of machine learning according to the first embodiment of the present invention.

[0023] Figure 5 Yes Figure 4 Flowchart of the test process for data set generation shown.

[0024] Figure 6 Yes Figure 5 Flowchart of the test printing process shown.

[0025] Figure 7A It is an explanatory diagram showing a standard random dot arrangement pattern serving as a reference.

[0026] Figure 7B It means for Figure 7A This is an explanatory diagram of a first example of a standard random dot arrangement pattern in which a first erroneous random dot arrangement pattern is intentionally added.

[0027] Figure 7C It means for Figure 7A An explanatory diagram of a second example of a standard random dot arrangement pattern in which a second erroneous random dot arrangement pattern is intentionally added.

[0028] Figure 7D It means for Figure 7A This is an explanatory diagram of a third example of a standard random dot arrangement pattern in which a third erroneous random dot arrangement pattern is intentionally added.

[0029] Figure 8 This is a structural diagram showing the structure of a neural network used in machine learning according to an embodiment of the present invention.

[0030] Figure 9 This is a flowchart showing a test printing process in the second embodiment of the present invention.

[0031] Figure 10 This is an explanatory diagram showing a dot arrangement pattern in which a dot omission pattern and a pattern without dot omission are superimposed for learning dot omission in the third embodiment of the present invention. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments, and various modifications and application examples within the technical concept of the present invention are also included within the scope of the present invention.

[0033] First, the structure and operation of a conventional continuous jet type charged control type inkjet printer will be briefly described. Furthermore, the principles of the present invention can be applied not only to the continuous jet type charged control type inkjet printer described below, but also to conventional office machine printers as needed.

[0034] exist Figure 1 The external structure of the inkjet printer is shown in FIG. Figure 1 In the embodiment, an inkjet printer body 1 includes a display 2 for displaying information. Ink is supplied to a print head 4 via a cable 3, and the determined print content is transmitted to the print head 4 via the cable 3. Based on this, ink droplets are continuously ejected, thereby printing on a print object 6 transported by a conveyor line 5 such as a conveyor belt.

[0035] exist Figure 2 The structure of an inkjet printer is schematically shown in FIG. Figure 2 Ink 8 stored in an ink container 7 is pressurized by an ink supply pump 9 and supplied to an ink nozzle 10. Periodically applying a voltage to a piezoelectric element 11 provided in the ink nozzle 10 excites the ink within the ink nozzle 10. The excited ink is ejected from the ink nozzle 10 as an ink column 12 and becomes ink droplets.

[0036] As the ink used for printing forms droplets, it is charged by charging electrode 13. Charged ink droplets 14 are deflected by the electric field generated in the deflection space between positive deflection electrode 15 and negative deflection electrode 16, and then land on printing object 6. Ink droplets 17 not used for printing are not charged and are not deflected, so they are collected by gutter 18.

[0037] In addition, Figure 1 The main body 1 of the inkjet printer houses Figure 2 The ink container 7 and ink supply pump 9 shown in FIG. Figure 1 The print head 4 contains Figure 2 The ink nozzle 10, the charging electrode 13, the positive deflection electrode 15, the negative deflection electrode 16, and the groove 18 are shown.

[0038] Example 1

[0039] Next, a first embodiment of the present invention will be described. Figure 3The figure shows the structure of the inkjet printing system with machine learning functions that constitutes this embodiment. A machine learning computing device 20 is connected to the print inspection device 19. Here, the print inspection device 19 and the machine learning computing device 20 are collectively referred to as the print inspection device 19. The structure and operation of the machine learning computing device 20 will be described in detail in the following embodiments. Furthermore, the machine learning computing device 20 can also be integrated with the print inspection device 19, as shown by the dotted line.

[0040] In the embodiment described below, the machine learning computing device 20 is integrally formed with the print inspection device 19. Furthermore, the print inspection device 19 and the inkjet printer 1 are capable of bidirectionally transmitting control information, and a dot pattern composed of a plurality of random dots for machine learning training is transmitted from the machine learning computing device 20 to the inkjet printer 1.

[0041] The inkjet printer 1 performs test printing according to the received dot arrangement pattern composed of a plurality of random dots, and uses the camera provided in the print inspection device 19 to capture the printed dot arrangement pattern, and uses the printed image for machine learning.

[0042] A first embodiment of the present invention is an inkjet printing system characterized by including: a random test printing function unit configured to print a dot pattern comprising a plurality of random dots based on a print area to be printed on an object; and an evaluation function training function unit configured to optimize an evaluation function of a neural network used to inspect a printed image obtained by photographing the printed dot pattern. This embodiment is particularly characterized by using a test image comprising a plurality of random dots for use in machine learning.

[0043] Figure 4 This flowchart shows the entire process of the inspection work performed by the print inspection device using machine learning. Machine learning itself is well known, and this embodiment uses a convolutional neural network (CNN) to perform machine learning.

[0044] The entire processing process of the inspection operation performed by the printing inspection device using machine learning is as follows: first, a training data set for machine learning is generated in the test process 301 for data set generation; then, the training data set generated in the test process 301 for data set generation is used in the machine learning process 302 to learn the neural network for inspection using the error backpropagation method; and then, in the application process 303, the neural network for inspection generated in the machine learning process 302 is used to perform the actual printing inspection operation.

[0045] Next, the random test printing function and the evaluation function training function, which are features of this embodiment, are described. The random test printing function is a function in the dataset generation test step 301 , and the evaluation function training function is a function in the machine learning step 302 .

[0046] exist Figure 5 4 shows a flowchart of the test process 301 for data set generation. First, in the test printing process 401, the inkjet printer 1 (see FIG. 4 ) is used in the printing area of the print object to be printed. Figure 3 ) to perform test printing. Then, in the test printing imaging step 402, the print inspection device 19 (refer to Figure 3 ) The test print printed on the print object is photographed. Next, in the data set generation step 403, a training data set for learning is generated.

[0047] exist Figure 6 4 shows a flowchart of the test printing process 401. First, in the random dot pattern acquisition process 501, a predetermined dot arrangement pattern consisting of a plurality of dots is acquired. The dot arrangement pattern will be described later.

[0048] The dot pattern is represented by a dot matrix that forms a single character. For example, in a dot matrix with 5 columns and 7 rows, a single ink droplet is printed in any of the 35 squares, forming the desired character. Of course, by printing ink droplets in randomly selected squares, a dot pattern composed of multiple random dots can be formed.

[0049] Here, the amount of practice data (training data) is determined by the type of dot arrangement pattern. Therefore, the more dot arrangement patterns there are, the higher the accuracy of subsequent machine learning will be, but the time consumed in the testing process will also increase accordingly. In this embodiment, 100 types of dot arrangement patterns are prepared for learning. The acquired dot arrangement pattern is a dot arrangement pattern composed of multiple points in which the presence or absence of points (the configuration position of the points) is randomly determined. Figures 7A to 7D An example of this is shown in .

[0050] like Figure 7A As shown, in the standard random dot arrangement pattern (standard dot arrangement pattern) 701, a plurality of dots are randomly arranged. Figure 7A The dots are shown in standard positions. Furthermore, the dot ratio (the ratio of dots to the total number of squares) in standard dot pattern 701 can be freely set; in this embodiment, it is set to 50%. That is, a random number of dots are present in half of the total number of squares. Furthermore, the number of dots in the vertical and horizontal directions (columns and rows) is determined by the size of the characters to be printed; in this embodiment, the number of dots in both the vertical and horizontal directions is 11.

[0051] Furthermore, the dot patterns also include intentionally added error dot patterns. This serves as training data for "defective" judgments during the inspection process. Generally speaking, inkjet printers have several characteristic "errors," and adding simulated "errors" is effective. In this example, three error dot patterns are used simultaneously. Figures 7B to 7D An example of an error dot arrangement pattern is shown.

[0052] Figure 7B This is a dot deviation error in dot pattern 702. This error occurs when only a specific dot 702e, enclosed by a dotted box, is printed slightly offset (misplaced). A one-dot deviation is equal to or less than half the distance between dots. When applying a one-dot deviation, a single dot is randomly selected and deviated by a randomly specified distance and angle relative to the standard dot pattern 701.

[0053] Figure 7C This is a row deviation error dot arrangement pattern 703. This is an error in which a specific row 703e, enclosed by a dotted box, deviates vertically. When assigning a row deviation, a row is randomly selected and deviated vertically by a randomly given distance relative to the standard dot arrangement pattern 701.

[0054] Figure 7D This is a dot missing error dot arrangement pattern 704. This is an error in which a specific dot 704e, enclosed by a dotted box, is missing. To make a dot missing, a dot is randomly selected and deleted from the standard dot arrangement pattern 701.

[0055] Three such erroneous dot patterns are pre-assigned to a random dot pattern. That is, a standard dot pattern or an erroneous dot pattern is assigned to a single dot pattern, thereby preparing multiple dot patterns. Thus, in this embodiment, 100 different dot patterns can be obtained.

[0056] return Figure 6 In the random dot pattern printing step 502, the dot pattern is test-printed on the print object. In this case, one dot pattern is assigned to each character to be printed. For example, if the character to be printed is "ABC", three dot patterns are printed in sequence.

[0057] Then, in Figure 5 In the test print imaging step 402, the print inspection device 19 is used to Figure 6 The dot arrangement pattern obtained by test printing in the random dot pattern printing step 502 is photographed. Then, when the photographing is completed, the data set generation step 403 is executed.

[0058] In the dataset generation step 403, a dataset is generated using the captured images. To use as a dataset, each image needs to be labeled (correct answer information). In this embodiment, binary labels are used. These labels correspond to "good" or "bad" for the printed dot pattern.

[0059] The label is assigned using a pre-prepared random dot pattern. The inkjet printer pre-shares the same dot pattern used for test printing. Therefore, the print inspection device can associate the captured image with the original random dot pattern.

[0060] The label assigned to an image is determined by whether or not an "error" has been added to the original dot pattern. If no "error" has been added, the image is labeled "good" (correct answer information). If an "error" has been added, the image is labeled "bad" (correct answer information). This allows for the creation of a dataset.

[0061] Then, return Figure 4 , the machine learning process 302 is described in detail. Here, the Figure 5 The dataset generated in the dataset generation step 403 is used to train the neural network for printing inspection. In this embodiment, as described above, a convolutional neural network (CNN) is used as the neural network for machine learning. Figure 8 A convolutional neural network is shown in . This convolutional neural network is a well-known structure.

[0062] First, an input image of a random dot pattern obtained by imaging is compressed into a 32×32 dot image in the input layer 801. This is a monochrome image in which each dot is a binary black and white image.

[0063] Next, we use CNN layer 802. The input is "32×32×1", the filter size is "3×3×1", and three filters are used. The filter weight size is "1ong" and is initialized using the "initial value of He". The same applies to subsequent layers. In addition, the stride size is "1", the padding size is "1", and the output is "32×32×3". The stride and padding are the same in subsequent CNN layers. The Relu function is used as the activation function, and the same applies to subsequent CNN layers.

[0064] The third layer is also CNN layer 803. The input is "32 × 32 × 3". The filter size is "3 × 3 × 3", and three filters are used. The output of these filters is "32 × 32 × 3".

[0065] The fourth layer is the Max Pooling layer 804. The pooling size is "2×2" and the output is "16×16×3." Similar layers are set after the Max Pooling layer 804.

[0066] The fifth layer is CNN layer 805. The input is 16×16×3. The filter size is 3×3×3, and the output is 16×16×3. Similarly, the sixth layer, CNN layer 806, the seventh layer, CNN layer 807, and the eighth layer, CNN layer 808, are all identical to CNN layer 805.

[0067] The ninth layer is the max pooling layer 809, with a pooling size of 2×2 and an output of 8×8×3. The final output layer 810 is a fully connected layer, with a binary output corresponding to the inspection result of "good" or "bad."

[0068] When the output result of the output layer 810 is different from the label (correct answer information), the weights and / or bias of the filter of the neural network are adjusted using the error back propagation method to make the output result consistent with the label, and training is performed. The error back propagation method is used in training. The number of data is "100", so the batch size (batch-size) is set to "1" and the number of training rounds (epochs, iterations) is set to "100". Figure 3 As shown, for the data set, training can be performed using an external machine learning computing device 20, or using a computing device inside the print inspection device 19 or the inkjet printer 1.

[0069] Furthermore, this training uses a loss function (cross entropy error, sum of squared error, etc.) to update weights and / or biases. Here, the weights and / or biases serve as evaluation functions for inspecting the actual printed image to be inspected based on a printed image obtained by photographing a print result of a random dot pattern.

[0070] Then, in Figure 4 In the application step 303 shown, the trained neural network is used to perform inspection. In this case, when an image of printed characters on a print object, captured by the print inspection device 19, is input, the trained neural network outputs a judgment of whether the printed characters are "good" or "bad."

[0071] Thus, in this embodiment, the printing inspection device has a machine learning function unit based on a neural network, and the machine learning function unit is configured to include: a random test printing function unit, which has a function of printing a dot arrangement pattern composed of a random plurality of dots based on various characters to be printed in the printing area of the printing object; and an evaluation function training function unit, which has a function of optimizing the evaluation function of the neural network used to inspect the printed image based on the printed image obtained by photographing the printed dot arrangement pattern.

[0072] In this way, a random dot arrangement pattern based on various characters to be printed can be test-printed on the printing object, and the evaluation function of the neural network can be optimized based on the random dot arrangement pattern. This improves the inspection accuracy and enables efficient inspection work.

[0073] Example 2

[0074] Next, a second embodiment of the present invention will be described. Compared to the first embodiment, this embodiment proposes a neural network training method that further improves inspection accuracy.

[0075] The first embodiment uses a pre-prepared dot pattern, but in this case the number of dot patterns is limited, and the same dot pattern must be used each time. Therefore, this embodiment proposes a method of generating a random dot pattern and performing test printing when higher-precision training is required.

[0076] exist Figure 9 Shown in Figure 5 Flowchart of the test printing process 401 shown. The characteristic of this embodiment is that the random dot arrangement pattern is dynamically generated.

[0077] First, in the random dot pattern generation step 901, the presence or absence of dots (the presence or absence of dots) is randomly selected for all dots. The probability of dot presence or absence can be freely determined; in this embodiment, it is set to 50%, similar to the first embodiment. Pseudorandom numbers are used as a random selection method. For example, methods for generating pseudorandom numbers using the Mersenne twister method are known.

[0078] Next, errors are added to the generated dot arrangement pattern in the error adding step 902. The error adding probability is 50%, and the error dot arrangement pattern is added as follows: Figures 7B to 7D When the dot arrangement pattern is determined, the dot arrangement pattern is printed in the random dot pattern printing step 903 in the same manner as in the first embodiment.

[0079] Next, in the random dot pattern sharing step 904, the dot pattern is shared with the print inspection device 19. This is necessary for generating the data set. Furthermore, the presence or absence of erroneous additions is also shared. The subsequent steps are the same as in the first embodiment, and the amount of test data can be freely determined. For example, if the trained network accuracy is insufficient with 100 test data items, additional test prints can be performed.

[0080] As described above, according to this embodiment, it is possible to train a neural network that can further improve inspection accuracy.

[0081] Example 3

[0082] Next, a third embodiment of the present invention will be described. Compared to the first embodiment, this embodiment proposes a neural network training method that further improves inspection accuracy.

[0083] In the first embodiment, the learning efficiency may be insufficient for errors caused by missing dots. This is because missing dots cannot be distinguished from the presence or absence of random dots.

[0084] So, in Figure 5 In the data set generation step 403 , composite dot arrangement pattern data 1001 is generated by superimposing (overlaying) the standard dot arrangement pattern 701 without adding dot errors and the data and dot missing error dot arrangement pattern 704 .

[0085] exist Figure 10 An example of a composite dot arrangement pattern 1001 obtained by superposition is shown in FIG. 1 . By superposition, the points 1001e of the missing portion surrounded by the dotted box are emphasized, which enables more efficient training of the neural network.

[0086] As described above, according to this embodiment, it is possible to train a neural network that can further improve inspection accuracy.

[0087] As described above, the present invention is characterized in that: a machine learning function unit based on a neural network is provided in the printing inspection device, and the machine learning function unit includes: a random test printing function unit, which has the function of printing a dot arrangement pattern composed of a random plurality of dots based on various characters to be printed in the printing area of the printing object; and an evaluation function training function unit, which has the function of optimizing the evaluation function of the neural network used to inspect the printed image based on the printed image obtained by photographing the printed dot arrangement pattern.

[0088] Thus, a random dot arrangement pattern (random dot arrangement pattern) based on various characters to be printed is test-printed on the printing object, and the evaluation function of the neural network is optimized based on this, thereby improving the inspection accuracy and performing efficient inspection operations.

[0089] Furthermore, the present invention is not limited to the aforementioned embodiments but includes various variations. The aforementioned embodiments are described in detail to facilitate understanding of the present invention and are not intended to necessarily include all of the described structures. Furthermore, a portion of the structure of one embodiment can be replaced with the structure of another embodiment, and the structure of one embodiment can be supplemented with the structure of another embodiment. Other structures can be added to, deleted from, or substituted for the structures of each embodiment.

[0090] Description of Reference Signs

[0091] 1...inkjet printer, 19...print inspection device, 20...machine learning operation device, 301...test process for data set generation, 302...machine learning process, 303...application process, 401...test printing process, 402...test printing and imaging process, 403...data set generation process, 501...random dot pattern acquisition process, 502...random dot pattern acquisition process, 801...input layer, 802, 803, 805~808...CNN layers, 804, 809...maximum pooling layers, 810...output layer.

Claims

1. An inkjet printing system comprising an inkjet printer for printing on a printing object; and a print inspection device for photographing characters printed by the inkjet printer and inspecting the photographed print image, wherein: The printing inspection device has a machine learning function unit based on a neural network. The machine learning function unit includes: a random test printing function unit having a function of printing a dot arrangement pattern including a plurality of random dots based on various characters to be printed in the print area of the print object, wherein the dot arrangement pattern is a dot arrangement pattern consisting of a plurality of dots with arrangement positions of the dots randomly determined; and The evaluation function training function unit has a function of optimizing the evaluation function of the neural network for inspecting the printed image based on the printed image obtained by photographing the printed dot arrangement pattern.

2. The inkjet printing system according to claim 1, wherein: The neural network is a convolutional neural network.

3. The inkjet printing system according to claim 2, wherein: The random test printing function unit sends the dot arrangement pattern to the inkjet printer, The inkjet printer performs test printing based on the dot arrangement pattern.

4. The inkjet printing system according to claim 3, wherein: The random dot arrangement pattern includes a standard dot arrangement pattern in which dots are located at standard positions and an erroneous dot arrangement pattern in which dots are not located at standard positions.

5. The inkjet printing system according to claim 4, wherein: The erroneous dot arrangement pattern includes at least one of the following dot arrangement patterns 1 to 3, wherein: The dot arrangement pattern 1 is a dot deviation error dot arrangement pattern in which one dot is randomly selected and deviates from the standard dot arrangement pattern by a randomly given distance and angle. The dot arrangement pattern 2 is a row deviation error dot arrangement pattern in which one row is randomly selected and deviates from the standard dot arrangement pattern in the vertical direction by a randomly given distance. The dot arrangement pattern 3 is a dot missing error dot arrangement pattern in which one dot is randomly selected and the selected dot is deleted from the standard dot arrangement pattern.

6. The inkjet printing system according to claim 1, wherein: The random test print function dynamically generates the dot arrangement pattern.

7. The inkjet printing system according to claim 6, wherein: In order to dynamically generate the dot arrangement pattern, the dot arrangement pattern is generated using pseudo-random numbers.

8. The inkjet printing system according to claim 5, wherein: The random test printing function unit generates a composite dot arrangement pattern obtained by superimposing the standard dot arrangement pattern and the dot-missing error dot arrangement pattern.

Images