Droplet weight estimating device, teacher data creation method, and inkjet printing system

Abstract

A droplet (D) weight estimating device (60) comprises a storage unit (61) and an executing unit (62). The storage unit (61) stores a trained model (LM) that has been machine trained in advance, using, as an input, image data itself obtained by imaging a droplet (D) ejected onto a printing target object (W), to output an estimated value indicating the weight of the droplet D captured in the image data. The executing unit (62) is capable of executing: acquisition processing for acquiring the image data in which the droplet (D) is captured; and estimation processing for inputting the acquired image data itself into the trained model (LM) in order to output the estimated value indicating the weight of the droplet (D) captured in the image data.

Description

Droplet weight estimation device, training data creation method, and inkjet printing system

[0001] The present disclosure relates to a droplet weight estimation device, a training data creation method, and an inkjet printing system.

[0002] The droplet weight estimation device disclosed in Patent Document 1 includes a memory unit and an execution unit. The memory unit stores correlation characteristics that correspond one-to-one to the diameter of ink droplets ejected onto a transparent substrate and the estimated value of the ejection volume per droplet from a head of an inkjet printing system. The execution unit acquires image data of the ink ejected onto the transparent substrate. This image data is an image of the ink droplets captured from directly above the droplets. The execution unit calculates the droplet diameter from the acquired image data. The execution unit uses the correlation characteristics to obtain an estimated value of the ejection volume per droplet from the calculated droplet diameter.

[0003] Japanese Patent Application Laid-Open No. 2005-238787

[0004] The droplet weight estimation device disclosed in Patent Document 1 assumes that the droplet shape when viewed from directly above is a perfect circle when calculating an estimated value of the ejection volume per droplet. However, the droplet shape is not always a perfect circle. Therefore, if the droplet has low circularity, the error between the estimated value and the true value will be large.

[0005] In order to solve the above problems, the present disclosure provides a droplet weight estimation device that includes a memory unit and an execution unit, wherein the memory unit stores a trained model that has been machine-learned to input image data of droplets ejected onto a printing object and output an estimated value indicating the weight of the droplets captured in the image data, and the execution unit is capable of performing an acquisition process that acquires the image data of the droplets, and an estimation process that inputs the acquired image data itself into the trained model and outputs the estimated value indicating the weight of the droplets captured in the image data.

[0006] The present disclosure also provides a droplet weight estimation device that includes a memory unit and an execution unit, wherein the memory unit stores relationship specification data that specifies the relationship between two or more input parameters extracted from image data in which droplets ejected onto a printing object are captured, and an estimated value indicating the weight of the droplets captured in the image data, one of the input parameters being a shading value that indicates the shade of color of the droplets reflected in the image data, and the execution unit is capable of executing an acquisition process that acquires each of the input parameters, and an estimation process that outputs the estimated value indicating the weight of the droplets captured in the image data based on the acquired input parameters and the relationship specification data.

[0007] The present disclosure also provides an inkjet printing system comprising a printing device that ejects droplets onto a printing object, an imaging device that captures images of the droplets ejected onto the printing object, a transport device that transports the printing object, a control device that controls the printing device, the imaging device, and the transport device, and a droplet weight estimation device, wherein the control device executes an ejection process that causes the printing device to eject the droplets onto the printing object, a transport process that causes the transport device to transport the printing object onto which the droplets have been ejected to a position where the image can be captured by the imaging device, and an imaging process that causes the imaging device to capture images of the droplets ejected onto the printing object after the transport process, and the execution unit of the weight estimation device acquires the image data of the droplets captured in the imaging process during the acquisition process.

[0008] The present disclosure also provides a method for creating teacher data, comprising: an ejection step of ejecting droplets onto a printing object; an imaging step of capturing images of the droplets ejected onto the printing object using an imaging device; and a teacher data creation step of creating teacher data, which is a set of image data of the droplets captured in the imaging step and data indicating the weight of the droplets captured in the image data.

[0009] The present disclosure also provides a printing device that ejects droplets onto a printing object, an imaging device that captures an image of the droplets ejected onto the printing object, a transport device that transports the printing object, a control device that controls the printing device, the imaging device, and the transport device, and a learning device that learns data indicating the weight of the droplets ejected onto the printing object, wherein the control device performs an ejection process that causes the printing device to eject the droplets onto the printing object, and after the ejection process, a transport device that causes the transport device to transport the printing object onto which the droplets have been ejected to a position where the image can be captured by the imaging device. and after the transport process, an imaging process in which the imaging device captures an image of the droplets ejected onto the printing object. The learning device is equipped with a memory unit and an execution unit, and the memory unit stores a machine learning model capable of machine learning. The execution unit executes a teacher data creation process in which teacher data is created as a set of image data captured in the imaging process and the data indicating the weight of the droplets captured in the image data, and a machine learning process in which the machine learning model learns the teacher data.

[0010] An estimate of the weight of the droplet is obtained that does not depend solely on the diameter of the droplet.

[0011] Fig. 1 is a perspective view of the inside of the housing of an inkjet printing system. Fig. 2 is a flowchart of control for estimating weight. Fig. 3 is a flowchart of a method for creating a trained model. Fig. 4 is an explanatory diagram for explaining the method for creating a trained model.

[0012] <Embodiments of a droplet weight estimation device, a training data creation method, and an inkjet printing system> An embodiment of a droplet weight estimation device, a training data creation method, and an inkjet printing system will be described below. Note that the drawings are schematic diagrams for ease of understanding, and components may be enlarged or omitted. Therefore, the dimensional ratios of the components may differ from the actual ones.

[0013] (Regarding the Inkjet Printing System) As shown in Figure 1, the inkjet printing system PS has a flat installation surface MS. The installation surface MS may be, for example, the bottom surface of a housing having an internal space, or the top surface of a housing on which an object or the like can be placed or placed.

[0014] Here, a specific direction parallel to the installation surface MS is defined as the first axis X. An axis parallel to the installation surface MS and perpendicular to the first axis X is defined as the second axis Y. An axis perpendicular to the first axis X and the second axis Y, i.e., an axis perpendicular to the installation surface MS, is defined as the third axis Z. Furthermore, a specific direction along the first axis X is defined as the first positive direction X1, and the direction opposite to the first positive direction X1 is defined as the first negative direction X2. A specific direction along the second axis Y is defined as the second positive direction Y1, and the direction opposite to the second positive direction Y1 is defined as the second negative direction Y2. Furthermore, a direction along the third axis Z toward which the installation surface MS faces is defined as the third positive direction Z1, and the direction opposite to the third positive direction Z1 is defined as the third negative direction Z2. Note that, hereinafter, "upper side" refers to the third positive direction Z1 side. "lower side" refers to the third negative direction Z2 side.

[0015] As shown in FIG. 1 , the inkjet printing system PS includes a printing device 10. The printing device 10 includes a support structure 11 and an XZ drive mechanism 12. The support structure 11 is fixed on an installation surface MS. The support structure 11 also includes a pair of pillars 11A and a rail 11B. The pair of pillars 11A extend from the installation surface MS in a third positive direction Z1. The pair of pillars 11A are aligned at a predetermined interval in the direction in which the first axis X extends. The rail 11B connects the upper ends of the pair of pillars 11A. In other words, the rail 11B extends in the direction in which the first axis X extends.

[0016] The XZ drive mechanism 12 is connected to the rail 11B of the support structure 11. The XZ drive mechanism 12 has a servo motor, a ball screw, gears, etc. (not shown). The XZ drive mechanism 12 is movable on the underside of the rail 11B along the longitudinal direction of the rail 11B. Therefore, the XZ drive mechanism 12 is movable relative to the support structure 11 in the direction in which the first axis X extends. Furthermore, the XZ drive mechanism 12 is extendable and contractible in the direction in which the third axis Z extends. Therefore, the lower end of the XZ drive mechanism 12 is movable relative to the support structure 11 in the direction in which the third axis Z extends.

[0017] 1, the printing device 10 includes a head 13, a plurality of nozzles 14, a tank 15, and piping 16. The head 13 is connected to the lower end of the XZ drive mechanism 12. Therefore, the head 13 is movable together with the XZ drive mechanism 12 in the direction in which the first axis X extends. The head 13 is also movable in the direction in which the third axis Z extends in accordance with the expansion and contraction of the XZ drive mechanism 12.

[0018] The head 13 has a substantially rectangular parallelepiped shape. The head 13 includes a pump (not shown), a flow path, and an injector inside. The pump can suck ink from the tank 15 via piping 16 (described later). The flow path is tubular. Ink sucked by the pump flows through the flow path. The injector can eject ink. The injector includes, for example, a piezoelectric element. That is, the injector ejects ink in response to the application of electricity to the piezoelectric element.

[0019] The tank 15 is capable of storing ink in its internal space. The ink stored in the tank 15 is, for example, pigment ink. The pipe 16 is tubular. One end of the pipe 16 is connected to the internal space of the tank 15. The other end of the pipe 16 is connected to a pump, a flow path, and an injector within the head 13.

[0020] The multiple nozzles 14 are tubular. The nozzles 14 are connected to the flow passages and injectors of the head 13. The nozzles 14 protrude from the bottom surface of the head 13. Therefore, the nozzles 14 are capable of ejecting ink from their tips, which is sucked in by the pump of the head 13 and ejected by the injector. The multiple nozzles 14 are aligned in a matrix at regular intervals on the bottom surface of the head 13. The bottom surface of the head 13 refers to the surface facing the installation surface MS of the head 13, i.e., the surface on the third negative direction Z2 side.

[0021] The inkjet printing system PS includes a transport device 30. The transport device 30 includes a stage 31 and a Y drive mechanism 32. The Y drive mechanism 32 is installed on an installation surface MS. The Y drive mechanism 32 is, for example, shaped like a rectangular box. Although not shown, the Y drive mechanism 32 includes a servo motor, a ball screw, gears, and the like. This allows the Y drive mechanism 32 to move relatively to the installation surface MS in the direction in which the second axis Y extends.

[0022] The stage 31 has a rectangular plate shape. The stage 31 is fixed to a surface of the Y drive mechanism 32 on the third positive direction Z1 side. The main surface of the stage 31 faces the third positive direction Z1. When the Y drive mechanism 32 is driven, the stage 31 can move together with the Y drive mechanism 32 relative to the installation surface MS in the direction in which the second axis Y extends. As described above, the nozzle 14 can also move relative to the support structure 11 in the direction in which the first axis X extends and the direction in which the third axis Z extends. Therefore, the relative positional relationship between the stage 31 and the nozzle 14 can be changed in any of the directions in which the first axis X extends, the second axis Y extends, and the third axis Z extends.

[0023] An object such as a printing object W can be placed on the main surface of the stage 31. The printing object W is a nonwoven fabric with intertwined fibers or a woven fabric with woven fibers. Specifically, the printing object W is printing paper. The stage 31 has a fixing device (not shown). The fixing device can fix the printing object W on the main surface of the stage 31.

[0024] The inkjet printing system PS includes an imaging device 40 and an illumination device L. The imaging device 40 is, for example, a CCD camera with a resolution of about 10 μm. Although not shown in the figure, the position of the imaging device 40 is fixed relative to the installation surface MS via a frame or the like.

[0025] 1, the imaging device 40 is located on the third positive direction Z1 side with respect to the installation surface MS. Furthermore, when viewed in the third negative direction Z2, the imaging device 40 is located outside the orbit of the head 13 driven by the XZ drive mechanism 12 and on the orbit of the transport device 30 driven by the Y drive mechanism 32. The lens of the imaging device 40 faces the third negative direction Z2 side, i.e., toward the installation surface MS. Therefore, when the stage 31 of the transport device 30 is located on the third negative direction Z2 side of the imaging device 40, the imaging device 40 can capture an image of the printing object W placed on the main surface of the stage 31.

[0026] The imaging device 40 is capable of capturing an image of the printing target W in full color. Specifically, the imaging device 40 has a Bayer filter and a light-receiving element such as a photodiode. Light emitted from the illumination device L is reflected by the printing target W and input to the light-receiving element via the Bayer filter. As a result, the imaging device 40 acquires the light intensity of the wavelength band corresponding to red, the wavelength band corresponding to blue, and the wavelength band corresponding to green contained in the reflected light. The peak wavelength of the wavelength band corresponding to red is approximately 630 nm. The peak wavelength of the wavelength band corresponding to blue is approximately 460 nm. The peak wavelength of the wavelength band corresponding to green is approximately 540 nm. The peak wavelength refers to the wavelength at which the intensity is maximum in the spectrum of light intensity relative to the wavelength. Therefore, the imaging device 40 is capable of capturing images using two or more types of light whose center wavelengths differ by 100 nm or more.

[0027] The image capturing device 40 converts the light intensities corresponding to the three wavelength bands into an RGB color model and records the converted data as image data. Therefore, the image data contains color data for two or more colors whose peak wavelengths differ by 100 nm or more.

[0028] The lighting device L can irradiate light in the direction in which the lens of the imaging device 40 faces, i.e., onto the droplets D ejected onto the printing target W. The lighting device L has blue LED (Light Emitting Diode), red LED, and green LED elements. The red LED has a peak wavelength of approximately 660 nm. The blue LED has a peak wavelength of approximately 450 nm. The green LED has a peak wavelength of approximately 545 nm. Therefore, the lighting device L can irradiate two or more types of light whose peak wavelengths differ by 100 nm or more. Furthermore, the lighting device L can irradiate light of two or more colors by adjusting the intensity of each LED element.

[0029] The inkjet printing system PS includes a control device 50. Although not shown, the control device 50 includes a control and calculation unit such as a CPU (Central Processing Unit), memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and peripheral circuits such as a clock circuit and a power supply circuit. The control device 50 is also capable of communicating with the printing device 10, the imaging device 40, and the transport device 30. The control device 50 controls the printing device 10, the imaging device 40, and the transport device 30.

[0030] Specifically, the control device 50 can control the injectors of the heads 13 of the printing device 10. That is, the control device 50 can control the amount of ink ejected from the nozzles 14, the timing of the ejection, etc. The control device 50 can control the amount of ink droplets D ejected from the nozzles 14 by controlling the voltage applied to the piezoelectric elements of the injectors. The amount of ink droplets D ejected at one time from one nozzle 14 is, for example, 0.001 pL or more and 100 pL or less.

[0031] The control device 50 controls the XZ drive mechanism 12 and the Y drive mechanism 32 to control the position of the stage 31 relative to the nozzle 14. Furthermore, the control device 50 controls the Y drive mechanism 32 of the transport device 30 to cause the transport device 30 to transport the print target W onto which the droplets D have been ejected to a position where it can be imaged by the imaging device 40. Therefore, by performing the above-mentioned controls, the control device 50 can perform inkjet printing on the surface of the print target W placed on the main surface of the stage 31.

[0032] (Weight Estimation Device) The inkjet printing system PS includes a weight estimation device 60. The weight estimation device 60 includes an execution unit 62 and a storage unit 61. Although not shown, the weight estimation device 60 also includes other peripheral circuits such as a clock circuit and a power supply circuit. The weight estimation device 60 is also capable of communicating with the control device 50.

[0033] The storage unit 61 is made up of a ROM and a RAM. The ROM of the storage unit 61 stores in advance programs for the execution unit 62 to execute various processes. The storage unit 61 also stores a trained model LM created by a method described below. The trained model LM in this embodiment has been trained in advance by machine learning to input image data of droplets D ejected onto a printing target W, and output an estimated value indicating the weight of the droplets D captured in the image data.

[0034] The execution unit 62 is a CPU. The execution unit 62 performs various processes using the RAM as a working memory in accordance with the programs stored in the storage unit 61. Specifically, the execution unit 62 uses the programs and the trained model LM to output an estimated value indicating the weight of the droplets D ejected onto the printing target W. In this embodiment, the "estimated value indicating the weight of the droplets D" is the weight itself. The unit of weight is "kgf (kilogram-force)."

[0035] (Estimating Droplet Weight) Next, a method for estimating the weight of a droplet D ejected onto a print target W by the inkjet printing system PS will be described below.

[0036] 2, the weight estimation is realized by a discharge process S101, a conveyance process S102, and an image capturing process S103 by the control device 50, and an acquisition process S104 and an estimation process S105 by the weight estimation device 60. As described above, the learned model LM is stored in advance in the storage unit 61 of the weight estimation device 60.

[0037] Before starting weight estimation, the conveying device 30 is positioned in advance directly below the head 13 and the nozzle 14, i.e., in the third positive direction Z1 relative to the head 13 and the nozzle 14. Printing paper is placed on the main surface of the stage 31 of the conveying device 30 as the printing object W. This printing paper is preferably the same type of paper as the printing object W used in the method for creating the trained model LM, which will be described later.

[0038] When weight estimation begins, the control device 50 first performs a discharge process S101. In the discharge process S101, the control device 50 causes the printing device 10 to discharge droplets D onto the printing object W. That is, the control device 50 controls the discharge mechanism of the head 13 to discharge a plurality of ink droplets D from the plurality of nozzles 14. The discharged ink droplets D adhere to the surface of the printing object W on the third positive direction Z1 side in a matrix at approximately regular intervals. The reason for the "approximately regular intervals" is that the actual droplet positions of the droplets D may differ from the ideal droplet positions due to the influence of external disturbances, etc.

[0039] As described above, the ink is a pigment ink. Therefore, the ink droplets D have varying shades of color. Shading of color is what is known as color unevenness. Note that if the ground on which the inkjet printing system PS itself is installed is not level, or if there is even a slight tilt, the color shading of the droplets D may be partially uneven. Furthermore, the droplets D have a generally spherical crown shape. Because the droplets D have a generally spherical crown shape, the outer edge of the droplets D is generally circular when viewed in the third negative direction Z2. However, strictly speaking, this outer edge shape is not circular but rather distorted. Specifically, this outer edge shape may be elliptical, or may have a wavy portion due to bleeding into the printing target W. Furthermore, when the droplets D are ejected from the nozzles 14, some of the droplets D may scatter. In addition, even if a portion of the droplet D ejected onto the printing object W by a single ejection based on the control of the execution unit 62 is scattered as described above, the droplet D that is contained within one area determined according to the distance between the nozzles 14 is considered to be one droplet D.

[0040] Next, the control device 50 performs a transport process S102. In the transport process S102, the control device 50 causes the transport device 30 to transport the printing target W onto which the droplets D have been ejected to a position where the printing target W can be imaged by the imaging device 40. Specifically, the control device 50 drives the Y drive mechanism 32 of the transport device 30 to move the transport device 30 and the printing target W relative to the nozzle 14. Then, the control device 50 moves the transport device 30 so that the printing target W is positioned approximately directly below the lens of the imaging device 40.

[0041] Next, the control device 50 performs an imaging process S103. In the imaging process S103, the control device 50 causes the imaging device 40 to capture an image of the droplets D ejected onto the printing target W. Specifically, the imaging device 40 captures an image of substantially the entire printing target W. The control device 50 then divides the entire image captured by the imaging device 40 into a matrix at a predetermined pitch, thereby generating multiple image data sets corresponding to the number of droplets D. This predetermined pitch is substantially the same as the pitch of the droplets D ejected in the ejection step S202. In other words, the control device 50 divides the entire image so that one droplet D exists in one divided image data set.

[0042] As described above, the spacing between droplets D is not constant in the strict sense, and therefore, in the image captured by the imaging device 40, the position of the droplet D may not be located at the center of the image. Furthermore, in the imaging step S206, the imaging device 40 captures a so-called full-color image. That is, as described above, the image data captured by the imaging device 40 includes color data for two or more colors whose wavelengths differ by 100 nm or more. Furthermore, the image data reflects the color of the droplet D and a shading value that indicates its shading.

[0043] Next, the execution unit 62 of the weight estimation device 60 performs an acquisition process S104. In the acquisition process S104, the execution unit 62 acquires the image data obtained in the imaging process S103 via the control device 50. This image data is stored in the storage unit 61.

[0044] Next, the execution unit 62 performs estimation processing S105. In estimation processing S105, the execution unit 62 inputs the acquired image data itself into the trained model LM, thereby outputting an estimated value indicating the weight of the droplet D captured in the image data. In this embodiment, the trained model LM outputs the weight itself as an estimated value indicating the weight of the droplet D. Through the above processing, the weight of the droplet D can be estimated.

[0045] (Regarding the method for creating a trained model) Next, a description will be given of a method for creating a trained model LM to be stored in the storage unit 61. As shown in Fig. 3, the method for creating a trained model LM includes a preparation step S201, a discharge step S202, a weight measurement step S203, a drying step S204, a transport step S205, an imaging step S206, a teacher data creation step S207, and a machine learning step S208.

[0046] At the stage of creating the trained model LM, the memory unit 61 of the weight estimation device 60 in the inkjet printing system PS described above does not store the trained model LM. That is, the memory unit 61 of the weight estimation device 60 stores a machine learning model MM that has not been subjected to machine learning. Hereinafter, the weight estimation device 60 will be referred to as a learning device 100 if it has the same configuration as the weight estimation device 60 described above but the memory unit 61 stores the machine learning model MM before machine learning. Note that this machine learning model MM is a regression model constructed by a known method using a model that uses a convolutional neural network such as ResNet (Residual Network). That is, the machine learning model MM is a regression model capable of machine learning using image data itself as an explanatory variable and a predetermined value as a target variable.

[0047] As shown in Fig. 3, the method for creating the learned model LM first performs a preparation step S201. As shown in Fig. 4, in the preparation step S201, a weighing scale S and a printing target W are placed on the main surface of the stage 31 of the conveying device 30. The weighing scale S measures the weight of an object placed on the upper tray. The printing target W is printing paper. The printing target W is placed on the upper tray of the weighing scale S. Also, in the preparation step S201, the control device 50 controls the XZ drive mechanism 12 and the Y drive mechanism 32 to align the printing target W in the third negative direction Z2.

[0048] Next, the ejection step S202 is performed. In the ejection step S202, the control device 50 performs the above-described ejection process S101. That is, the control device 50 causes the printing device 10 to eject droplets D onto the printing object W. As described above, the droplets D are deposited in a matrix pattern at approximately regular intervals on the printing object W. Furthermore, the droplets D are approximately spherical crown-shaped.

[0049] Next, a weight measurement step S203 is performed. In the weight measurement step S203, a weigh scale S is used to obtain data indicating the weight of the droplets D ejected onto the printing target W. Specifically, the total weight of the droplets D ejected onto the printing target W is measured using the weigh scale S. Then, the total weight of the droplets D is divided by the number of droplets D to calculate the weight per droplet D as the "data indicating weight." Note that the total number of times droplets D were ejected in the ejection step S202 is considered to be the "number of droplets D."

[0050] Next, the drying step S204 is performed. In the drying step S204, the droplets D whose weights have been measured in the weight measurement step S203 are left to stand for a predetermined time. The predetermined time may be, for example, several hours to several days. This removes most of the volatile components in the droplets D, leaving the pigment on the surface of the printing paper. As a result, the shape of the droplets D on the printing paper is substantially fixed. Note that, for convenience, hereinafter, droplets D from which most of the solvent has been removed after the drying step S204 will also be referred to as "droplets D."

[0051] Next, a transport step S205 is performed. In the transport step S205, the control device 50 performs the transport process S102 described above. That is, as shown in FIG. 4 , the control device 50 causes the transport device 30 to transport the printing target W onto which the droplets D have been ejected to a position where the image can be captured by the image capturing device 40.

[0052] As shown in Fig. 3, next, an imaging step S206 is performed. In the imaging step S206, the control device 50 performs processing similar to the imaging process S103 described above. That is, the control device 50 causes the imaging device 40 to capture an image of the printing target W onto which the droplets D have been ejected. The control device 50 also generates image data by dividing the captured entire image. That is, one image data contains an image of one droplet D.

[0053] Next, a teacher data creation step S207 is performed. In the teacher data creation step S207, first, the execution unit 62 of the learning device 100 performs the acquisition process S104 described above. That is, the execution unit 62 acquires image data of the droplet D that was imaged by the imaging device 40 in the imaging step S206 and divided by the control device 50. Next, the execution unit 62 performs a teacher data creation process. In the teacher data creation process, the execution unit 62 stores a pair of the acquired image data and data indicating the weight of the droplet D imaged in the image data in the memory unit 61 as teacher data for machine learning for the machine learning model MM.

[0054] More specifically, the execution unit 62 first acquires one of the divided image data from the control device 50. Next, the execution unit 62 labels the acquired image data with the data on the weight of the droplet D calculated in the weight measurement step S203. By performing this labeling for all of the divided image data, the execution unit 62 creates a data set as training data. The execution unit 62 stores the training data in the storage unit 61.

[0055] Next, a machine learning step S208 is performed. In the machine learning step S208, the execution unit 62 of the learning device 100 performs machine learning processing. That is, the execution unit 62 performs machine learning on the above-mentioned machine learning model MM. Specifically, the execution unit 62 uses each image data itself of the teacher data created in the teacher data creation step S207 as an explanatory variable and the weight calculated in the weight measurement step S203 as a target variable, and causes the above-mentioned machine learning model MM to perform machine learning. Through the above steps, a learned model LM is created. Then, when the learned model LM created by the execution unit 62 is stored in the memory unit 61, the learning device 100 becomes the weight estimation device 60.

[0056] (Effects of this embodiment) (1) In the above embodiment, the execution unit 62 of the weight estimation device 60 outputs an estimated value indicating the weight of the droplet D using a trained model LM that has been subjected to machine learning. The trained model LM has been trained by machine learning to output an estimated value indicating the weight of the droplet D using the image data itself as input. The image data of the droplet D reflects elements such as the shade of the droplet D, the size of the droplet D, the shape of the droplet D, and the position of the droplet D within the image data. Therefore, even if the circularity of the droplet D is low, the weight can be estimated with higher accuracy than when the weight of the droplet D is estimated from some elements extracted from the image data.

[0057] (2) In the above embodiment, the image data includes three or more colors. This allows the image data to contain more information than when a grayscale image or a monochrome image is used. In other words, since more information is available for estimation, the accuracy of estimating the weight of the droplet D is improved.

[0058] (3) In the above embodiment, the material of the printing target W is paper. The droplets D are amorphous immediately after being ejected from the nozzle 14. Therefore, by allowing the printing target W to absorb a portion of the solvent and volatile components of the droplets D, the shape of the droplets D is substantially fixed. In other words, it is possible to prevent the shape of the droplets D from changing due to disturbances after they adhere to the printing target W. By creating training data in advance using such image data and using similar image data when obtaining an estimated value indicating the weight of the droplets D, the accuracy of estimating the weight of the droplets D can be significantly improved.

[0059] (4) In the above embodiment, the control device 50 can execute the transport process S102. Therefore, the inkjet printing system PS can perform the process from ejecting the droplets D to capturing an image of the droplets D and estimating their weight. This reduces the amount of work required compared to manually moving the droplets D after they are ejected.

[0060] Furthermore, the volatile components contained in the droplets D evaporate or soak into the print target W immediately after the droplets D are ejected. By using a single inkjet printing system PS to perform the entire process from ejection of the droplets D to imaging, it is possible to reduce the variation in the state of multiple droplets D when creating training data multiple times. In particular, it is possible to reduce the error for each print target W compared to when the print target W is transported manually.

[0061] (5) In the above embodiment, the learning device 100 creates training data that is a combination of the image data itself and data indicating the weight of the droplet D. Therefore, compared to creating training data based only on the diameter of the droplet D, for example, it is possible to create a trained model LM with higher estimation accuracy.

[0062] <Modifications> The above embodiment and the following modifications can be implemented in combination with each other within the scope of technical compatibility.

[0063] (Modifications to the Overall Configuration of the Inkjet Printing System) The configuration of the inkjet printing system PS can be modified as appropriate to suit the intended use of the inkjet printing system PS. For example, in addition to the configuration of the above embodiment, a drying mechanism for drying the ink after printing may be included. The inkjet printing system PS may be applied to so-called home inkjet printers or industrial inkjet printers.

[0064] The configuration of the head 13 and the nozzles 14 is not limited to the example in the above embodiment. For example, the method of ejecting ink from the nozzles 14 may be any method, such as a thermal method or an electrostatic method.

[0065] The type of droplets D is not limited to pigment ink. The droplets D may be dye ink, a liquid such as water that has no color or shade of color, or a functional material ink that contains, for example, metal and resin.

[0066] The configuration of the transport device 30 for transporting the printing object W is not limited to the example of the above embodiment. For example, the transport device 30 may be movable in the direction along the first axis X and the direction along the third axis Z.

[0067] The configuration of the imaging device 40 is not limited to the example of the above embodiment. For example, the imaging device 40 may be a hyperspectral camera. In this case, the imaging device 40 can obtain information corresponding to several tens of wavelength bands, thereby improving the accuracy of the estimated value indicating the weight of the droplet D by the weight estimation device 60. Furthermore, even if the droplet D is colorless, such as water, the weight of the droplet D can be estimated.

[0068] The type of light source and light intensity of the illumination device L may be changed as appropriate depending on the type of droplet D, etc. For example, the imaging device 40 may have multiple types of light sources, such as a halogen lamp or a xenon lamp. Even in this case, it can be said that the illumination device L is capable of irradiating at least two types of light whose peak wavelengths differ by 100 nm or more. Furthermore, the learning device 100 may perform machine learning using image data obtained by irradiating and imaging with light in each wavelength band as training data, and may also irradiate and image with light in the same wavelength band when estimating the weight of the droplet D. By increasing the amount of information used for estimation, the estimation accuracy of the estimated value indicating the weight of the droplet D can be improved.

[0069] Furthermore, the illumination device L does not have to be capable of emitting at least two types of light whose peak wavelengths differ by 100 nm or more. That is, the illumination device L may be capable of emitting only monochromatic light. The imaging device 40 may be capable of capturing an image of the printing target W in only one wavelength band, or may be capable of capturing an image of the printing target W in three or more wavelength bands.

[0070] The physical configurations of the control device 50 and the weight estimation device 60 are not limited to the examples in the above embodiment. For example, the control device 50 and the weight estimation device 60 may be physically the same device. That is, the CPU that is the control and calculation unit of the control device 50 and the CPU that is the execution unit 62 of the weight estimation device 60 may be the same CPU. In this case, it is sufficient that one CPU is capable of controlling the printing device 10, the imaging device 40, and the conveying device 30, and is also capable of executing the acquisition process S104 and the estimation process S105.

[0071] (Modifications Related to Droplet Weight Estimation) In estimating the weight of a droplet D, it is sufficient that the execution unit 62 of the weight estimation device 60 can at least execute the acquisition process S104 and the estimation process S105. For example, the control device 50 does not need to execute the transport process S102, and after the discharge process S101, the printing target W may be moved manually and an image may be taken by the imaging device 40.

[0072] In estimating the weight of the droplets D, the printing target W does not have to be a nonwoven or woven fabric. For example, the printing target W may be a sheet-like object made of paper, such as thermal paper, that changes color upon chemical reaction with ink, glass, or PET (polyethylene terephthalate). Even in these cases, when estimating the weight of the droplets D, it is preferable to use a printing target W of the same type as the printing target W used to create the training data.

[0073] In estimating the weight of the droplets D, the transport process S102 is not limited to a method of moving the stage 31. For example, the control device 50 may move the imaging device 40 relative to the stage 31. That is, for example, the imaging device 40 may be connected to the rail 11B of the support structure 11, and the printing target W onto which the droplets D have been ejected may be moved relatively to a position where it can be imaged by the imaging device 40 by driving the XZ drive mechanism 12.

[0074] Instead of the trained model LM in the above embodiment, the storage unit 61 of the weight estimation device 60 may store relationship definition data that defines the relationship between two or more input parameters extracted from image data capturing an image of droplets D ejected onto a print target W and an estimated value indicating the weight of the droplets D captured in the image data. The relationship definition data is data that determines the relationship between these input parameters and the estimated value indicating the weight of the droplets D using a multivariate analysis method or the like.

[0075] However, one of the two or more input parameters is a gradation value that represents the gradation of the color of the droplet D in the image data. The gradation value is a value calculated based on the pixel value of one pixel in the image data. For example, the pixel value is a value that expresses the intensity of each of the red, green, and blue components in multiple gradations. The multiple gradations are, for example, 256 gradations. The gradation value is the sum of the intensities of pixels of a specific color within one image data. In this case, the execution unit 62 only needs to be able to execute an acquisition process that acquires each input parameter and an estimation process that outputs an estimated value indicating the weight of the droplet D captured in the image data based on the acquired input parameters and related specified data.

[0076] Specifically, after executing the imaging process S103 in the above embodiment, the execution unit 62 acquires image data from the imaging device 40. Next, the execution unit 62 acquires two or more input parameters from the image data. One of the input parameters is a gray value. Another input parameter is, for example, the area of ​​the droplet D in the image data. The area of ​​the droplet D can be calculated as the total number of pixels in the image data that have a pixel value equal to or greater than a predetermined value. The execution unit 62 inputs these input parameters into related specification data, thereby outputting an estimated value indicating the weight of the droplet D.

[0077] The relationship specification data may be a trained model that has been machine-learned in advance to input two or more of the input parameters and output the estimated value. In this case, the execution unit 62 may output the estimated value by inputting two or more input parameters to the trained model. As in the above embodiment, in this modified example, the model can be trained using a data set of two or more of the input parameters and the weight of the droplet D measured in the weight measurement step S203 as training data.

[0078] Furthermore, the relationship specification data may be a regression model in which two or more parameters are explanatory variables and a predetermined value is a target variable. The entity that extracts the input parameters from the image data does not matter. For example, the execution unit 62 may extract the input parameters according to an image processing program, or a user may measure and extract the input parameters while visually viewing the image.

[0079] The "estimated value indicating weight" output by the weight estimation device 60 is not limited to weight itself. For example, it may be the mass of the droplet D or the volume of the droplet D. The "estimated value indicating weight" may also be the voltage applied to the piezoelectric element of the injector. The greater the voltage applied to the piezoelectric element, the greater the weight of the droplet D per injection. In other words, there is a strong positive correlation between the voltage applied to the piezoelectric element of the injector and the weight of the droplet D. Therefore, it can be said that the voltage applied to the piezoelectric element indicates the weight of the droplet D. The execution unit 62 can output these estimated values ​​by using these values ​​as training data. When the estimated value is mass, the unit is "g (gram)" or the like. When the estimated value is volume, the unit is "L (liter)" or the like. When the estimated value is voltage, the unit is "V (volt)" or the like.

[0080] The "estimated value indicating the weight" output by the weight estimation device 60 is not limited to an estimated value indicating the weight of one droplet D. For example, an estimated value indicating the total weight of droplets D ejected from all the nozzles 14 of one head 13 may be output.

[0081] (Regarding a modified example of the method for creating a trained model) The storage unit 61 of the learning device 100 may store a machine learning model MM capable of machine learning. That is, the storage unit 61 of the learning device 100 may store a trained model LM as the machine learning model MM, and in the machine learning step S208, machine learning may be performed again on the trained model LM that has already been subjected to machine learning.

[0082] The preparation performed in the preparation step S201 is not limited to the example of the above embodiment. For example, a petri dish for measuring weight may be placed on the stage 31 as the printing object W. Furthermore, if a value other than weight is measured as a value indicating weight, the weight scale S does not need to be prepared.

[0083] In the weight measurement step S203, the weight of one droplet D may be measured as data indicating the weight. Alternatively, the weight of the droplet D after the drying step S204 may be measured. The operations related to the drying step S204 are not limited to those of the above embodiment. For example, the printing target W may be transported in the transport step S205, and then the drying step S204 may be performed. Alternatively, the printing target W may be manually moved from the stage 31 to another location, and the droplet D may be dried. The transport step S205 and the imaging step S206 may be performed without performing the drying step S204. Even in these cases, it is sufficient to obtain the image data itself or two or more parameters as the data to be used as training data.

[0084] In the teacher data creation step S207, the execution unit 62 of the learning device 100 may acquire image data from the imaging device 40 and then acquire two or more parameters from the image data. Then, a set of the two or more parameters and data indicating the weight of the droplet D may be used as teacher data. In this case, however, one of the parameters is a shading value that indicates the shading of the color.

[0085] In the teacher data creation step S207, image processing may be performed as preprocessing before inputting the image data into the learned model LM. This preprocessing may include, for example, converting the image data to grayscale, rotating the image data, adjusting the contrast of the image data, etc. Note that, when using relational specifying data that does not utilize machine learning in estimating the weight of the droplet D, these preprocessing steps may be performed before creating the relational specifying data.

[0086] Before the machine learning step S208, the series of steps from the preparation step S201 to the teacher data creation step S207 may be performed two or more times. In this case, for example, in the ejection step S202, the amount of ink droplets D associated with the weight may be changed by controlling the voltage applied to the piezoelectric element of the ejection mechanism. By acquiring image data under such diverse conditions, the accuracy of estimation by the learned model LM can be improved.

[0087] <Supplementary Notes> The following describes the technical ideas that can be understood from the above-described embodiments and modified examples. [1] A droplet weight estimation device including a storage unit and an execution unit, wherein the storage unit stores a trained model that has been machine-learned to receive image data of droplets ejected onto a printing target as input and output an estimated value indicating the weight of the droplets captured in the image data, and the execution unit is capable of executing an acquisition process that acquires the image data of the droplets, and an estimation process that inputs the acquired image data into the trained model and outputs the estimated value indicating the weight of the droplets captured in the image data.

[0088] [2] A droplet weight estimation device comprising a memory unit and an execution unit, wherein the memory unit stores relationship definition data that defines the relationship between two or more input parameters extracted from image data of droplets ejected onto a printing object and an estimated value indicating the weight of the droplets imaged in the image data, one of the input parameters being a shading value that represents the shade of color of the droplets reflected in the image data, and the execution unit is capable of executing an acquisition process that acquires each of the input parameters, and an estimation process that outputs the estimated value indicating the weight of the droplets imaged in the image data based on the acquired input parameters and the relationship definition data.

[0089] [3] The relationship specification data is a trained model that has been machine-learned in advance to input two or more of the input parameters and output the estimated value, and the execution unit outputs the estimated value by inputting the input parameters into the trained model in the estimation process.

[0090] [4] The droplet weight estimation device according to any one of [1] to [3], wherein the image data has color data for two or more colors with peak wavelengths that differ by 100 nm or more. [5] The droplet weight estimation device according to any one of [1] to [4], wherein the printing object is a nonwoven fabric with entangled fibers or a woven fabric with woven fibers.

[0091] [6] An inkjet printing system comprising: a printing device that ejects droplets onto a printing object; an imaging device that captures images of the droplets ejected onto the printing object; a transport device that transports the printing object; a control device that controls the printing device, the imaging device, and the transport device; and the droplet weight estimation device described in any one of [1] to [5], wherein the control device executes an ejection process that causes the printing device to eject the droplets onto the printing object; a transport process that, after the ejection process, causes the transport device to transport the printing object onto which the droplets have been ejected to a position where they can be captured by the imaging device; and an imaging process that, after the transport process, causes the imaging device to capture images of the droplets ejected onto the printing object, and the execution unit of the weight estimation device acquires the image data of the droplets captured in the imaging process in the acquisition process.

[0092] [7] An inkjet printing system as described in [6], further comprising an illumination device capable of irradiating the droplets ejected onto the printing object with two or more types of light having peak wavelengths that differ by 100 nm or more, and the imaging device is capable of imaging with the two types of light.

[0093] [8] A method for creating teacher data, comprising: an ejection step of ejecting droplets onto a printing object; an imaging step of capturing images of the droplets ejected onto the printing object using an imaging device; and a teacher data creation step of creating teacher data, which is a set of image data of the droplets captured in the imaging step and data indicating the weight of the droplets captured in the image data.

[0094] [9] A printing apparatus that ejects droplets onto a printing object, an imaging device that captures an image of the droplets ejected onto the printing object, a transport device that transports the printing object, a control device that controls the printing apparatus, the imaging device, and the transport device, and a learning device that learns data indicating the weight of the droplets ejected onto the printing object, wherein the control device performs an ejection process that causes the printing apparatus to eject the droplets onto the printing object, and a transport process that causes the transport device to transport the printing object onto which the droplets have been ejected to a position where the image can be captured by the imaging device after the ejection process. and after the transporting process, an imaging process in which the imaging device captures an image of the droplets ejected onto the printing object, wherein the learning device comprises a memory unit and an execution unit, the memory unit stores a machine learning model capable of machine learning, and the execution unit executes a teacher data creation process in which teacher data is created as a set of image data captured in the imaging process and the data indicating the weight of the droplets captured in the image data, and a machine learning process in which the machine learning model learns the teacher data.

[0095] MS: installation surface; 10: printing device; 12: XZ drive mechanism; 30: transport device; 31: stage; 32: Y drive mechanism; 40: imaging device; 50: control device; 60: weight estimation device; 61: storage unit; 62: execution unit; PG: estimation program; LM: trained model; D: droplet; W: printing object; 100: training device

Claims

1. A droplet weight estimation device comprising a memory unit and an execution unit, wherein the memory unit stores a trained model that has been machine-learned to input image data of droplets ejected onto a printing target and output an estimated value indicating the weight of the droplets captured in the image data, and the execution unit is capable of executing an acquisition process that acquires the image data of the droplets, and an estimation process that inputs the acquired image data into the trained model and outputs the estimated value indicating the weight of the droplets captured in the image data.

2. A droplet weight estimation device comprising a memory unit and an execution unit, wherein the memory unit stores relationship definition data that defines the relationship between two or more input parameters extracted from image data of droplets ejected onto a printing object and an estimated value indicating the weight of the droplets imaged in the image data, one of the input parameters being a shading value that represents the shade of color of the droplets reflected in the image data, and the execution unit is capable of executing an acquisition process that acquires each of the input parameters, and an estimation process that outputs the estimated value indicating the weight of the droplets imaged in the image data based on the acquired input parameters and the relationship definition data.

3. The droplet weight estimation device described in claim 2, wherein the relationship specification data is a trained model that has been machine-learned in advance to output the estimated value using two or more of the input parameters as input, and the execution unit outputs the estimated value by inputting the input parameters into the trained model in the estimation process.

4. A droplet weight estimation device according to any one of claims 1 to 3, wherein the image data has color data for two or more colors whose peak wavelengths differ by 100 nm or more.

5. The droplet weight estimation device according to any one of claims 1 to 4, wherein the printing object is a nonwoven fabric having intertwined fibers or a woven fabric having woven fibers.

6. An inkjet printing system comprising: a printing device that ejects droplets onto a printing target; an imaging device that captures an image of the droplets ejected onto the printing target; a transport device that transports the printing target; a control device that controls the printing device, the imaging device, and the transport device; and a droplet weight estimation device as described in any one of claims 1 to 5, wherein the control device executes the following: an ejection process that causes the printing device to eject the droplets onto the printing target; a transport process that, after the ejection process, causes the transport device to transport the printing target onto which the droplets have been ejected to a position where the image can be captured by the imaging device; and an imaging process that, after the transport process, causes the imaging device to capture an image of the droplets ejected onto the printing target, and wherein the execution unit of the weight estimation device acquires the image data of the droplets captured in the imaging process in the acquisition process.

7. The inkjet printing system described in claim 6, further comprising an illumination device capable of irradiating the droplets ejected onto the printing object with two or more types of light having peak wavelengths differing by 100 nm or more, and the imaging device is capable of imaging with the two types of light.

8. A method for creating teacher data comprising: an ejection step of ejecting droplets onto a printing substrate; an imaging step of capturing an image of the droplets ejected onto the printing substrate with an imaging device; and a teacher data creation step of creating teacher data which is a set of image data of the droplets captured in the imaging step and data indicating the weight of the droplets captured in the image data.

9. A printing device that ejects droplets onto a printing target, an imaging device that captures an image of the droplets ejected onto the printing target, a transport device that transports the printing target, a control device that controls the printing device, the imaging device, and the transport device, and a learning device that learns data indicating a weight of the droplets ejected onto the printing target, wherein the control device executes: an ejection process that causes the printing device to eject the droplets onto the printing target, a transport process that causes the transport device to transport the printing target onto which the droplets have been ejected to a position where the image can be captured by the imaging device after the ejection process, and an imaging process that causes the imaging device to capture an image of the droplets ejected onto the printing target after the transport process, wherein the learning device includes a memory unit and an execution unit, wherein the memory unit stores a machine learning model capable of machine learning, and the execution unit executes: a teacher data creation process that creates teacher data that is a set of image data captured in the imaging process and the data indicating the weight of the droplets captured in the image data, A machine learning process for training the machine learning model to learn the training data.

Images