Method for generating a training data set for an artificial neural network, method for training an artificial neural network, device, and computer program product

Abstract

The invention relates to a method for generating (13) a training data set (1) for an artificial neural network, wherein the artificial neural network is designed to determine a second contact pressure value (7), which is associated with a printed mark (3), from a first image (4) of the printed mark (3), the method having the following steps: - providing (10) a first image (4) of at least one printed mark (3), the printed mark (3) being circular in particular; - providing (11) a first contact pressure value (6), the first contact pressure value (6) being associated with the imprint of the printed mark (3); - transforming (12) the first image (4) onto a second image (5) by means of a coordinate transformation, the coordinate transformation being designed such that the printed mark (3) detected using the first image (4) is rectangular within the second image (5) and; - generating (13) the training data set (1), the training data set (1) comprising at least the second image (5) and the corresponding first contact pressure value (6). The invention further relates to a method for training (14) an artificial neural network, to a device, and to a computer program product.

Description

[0001] 202500157

[0002] 1

[0003] Description

[0004] Method for generating a training dataset for an artificial neural network, method for training an artificial neural network, device and computer program product

[0005] The invention relates to a method according to the preamble of claim 1, a method according to the preamble of claim 6, a device according to the preamble of claim 7 and a computer program product according to the preamble of claim 13.

[0006] The development of closed-loop control systems for industrial machinery is becoming increasingly important with a view to achieving cost savings. Control loops, as known from control engineering, can already be used for the simple control of a single controlled variable.

[0007] Complex applications are conceivable where there is more than one controlled variable, and a purely control engineering approach can reach its limits. These limits can include not only performance but also increasing costs for the development and operation of a single complex control loop or a chain of multiple control loops. In such cases, sensors continuously record machine performance and transmit it to a processing unit, which comprises the control loop. The processing unit evaluates the data and, in particular, updates machine configuration data to ensure or improve product quality.

[0008] In addition to control engineering approaches, artificial neural networks and / or related technologies, collectively known as artificial intelligence (AI), can achieve similar results, but with potential savings in development and operation. One hurdle in developing AI-based solutions is the provision of training data. Even collecting training data presents challenges, which is why subsequent, more efficient handling or a potential reduction in the need for training data can be beneficial. This can lead to more efficient configuration and / or control of the respective machines, particularly printing devices.

[0009] 2

[0010] The present invention is based on the objective of controlling a printing device more efficiently.

[0011] The problem is solved by a method with the features of independent claim 1, and by a device with the features of independent claim 7. Advantageous embodiments and further developments of the invention are specified in the dependent claims.

[0012] The inventive method for generating a training data set for an artificial neural network, wherein the artificial neural network is designed to determine a second contact pressure value associated with a print mark from a first image of the print mark, comprises the following steps:

[0013] - Providing a first image of at least one print mark, wherein the print mark is in particular circular form;

[0014] - Providing an initial contact pressure value, wherein the initial contact pressure value is associated with the imprinting of the print mark;

[0015] - Transforming the first image onto a second image by means of a coordinate transformation, wherein the coordinate transformation is designed such that the print mark captured by means of the first image is rectangular within the second image and;

[0016] - Generating the training dataset, wherein the training dataset includes at least the second image and the associated first contact pressure value.

[0017] The training data set can include, in addition to the second image and the associated first contact pressure value, a surface temperature associated with the second image, particularly at the time of a printing process. The training data set can also include a counter indicating how many print marks have already been printed at a given point in time, whereby the counter of the printing device can be reset to zero during commissioning, tool changes, and / or maintenance.

[0018] The association can be understood as follows: particularly during a printing process, the respective contact pressure, which depends on the contact pressure value, is related to the image of a printed picture. The printed picture can be a result of applying this contact pressure. 202500157

[0019] 3

[0020] The artificial neural network can exhibit multimodal capability, in particular by using input and / or output parameters from the vibroacoustics of the printing press and / or image data and / or temperature data and / or electrical properties of a workpiece for inference by the artificial neural network.

[0021] Print marks can be colored, particularly cyan, magenta, yellow, in shades of gray, and / or black and white. Print marks can serve as a reference point for print quality in a printing process. They can be designed to be invisible to humans when exposed to visible light, which can range in wavelength from 780 nm to 380 nm. These print marks can also be made visible to humans by exposure to ultraviolet light. This can be advantageous for concealing print marks in a printed product under daylight conditions.

[0022] The print quality of printed marks can be determined, in particular, by the uniformity of ink application and / or edge sharpness. Achieving a required color depth can be an analytical criterion for the uniformity of ink application.

[0023] The contact pressure value indicates the amount of pressure with which a printing tool, especially a punch, was pressed against a surface to be printed. The contact pressure can also be in the form of tensile pressure, as occurs, for example, in a tensile-compressive forming process and / or in deep drawing.

[0024] Coordinate transformation allows the calculation of new coordinates for a point within a coordinate system. A coordinate transformation between Cartesian and polar coordinates can be particularly advantageous. Coordinate transformations include, in particular, scaling, shearing, and / or rotating points and / or polygons consisting of multiple points.

[0025] The present invention provides a method for advantageously generating the training dataset for an artificial neural network. This is because the coordinate transformation of the first image to the second image yields a rectangular image. The transformation, in particular of a circular print mark image, to a rectangular image advantageously enables an image with a smaller number of pixels and no information.

[0026] 4

[0027] In other words, a rectangular image makes better use of the pixel area defined by common digital image formats than a circular image. This better utilization of the pixel area reduces the number of empty pixels and, advantageously, results in lower storage requirements. Furthermore, the transformation, particularly of the rectangular print mark, to the rectangular image makes it easier for the artificial neural network to train the print mark's detectable features. This is because the features in the rectangular image are essentially straight edges. In contrast, in the original circular image, the same features can be circular arcs, which are more complex for an artificial neural network to detect.This complexity is due to the larger number of parameters required to describe a curve compared to a straight line.

[0028] The inventive method for training the artificial neural network is characterized in that the artificial neural network is trained with the training data set.

[0029] This advantageously allows for more efficient training of the neural network using the training dataset. This is because reducing the number of pixels without information and thereby increasing the proportion of pixels containing information results in more efficient training of the artificial neural network.

[0030] The inventive method for generating training data offers similar, equivalent and equivalent advantages.

[0031] The device according to the invention comprises a printing device for printing print marks, a computing unit, a control unit and / or a detection unit, wherein the detection unit is configured to detect a first image of a print mark and provide it to the computing unit.

[0032] characterized by the fact that,

[0033] - the computing unit is designed to transform the first image onto a second image by means of a coordinate transformation, wherein the coordinate transformation is such that 202500157

[0034] 5

[0035] The design is such that the print mark captured by means of the first image is rectangular within the second image;

[0036] - the computing unit comprises an artificial neural network, wherein the artificial neural network is configured to use at least a second image as an input value and to determine a second contact pressure value as an output value; and

[0037] - the control unit is designed to control the contact pressure of the pressure device depending on the determined second contact pressure value.

[0038] The control unit can be, in particular, a programmable logic controller (PLC) and can, in addition to a control function, also provide a regulation function. In other words, the control unit can also be designed as a regulation unit.

[0039] The inventive method for generating training data offers similar, equivalent and equivalent advantages.

[0040] The printing device can be designed for printing printed media and / or for producing imprints on packaging.

[0041] Furthermore, the printing device can be designed as a pressing device that exerts a contact pressure on a surface, particularly a metallic one. In this case, the printing marks are equivalent to the resulting impressions. Here, the impressions are formed by pressing down a punch.

[0042] Furthermore, the pressure device can be designed as a deep-drawing device, whereby instead of a contact pressure a tensile pressure occurs, as occurs in a tensile-compression forming process and / or in deep drawing.

[0043] The processing unit can be housed in a common enclosure with the control unit and / or mounted on the printing device. Alternatively, the processing unit can be housed in a control cabinet. The processing unit can also be implemented as a virtual processing unit in the cloud. This advantageously eliminates the need for a local processing unit, provided the required latency between the control unit and the virtual processing unit is met.

[0044] The detection unit can be, in particular, a camera unit that provides optical images (photos) and / or thermal images. The detection unit can be 202500157

[0045] 6

[0046] furthermore, it may be designed as a time-of-flight camera unit and / or a LiDAR scanner unit, which captures and / or provides a height profile of the surface of the print mark.

[0047] The detection unit can be configured for imaging using X-rays or ultrasound. This advantageously allows for the provision of images of printed conductor structures, particularly for RFID technology, made of metallic or ceramic conductors or printed marks encompassing them. The use of X-rays or ultrasound can be especially beneficial for multi-layered printed products where optical imaging only captures the surface.

[0048] Imaging using X-rays or ultrasound can be designed as layer-by-layer imaging and can be performed either as an end-of-line inspection after layer-by-layer printing or already in situ between the individual print runs.

[0049] This allows, advantageously, the depth and / or height of a printed structure to be measured in addition to its width. The artificial neural network can then use this width, depth, and / or height for inference and determine the contact pressure.

[0050] The invention provides a device for the advantageous control of the contact pressure of a printing device. This is because, through inference by the artificial neural network, it is possible to deduce the contact pressure applied by the printing device, thereby eliminating the need for a more costly and time-consuming interpretation of the first print mark image by the machine operator.

[0051] The computer program product according to the invention comprises instructions which, when the program is executed by a computing unit, in particular a computer, cause it to execute the inventive method for generating a training data set for an artificial neural network and / or the inventive method for training an artificial neural network and / or steps of both methods.

[0052] The inventive method for generating training data offers similar, equivalent and equivalent advantages.

[0053] According to an advantageous embodiment of the invention, the coordinate transformation is carried out using polar coordinates.

[0054] 7

[0055] This advantageously makes it possible to transform circular and / or circular images with fewer empty pixels into a rectangular image. This, in turn, allows the second image to be created with a lower storage requirement.

[0056] The coordinate transformation using polar coordinates can be particularly advantageous for circular print marks, where the radius of the polar coordinates extends along the radial extent of the circular print mark.

[0057] For circular and / or circle-like and / or other N-sided printed marks, for the coordinate transformation a centroid of the printed mark can be located on a coordinate origin of a polar coordinate system and a line segment of a point and / or area of ​​the printed mark furthest from the centroid can be used as the radius for the coordinate transformation.

[0058] According to an advantageous embodiment of the invention, the training data set comprises a plurality of tuples and / or pairs of values, each consisting of the second image and the first contact pressure value.

[0059] In other words, each tuple comprises a second image and an associated first contact pressure.

[0060] This advantageously allows the respective second image and the first contact pressure to be related. The respective pair of values ​​and / or tuple links the two values. Furthermore, higher-dimensional n-tuples can be provided for the training and inference of multimodal artificial neural networks.

[0061] In an advantageous embodiment of the invention, the print mark captured by means of the first image is rectangular or rectangular within the second image.

[0062] This advantageously makes it possible to save the second image in all common computer-based image formats, without requiring empty pixels to fill the image area. 202500157

[0063] 8

[0064] According to an advantageous embodiment of the invention, the printed mark is circular or shaped as a circle or as a convex N-gon.

[0065] This advantageously makes it possible to transform the image of the first print mark from polar coordinates to Cartesian coordinates into a rectangular image with even fewer empty pixels. Thus, the second image can advantageously be generated with less storage space required.

[0066] According to an advantageous embodiment of the invention, data points taken from the first image and / or second image are provided and the training data set is generated, wherein the training data set comprises at least the first image and / or the second image and / or the data points and / or the associated first contact pressure value.

[0067] The data points extracted from the images can be individual numerical values, which can be assigned in particular to the individual pixel and / or an image area consisting of several pixels and / or to the entire image.

[0068] This advantageously allows only the data points extracted from the images and the associated contact pressure values ​​to be included in the training dataset, instead of the first and / or second images. The advantage here is that the training dataset can contain significantly less data and requires considerably less computing power for training with such a dataset.

[0069] According to an advantageous embodiment of the invention, the provided data points comprise at least a first mean color value of the respective image and / or at least a second mean color value for at least the individual image area of ​​the respective image and / or a threshold value for the respective color value.

[0070] The color value for each pixel, and in particular its color intensity, can be represented using color models such as RGB and / or CMYK. Cameras typically capture images in the RGB color model, and printing devices typically use the CMYK color model. 202500157

[0071] 9

[0072] This advantageously makes it possible to adequately represent characteristic properties of the images, in particular color gradients and / or edge gradients, in the training data set despite the possible reduction in the size of the training data set.

[0073] Furthermore, a conversion between the RGB color model and the CMYK color model can be performed.

[0074] This makes it advantageously possible, especially with the CMYK color model, to encode the color intensity in a channel.

[0075] According to an advantageous embodiment of the invention, the detection unit is configured to detect a plurality of print marks by means of the first image, wherein the processing unit is configured to process the first image by means of segmentation, so that at least one of the detected print marks can be provided in a new first image.

[0076] Image segmentation is a subfield of image processing. Here, adjacent pixels are grouped into conceptually related regions based on a homogeneity criterion. In other words, the initial image is segmented into individual print marks. Furthermore, if defined patterns, especially circles, are present, the segmentation can be implemented as pattern recognition.

[0077] This advantageously makes it possible to provide multiple new first images by capturing a single first image. It is advantageously necessary to capture only exactly one first image.

[0078] In an advantageous further development of the invention, the computing unit is configured to determine a further contact pressure value from at least two contact pressure values ​​determined by the artificial neural network.

[0079] This advantageously eliminates the need for a more time-consuming and costly evaluation, especially a manual evaluation, of each contact pressure value.

[0080] According to an advantageous embodiment, the processing unit is configured to determine the further contact pressure value as an average of at least two contact pressure values ​​determined by the artificial neural network. 202500157

[0081] 10

[0082] The average can also be expressed as a mean. In particular, the mean can be a harmonic mean, geometric mean, arithmetic mean, weighted mean, root mean square, and / or a Gastwirt-Cohen mean.

[0083] This advantageously makes it possible to provide plausibility checks for multiple contact pressure values ​​using a further (combined) contact pressure value. This is because, in particular, a contact pressure value can be determined for each new first image segmented from a first image. Furthermore, since the two contact pressure values ​​are related through their common first image, a larger difference between at least two determined contact pressure values ​​can be visualized on average and advantageously indicate a lack of plausibility.

[0084] In an advantageous embodiment of the invention, the control unit is designed to compare the second contact pressure value and / or the further contact pressure value with at least one threshold value and to control the contact pressure of the printing device depending on this.

[0085] This makes it advantageously possible to automate the control of the contact pressure, whereby this control is faster than manual control and this evaluation can take place during the ongoing printing process.

[0086] In other words, monitoring and / or manual recalibration of the contact pressure during the printing process, or even an interruption between two printing processes, is advantageously eliminated.

[0087] According to an advantageous embodiment, the control unit is designed to trigger an increase in the contact pressure if the threshold value for the contact pressure is undershot and / or to reduce the contact pressure if the threshold value is exceeded.

[0088] This advantageously makes it possible to produce a print result with essentially constant quality. Furthermore, it is advantageously possible to increase quality by changing the threshold value depending on new quality characteristics. A combination of several threshold values ​​that can be exceeded and fallen below also advantageously allows for the definition of value ranges.

[0089] 11

[0090] Further advantages, features, and details of the invention will become apparent from the exemplary embodiments described below and from the drawings. These show, schematically:

[0091] Figure 1 shows a representation of a data processing pipeline comprising the generation of a training dataset, the training of an artificial neural network, and the execution of the artificial neural network; and

[0092] Figure 2 shows a block diagram illustrating the data flows between a printing device, a capture unit, a computing unit and a control unit.

[0093] Similar, equivalent or equivalent elements may be provided with the same reference symbols in one or more of the figures.

[0094] Figure 1 schematically shows the data processing pipeline with a provision 10 of the first image 4 of the print mark 3. This first image 4 is subjected to a coordinate transformation 12, whereby a second image 5 of the print mark 4 is generated as a result.

[0095] The generation of training data set 13 is achieved by combining at least one initial contact pressure value 6 provided in step 11 with the associated second image 5 of the print mark 4. This combination can be achieved by forming value pairs and / or tuples from the second image 5 and the initial contact pressure value 6. In addition to the contact pressure value 6 and the second image 5, further information can also be combined. In particular, process temperature and / or humidity and / or the aging of a printing die or, more generally, a tool head, are additionally taken into account.

[0096] Training dataset 1 serves for training the artificial neural network. This training dataset can be enriched with synthetic data, particularly from a simulation. Furthermore, it is possible to remove a portion of training dataset 1 and provide it in a separate dataset. This separate dataset can be used for the subsequent evaluation of the artificial neural network. A preferred distribution is 20% evaluation data and 80% training data. Alternatively, a distribution of 70% training data and 15% training data is also possible.

[0097] 12

[0098] Validation data and 15% test data are possible, whereby the validation data are used to assess the quality of the artificial neural network after a training step.

[0099] Version 15 of the artificial neural network provides that at least one second image 5 transformed in step 12 serves as input for the artificial neural network, whereby the first image 4 of the print mark 3 is captured in step 18 and again serves as input for the transformation in step 12.

[0100] The execution 15 of the artificial neural network, taking into account the respective input value, leads to an inference, whereby at least one second contact pressure value is provided as an output value. Finally, the contact pressure of a pressure device is controlled 17 by means of the second contact pressure value.

[0101] Figure 2 schematically shows a block diagram representing the data flows between the printing device 2, the acquisition unit 19, the computing unit 9 and the control unit 8.

[0102] The printing device 2 provides sensor data 20 for the control unit 8.

[0103] The control unit can periodically and / or depending on the sensor data 20 trigger the acquisition 10a of the first image of the print mark in step 21. If the acquisition unit 19 is a camera unit, it can be aligned with print marks passing through the printing device 2.

[0104] Once the capture unit 19 has detected a print mark, it provides the first image to the processing unit 9 in step 18. Provisioning in step 18 can be done using the second image, particularly if the capture unit 19 is configured for corresponding preprocessing, especially the transformation of images.

[0105] The result of inference 16 of the artificial neural network depends on the second print mark image, which is used as the input value. For this purpose, the first print mark image is provided in step 18 and transformed into the second print mark image by processing unit 9. If step 18 already provides the second image, the transformation by processing unit 9 can be omitted.

[0106] 13

[0107] The result provided by inference 16, or rather the output value of the artificial neural network, is the second contact pressure value. This second contact pressure value is used by the control unit as a manipulated variable for controlling 17 the contact pressure of the pressure device 2.

[0108] Although the invention has been further illustrated and described in detail by the preferred embodiments, the invention is not limited by the disclosed examples, nor can other variations be derived from them by a person skilled in the art without departing from the scope of protection of the invention. 202500157

[0109] 14

[0110] Reference symbol list

[0111] 1 training data set

[0112] 2 Printing device

[0113] 3 Print mark

[0114] 4 First image of the print mark

[0115] 5 Second image of the printed mark

[0116] 6 First contact pressure value

[0117] 7 Second pressure value

[0118] 8 Control unit

[0119] 9 Calculation unit

[0120] 10. Providing the first image of the print mark

[0121] 10a Capturing the first image of the print mark

[0122] 11. Providing the initial contact pressure value

[0123] 12. Transformation of the first image to the second image using coordinate transformation

[0124] 13. Generating the training dataset

[0125] 14 Training the artificial neural network

[0126] 15. Implementation of the artificial neural network

[0127] 16 Inference / Determining and providing the second contact pressure value 17 Controlling the contact pressure of the printing device using the second contact pressure value

[0128] 18. Capturing and providing the first print mark image

[0129] 19 recording units

[0130] 20 Provision of sensor data

[0131] 21. Triggering the capture of the first image

Claims

202500157 15 Patent claims 1. Method for generating (13) a training data set (1) for an artificial neural network, wherein the artificial neural network is designed to determine a second contact pressure value (7) associated with a print mark (3) from a first image (4) of the print mark (3), comprising the following steps: - Providing (10) a first image (4) of at least one print mark (3), wherein the print mark (3) is in particular circular in form; - Providing (11) a first pressure value (6), wherein the first pressure value (6) is associated with the imprinting of the print mark (3); - Transforming (12) the first image (4) onto a second image (5) by means of a coordinate transformation, wherein the coordinate transformation is designed such that the print mark (3) captured by means of the first image (4) is rectangular within the second image (5) and; - Generating (13) the training data set (1), wherein the training data set (1) includes at least the second image (5) and the associated first contact pressure value (6).

2. Method according to claim 1, characterized in that the coordinate transformation is carried out using polar coordinates.

3. Method according to one of the preceding claims, characterized in that the training data set (1) comprises a plurality of tuples and / or pairs of values ​​consisting of the second image (5) and the first contact pressure value (6).

4. Method according to one of the preceding claims, characterized in that the print mark (3) captured by means of the first image (4) is rectangular or rectangular within the second image (5).

5. Method according to one of the preceding claims, characterized in that the print mark (3) is circular or as a circle or as a convex N-gon.

6. A method according to one of the preceding claims, characterized in that data points extracted from the first image (4) and / or second image (5) are provided and the training data set (1) is generated, wherein the training data set (1) comprises at least the first image (4) and / or the second image (5) and / or the data points and / or the associated first contact pressure value (6). 16 7. Method according to claim 6, characterized in that the provided data points comprise at least one first mean color value of the respective image (4, 5) and / or at least one second mean color value for at least one image area of ​​the respective image (4, 5) and / or a threshold value for the respective color value.

8. Method for training (14) an artificial neural network, characterized in that the artificial neural network is trained (14) with the training data set (1) according to one of the preceding claims.

9. Device comprising a printing device (2) for printing print marks (3), a computing unit (9), a control unit (8) and / or a detection unit (19), wherein the detection unit (19) is configured to detect a first image (4) of a print mark (3) and provide it to the computing unit (9), characterized by the fact that, - the computing unit (9) is configured to transform the first image (4) onto a second image (5) by means of a coordinate transformation (12), wherein the coordinate transformation is configured such that the print mark (3) captured by means of the first image (4) is rectangular within the second image (5); - the computing unit (9) comprises an artificial neural network, wherein the artificial neural network is configured to use at least one second image (5) as an input value and to determine and / or provide a second contact pressure value (7) as an output value; and - the control unit (8) is designed to control the contact pressure of the pressure device (2) depending on the determined second contact pressure value (7) (17).

10. Device according to claim 9, characterized in that the detection unit (19) is configured to detect a plurality of print marks (3) by means of the first image (4), wherein the processing unit (9) is configured to process the first image (4) by means of segmentation, such that at least one of the detected print marks (3) is available in a new first image (4). 17 11. Device according to claim 9 or 10, characterized in that the computing unit (9) is configured to determine a further contact pressure value from at least two contact pressure values ​​determined by the artificial neural network.

12. Device according to claim 11, characterized in that the computing unit (9) is configured to determine the further contact pressure value as an average of at least two contact pressure values ​​determined by the artificial neural network.

13. Device according to one of claims 9 to 12, characterized in that the control unit (8) is configured to compare the second contact pressure value (7) and / or the further contact pressure value with at least one threshold value and to control the contact pressure of the pressure device (2) depending on this (17).

14. Device according to claim 13, characterized in that the control unit (8) is configured to trigger an increase in the contact pressure when the threshold value for the contact pressure is undershot and / or to reduce the contact pressure when the threshold value is exceeded.

15. Computer program product comprising instructions which, when the program is executed by a computing unit (9), in particular a computer, cause it to execute a method and / or steps of the method according to any one of claims 1 to 5.

Images