Method of generating a training data set, method of training an artificial neural network, device

Abstract

A method for generating a training dataset, a method for training an artificial neural network, and an apparatus are disclosed. A method for generating a training dataset for training an artificial neural network is proposed, comprising the following steps: - providing a first image; - providing first parameters, wherein the first parameters are associated with the first image; - providing at least one second image as a segment of the first image, wherein the segment is determined by element-wise segmentation of the first image; - generating the training dataset, wherein the training dataset contains at least the second image and the associated first parameters. The invention also relates to a method for training an artificial neural network, an apparatus, and a computer program product.

Description

Technical Field

[0001] The present invention relates to a method according to the preamble of claim 1, a method according to the preamble of claim 9, an apparatus according to the preamble of claim 10, and a computer program product according to the preamble of claim 14. Summary of the Invention

[0002] The purpose of this invention is to control production machines more efficiently.

[0003] This objective is achieved by a method having the features of independent claim 1, a method having the features of independent claim 9, an apparatus having the features of independent claim 10, and a computer program product having the features of independent claim 14. Advantageous designs and extensions of the invention are described in the dependent claims.

[0004] The method according to the present invention for generating a training dataset for training an artificial neural network includes the following steps: - Provide the first image; - Provide a first parameter, wherein the first parameter is associated with a first image; - Provide at least one second image as a fragment of the first image, wherein the fragment is determined by element-wise segmentation of the first image; - Generate the training dataset, which contains at least the second image and the associated first parameters. Its features are, The element-wise segmentation of the first image is performed progressively, wherein image regions with fixed height and width are covered by pixels of the first image in a pixel-wise and / or pixel-wise and / or row-wise and / or column-wise manner, and the covered regions in the respective steps are provided as segments, wherein, for each segment: - Determine uniformity, especially the uniformity of color values ​​per pixel; and / or - Determine the color intensity, especially the average sum of the normalized sums of all RGB values; and / or - Perform edge detection, specifically to determine the strength of the edges. Specifically, based on the determined uniformity and / or color intensity and / or the presence of edges, progressively provided segments are selected as second images, and these segments are used to generate the training dataset.

[0005] In addition to the second image and the associated first parameter, the training dataset may also include other parameters, where the first parameter and / or these other parameters may include the surface temperature associated with the second image, particularly the surface temperature at the moment of the production step. Counter states, indicating the number of times the production step has occurred, may also be used as parameters.

[0006] Artificial neural networks can be used as architectures such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Multi-Layer Perzeptron (MLP), or Transformer models.

[0007] These images can be data structures, especially arrays and / or tables.

[0008] Color intensity describes the purity or intensity of a color. In other words, color intensity indicates how much a color deviates from its pure grayscale value. Higher saturation makes the color appear purer and more vibrant. Taking the RGB color encoding of image points as an example, color intensity can be calculated using the following formula: MAX is the maximum value of the RGB color point, MIN is the minimum value of the RGB color point, and S is in the range of [0, 1]. Here, S = 0 is the grayscale value of no color, and S = 1 represents the maximum color saturation.

[0009] Edge detection can include methods for identifying significant intensity changes in an image. Here, an edge can be mathematically defined as a local discontinuity in an image function. One edge detection method is the Canny algorithm.

[0010] Uniformity can be a texture metric, specifically referring to the uniform distribution of color values ​​and / or color intensities at points in an image.

[0011] The present invention describes a method for advantageously providing a training dataset for training an artificial neural network. This is because the training dataset provided according to the present invention eliminates the need to record the complete first image, and only a smaller second image can be recorded. Furthermore, only the second image that meets the criteria of uniformity and / or color intensity and / or the presence or absence of edges is recorded in the training dataset. Thus, portions of the corresponding images can be recorded in the training dataset in a highly selective manner, making the training dataset advantageously enable more efficient training of the artificial neural network.

[0012] The method for training an artificial neural network according to the present invention is characterized in that the artificial neural network is trained using the training dataset.

[0013] This training can be provided using supervised learning methods.

[0014] The method for generating training data according to the present invention produces similar, equivalent, and equivalent advantages.

[0015] The apparatus according to the present invention includes a production machine, a computing unit, a control unit, and / or a data acquisition unit, wherein the detection unit is designed to acquire a first image and provide the first image to the computing unit. Its features are, - The computing unit includes an artificial neural network, wherein the artificial neural network is designed to: use at least one first image as input value and provide second parameters as output value; and - The control unit is designed to control the control parameters based on the determined second parameter.

[0016] Production machines can be printers and / or conveyor systems (Fördertechnik).

[0017] The computing unit can be an industrial computer and / or an edge device, or it can be a virtual computing unit in the cloud.

[0018] The control unit can be a programmable logic controller.

[0019] The acquisition unit can be an optical camera and / or an ultrasonic sensor and / or a laser scanner, especially based on the principles of time-of-flight (ToF) or light-detection-and-ranging (Lidar).

[0020] The method for generating training data according to the present invention produces similar, equivalent, and equivalent advantages.

[0021] The computer program product according to the invention includes instructions that, when executed by a computing unit, in particular a computer, cause the computing unit, in particular the computer, to perform the method and / or the steps of the method according to any one of claims 1 to 9.

[0022] The method for generating training data according to the present invention produces similar, equivalent, and equivalent advantages.

[0023] According to an advantageous embodiment of the invention, the first image is segmented element-wise by dividing it into square segments, wherein these square segments are provided as the second image.

[0024] Advantageously, these segments can be provided even faster because the segmentation is performed in fewer steps, especially fewer steps than pixel-by-pixel segmentation.

[0025] In an advantageous extension of the invention, the second image is provided at a pixel resolution of 224×224 to 227×227.

[0026] Pixel resolution describes the number of image points (pixels) in a digital image in both the horizontal and vertical directions.

[0027] Therefore, artificial neural networks that have already been trained using images at these pixel resolutions can be further trained more efficiently by taking advantage of the provided training data.

[0028] According to an advantageous design of the invention, when selecting a second image from the determined segments, segments with the highest color intensity and / or the highest uniformity and / or the most edges are selected based on at least one threshold, and these segments are used to generate the training dataset.

[0029] Advantageously, it is possible to define a threshold that can be adjusted to respond to changes in the parameter.

[0030] In an advantageous extension of the invention, a fragment containing at least a portion of a pattern (especially a sign and / or symbol and / or title) is selected as a second image, and these fragments are used to generate the training dataset.

[0031] From an image processing perspective, patterns, symbols, logos, or titles can be recurring patterns and / or image characteristics.

[0032] Therefore, the recognition of recurring patterns can be advantageously used as a strength in image processing. In other words, knowledge about recurring patterns is used to simplify, and especially retrain, the process. This is because some images, despite their differences, share these patterns. Consequently, training on these commonalities can advantageously simplify the retraining of artificial neural networks, or allow existing networks to be used directly.

[0033] According to an advantageous design of the invention, segments provided stepwise are selected as second images by means of a quality score, and these segments are used to generate the training dataset, the quality score depending on the determined uniformity and / or color intensity and / or the presence of edges.

[0034] This quality score can represent a threshold, at which images can be selected. To integrate various metrics (uniformity, color intensity, and the presence of edges), this quality score can be a weighted sum.

[0035] This weighted sum can be adjusted by changing the weights to give different degrees of weight to each indicator.

[0036] Therefore, it is advantageous to respond to changes in parameters by means of changes in weights, especially by means of dynamic changes. This is particularly advantageous when consistent quality needs to be achieved during the operation of production machines.

[0037] In an advantageous extension of the invention, the training dataset includes location information of the second image, wherein the location information relates to a position within the first image, and wherein the location information is provided, in particular, row-by-row and / or column-by-column.

[0038] This location information can indicate the position of the second image within the first image. Furthermore, this location information can be relative to the camera's position, allowing the camera to be aligned accordingly.

[0039] Therefore, when acquiring the first image, this location information allows for the advantageous acquisition of only the selected area, thereby reducing the amount of data. This is because the location information allows for the exclusion of other areas from the acquisition process.

[0040] According to an advantageous design of the present invention, the first parameter is the contact pressure value.

[0041] Contact pressure values ​​can represent the contact pressure of the printing plate and / or die, especially the contact pressure of the printing plate and / or die of a printer and / or printing press.

[0042] This allows for more efficient control of electromechanical equipment that can be subjected to variable pressure.

[0043] In an advantageous extension of the invention, the computing unit provides a dataset that links object features that can be processed by an artificial neural network to the locations of those object features, wherein the acquisition unit can be aligned by means of the computing unit according to the corresponding locations, so that the acquisition unit acquires the corresponding object features in the first image when acquiring the first image.

[0044] The key features of an object can be specific metrics (uniformity, color intensity, and the presence of edges). Object features can be assigned locations relative to the acquisition unit and / or a higher-level reference frame, particularly the surrounding space (rooms, halls, or buildings).

[0045] The alignment capability of the acquisition unit can be designed such that the acquisition unit is aligned by computation and based on its position in the dataset. One-time alignment of the acquisition unit is also a possible design, which allows acquisition from all possible positions within the dataset.

[0046] Therefore, advantageously, the acquisition of the first image can be provided with less data. This is because: regions with object features can be preferably acquired.

[0047] According to an advantageous design of the invention, the computing unit provides a dataset that links object features that can be processed by an artificial neural network with the locations of these object features in a first image to be acquired, such that the acquisition unit only acquires regions of the first image that have object features present in the dataset.

[0048] This advantageously reduces the amount of data acquired when capturing the first image. This is because the computing unit uses virtual cropping to remove regions lacking object features and only captures regions of the first image that possess object features present in the dataset.

[0049] In an advantageous extension of the invention, the production machine is designed as a printing apparatus, wherein the second parameter is a contact pressure value, and the control parameter is designed as the contact pressure, which can be controlled by means of the contact pressure value.

[0050] The printing device can be a printer for printed materials.

[0051] This allows for more efficient control over the printer. Attached Figure Description

[0052] Other advantages, features, and details of the invention will become apparent from the embodiments described below and from the accompanying drawings. Hereinafter: Figure 1 The data pipeline is illustrated schematically as a block diagram, comprising: generating training data, using the training data to train an artificial neural network, and using the artificial neural network to control control parameters; and Figure 2 The device is illustrated in a block diagram and includes a production machine, a computing unit, a control unit, and a data acquisition unit. Detailed Implementation

[0053] Figure 1 A schematic diagram of a data pipeline as an example is shown in block diagram form. At the core of this data pipeline is training dataset 1, which provides data 6 and 9, and continues to use training dataset 1 to train an artificial neural network 10.

[0054] Data is provided through two different methods: providing 5 (first parameter 6) and providing 7 (first image 2).

[0055] The first parameter (6) provided (5) can include sensor data such as pressure, temperature, speed, rotational speed, etc. Another possibility is to provide machine configuration data, especially the model and / or the date of the last maintenance.

[0056] Providing the first image 2 can be achieved by means of an acquisition unit, especially an optical camera.

[0057] The element-wise segmentation 8 of the first image 2 can be performed as follows: an image region with a fixed height and width is covered by pixels of the first image 2 in a pixel-wise and / or pixel-wise group and / or row-wise and / or column-wise manner, and the covered region in the corresponding step is provided as segment 3.

[0058] Segment 3 was selected as the second image 4 using at least one of the following metrics: 1. Uniformity: Determines the uniformity of the distribution of color values ​​and / or color intensities of image points.

[0059] 2. Color Intensity: Determine the intensity / purity of color pixel by pixel, especially the average of the normalized sums of the RGB values ​​of the image points.

[0060] 3. Existence of edges: Edge detection algorithms such as the Canny algorithm can be used to extract edges from the fragment. Then, the number of edges and / or the length of the edges can be determined.

[0061] To provide the second image 4, suitable segments 3 are selected. This selection can be performed through evaluation logic, where these metrics are compared individually to corresponding thresholds, and segments are selected or not selected based on whether they exceed or fall below the threshold. In particular, it is feasible to satisfy only one metric, at least one metric, or all metrics, especially other logical links that exceed the threshold. Alternatively, a single threshold can be used, specifically as a quality score, which is determined using a weighted sum of all three metrics. In this case, a value is calculated for each metric, weighted using prescribed weights, and the sum is applied to all metrics. The result is a weighted sum that can be compared to the threshold, and as a result, the corresponding segment is provided as the second image.

[0062] Training dataset 1 is generated based on these two sources 6 and 9. In this case, the first parameter 5 is linked to at least one second image 4, thus, these first parameters and the second image are associated with each other. The same association exists between the first parameter 5 and the first image 2, whereby this association transfers to the second image 4. The association can be understood as follows: the first parameter 5 was acquired at a time similar to that of the first image 2; and / or the control performed by means of the first parameter 5, especially the control of the machine, produces the result acquired in image 2.

[0063] An artificial neural network can be trained using a training dataset 1 and a supervised learning method. This training 10 can be retraining, where an already trained artificial neural network is further trained. Retraining aims to: retrain the artificial neural network for new, specific use cases. Advantageously, this retraining is less time-consuming and / or less costly compared to completely training a new artificial neural network.

[0064] Inference 11 is performed using an artificial neural network by providing a first image 2 as input, wherein the first image 2 does not need to be the same first image 2 from the training dataset. Instead, it can be another first image 2 with similar features, especially with similar metrics. The artificial neural network provides 12 second parameters as output values.

[0065] According to this embodiment, the provided second parameter 12 can be used to control the control parameter 13.

[0066] An example of this control 13 is: controlling the printer's contact pressure according to a second parameter, which can in particular be a contact pressure value as a target parameter.

[0067] Another example of this control 13 is: controlling the temperature and / or baking time of an automatic baking machine according to a second parameter, which can in particular be a temperature value and / or baking time value as a target parameter.

[0068] Figure 2 A block diagram is schematically shown, which reproduces the data flow between control unit 14, acquisition unit 16, production machine 17 and computing unit 15.

[0069] Control unit 14 can periodically and / or trigger the acquisition 18 of the first image using data stream 20 based on the acquisition of the first parameter via data stream 23. If acquisition unit 16 is a camera unit, the camera unit can be aimed at the product passing through production machine 17. If acquisition unit 16 has acquired the first image, it provides the first image to computing unit 15 via data stream 19.

[0070] According to this embodiment, the computing unit 15 is designed to provide an artificial neural network and supply the same first image as input value to the artificial neural network. Then, through inference 11, the artificial neural network provides 21 second parameters to the control unit 14.

[0071] According to this embodiment, the control unit 14 is designed to control the control parameters 22 by means of a second parameter. This control 22 can be performed by connecting to the production machine 17 via common communication protocols (especially Modbus, CAN bus, OPC-UA, MQTT, etc.).

[0072] In one or more of these figures, similar, equivalent, and related elements may be provided with the same reference numerals.

[0073] Although the present invention has been further illustrated and described in detail through preferred embodiments, the present invention is not limited to the disclosed examples, or other variations can be derived by those skilled in the art without departing from the scope of protection of the present invention.

[0074] List of reference numerals 1. Training Dataset 2 First Image 3 segments 4 Second Image 5 First parameter 6. Provide the first parameter 7. Provide the first image 8. Perform element-wise segmentation on the first image. 9. Provide a second image 10 Training Artificial Neural Networks 11. Reasoning using artificial neural networks 12. Provide a second parameter 13. Controlling the control parameters by means of a second parameter. 14 Control Unit 15 Computing Units 16 Acquisition Units 17. Production machinery 18. Acquire the first image. 19. Provide a second image. 20 Trigger data collection 21. Provide a second parameter 22. Controlling the control parameters using the second parameter. 23. Provide the first parameter.

Claims

1. A method for generating a training dataset for training an artificial neural network, the method comprising the following steps: - Provide the first image; - Provide the first parameter, where, The first parameter is associated with the first image; - Provide at least one second image as a fragment of the first image, wherein the fragment is determined by element-wise segmentation of the first image; - Generate the training dataset, wherein the training dataset contains at least the second image and the associated first parameters. Its features are, The element-wise segmentation of the first image is performed progressively, wherein image regions with fixed height and width are covered by pixels of the first image in a pixel-wise and / or pixel-wise and / or row-wise and / or column-wise manner, and the covered regions in the corresponding steps are provided as segments, wherein, for each segment: - Determine uniformity, especially the uniformity of color values ​​per pixel; and / or - Determine the color intensity, especially the average sum of the normalized sums of all RGB values; and / or - Perform edge detection, specifically to determine the strength of the edges. Specifically, based on the determined uniformity and / or color intensity and / or the presence or absence of edges, progressively provided segments are selected as second images, and these segments are used to generate the training dataset.

2. The method according to claim 1, characterized in that, The first image is segmented element-wise by dividing it into square segments, which are then provided as the second image.

3. The method according to any one of the preceding claims, characterized in that, The second image is provided at a pixel resolution of 224×224 to 227×227.

4. The method according to any one of the preceding claims, characterized in that, When selecting a second image from the determined segments, the segment with the highest color intensity and / or the highest uniformity and / or the most edges is selected according to at least one threshold, and the segment is used to generate the training dataset.

5. The method according to any one of the preceding claims, characterized in that, Select a fragment containing at least a portion of a pattern, especially a logo and / or symbol and / or title, as a second image, and use said fragment to generate the training dataset.

6. The method according to any one of the preceding claims, characterized in that, The quality score is used to select progressively provided segments as second images, and these segments are used to generate the training dataset, the quality score depending on the determined uniformity and / or color intensity and / or the presence of edges.

7. The method according to any one of the preceding claims, characterized in that, The training dataset contains location information of the second image, wherein the location information relates to the location within the first image, and wherein the location information is provided, in particular, row-by-row and / or column-by-column.

8. The method according to any one of the preceding claims, characterized in that, The first parameter is the contact pressure value.

9. A method for training an artificial neural network, characterized in that, The artificial neural network is trained using the training dataset according to any one of the preceding claims.

10. An apparatus comprising a production machine, a computing unit, a control unit, and / or a data acquisition unit, wherein, The acquisition unit is designed to acquire a first image and provide the first image to the computing unit. Its features are, - The computing unit includes an artificial neural network, wherein the artificial neural network is designed to: use at least one first image as input value and provide second parameters as output value; and - The control unit is designed to control the control parameters according to the determined second parameter.

11. The device according to claim 10, characterized in that, The computing unit provides a dataset that links object features that can be processed by the artificial neural network with the positions of those object features. The acquisition unit can align itself with the corresponding position using the computing unit, so that the acquisition unit acquires the corresponding object features in the first image when acquiring the first image.

12. The device according to claim 10 or 11, characterized in that, The computing unit provides a dataset that links object features that can be processed by the artificial neural network with the location of the object features in the first image to be acquired, so that the acquisition unit only acquires regions of the first image that have object features present in the dataset.

13. The device according to claims 10 to 12, characterized in that, The production machine is designed as a printing device, wherein the second parameter is a contact pressure value, and the control parameter is designed as a contact pressure, which can be controlled by means of the contact pressure value.

14. A computer program product comprising instructions that, when executed by a computing unit, particularly a computer, cause the computing unit, particularly the computer, to perform the method and / or the steps of the method according to any one of claims 1 to 9.

Images