Device and method for tire testing

Abstract

Device for tire testing comprising a tire testing device by which at least one image of at least one area of ​​a tire is generated, and with an evaluation device for evaluating the at least one image, wherein the tire testing device performs a shearographic measurement to generate phase difference images, characterized in that at least one evaluation function of the evaluation device operates by means of machine learning (ML), and here by means of deep learning (DL), wherein the evaluation device performs a separation between a tire area and a background, wherein the separation between the tire area and the background is carried out by at least one ML model.

Description

[0001] The present invention relates to a tire testing device comprising a tire testing device that generates at least one image of at least one area of ​​a tire, and an evaluation device for evaluating the at least one image. For example, the tire testing device can be an optical tire testing device, in particular a shearographic tire testing device.

[0002] Such a device is known, for example, from document EP 2 851 670 A2.

[0003] In many cases, the evaluation of the images captured by the tire testing device is still performed manually by an operator, who uses the images to identify defects on the tire and classifies the tire based on the identified defects. Initial approaches to automating the evaluation of the images are based on image analysis and pattern recognition algorithms, which operate, for example, based on grayscale statistics or edge detection. However, these approaches have so far proven insufficiently reliable in detecting defects and also require considerable effort to adapt to the specific measurement situation or the tire being tested.

[0004] Further state of the art, partly also on the use of machine learning, is known from the following publications: US 2001 / 0 040 682 A1, DE 10 2018 104 857 A1, ODENTHAL, Cordula: Deep Learning in Image Processing - AI Finds Surface Defects. In: A&D Automation Digitization, a web magazine from Industr.com. October 30, 2018. URL: https: / / www.industr.com / de / ki-findet-oberflaechenfehler-2348187, US 2017 / 0 124 784 A1, KR 10 2018 0 052 180 A, KR 10 1 504 345 B1, CN 1 09 738 452 A, HEINZ, Sebastian: Deep Learning Basics - Part 1: Introduction. URL: https: / / www.statworx.com / content-hub / blog / deep-learning-teil-1-einfuehrung, October 16, 2017, CHANG, Chuan-Yu ; WANG, Wei-Chun: Integration of CNN and faster R-CNN for tire bubble defects detection. In: Advances on broadband and wireless computing, communication and applications : Proceedings of the 13th International conference on broadband and wireless computing, communication and applications (BWCCA-2018).Cham : Springer, 2019 (Lecture notes on data engineering and communications technologies ; 25). pp. 285-294. - ISBN 978-3-030-02612-7; Springer Verlag: Springer Link - Integration of CNN and FASTER R-CNN for Tire Bubble Defects Detection. URL: https: / / link.springer.com / chapter / 10.1007 / 978-3-030-02613-4_25#citeas.

[0005] The object of the present invention is therefore to provide an improved device and method for tire testing. In particular, the evaluation is to be improved.

[0006] This object is achieved by a device according to claim 1 and a method according to claim 11. Preferred embodiments of the present invention are the subject of the dependent claims.

[0007] The present invention comprises a tire testing device comprising a tire testing device that generates at least one image of at least one area of ​​a tire, and an evaluation device for evaluating the at least one image. It is provided that at least one evaluation function of the evaluation device operates using machine learning (ML).

[0008] The inventors of the present invention have discovered that machine learning is particularly well-suited to the specific requirements that must be met in the automatic evaluation of images during tire testing. The use of machine learning enables, on the one hand, robust and precise evaluation, and, on the other hand, easier adaptation to new tire types and / or measurement situations.

[0009] For machine learning, any machine learning method can initially be used within the scope of the present invention. For example, the use of statistical models is conceivable.

[0010] According to the present invention, the at least one evaluation function operates with deep learning (DL). Deep learning, in the context of the present invention, refers to the use of an artificial neural network. Preferably, an artificial neural network with a plurality of intermediate layers between the input layer and the output layer is used.

[0011] In one possible embodiment of the present invention, the artificial neural network used has an encoder-decoder architecture. Furthermore, the artificial neural network can include skip connections, which make results from earlier transformation layers available to corresponding later layers and, in particular, make results from encoder layers available to directly associated decoder layers, e.g., decoder layers with the same number of pixels in the input.

[0012] In one possible embodiment of the present invention, the artificial neural network used comprises residual blocks that transform inputs from multiple layers and additionally calculate the output with the input to obtain the block's result. In one possible embodiment, multiple residual blocks can be connected in series.

[0013] The above-mentioned designs of the artificial neural network can also be combined with each other.

[0014] According to the present invention, the evaluation device separates a tire area from a background. This takes into account the fact that the images typically show not only the tire but also a background, which may, for example, also include holding devices.

[0015] According to the invention, the separation between the tire area and the background is carried out by at least one ML model.

[0016] For example, deep learning segmentation can be used to separate the tire area and the background pixel by pixel.

[0017] In one possible embodiment of the present invention, in addition to the use of machine learning to separate the tire area from the background, a parameterization of the dividing line between the tire area and the background can be used. The parameterization can be implemented, for example, using a primitive such as a spline, a circular arc, or an elliptical arc.

[0018] In a first variant, a fit with a primitive such as a spline, a circular arc or an elliptical arc can be carried out in the pixel-by-pixel separation generated by machine learning in order to incorporate known knowledge about the circular shape of the tire surface.

[0019] In a second variant, a direct estimation of the parameters of the primitives from the recording can be carried out using machine learning.

[0020] The parameterization can be used in particular for the evaluation of images of the side surface of a tire, since the tire area can always be approximated by a circular segment.

[0021] In one possible embodiment of the present invention, the evaluation device performs a localization of defect areas and / or a classification of defects and / or the tire.

[0022] When localizing defect areas, for example, pixel-precise segmentation and / or a rough definition of defect locations on the image can be carried out.

[0023] When classifying defects, they can be divided into classes that describe, for example, the type and / or size of the defects. This can be done either based on the localization of defect areas and / or directly from the images.

[0024] Within the framework of tire classification, tires can be divided into defect classes, for example, tires without defects and tires with defects. Tires with defects can be further divided into several classes that differ in terms of the type, number, and / or severity of the defects. Tire classification can be based on a classification of the defects in the tires, on the localization of defect areas, and / or directly from the images.

[0025] The manner in which the localization and / or classification is carried out is initially not further restricted within the scope of the present invention.

[0026] For example, one, several, or all of the localization and / or classification processes described above can be performed algorithmically, i.e., without the use of machine learning, or manually by an operator. For example, the segmentation of defect regions can be performed algorithmically, e.g., by edge detection.

[0027] In a preferred embodiment of the present invention, however, the localization and / or classification is performed by at least one ML model. In particular, at least one of the above-described localization and / or classification processes can be performed by an ML model. In one possible embodiment, several or all of the above-described localization and / or classification processes can also be performed by at least one ML model.

[0028] In one possible embodiment of the present invention, the localization, classification and / or separation between tire area and background is solved simultaneously by a single ML model.

[0029] In particular, the separation between tire area and background on the one hand and the localization and / or classification on the other hand can be solved simultaneously by a single ML model.

[0030] In one possible embodiment of the present invention, the evaluation device makes a decision as to whether the tire is a reject based on the location of defect areas and / or the classification of defects in a tire. This can, for example, be a yes / no statement or a reject probability can be specified. The statement can be used directly to control a sorting station in the production plant and to sort out reject tires, or it can be reviewed again by a human inspector.

[0031] In one possible embodiment of the present invention, the defect areas and / or defects are assessed with regard to their rejection relevance using one or more tolerance rules.

[0032] In particular, within the scope of the present invention, the localization of defect areas and / or the classification of defects in a tire is carried out using machine learning, while the decision based on these results as to whether the tire is a reject or not is rule-based. This makes the decision-making process transparent.

[0033] A tolerance rule can, for example, specify which defect areas and / or defects are tolerated and which defect areas and / or defects lead to the decision that the tire is reject. For example, defect areas and / or defects can be compared with threshold values ​​based on one or more of their properties, such as size, radius, length, and / or contrast.

[0034] In one possible embodiment of the present invention, different tolerance rules are used for different test areas of the tire. The tolerance rules applied to a defect area and / or a defect therefore depend in particular on the area of ​​the tire in which the defect area and / or defect is located. For this purpose, the tire can be divided into test regions, each of which is assigned different tolerance rules.

[0035] In one possible embodiment of the present invention, different error tolerances are used depending on the test region. For example, the threshold values ​​with which one or more properties of the defect areas and / or defects are compared can vary depending on the test region in which the defect area and / or defect is located.

[0036] For example, the tire's sidewalls and tread can form different test regions. Furthermore, the tire's sidewalls can also be divided into different test regions.

[0037] In one possible embodiment of the present invention, the device comprises a user interface by means of which the tolerance rules can be specified and / or changed.

[0038] In one possible embodiment of the present invention, the device comprises a user interface by means of which the test regions can be specified and / or changed.

[0039] In one possible embodiment of the present invention, the device comprises an interface via which a user can prepare images recorded and / or simulated by the tire testing device by annotation, in particular by manual separation, localization and / or classification, wherein the device further comprises a training module which allows training of the ML model by means of the images prepared by the user.

[0040] The interface allows the creation of training data for training the machine learning process.

[0041] In one possible embodiment of the present invention, the device comprises a pre-trained ML model, wherein the interface and the training module allow further training during installation and / or during operation of the device. This reduces the effort required to train the ML model, as it is already pre-trained. On the other hand, the device can still be adapted to the specific evaluation task, e.g., the tire types to be tested and / or the measurement situation, during installation and / or during operation. Preferably, the device is designed such that the relearning process can also be carried out multiple times, for example, if the recording environment changes, new products need to be tested, new defect classes need to be detected, and / or other new information needs to be provided to the ML model.

[0042] In one possible embodiment of the present invention, the device comprises an evaluation station for manual evaluation of images, with the evaluation station being supported by the ML model. In this embodiment, the final evaluation is therefore not entirely fully automated, but at least partially performed by an operator. However, this operator is supported by automated subprocesses.

[0043] In a first variant, the evaluation station can be designed so that all tires are also inspected by a single operator. For example, the evaluation function can offer an evaluation, such as a classification, which can be either confirmed or changed by the operator.

[0044] In a second variant, the evaluation station can be designed so that only a portion of the tires are inspected by an operator, while another portion is inspected solely by the evaluation function. In particular, only those tires for which the evaluation function indicates a high uncertainty factor can be presented to an operator for inspection.

[0045] In one possible embodiment of the present invention, the evaluation station displays images differently depending on a localization and / or classification performed with the aid of the ML model. For example, images showing defects are prioritized and / or highlighted in the display. Furthermore, an uncertainty factor of the ML model can be taken into account. For example, images whose evaluation function outputs a high uncertainty factor can be prioritized and / or marked and / or highlighted in the display.

[0046] In one possible embodiment of the present invention, the device displays tires differently depending on a localization and / or classification carried out with the aid of the ML model. For example, tires for which no defects were detected and / or a low uncertainty factor exists with regard to the automatic classification can no longer be displayed at all or can only be displayed with a lower priority than tires for which defects were detected and / or a high uncertainty factor exists with regard to the automatic classification.

[0047] In a further possible embodiment, the tire inspection is carried out at least in one operating mode of the device for all tires exclusively automatically by the evaluation function and without additional human inspection.

[0048] The present invention is used to evaluate images from a tire testing device that performs a shearographic measurement to generate phase difference images. The images evaluated according to the invention can therefore, in particular, be phase difference images.

[0049] In one possible embodiment of the present invention, the tire testing device subjects the tires to a compressive load. The images can reflect tire deformation caused by the compressive load. In particular, the tires are subjected to negative pressure.

[0050] In one possible embodiment of the present invention, the tire testing device comprises a pressure chamber, in particular a vacuum chamber. Preferably, one or more measuring heads for shearographic measurement of the tread and / or the side surfaces of the tire are arranged in the pressure chamber.

[0051] In one possible embodiment of the present invention, the tire testing device checks the tread and / or the side surfaces of the tire.

[0052] In one possible embodiment of the present invention, the tire testing device is arranged in a tire transport line. At least one tire turner can be provided to allow inspection of both sides of the tire from the outside.

[0053] The present invention further comprises a method for evaluating images from a tire testing device as described above, comprising the steps: - Recording images by the tire testing device; and - Evaluating the images using an ML model, whereby a separation is made between a tire area and a background, whereby the separation between the tire area and the background is made by at least one ML model.

[0054] This results in the same advantages which have already been described in more detail above with regard to the device according to the invention.

[0055] In one possible embodiment of the present invention, the following steps are carried out in advance: - processing of recordings by annotation, in particular by separating, localizing and / or classifying recordings to generate initial training data; - Training the ML model using the first training data.

[0056] In possible alternative designs, however, the processing of the recordings may be omitted. For example, in this case, the ML model can simply learn meaningful descriptions of the recordings, for example, using unsupervised or self-supervised learning methods.

[0057] In one possible embodiment of the present invention, the ML model is delivered pre-trained and after installation of the device, the following further steps are carried out: - Recording images by the tire testing device - Processing of the recordings by separating, localizing and / or classifying recordings to generate second training data; - Retraining an ML model using the second training data.

[0058] In a possible embodiment of the present invention, such retraining is carried out several times, for example, when the recording environment changes, new products are to be tested, new defect classes are to be detected, or other new information must be provided to the ML model.

[0059] In one possible embodiment of the present invention, images of tires without defects are used to generate training data, and a distinction is made between a tire area and a background in these images. Since defects on tires occur relatively rarely, creating a sufficient amount of training data with defects is complex. This effort can be reduced by using images that show tires without defects, at least for learning the distinction between the tire area and a background.

[0060] In a further possible embodiment of the present invention, training data are generated by simulation.

[0061] The method according to the invention is preferably carried out as described in more detail above with regard to the device. In particular, the method can be carried out using the device described above.

[0062] The present invention further includes software for implementing a device as described above. In particular, the software may include an ML model.

[0063] By means of the software, a method as described above can preferably be carried out, in particular if the software runs on the device according to the invention.

[0064] The software may be stored on a data carrier or memory.

[0065] The device according to the invention preferably comprises at least one microprocessor and a memory on which the software according to the invention is stored, wherein the software comprises commands by which the microprocessor controls the device to implement the functions according to the invention.

[0066] The tires tested according to the invention are in particular rubber tires.

[0067] The present invention will now be described in more detail with reference to embodiments and drawings.

[0068] Showing: Fig. 1 shows a first embodiment of a device according to the invention, Fig. 2 a measurement of a side surface of a tire and a corresponding recording in an embodiment of the present invention, Fig. 3 a measurement of a tread of a tire and a corresponding recording in an embodiment of the present invention, Fig. 4 examples of different classes of defects, Fig. 5 a schematic diagram of an embodiment of a method according to the invention, Fig. 6 Images of the tread and a side surface of a tire with a separation between the tire area and the background and a localization of defects carried out by an embodiment of a device according to the invention, Fig. 7 a second embodiment of a device according to the invention with an external interface, Fig. 8 a first embodiment of an ML model and Fig. 9 a second embodiment of an ML model.

[0069] Fig. 1 shows an embodiment of a device according to the invention with a tire testing device 1.

[0070] In the exemplary embodiment, the tire testing device is arranged in a transport path, of which transport sections 4 and 5, which are arranged in front of and behind the tire testing device 1, are Fig. 1. Therefore, in the exemplary embodiment, the tires 6 pass through the tire testing device on the transport route. Furthermore, a turning device 2 is provided to turn the tire between two test runs in the tire testing device.

[0071] In the exemplary embodiment, the tires are delivered lying down on the transport path, turned 180° by the turning device if necessary, and tested in the tire testing device in at least one test run. The tires are preferably also tested lying down in a test run. The tire testing device can have an internal transport path on which the tires are moved through the tire testing device and on which the tires lie during the testing process.

[0072] However, other arrangements for loading the tire testing device with tires are also conceivable within the scope of the present invention. For example, the tire testing device can be loaded manually.

[0073] In one possible embodiment of the invention, the tire testing device 1 is a testing device in which the tire is subjected to a compressive load, and the resulting deformations are measured. In particular, the tire can be subjected to a negative pressure load.

[0074] The test is primarily performed using holographic and / or shearographic techniques. One possible test procedure may involve first recording a phase image when the tire is subjected to a first compressive load, then changing the compressive load, and then recording a second phase image when the tire is subjected to a second compressive load. A phase difference image is created from the two phase images, which can be evaluated to detect imperfections and / or defects in the tire, as these deform under the changed compressive load. This procedure can be repeated several times during a test to test different circumferential areas of the tire.

[0075] In the exemplary embodiment, the tire testing device 1 comprises a pressure chamber 10, in particular a vacuum chamber. This preferably has at least one opening 11 through which the tires can be moved into and out of the pressure chamber, and which can be sealed hermetically to change the pressure in the pressure chamber. In the exemplary embodiment, the tires are transported into the pressure chamber 10 on a first side, tested, and, after the test, transported out of the tire testing device on the opposite side.

[0076] The tire testing device comprises one or more measuring heads 7, which are preferably movable and / or rotatable via a traversing arrangement 8. The measuring heads are, in particular, holographic and / or shearographic measuring heads. By rotating the measuring head(s), different circumferential areas of a lying tire can be detected. Alternatively or additionally, it would also be conceivable to rotate the tire to test different circumferential areas.

[0077] However, in alternative designs, other optical testing methods may also be conceivable.

[0078] The tire testing system is preferably designed for a so-called bead-to-bead tire test, in which both side surfaces of the tire are inspected at least from the outside, and the tire tread is inspected at least from the inside. Since one of the two side surfaces is usually resting on a support during a test run, preventing it from being inspected from the outside, the entire tire test can be performed in two separate test runs, between which the tire is rotated 180°.

[0079] Alternatively, only the side surfaces, one of the side surfaces or the tread, as well as any combination thereof, can be tested.

[0080] Fig. 2 shows an inspection of a side surface 12 of the tire 6 by a measuring head 7 with a measuring field 19. The image 20 taken by the measuring head comprises an outer background 13, the upper sidewall 12 from the outside, the upper bead 14, the lower sidewall 15' from the inside, the lower bead 14' and the inner background 13'.

[0081] Fig. Figure 3 shows an inspection of an inner tread 16 of the tire 6 by a measuring head 7' with a measuring field 19'. The image 20' captured by the measuring head 7' includes the upper bead 14, the upper sidewall 15 from the inside, the tread 16 from the inside, and the lower sidewall 15' from the inside.

[0082] Fig. 4 shows the appearance of typical defects 21 to 23, which are to be detected by a tire test on the image taken by the tire testing device, in particular a phase difference image.

[0083] The inventive evaluation of the images of the tire produced by the tire testing device serves to localize and / or quantify defects on the image of the tire.

[0084] The current state-of-the-art workflow used for this purpose was as follows: 1. Data recording using shearography 2. Manual inspection or 1. Data recording using shearography 2. Extraction of hand-coded features (e.g. histogram of gray values, edge orientations, ...) 3. Simple analysis methods (e.g. threshold) on the feature representation to distinguish background vs. tire and analysis of the defect regions in the tire area

[0085] However, this had the following disadvantages: ▪ Manual testing is necessary, which is time-consuming and can vary in accuracy and is therefore subjective. ▪ Automated tests based on simple threshold operations are too imprecise for fully automated use in the application. ▪ Automated tests based on simple image processing methods (e.g. hand-coded features) are time-consuming to implement, cannot be transferred to new device types without significant effort (in contrast to learning-based methods) and are often too inaccurate for the application. ▪ The position of the tire in the analysis tool can vary, which can only be taken into account in simple analysis procedures with considerable effort (for example, through complex calibration and measuring of the system parameters). • Tires are seen from above and from the side and show varying backgrounds, possibly with additional objects (metal arms, etc.), which is too complex for simple analysis methods

[0086] In order to solve these problems, the device according to the invention therefore has an evaluation device for evaluating the images generated by the testing device, in particular for evaluating the phase difference images of the shearographic testing device, wherein an ML model is used for the evaluation.

[0087] In particular, the ML model can be used to predict the tire region to be inspected, i.e., the separation between the background and the tire area, and / or the (subsequent) localization and / or classification of the defect regions. In particular, the ML model can comprise a trained segmentation model and / or a trained classifier. The classification can include quantifying the defect regions, e.g., determining the area and / or extent of a defect region.

[0088] The advantages of ML over known methods such as a simple image processing solution or a simple computer vision solution can be summarized as follows: ◯ Robust and precise prediction of test region and defects (especially through joint learning of the image representation and the predictor / classifier). ◯ Consideration of the global context can be learned without additional effort. ◯ Adaptation to new tire types, changes in mounting, etc. can be done with little effort through new training (e.g. by service personnel or customer) and does not require algorithmic adjustments with great implementation effort.

[0089] Fig. 5 shows an embodiment of a method according to the invention in which an ML model is used to evaluate the recordings.

[0090] A training phase 30 includes the creation 31 of recordings, an annotation 32 of errors in the recordings to generate training data and the training 33 of an ML model or machine learning process.

[0091] To create the images, they are preferably acquired in step 31, particularly using shearography. Alternatively, the images can also be created using a simulation. Annotation is performed manually and can be supported by conventional computer vision if necessary.

[0092] By training the ML model, its model parameters are determined.

[0093] During normal operation 40 of the device, data recording 41 is again performed using an optical measurement method, in particular shearography. The evaluation 42 of the recordings is performed with the aid of the ML model learned in the training phase 40. In particular, the output parameters are automatically determined using the learned model parameters.

[0094] In a possible embodiment of the present invention, it is provided that the ML model does not only receive a single input image which is evaluated, but rather an image stack, e.g. an image stack which shows the same area of ​​the tire in different resolutions.

[0095] The evaluation device can, on the one hand, separate the background from the image areas to be inspected, i.e., the tire area (step 1), and, on the other hand, perform defect region analysis and / or classification (step 2). At least one of these steps uses a machine learning method, particularly in the form of a neural network or deep learning. Within the scope of step 1, the tire area can also be further divided into the different areas of a tire, discussed in more detail above, which may be visible in the image.

[0096] For example, there are the following possible solutions for step 1 (especially the background separation): - L1: Classical algorithmic solution (for example via gray value statistics); - L2: ML learning-based solution (for example, using deep learning segmentation to separate pixel-by-pixel foreground from background and / or different tire areas); - L3: Combination of L1 or L2 with fitting of a primitive (e.g. spline) to incorporate known knowledge about the circularity of the test surface; - L4: Direct estimation of parameters of a primitive (e.g. spline parameters) from the image using a deep learning method (e.g. for the evaluation of side shots of tires in which the foreground can always be approximated by a circular segment).

[0097] Furthermore, the following possible solutions exist for step 2 (defect region analysis and / or classification): - K1: Classical algorithmic segmentation of defect regions (e.g. via edge features or similar); - K2: ML learning-based pixel-precise segmentation and, if necessary, classification of defect locations (e.g. using deep learning for image segmentation); - K3: ML learning-based detection of defect locations and description using rectangular bounding boxes or other parametrically describable geometric primitives such as circles or ellipses (e.g. using deep learning methods for object recognition).

[0098] Options L2 and K2 can also be solved with a single ML model. The input here is the image, specifically a shearography image. The output is an image in which a pixel-by-pixel distinction has been made between the background, tire, and one or more defect types.

[0099] The options (L4 and K2) or (L4 and K3) can also be solved simultaneously with a joint ML model (so-called multi-task learning, joint prediction of a foreground-background distinction and a defect region segmentation).

[0100] Fig. 6a and Fig. 6b show exemplary outputs of a device according to the invention for receiving a running surface ( Fig. 6a) and a side surface ( Fig. 6b).

[0101] Here, a separation between tire area 16, 12 and background 13, 13' is performed using a primitive 50, as well as a localization and classification of defects 21, 22, and 23. Localization is performed here using bounding boxes. Alternatively, localization could also be performed pixel-by-pixel.

[0102] According to the invention, an ML model is used in at least one of steps 1 and / or 2, ie at least L2, L3 or L4 or K2 or K3.

[0103] The ML model can be used and designed independently of or in combination with the above-mentioned approaches as follows: Solution 1 (only replacing the previous analysis)Description • Input: Images, especially from the shearography process • Output: Defect map (binary or multi-class, scalar or class probabilities) and / or Region of Interest (ROI) mask of the tire surfaces • The ML model learns the prediction of the output from the input Solution 2 (direct prediction of the relevant metrics)Description • Input: Images, especially from the shearography process • Output: Estimation of defect sizes and defect position (e.g. in mm), ROI mask of the tire surfaces • The ML model learns the prediction of the output from the input. Solution 3 (only learning the feature representations in the existing workflow)Description • Input: Images, especially from the shearography process • Output: Defect map (binary or multi-class, scalar or class probabilities) and / or Region of Interest (ROI) mask of the tire surfaces • An involved ML model simply learns meaningful descriptions of the shearography data, e.g., using unsupervised learning (AutoEncoders, RBM). The rest of the state-of-the-art workflow remains unchanged, i.e., the descriptions learned by the ML model are used in a computer vision process instead of hand-encoded features. In this case, annotation can be omitted, reducing the initial annotation effort.

[0104] There are different variants for generating training data: Variant 1 (learning from measured data)

[0105] As training data, corresponding pairs of {image, annotation} are required. Images obtained through measurements are therefore annotated accordingly. Depending on the solution (1-3), the annotation can be a pixel-wise defect map, a defect probability map, a binary and multi-class version of the latter, or simply a scalar or vector of the target variable. Variant 2 (learning from simulated data)

[0106] Like variant 1, except that the images are obtained from a simulation process that appropriately simulates or approximates the shearography image. Variant 3 (Learning from measured and simulated data)

[0107] Learning from measured and simulated data can be done cyclically, can be mixed, can be done as a two-step process (first V1 then V2).

[0108] Regardless of how the training data is generated, the ML model can be trained with data from different “modalities”, e.g. with data from several different shearography systems, or different lighting settings.

[0109] The training data should be as representative as possible, i.e. it should have at least one and preferably several of the following properties: - Different times - Various physical measuring machines - Different types of defects - Different types of defects - Rated by various annotators - Different tire types

[0110] This applies to both actual recorded and simulated data.

[0111] The annotations should be consistent, i.e., the annotations should be as consistent as possible and / or allow an estimation of the annotation inaccuracy and / or the acquisition process should be stationary.

[0112] The device can be equipped with additional design features in possible configurations. The possibilities described below can be combined with all the options and variants described above. 1. Design feature “Continuous training + interface”

[0113] It's practically impossible to have representative data available for all future use cases during development. In particular, systems may be installed differently from customer to customer, different environments with background structures may exist, different settings for data acquisition may be configured, etc.

[0114] In one possible embodiment of the present invention, the device is therefore designed in such a way that the customer can relearn the previously trained ML model. This can be done, for example, during an installation phase, or later at regular or irregular intervals.

[0115] In particular, the relearning process can also be executed multiple times, for example when the recording environment changes, new products need to be tested, new defect classes need to be detected, or other new information needs to be provided to the ML model.

[0116] A corresponding embodiment is shown in Fig. 7, wherein the device initially has the functions described above with regard to Fig. 1 described components. To enable relearning, an interface can be provided to create annotations for newly recorded data, e.g. as an annotation GUI on an operator PC 60, which is arranged on the tire testing device, or via an external computer 61, which is connected to the device via a network 62, see Fig. 7. The interface on the external computer 61 can be implemented, for example, via a browser solution.

[0117] In addition, a computing unit 63 (with GPU) can be available on which the relearning can take place, e.g. in the cloud or on a local PC.

[0118] Algorithmically or technically, methods from the field of transfer learning can be used for relearning. Specifically for deep neural networks, relearning can be achieved through fine-tuning, for example, in which the previous parameter values ​​of the originally trained model are used as a starting solution for retraining at the customer's site. Through fine-tuning, the entire learning process can be completed faster and with less data than was necessary for the initial training during development. 2. Design feature "Prioritization of learning data"

[0119] During the above-described relearning or continuous learning during installation, operation and / or running-in of the device by adapting the ML processes, an unlimited amount of data cannot be annotated due to time and cost constraints.

[0120] Therefore, in one possible embodiment of the present invention, the newly acquired data can be automatically sorted by the device according to relevance and made available accordingly. This makes it possible to annotate only the most relevant data. The uncertainty of the initially learned system can be a measure of relevance.

[0121] Here, too, hardware and / or software for annotation and training can be provided, either locally or via cloud-like connections 3. Design feature “ML method without pixel-wise annotations”

[0122] Pixel-precise image annotation is often too costly to be performed on a large scale.

[0123] According to one possible embodiment of the present invention, the device is therefore designed such that training data can be used for which annotation is implemented only at the image label level. For example, the annotation can be performed based on whether an image contains a defect or not, or one or more defect classes can be assigned to an image. A correspondingly designed interface is preferably provided for this purpose.

[0124] In one embodiment, images annotated in this way are used to train an image classifier, which predicts this yes / no answer and / or classification for new images and, if necessary, subsequently visualizes in the image which regions were relevant for the decision-making process for the ML classifier (e.g., Gradcam, GuidedBackprob). Thus, in addition to the classification result, a segmentation-like map is generated without the need for prior segmentation annotations. 4. Design feature “Hybrid operation mode visual inspector + machine”

[0125] It is to be expected that ML methods will not be sufficiently accepted initially, especially since 100% accuracy cannot be achieved initially, but extremely low error rates are required for application.

[0126] In one possible embodiment of the present invention, a hybrid solution consisting of a machine and a visual inspector is therefore implemented. The device provides an interface through which a visual inspector can check the prediction generated by the device based on an ML model and, if necessary, correct it. In particular, an automated pixel-by-pixel ML analysis can be coupled with manual verification.

[0127] In addition to the prediction generated using an ML model, the device preferably also outputs a value and / or an indication of the confidence that the ML model had in making the prediction.

[0128] For example, the visual inspector can accept the ML prediction for images without further scrutiny if the model has high confidence. The visual inspector only needs to verify / correct predictions for examples for which the ML system has high internal uncertainty. 5. Design feature “fully automated final testing”

[0129] The current workflow with manual visual inspection can be too time-consuming for high-frequency inspection processes, such as 24 / 7 inspection sequences where visual inspectors are not always available in 24 / 7 shifts. The overall workflow "hybrid operation mode visual inspector + machine" described in the previous feature can also be too time-consuming, even if only a few (uncertain) images need to be manually verified.

[0130] According to a possible embodiment of the present invention, the test is therefore carried out exclusively by an ML method which directly predicts the final test decision (e.g. as a classification model or with additional prediction of the segmentations as described above). 6. Design feature “Rule-based committee decision”

[0131] In one possible embodiment, the evaluation device first uses an ML model to determine a segmentation map and / or a classification of the defects, e.g. a prediction of the location, size, type and / or number of defects.

[0132] This result is then converted into the final statement as to whether the tire is reject (in particular in the form of a yes / no decision) based on certain tolerance rules.

[0133] For this purpose, the evaluation device assesses the identified defect regions and / or defects using a rule-based toleration (e.g. regarding radius, length, contrast, ...) with regard to their scrap relevance.

[0134] In one possible design, the assessment of defect relevance for passing or failing can be dependent on test regions. In particular, error tolerances can be set depending on test regions.

[0135] For example, this can result in different requirements for a bubble size on the tire side than on the tire tread side.

[0136] In the following, some ML models are described which can be used in the context of the present invention.

[0137] The simplest way to learn an ML model for data transformation is through statistical models (almost any parametric distribution is conceivable). However, in practice, this often does not lead to sufficient accuracy. Therefore, deep learning methods are preferred within the scope of the present invention.

[0138] In one possible embodiment of the present invention, an encoder-decoder architecture is used as the ML model. In particular, such an encoder-decoder architecture can be used for pixel-precise separation of background and tire areas and / or for pixel-precise segmentation of defects. For example, the image can be transformed into a class probability map.

[0139] In one possible implementation, the encoder-decoder architecture can be extended with skip connections, which allow for the accurate description of even the smallest details in image data. Well-known ML models for this purpose include ResNet, DenseNet, and RefineNet.

[0140] The encoder-decoder architecture can be designed as a U-Net, in which the encoder side reduces the sample size in several steps, while the decoder side increases the sample size again.

[0141] Fig. Figure 8 shows an embodiment of such a model for transforming recordings 70 into pixel-precise segmentations 71. The embodiment initially comprises an encoder path 73 consisting of several levels 74, 74', 74'', 74''' of layers 80, which reduce the sample size. The decoder path 75 also comprises corresponding levels 74, 74', 74'', 74''' of layers 80, which increase the sample size again.

[0142] In the exemplary embodiment, this classic encoder-decoder architecture is generalized in that results from early transformation layers are also available to later layers via skip connections (represented by horizontal arrows 72). In the exemplary embodiment, the skip connections connect layers of the encoder path 73 and the decoder path 75 of the same level. This preserves, in particular, the spatial information, leading to greater accuracy. Furthermore, the intermediate result of each path can be interpreted as a feature, which generalizes the mapping to a central feature collection.

[0143] Instead of a U-Net, block-wise processing with skip connections can be provided.

[0144] Fig.Figure 9 shows an example representation of a residual block, which transforms inputs with multiple layers and additionally calculates the output with the original input to obtain the final result of the block.

[0145] Several such residual blocks can be connected in series. The exclusive data flow from one layer to the next is thus generalized.

[0146] Dense blocks can also be used. In a dense block, inputs are transformed using multiple layers. Each layer receives the (computed) results of all previous layers and forwards its output to the next layer and the final output.

[0147] Block processing can be combined with a U-Net.

[0148] Furthermore, DenseNet models can consist of multiple dense blocks. It thus generalizes ideas from ResNet and U-Net by generalizing the data flow and its processing, which positively impacts both the accuracy and learnability of the models.

[0149] Furthermore, a RefineNet block can be used, which simultaneously receives inputs from a modality at multiple resolution levels, processes them individually with ResNet blocks, fuses their results, and finally transforms this intermediate result in a denset-like manner. Likewise, a RefineNet model can be used that combines multiple RefineNet blocks to realize highly precise, learnable transformations.

[0150] In particular, encoder-decoder architectures, with or without skip connections, and in particular such encoder-decoder architectures as just described, can be used for solution 1 described above.

[0151] Within the framework of Solution 2 described above, the prediction of the target size can be viewed as a regression problem in one possible embodiment. Since multiple defects can occur in an image, it can also be provided that the number of defects is also predicted, i.e., a list of defects with the corresponding size and prediction probability for each image. Possible detection methods such as YOLO, SSD, YOL0-9000, or Mask R-CNN can be adapted accordingly by predicting the defect size in mm and the class membership instead of the object size in pixels and the class membership predicted there.

[0152] Within the framework of Solution 3 described above, AutoEncoders or Restricted Boltzmann Machines (RBMs) can be used, which allow for the unsupervised learning of image representations. Alternatively, supervised methods can be trained as described for Solution 1, but then only their activations are used as representations of image data and integrated into the existing workflow (e.g., replacing the handcrafted features).

[0153] In summary, the present invention provides a device and a method by which images of tires, in particular phase difference images from shearography systems for tires, are automatically evaluated, defects are detected, their position and size are recorded and / or they are divided into defect classes, and in particular also differentiated from sensor artifacts.

[0154] Phase difference images from shearography systems for tires are available in different views of the tire: They can depict sectors of the tread and / or sidewall and at least one bead. An entire tire can also be visible. The image areas are thus divided into the individual areas: tread, sidewall (near and far), bead area (near and far), and background, each of which can be partially or completely missing.

[0155] Existing methods detect different image areas and defects based on predefined criteria and thresholds for local or global differences in image contrast or pattern recognition parameters. Specifying thresholds is error-prone and complex.

[0156] The invention, on the other hand, uses a self-learning method that works without manual specification of threshold values ​​and can therefore be used in a wide range of applications.

[0157] This makes it possible to automatically detect and classify defects during the production and retreading of any pneumatic tire. This detection system can support or even replace a system operator in evaluating the inspected tire. The defect assessment is trained by testing tires with known defect patterns.

[0158] Time-consuming and costly parameter determination is eliminated; error detection is continuously optimized and objectified and automatically adapted to the needs of each customer.

Claims

[1] A device for testing tyres comprising a tyre testing device, by means of which at least one image of at least one area of a tyre is produced, and comprising an evaluation device for evaluating the at least one image, wherein the tyre testing device carries out a shearographic measurement for generating phase difference images, characterized by that at least one evaluation function of the evaluation device works by means of machine learning (ML), and in this case by means of deep learning (DL), wherein the evaluation device carries out a separation between a tire area and a background, wherein the separation between the tire area and the background is carried out by at least one ML model. [2] Device according to claim 1, wherein additionally a parameterization of the dividing line between the tire area and the background is used. [3] Device according to one of the preceding claims, wherein the evaluation device carries out a localization of defect areas and / or a classification of defects and / or of the tire. [4] Device according to claim 3, wherein the localization and / or classification is carried out by at least one ML model, wherein preferably localization, classification and / or separation between tire area and background are solved simultaneously by a single ML model. [5] Device according to claim 3 or 4, wherein the evaluation device makes a decision on the basis of the localization of defect areas and / or the classification of defects of a tire as to whether the tire is reject or not, for which purpose the defect areas and / or defects are preferably assessed with regard to their reject relevance by means of one or more tolerance rules, wherein different tolerance rules are preferably used for different test areas of the tire and, in particular, different error tolerances are used depending on the test region. [6] Device according to one of the preceding claims, with an interface via which a user can prepare images recorded and / or simulated by the tire testing device by annotation, in particular by manual separation, localization and / or classification, wherein the device further comprises a training module which allows training of the ML model by means of the images prepared by the user. [7] Device according to claim 6, comprising a pre-trained ML model, wherein the interface and the training module allow further training upon installation and / or during operation of the device, wherein the interface and the training module preferably allow multiple execution of the relearning process. [8] Device according to one of the preceding claims, with an evaluation station for the manual evaluation of images, wherein the evaluation station is supported by the ML model, wherein in particular all tires or some of the tires are also tested by an operator in addition to the test by the evaluation function, wherein preferably the evaluation station displays images and / or tires differently depending on a localization and / or classification carried out with the aid of the ML model, and in particular takes into account an uncertainty factor of the ML model. [9] Device according to one of the preceding claims, wherein the tire testing is carried out at least in one operating mode of the device for all tires exclusively in an automated manner by the evaluation function and without additional human testing. [10] Device according to one of the preceding claims, wherein the tire testing device subjects the tires to a pressure load and in particular has a pressure chamber. [11] Method for evaluating images from a tire testing device using a device according to one of the preceding claims, comprising the steps: - Recording images by the tire testing device; and - Evaluating the images using an ML model, whereby a separation is made between a tire area and a background, whereby the separation between the tire area and the background is made by at least one ML model. [12] Method according to claim 11, wherein the following steps have been carried out beforehand: - processing of recordings by annotation, in particular by separating, localizing and / or classifying recordings to generate initial training data; - Training the ML model using the first training data. [13] Method according to claim 11 or 12, wherein the ML model is delivered pre-trained and the following steps are carried out after installation of the device: - Recording images by the tire testing device - Processing of the recordings by separating, localizing and / or classifying recordings to generate second training data; - Retraining an ML model using the second training data. [14] Method according to claim 11, 12 or 13, wherein images of tires without defects are used in the generation of training data and a separation is made between a tire area and a background in these images. [15] Software for implementing a device according to one of claims 1 to 10, by means of which a method according to one of claims 11 to 14 can preferably be carried out when it runs on the device.

Images