qualitative or quantitative characterization of the coating surface

Abstract

This invention relates to a method for qualitatively and / or quantitatively characterizing a coating surface, the method comprising: providing a defect identification program (124) designed to identify coating surface defect types (1204, 1206, 1306, 1308), the defect identification program including a prediction model trained based on training images (604, 606) acquired within a predetermined distance range and / or a predetermined image acquisition angle range relative to a coating surface depicted in training images; determining, by the defect identification program, whether at least one camera (134) effectively connected to the defect identification program is located within the predetermined distance range and / or the predetermined image acquisition angle range relative to the currently presented coating surface; and, based on the determination result, generating an adjustment of the position of the at least one camera so that the camera is located within the predetermined distance range. Feedback signals within the range and / or the predetermined image acquisition angle range, and / or automatically adjusting the relative distance between the at least one camera and the coating surface such that the camera position is within the predetermined distance range from the coating surface, and / or automatically adjusting the angle of the at least one camera such that the camera position is within the predetermined image acquisition angle range; allowing the camera to acquire digital images (604, 606, 1202) of the coating surface only when the camera is within the predetermined distance range and / or the image acquisition angle range; processing (102) the digital images (604, 606, 1202) of the coating surface (162-168) by a defect identification program for identifying coating surface defects; outputting (104) a characterization of the coating surface by the defect identification program, and calculating the characterization in relation to the identified coating surface defects.

Description

Technical Field

[0001] This invention relates to the identification of coating defects and the characterization of coating surfaces, particularly coating surfaces based on coating compositions used in paints, varnishes, printing inks, abrasive resins, pigment concentrates or other coating compositions. background

[0002] Paint and varnish coatings can have a variety of defects that adversely affect the appearance or technical performance of the coated object. Coating defects can include, for example, foaming, pitting, clouding, leveling problems, corrosion, wetting problems, pigment floating (slippage), sagging, clumping, or bubble formation. Multiple defects may occur simultaneously and interact with each other. To investigate and avoid these problems, the formulation is tested on a substrate and defects are examined during formulation development. Depending on the intended application, different substrates such as wood, plastics, paper / cardboard, glass, or metal are used. Furthermore, different substrate pretreatments are possible and can complicate matters further. Due to the large number of interdependent process parameters, coating compositions, pretreatment methods, and substrate types, it is currently impossible to predict whether a particular coating composition will provide a coating of acceptable quality when applied to a particular substrate. Therefore, coating surface quality can currently only be determined retrospectively.

[0003] Currently, defects are assessed by humans, such as employees, through visual inspection. This purely visual assessment is often very coarse-grained, highly subjective, and difficult to reproduce. Therefore, defect identification and coating surface quality assessment may require extensive experience from employees, but can vary considerably from person to person, making it difficult to compare results. Furthermore, manual assessment of coating surfaces is time-consuming and expensive.

[0004] Overview

[0005] This invention aims to provide an improved method and corresponding system for characterizing surface defects in coated surfaces, as specifically described in the independent claims, and the use of the resulting procedures and information in a coating composition production environment. Examples are given in the dependent claims. The embodiments of the invention can be freely combined with each other, provided they are not mutually exclusive.

[0006] In one aspect, the present invention relates to a method for qualitatively and / or quantitatively characterizing a coated surface. The method includes:

[0007] - A digital image of the coating surface is processed by a defect identification program configured to identify the types of defects on the coating surface, for example, by identifying patterns, each pattern representing one type of coating surface defect; and

[0008] - Outputs a characterization of the coating surface, which is calculated in relation to coating surface defects identified by a defect identification procedure.

[0009] Embodiments of the present invention may have the advantage of providing characterization of the coated surface in a reproducible, objective, and rapid manner through a defect identification procedure.

[0010] For example, the output representation might indicate that the coated surface has no defect type D1, but may contain defect types D2 and D3. The output could also indicate that the coated surface has no coating defects at all. According to other examples, the output can be more specific and also indicate the amount or extent of coating defects, for example by indicating that the surface contains minor defects of type D2 and severe defects of type D3. In other examples, the representation may include a numerical representation of the defect extent and / or an indication in a digital image of individual pixels representing a specific type of defect or a specific instance belonging to a specific defect type.

[0011] Embodiments of the present invention offer the advantage that the presence and / or location of defects on the coating surface are detected fully automatically and used to automatically calculate the coating surface characterization based on the type and / or extent of one or more defects identified in an image analysis program. Therefore, a large number of digital images can be evaluated and labeled fully automatically with the calculated characterization of their respective depicted coating surfaces. This can be particularly useful in high-throughput environments used for testing and / or producing coating compositions. The automated determination of coating surface characterization improves the transparency and reproducibility of the evaluation of coating surface quality and other properties.

[0012] Embodiments of the present invention may offer additional advantages, such as the ability to use automatically identified coating surface defects and the resulting coating surface characterizations as a database for performing a variety of different forms of data analysis. In particular, computational characterizations can be used as qualitative and / or quantitative indicators of the quality and suitability of the coating surface and coating composition, the quality or suitability of the coating composition production process, and / or the quality or suitability of the coating application process (including surface pretreatment processes, if any, substrate type, and / or application equipment) used to produce the coating composition and coating surface.

[0013] Automated calculations of qualitative and / or quantitative coating surface characterization enable the automated analysis of large amounts of data and ensure the similarity of surface quality characterizations related to different coating compositions, coating production process parameters, and / or coating application process parameters. This is particularly useful in environments where many different types of coating compositions are produced and tested for identifying the optimal coating composition for a particular substrate or use case scenario.

[0014] Embodiments of the present invention may be particularly advantageous in the production environments of paints, varnishes, printing inks, abrasive resins, pigment concentrates, and other coatings, because reproducible, objective characterization of the surface properties of the respective coating surfaces was previously unavailable. Due to the subjectivity of the manual description of coating surfaces and the large number of components and their interactions, the quality of coating surfaces cannot be predicted.

[0015] For example, foaming is a common phenomenon in the production and application of coatings and printing inks. Foaming is caused by introducing gas into liquid materials. This can occur in the following ways:

[0016] • Air is introduced during the production process through stirring and mixing machinery.

[0017] • Air is displaced during the wetting process of pigments and fillers.

[0018] • Air is mechanically introduced during applications such as rolling, spraying, and printing.

[0019] • Displace air when coating porous substrates.

[0020] Dried foam can leave surface defects (such as bubble defects) in the paint film. Foam not only affects visual appearance but also weakens the protective function of the coating. In fact, all components in a paint formulation positively or negatively influence foaming behavior. The substrate and application method also affect foaming behavior. Therefore, defoamers are often an essential component in coating composition formulations.

[0021] The applicant has observed that the effectiveness of a defoamer in a coating composition depends on its partial incompatibility with the coating medium. This allows defoamer droplets to form in the system without causing film or surface defects due to excessive incompatibility. Therefore, a key characteristic of all defoamers is their established, controllable incompatibility with the medium to be defoamed. Defoamers with too high compatibility do not migrate specifically into the foam sheet; they are present throughout the entire coating film. The defoaming effect is either low or nonexistent. Too high incompatibility causes problematic coating defects such as turbidity or pitting. Therefore, selecting the right defoamer is a “balancing act” between compatibility and incompatibility. Thus, defoaming is always a trade-off between efficiency and compatibility.

[0022] Because there are many different coating systems, there is no single defoamer that is optimally suited to all formulations. A wide variety of defoamers are needed to ensure a suitable product for any purpose. The defoaming effect can be fine-tuned by varying the dosage: generally, the more defoamer used, the better the defoaming effect. However, this can also increase defects (such as pits), or more precisely, defects may become more visible. Pitting defects are defects stemming from component incompatibility, which can be caused, for example, by too much defoamer and / or the wrong type of defoamer. Reducing the amount of defoamer can prevent film defects, but in some cases, the defoaming effect may be insufficient. Furthermore, the applicant has observed that in some coating compositions, increasing the amount of defoamer beyond a certain point can actually weaken the defoaming effect. Therefore, identifying one or more defoamers that provide a defect-free or minimally defective coating surface is an extremely complex task.

[0023] According to embodiments of the present invention, the coating composition used to generate the coating surface comprises a combination of two or more different defoamers, the types and amounts of which are selected to achieve a compromise between defoaming efficiency and compatibility / miscibility with other components of the same composition.

[0024] Therefore, selecting the type and amount of one or more defoamers in a coating composition is an extremely complex task. This is further complicated by the fact that the selected type and amount of defoamer are related to the type and amount of other components in the coating composition, coating production process parameters, and / or coating application process parameters. Even a small change in one of these parameters can significantly affect the type and extent of surface defects observed on the respective coating surface. To date, the manual characterization of coating surface quality has been a major obstacle to identifying variations related to the coating composition and associated process parameters that contribute to improvements in coating surface quality.

[0025] The applicant has observed that automated detection of coating defects of specific defect types, such as bubble defects and pit defects, can allow for further training of another machine learning model (such as models M2, M3 described below with respect to some embodiments of the invention) based on data containing various coating composition specifications associated with the respective observed coating defects. This training can automatically calculate and output specifications for an improved version of the coating composition (M3) or calculate and output the expected coating surface characterization, such as coating defects, for a given (complete or incomplete) coating composition (M2). For example, the machine learning model may learn during the training phase that bubbles on the coating surface indicate an ineffective defoamer, while pit defects on the coating surface indicate that the defoamer is incompatible with the coating medium. In the former case, the model may suggest replacing the currently used defoamer with a more effective one. In the latter case, the model may suggest replacing and / or supplementing the currently used defoamer with a defoamer that is predicted to be more compatible with the coating medium. If the coating composition already contains an "effective" defoamer DF1 and a "compatible" defoamer DF2 in a specific ratio r = DF1:DF2, the model may suggest increasing the ratio in the former case and decreasing the ratio in the latter case.

[0026] Therefore, according to embodiments of the present invention, a composition specification prediction model is used to predict the ratio of at least two different defoamers, particularly the ratio of a first defoamer with high defoaming efficiency and low coating medium compatibility to a second defoamer with low defoaming efficiency and high coating medium compatibility, thereby predicting the ratio to minimize the occurrence of bubble defects and pit defects.

[0027] Other defects may be caused by another component and / or production process parameters.

[0028] According to some embodiments, the characterization of the coating surface output by the defect identification program includes a fine-grained quantitative characterization, such as values ​​within a continuous scale or a range of at least 10 different predetermined values. The defect identification program can be configured to convert the fine-grained quantitative characterization into a coarse-grained quantitative characterization so that the automatically calculated characterization is comparable to an existing coarse-grained dataset. For example, the existing coarse-grained dataset might be manually created. The coarse-grained quantitative characterization can be a set of fewer than 10 different predetermined values ​​or a range of values. Automated conversion can increase the data base for various data analysis or machine learning purposes by demonstrating a mixture of manually labeled and automatically labeled digital images of coating surfaces that are similar to each other.

[0029] According to other embodiments, the output of the defect identification program can be coarse-grained from the outset. For example, the output could be an image classification output containing only image category labels such as "defect-free coating surface", "coating surface with (minor) bubble defects", or "coating surface with (minor) bubble defects and (severe) pinhole defects".

[0030] According to an embodiment, the method includes calculating a measure of identified defects for calculating qualitative and / or quantitative characterization of the coating surface using a defect identification procedure; and outputting the qualitative and / or quantitative characterization of the coating surface.

[0031] For example, a coating surface may include two or more defects of one or more different types and degrees of variation. The calculated qualitative and / or quantitative measures of each defect can be used, for example, to aggregate automatically acquired measures of automatically identified defects within the coating surface to obtain a qualitative and / or quantitative characterization of the coating surface.

[0032] The automatic identification and measurement of individual defects offers the advantage of objectifying the quantity, type, size, and other properties of each individual defect on the coating surface. Embodiments of the present invention exhibit significantly less subjectivity compared to prior art methods based on manual / visual assessments of individual defects and / or coating surface quality by workers. Visual assessments are not quantitative and only allow for a very rough classification of the results according to coarse-grained quality grades or coarse-grained grading systems.

[0033] According to an embodiment, a defect measure is or includes a quantitative measure selected from the group consisting of: defect area (in absolute value or relative to the size of the coating surface depicted in the image), number of bubbles, dents or other defect-related patterns observed in the image, and the maximum, minimum, and / or average size of bubbles or dents or other patterns representing defects observed in the image.

[0034] In addition, or alternatively, defect measurement is or includes quantitative measurement. Quantitative measurement can in particular identify the type of defect. For example, defect types can be selected from the following group, which includes: pitting defects, scratches, adhesion failure defects, alligator cracks, bleeding defects, blistering defects, efflorescence defects, run-through defects, bubble defects, cathodic disbondment defects, fine cracks, shrinkage defects, wire drawing defects, cracking defects, crazing defects, claw-shaped wrinkles, delamination defects, fading defects, peeling defects, exposed substrate defects, heat loss defects, impact damage defects, interlayer fouling defects, mud cracking defects, orange peel defects, flaking defects, pinhole defects, wavy coating defects, sagging defects, sheet rust defects, pitting rust defects, rust spot defects, dent defects, settling defects, skinning defects, solvent undercut defects, solvent bubble defects, stress cracking defects, underfilm corrosion defects, and wrinkling defects.

[0035] According to an embodiment, the method further includes:

[0036] - Identify at least one type of coating surface defect to be identified;

[0037] - Automatically determine one or more illumination angles and / or one or more image acquisition angles that allow the acquisition of digital images, which allows the defect identification program to calculate the characterization of the coating surface depicted in the image with respect to the at least one determined defect type;

[0038] - Position one or more light sources at a defined illumination angle relative to the coating surface; and / or

[0039] - Position one or more cameras (preferably one camera) at a defined one or more image acquisition angle relative to the coating surface;

[0040] - After positioning the light source, camera, and coating surface relative to each other, the camera is used to acquire a digital image of the coating surface.

[0041] As a supplement to or alternative to the automatic positioning light source, the defect identification program can be configured to generate a feedback signal indicating that one or more light sources and the coating surface are adjusted relative to each other and / or that such adjustment can be made such that the illumination angle is within the predetermined illumination angle range for identification.

[0042] The illumination angle and / or image acquisition angle suitable for detecting surface defects can vary depending on the type of defect and / or the type of substrate used.

[0043] For example, to detect bubble defects, it is best to use a tilted lighting angle, such as between 0° and 70°, especially about 45°, and between 110° and less than 180°, especially about 135°, because this provides sufficient contrast for the bubbles to evoke shadows.

[0044] The image acquisition angle used for detecting bubble defects can preferably be between 0° and 360°, preferably between 0° and 180°, and more preferably any angle between 10° and 170°.

[0045] Conversely, pitting defects in the coating surface can be identified and characterized using substantially orthogonal image acquisition angles and orthogonal (background) illumination angles. For example, to detect pitting defects, it is preferable to use an image acquisition angle between 70° and 110°. For transparent substrates (such as glass or transparent plastic), the preferred illumination angle is between 250° and 290°, and one or more light sources are backlit (transparent light-emitting elements) positioned so that the light emitted by the light source passes substantially orthogonally through the transparent substrate and coating surface, and is then captured by one or more cameras. Substantially orthogonal illumination angles reduce light reflection and allow determination of coating thickness and / or pit depth at the bottom and sides of the pits. For opaque substrates, other illumination angles between 0° and 180° can be used, preferably orthogonal illumination angles, such as those between 70° and 110°.

[0046] Embodiments of the present invention that control the relative positions of one or more cameras, one or more light sources, and the coating surface, as well as the illumination angle and image acquisition angle, can have the advantage that the digital images provided as input to the defect identification program are ensured to be acquired from a similar angle and under similar lighting conditions, allowing for the identification and characterization of the drawn defects. When the defect identification program is based on a machine learning method, embodiments of the present invention that control the relative positions, illumination angle, and image acquisition angle can ensure that the conditions used to acquire the digital images are similar to the conditions used to obtain the training images from which the defect identification program is derived. By imposing strict control on the image acquisition process, embodiments can ensure that the acquired digital images are similar and can be reproducibly and accurately processed by the defect identification program.

[0047] According to an embodiment, the processing of digital images further includes:

[0048] - A digital image is classified using a defect identification procedure regarding the type and / or quantity of surface defects depicted therein and / or based on image semantic segmentation and / or object detection and / or image instance segmentation according to one or more types of surface defects depicted therein. This automatically assigns one or more labels to the entire digital image, image regions, and / or individual pixels, each label indicating the type of defect identified in the digital image; and

[0049] - Output one or more assignment tags.

[0050] According to some embodiments, a defect identification program (which may be based on a publicly available program such as Mask R-CNN and may also include one or more additional program modules or functions, such as for generating a GUI, for exchanging data with a network, database and / or THE, or for performing further prediction tasks) is configured to perform image classification and / or image semantic segmentation based on the type and / or type of one or more surface defects depicted therein, and / or perform object detection and / or instance segmentation of the image.

[0051] Different methods can be used to generate different types of labels for different types of defects and / or application scenarios. In some examples, combinations of two or more different types of labels can be assigned to images, image regions, or pixels.

[0052] For example, image classification is the process of classifying digital images of coating defects depicted within them. Category labels might look like "defect-free surface," "surface with bubble defects," or "surface with severe bubble defects." Category labels do not indicate the location or number of individual defect instances.

[0053] Semantic segmentation is the process of identifying one or more different defect types of coating defects at the pixel level, where information about the contours of individual defect instances is not available. This means that semantic segmentation provides information about the location of different defect types at the pixel level, but cannot identify individual defect instances when multiple instances of the same type overlap. A coating defect semantic label might look like "a bubble defect identified at pixel {xy coordinates of all pixels in a digital image depicting a bubble defect}" or "a pit defect identified at pixel {xy coordinates of all pixels in a digital image depicting a bubble defect}". Coating defect semantic labels mark the location of one or more defect types identified in a pixel-level image, thus allowing for a coarse quantification of defect severity. Defect type semantic labels can have the advantage of providing sufficient detail to identify the location and severity of one or more different defect types, but cannot allow for the identification of individual defect instances.

[0054] Object detection, as used in this paper, is the process of identifying one or more instances of coating defects of different types and their approximate location in a digital image. A coating defect object label might look like "Bubble defect instance #234, at pixel {xy coordinates of the (typically rectangular) bounding box containing this bubble defect instance #234}" or "Bubble defect instance #554, at pixel {xy coordinates of the (typically rectangular) bounding box containing this bubble defect instance #554}". Coating defect object labels mark individual instances of one or more defect types identified in an image, and also mark the approximate location of these defect instances in the image, but not at the pixel level, but at the bounding box level. Coating defect instance labels can be graphically represented in the form of a digital image, highlighting the bounding boxes predicted to contain instances of the identified defect types. Defect type object labels can have the advantage of providing a wealth of detailed information, allowing for the quantification of a particular defect type and the counting of defect type instances. Location information based on coarse-grained bounding boxes facilitates the storage and processing of these labels.

[0055] Defect instance segmentation is the process of identifying instances of one or more different defect types and their respective locations in an image at the pixel level. A coating defect instance label might look like "Bubble defect instance #234, xy coordinates of a set of pixels in the spot depicting this bubble defect instance #234" or "Bubble defect instance #554, xy coordinates of a set of pixels in the spot depicting this bubble defect instance #554". The coating defect instance label marks individual instances of one or more defect types identified in the image, and also marks the pixel locations of these defect instances in the image. The coating defect instance label can be represented graphically as a digital image, highlighting image segments representing instances of the identified defect types. For example, different segments can be graphically represented as segments with a specific color different from the color of the intact coating surface. In some embodiments, different segments representing different instances of the same type of defect have different colors. Additionally or alternatively, different segments representing instances of different defect types may have different colors. Defect type instance labels can have the advantage of providing a large amount of detailed information, but the storage and processing of these labels may require more resources.

[0056] Identifying the approximate (bounding box) or detailed (pixel-based) location of defect types and / or instances of defect types can have the advantage that the defect identification program can easily further process the location information, such as for calculating the fraction of image pixels covered by the defect, for calculating and highlighting the identified defect, for example, with ellipses, circles, ridges, contours, or segment edges. On the other hand, providing a graphical representation of pattern instances can have the advantage that the identified defects can be easily confirmed by humans. For example, the defect identification program can generate a graphical user interface (GUI) configured to display image segments identified as representing specific types of coating defects using their respective colors or shading. For example, image segments identified as representing bubbles or dents can be highlighted with a specific color, such as yellow. Alternatively, only the segment boundaries can be highlighted. Providing a combination of coordinate information and graphical representation can have the advantage that the output of the defect identification program can be easily processed and interpreted by both software and humans.

[0057] The applicant observed that region proposal networks, such as those provided by the Mask R-CNN program, can accurately identify and characterize a variety of coating defect types. However, according to some embodiments, other image analysis methods, particularly speckle detection methods, are used as an alternative to the Mask R-CNN program to identify coating defects and / or characterize coating surfaces including these defects.

[0058] For example, the YOLO neural network (J. Redmon and A. Farhadi: "Yolov3: Stepwise Improvement", arXiv, 2018) can be used to analyze digital images to identify coating defects.

[0059] Based on other examples, the applicant observed that one or more of the following spot detection image analysis methods can accurately identify and / or characterize one or more coating defect types, especially bubble defects and / or pit defects:

[0060] • Simple threshold setting: This method involves setting hard pixel color boundaries to segment a binary image into defects and background. "Color" can be the intensity value of a monochrome image or the intensity value of a color channel in a multi-channel image.

[0061] • Otsu threshold setting: This method includes automatically calculating a global threshold from the intensity histogram of the bimodal image.

[0062] • Adaptive or dynamic threshold setting: This method involves calculating thresholds for multiple small regions of the image, resulting in different thresholds for different regions.

[0063] According to other embodiments, one or more of the following spot detection methods are employed:

[0064] • "Finding contours": This method includes detecting contours, which are all points with similar intensity in an image;

[0065] • Edge detection: This method involves using measured edges, which can be described as discontinuous local features, to separate the background and blob;

[0066] • Clustering: This method involves using clustering techniques such as k-means to divide the image into similar pixel segments;

[0067] • Watershed Transformation: Watershed transformation methods include topographic representation. Figure 1 The image being processed is such that the brightness of each point represents its height, and a line extending along the top of the ridge is found.

[0068] According to an embodiment, the digital image is preprocessed to improve the accuracy of subsequent image processing steps. For example, image preprocessing may include one or more of the following:

[0069] • Flooding Fill: This method involves adjusting neighboring values ​​in the image based on their similarity to the initial seed point to minimize noise and improve blob detection;

[0070] • Morphological transformation: This method includes performing basic morphological operators erosion and dilation and reducing or increasing the number of pixels used for features on the image, such as edges;

[0071] • Smoothing: This method includes image preprocessing to remove noise, for example, by using a Gaussian filter.

[0072] According to an embodiment, the method further includes installing and / or instantiating the defect identification program on a data processing system including a graphical user interface (GUI). The data processing system may include or be effectively coupled to a camera. The defect identification program can be configured to generate the GUI, which is displayed to the user via the screen of the data processing system. In response to user actions using the GUI, the defect identification program acquires images of the coating surface via the camera. The defect identification program processes the acquired images as images to automatically identify defect patterns, calculate defect pattern measures, and calculate qualitative and / or quantitative characterizations of the coating surface. The defect identification program then executes the output of the calculated characterizations via the GUI or another output interface of the data processing system.

[0073] According to an embodiment, the defect identification program is effectively integrated with the camera and designed for use with

[0074] - Determine whether the camera is located within a predetermined distance range and / or a predetermined image acquisition angle range relative to the coating surface; for example, this distance range and / or angle range may be adjusted to allow image acquisition from a relative position similar to that used to acquire training images to generate a predictive model for a defect identification procedure; alternatively or additionally, the predetermined distance range may be adjusted to allow image acquisition with at least a predetermined minimum resolution (pixels / drawn surface area); and

[0075] Based on the determined results:

[0076] - Generate feedback signals for the user and / or camera to determine whether camera position adjustment is needed; and / or

[0077] - Automatically adjust the relative position of the camera and the coated surface; and / or

[0078] - Selectively allow the camera to capture images when the camera is within a predetermined distance range and / or image acquisition angle range.

[0079] This can have the following advantages: if the current settings of the image acquisition system do not allow the acquisition of digital images under the conditions required by the defect identification program to perform accurate and reproducible defect identification and coating surface characterization processes, then the acquisition and analysis of images of the coating surface are prohibited from the outset.

[0080] For example, the predetermined distance range and / or predetermined image acquisition angle range may be the following ranges, which are explicitly or implicitly given before a digital image of the currently presented coating surface is acquired.

[0081] The resolution of a digital image specifies the number of pixels used to depict a coated surface area of ​​a certain size within that image. For example, resolution can be given as pixels per square centimeter or pixels per square inch.

[0082] According to some examples, a predetermined minimum resolution can be provided in the form of an absolute resolution value, such as a surface drawn at least 5px × 5px / 0.05 square millimeters, where "px" is a pixel. More preferably, the predetermined minimum resolution is a surface drawn at least 10 pixels × 10 pixels / 0.05 square millimeters, or at least 15 pixels × 15 pixels / 0.05 square millimeters.

[0083] According to other examples, a predetermined minimum resolution is provided in the form of a range of absolute resolution values. The lower limit of the range may be, for example, a surface drawn of at least 5 pixels × 5 pixels / 0.05 square millimeters, or a surface drawn of at least 10 pixels × 10 pixels / 0.05 square millimeters, or a surface drawn of at least 15 pixels × 15 pixels / 0.05 square millimeters. The upper limit of the range may be, for example, a surface drawn of 10.000 × 10.000 pixels / 0.05 square millimeters.

[0084] For example, for the defect type "microbubble", the minimum resolution can be a surface drawn at 8 pixels × 8 pixels / 0.05 square millimeters. As used in this article, "microbubble defect" refers to a bubble defect characterized by a bubble diameter of less than 0.5 mm.

[0085] Additionally or alternatively, for the defect type "macrobubble", the minimum resolution can be a surface drawn at 5 pixels × 5 pixels / 0.6 square millimeters. As used herein, "macrobubble defect" refers to a bubble defect characterized by a bubble diameter of at least 0.5 mm.

[0086] Alternatively, for the defect type "pit", the minimum resolution can be an 8-pixel × 8-pixel / 0.5 square millimeter surface.

[0087] According to some embodiments, the defect identification program is configured to computationally reduce the resolution of a digital image when the resolution exceeds an upper resolution range, such that the reduced digital image resolution falls within a predetermined resolution range. This may have the advantage that the image with the reduced resolution range has the same or sufficiently similar resolution as the training images used to train the prediction model. This ensures that the defect identification program can accurately perform predictions even when the resolution of at least one camera should be much higher than the training image resolution (considered a source of error).

[0088] According to an embodiment, the predetermined minimum resolution and / or predetermined resolution range are specific to the defect type. For example, for each of one or more different defect types, a predetermined minimum resolution or resolution range may be stored in the storage medium in association with a defect type identifier for that defect type.

[0089] The applicant has observed significant size variations among different coating defect types. The applicant further unexpectedly observed that the resolution of the training images, adapted to the size of the coating defect types to be learned during training, can strongly influence the computational resources required during training and the accuracy of the generated predictive model: if the resolution is too high, training will require significant CPU capacity, memory, and time, potentially leading to suboptimal results in some cases. If the resolution is too low, the accuracy of the trained model will be poor. However, because different defect types have different sizes and / or may have different degrees of wire clamping, the minimum and / or optimal resolution range to be used during training and testing has been observed to depend on the defect type.

[0090] According to an embodiment, the defect identification program dynamically receives the selection of one or more defect types to be identified, and dynamically determines, for each selected defect type, the distance between at least one camera and the surface, ensuring that the resolution of the image to be acquired is at least the minimum resolution stored in the data storage medium associated with a defect type identifier. Alternatively, for each selected defect type, the defect identification program dynamically determines the distance between at least one camera and the surface, ensuring that the resolution of the image to be acquired represents the optimal resolution range for that defect type and is stored in the data storage medium associated with a defect type identifier. The defect identification program then triggers relative movement between at least one camera and / or the surface being presented, allowing the camera to acquire images at a resolution that is at least the minimum resolution specific to the defect type and / or within the optimal resolution range for that defect type. When multiple defect types are selected, the relative distance is repeatedly adjusted for each selected defect type. Additionally or alternatively, the defect identification program outputs a feedback signal that allows the user to adjust the relative position manually or semi-automatically.

[0091] The predetermined image acquisition angle range may depend, for example, on the image acquisition angle of at least one camera used to acquire training images, and can be selected such that the image acquisition angle of at least one camera to be used to acquire images of the currently presented coated surface is sufficiently similar to the image acquisition angle used to acquire training images. The term "similar" may, for example, mean having the same image acquisition angle as the image acquisition angle used to acquire training images + / - < 5%, or having the same image acquisition angle + / - < 10%, or having the same image acquisition angle + / - < 15%, or having the same image acquisition angle + / - < 20%, or having the same image acquisition angle + / - < 40%.

[0092] According to some examples, the projection distance range is the distance range between at least one camera (for acquiring a digital image of the currently presented coated surface) and the presented coated surface, which will result in the acquired digital image having a resolution of at least a minimum resolution, wherein the minimum resolution is a defect type-specific minimum resolution associated with a particular defect, wherein the prediction model is trained to identify the defect type based on training images having at least said minimum resolution.

[0093] Dynamically adjusting the camera-surface distance can be advantageous because the type and / or configuration of at least one camera used to acquire a digital image of the currently presented coated surface may differ from the type and / or configuration of the camera used to acquire training images. Automatically determining whether the distance of the at least one camera to the surface region allows for the acquisition of a digital image with a resolution of at least a predetermined minimum resolution or within a predetermined resolution range allows assurance that the camera is allowed to acquire a digital image only if the predictive model is likely to be able to identify one or more different types of coating defects. Similarly, automatically determining whether the at least one camera has an image acquisition angle that allows for the acquisition of digital images from angles similar to those used for acquiring training images allows assurance that the camera is allowed to acquire digital images only if the acquisition angle is similar to that used for acquiring training images.

[0094] According to an embodiment, the predetermined image acquisition angle range is specific to the defect type. For example, for each of one or more different defect types, the respective predetermined image acquisition angle range can be stored in a storage medium in association with a defect type identifier.

[0095] According to some embodiments, the training images used to train the predictive model for a defect identification procedure depict not only the coating surface but also a reference object of known size. For example, the reference object may be positioned on or next to the coating surface. The reference object can be, for example, a coin, a piece of paper, or any object of known size. Reference objects of the same type are placed on or next to the coating surface currently presented to at least one camera. According to some embodiments, during the training phase of the predictive model, the number of pixels used to depict the reference object in each training image is determined to calculate the resolution of each training image in relation to the known reference object size and the determined number of reference object pixels. According to some examples, the resolution calculation is based, for example, on preprocessed training images stored as part of the training data. In some examples, a minimum resolution used to obtain the training images (for acquiring all training images or only images for one or more specific defect types) is used as a lower bound of a predetermined resolution range.

[0096] According to some embodiments, the defect identification program can be configured to determine the expected resolution of an image before acquiring an image of the currently presented coated surface, assuming a current distance between the at least one camera and the presented surface area and / or assuming a current configuration of the at least one camera. For example, the defect identification program may be communicatively coupled to at least one camera and can request the expected resolution directly from the at least one camera, assuming the current distance and / or configuration. According to other embodiments, the expected resolution is read from or calculated based on a configuration file or other data source, including the current settings of the at least one camera.

[0097] According to other embodiments, the expected resolution is determined based on a reference object. A reference object of known size is located on or next to the coated surface currently presented to at least one camera. The at least one camera captures a preview image depicting at least the reference object and determines the number of pixels depicting the reference object in the preview image. The determined number of pixels in the preview image depicting the reference object is the expected resolution of the at least one camera in its current state. The expected resolution is compared to a predetermined resolution range to determine whether the current expected resolution is within the predetermined resolution range. For example, if the current distance between the at least one camera and the surface is determined to be below a predetermined minimum resolution (and / or outside the expected distance range), this can be used as an indication that the expected resolution will not allow the defect identification procedure to accurately identify coating defects.

[0098] When the expected resolution is equal to or higher than the minimum resolution, the user is allowed to manually capture digital images through the at least one camera, or to have the at least one camera automatically capture images.

[0099] In cases where the expected resolution is lower than a predetermined minimum resolution (and / or outside the predetermined resolution range), the distance between at least one camera and the currently presented coated surface is changed such that the changed distance is within the expected distance range, thereby ensuring that the expected resolution of the camera at the changed position is equal to or higher than the minimum resolution (and / or within the predetermined resolution range).

[0100] According to an embodiment, the image acquisition angle is similarly adjusted. For example, computational recognition of the shape of the profile, camera interface, and / or reference object depicted in the preview image (which only occurs when viewed from a specific angular range) is used to determine the expected image acquisition angle, assuming the current distance, position, and orientation of at least one camera.

[0101] Feedback signals can help users correctly position the camera and / or modify the settings of the image acquisition system to ensure suitable image acquisition conditions. For example, the feedback signal can show the user in which direction or at what angle they must move or rotate the camera to position it appropriately relative to the coated surface. In some cases, feedback is output via a GUI. Alternatively or supplementarily, feedback signals may include audible alarms.

[0102] For example, the user, application, or configuration file can select or determine the types of defects to be identified, such as through a GUI or configuration file. The selection or determination of the types of defects to be inspected and identified in an image can thus determine which image acquisition angles and / or distances are considered permissible for identifying specific types of defects. This can improve defect identification accuracy, as it is observed that different types of defects can be best identified under varying lighting and / or image acquisition conditions.

[0103] According to other embodiments, a corresponding feedback signal is generated and provided to the image acquisition unit of an apparatus for automatically producing and / or testing coating compositions. The feedback signal may include control commands that cause the light source and / or camera included in the image acquisition unit to automatically change their relative position, illumination angle, and / or image acquisition angle relative to the coating surface, such that the image acquisition unit is allowed to acquire a digital image of the coating surface as required by the defect identification process within the unit for proper image analysis.

[0104] For example, the feedback signal could be a signal for a human user to allow manual or semi-automatic correction of the camera position (distance and / or orientation). The feedback signal may additionally or alternatively include machine commands to allow the controller unit to automatically or semi-automatically correct the camera position.

[0105] Feedback signals may be, for example, an overlay image superimposed on a preview image of the coated surface captured by at least one camera, wherein the overlay image contains one or more GUI elements indicating whether the at least one camera is in a position suitable for capturing an image that can be successfully processed by the defect identification program.

[0106] According to an embodiment, the defect identification procedure is selected from the group consisting of:

[0107] - Application, wherein the data processing system is a fixed or portable data processing system, such as a general-purpose data processing device and especially a portable telecommunications device such as a laptop, smartphone, tablet, or other general-purpose data processing device can be used as a portable data processing system. The camera can be a built-in, internal device camera or an external camera such as a digital camera or camcorder. The advantage of using a general-purpose data processing system is that no additional dedicated hardware is required to enable company employees to perform accurate and repeatable quality checks on the coating composition and / or coating surface produced by the company.

[0108] - Applications and data processing systems are portable or stationary devices designed specifically for coating surface quality inspection. For example, specially designed quality inspection devices may include additional components such as cameras and / or lighting sources, whose position relative to the coating surface can be controlled by a defect identification program. Embodiments of the present invention can ensure that these devices can automatically identify and characterize coating surface defects and the quality of the inspected coating surface in a reproducible and accurate manner.

[0109] - The application and data processing system is a high-throughput (HT) facility (also known as a "high-throughput equipment - HTE") for the automated or semi-automated production of coatings. In particular, the high-throughput equipment can be a device that includes an automated image acquisition unit as described herein with respect to an embodiment of the invention. Using a defect identification program in an HT equipment environment may be particularly advantageous because the HT equipment can automatically produce and test many different coating compositions and coating surfaces, thereby creating a large amount of data that can again be used to train machine learning models to identify and / or predict coating compositions with the desired coating quality characterization. The automated generation and storage of qualitative and / or quantitative coating characterizations in an HT equipment environment allows for consideration of coating defects and coating quality characterizations in a variety of big data applications, especially machine learning-based predictions, which cannot be done based on manually created, subjective, and inconsistent quality labels.

[0110] - Web applications downloaded over the network and / or permanently or temporarily instantiated, such as a server providing a Java application that implements a defect detection program via the Internet;

[0111] - Programs that execute in a browser, such as JavaScript programs;

[0112] A server program instantiated on a server computer, which is effectively coupled via network communication to a client program instantiated on a client data processing system (e.g., a general-purpose computer, a device specifically designed for quality inspection, or a smartphone). For example, the client program could be configured to control one or more cameras to acquire digital images of one or more coated surfaces. The acquired digital images are sent via the network to the server program for image processing, and the processing results, including coating surface characterization, are output via the network to the client program. Furthermore, the client program can be configured to display the results provided by the server program.

[0113] "Portable devices" can be handheld devices like smartphones, but according to some examples, portable devices can also be larger devices that weigh up to the point where they can be carried by a person for at least several meters without the aid of technology. Typically, such devices weigh less than 50 kilograms, and especially less than 40 kilograms.

[0114] Combinations of the above embodiments are feasible. For example, the defect identification program can be implemented as a client-server system, where the client portion is implemented as a web application or browser executor and instantiated on a data processing system, responsible for acquiring digital images with sufficient quality and in the appropriate context, while the server portion is instantiated on a remote server computer and responsible for performing image analysis to identify coating defects and provide coating surface characterization. According to another example, the client program and client data processing system can be part of an apparatus for preparing and / or testing coating compositions, such as an HTE. The server program can also be, for example, a (remote) part of the HTE, serving as a server for multiple client programs belonging to multiple different HTEs and / or may not be part of an HTE. The server computer system can be a monolithic computer system or a distributed computer system, such as a cloud computer system.

[0115] According to an embodiment, the defect identification procedure includes a predictive model that has learned from training data to identify predetermined patterns during a training step performed by a machine learning program. In particular, the machine learning program may be a neural network.

[0116] According to an embodiment, the training data includes multiple labeled training images of multiple coated surfaces. For example, the training images may include digital images depicting coated surfaces obtained by applying coating compositions to many different types of substrates (e.g., wood, plastic, metal, paper, etc.) according to many different coating application protocols (e.g., once or multiple times, at different temperatures, using techniques such as spraying, smearing, painting, or dipping, after different types of pretreatment of the substrate, etc.). Additionally or alternatively, the training images may include digital images depicting coated surfaces obtained by applying many different coating compositions, where the different coating compositions have been obtained by combining different types and / or quantities of components and / or combining the components according to different production process parameters (e.g., mixing time, mixing temperature, mixing speed, etc.) and / or different application process parameters. Therefore, the training data can encompass a vast multidimensional data space covering a wide variety of substrates, coating compositions, coating composition production parameters, and coating application protocol parameters. The method also includes storing the training data in a database. Initially, the training data will be manually labeled. In subsequent training steps, the training data can be expanded with additional digital images of the coating surface, which have been automatically labeled and preferably checked or corrected by an annotator. The labels preferably include pixel-based indications of coating defect boundaries, coating defect types, and one or more quantitative and / or qualitative characteristics of the coating surface, which in some cases may be the same measure as one or more coating defects included in the respective training images.

[0117] According to an embodiment, the machine learning program is a neural network or a set of neural networks including a region proposal network. The region proposal network is configured to generate anchors in the input image to propose whether the anchors may contain an object (one of the defect patterns). An "anchor" may refer to a sub-region of the input image whose anchor size covers the size of the defect pattern to be detected.

[0118] For example, the average size of certain types of defects, such as bubbles, foam depressions, etc., may already be known. The size range of these defects can be used as the expected size range for the defect pattern. The region proposal network is configured to create and detect anchor points whose sizes cover the expected size range of one or more defect types to be identified, in order to detect defects (such as “bubbles” and “pits”). According to some embodiments, multiple different anchor point sizes are defined by the user during training, for example, anchor points from a set of anchor points with sizes of 8×8 pixels, 16×16 pixels, 32×32 pixels, 64×64 pixels, and 128×128 pixels can be used.

[0119] The “Region Proposal Neural Network” (RPN) used in this paper is a neural network configured to operate on many different sub-regions of the input image, called “anchors”, which are adapted to make “proposals” for objects appearing in the sub-regions.

[0120] According to a preferred embodiment, the region proposal neural network is a region proposal network included in the “Mask R-CNN” model, which is described, for example, in “Mask R-CNN” (Kaiming He and Georgia Gkioxari and Piotr Dollár and Ross Girshick, 2017, eprint 1703.06870, arXiv:1703.06870). “Mask R-CNN” is a flexible neural network-based procedure designed to efficiently detect objects in images while generating a high-quality segmentation mask for each instance. The Mask R-CNN procedure extends Faster R-CNN by adding a branch for predicting object masks in parallel with an existing branch for bounding box prediction. Mask R-CNN has four main outputs: a class label, a score, and a bounding box for each candidate object (candidate defect type instance). Additionally, an object mask can be provided. Mask R-CNN adds only a small amount of overhead to Faster R-CNN. The applicant observed that Mask R-CNN is particularly suitable for identifying many different types of coating surface defects because it is easily generalized to many different tasks within the same framework. Mask R-CNN can... https: / / github.com / matterport / Mask_RCNNDownload it here. The online article "Splash of Color: Instance Segmentation with Mask R-CNN and TensorFlow" by Waleed Abdulla, written on March 20, 2018, describes the use of Mask R-CNN for instance segmentation of other object types. This article is available online at https: / / engineering.matterport.com / splash-of-color-instance-segmentation-with-mask-r-cnn-and-tensorflow-7c761e238b46. The Faster R-CNN method was developed by Shaoqing Ren and Kaiming He, along with Ross Girshick and Jan Sun in "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks" (2015, eprint 1506.01497), which is available online at [link missing]. https: / / arxiv.org / abs / 1506.01497 It is described in the document (obtained).

[0121] In one example, a neural network consists of a "backbone" network and a Region Proposal Network (RPN). The "backbone" network can be a standard convolutional neural network (typically ResNet50 or ResNet101), which is first applied to the input image and used as a feature extractor. The first few layers of the "backbone" network detect low-level features (edges and corners), while the later layers detect higher-level image features in sequence. The RPN is preferably trained to generate region proposals based on feature vectors derived from the input image provided by the backbone neural network.

[0122] The operation of Mask R-CNN can be summarized as follows:

[0123] The image is processed through a "backbone network" to generate feature maps;

[0124] • The Region Proposal Network (RPN) generates multiple Regions of Interest (RoIs) using a lightweight binary classifier. It does this by utilizing anchor boxes positioned above the image. An "anchor point" or "anchor box" can be understood as a set of bounding boxes covering a certain size range to capture the scale and aspect ratio of a specific defect type to be detected. In various embodiments, multiple different anchor points are processed in parallel on a CPU (Central Processing Unit) or GPU (Graphics Processing Unit). This significantly improves processing speed for detecting defect instances of one or more different defect types. Anchor boxes also facilitate the detection of multiple objects, objects of different scales, and overlapping objects without scanning the image with a sliding window that computes a separate prediction at each potential location. This enables real-time object detection. For each anchor point, the RPN predicts whether the anchor point contains an object (foreground class) or does not contain an object (background class). Furthermore, for all foreground anchor points, bounding box refinement is performed to better fit the object (coating defect). These regions are also referred to as Regions of Interest (RoIs).

[0125] • For each RoI, predict one proposal. Each proposal is a combination of the RoI probability score describing a specific type of object (e.g., 93% for bubble defects) and the object's category / label (e.g., "bubble defect", "pit defect", or "background").

[0126] • Further refine the region of interest by performing additional bounding box refinement steps.

[0127] Furthermore, segmentation masks are predicted for positive anchor points (containing the object's region of interest), indicating the identified defect instances at the pixel level. For example, Mask R-CNN has been observed to identify and characterize foam defects with high accuracy.

[0128] The steps described above illustrate how existing and / or trained defect identification programs can be applied to new (test) images that do not contain labels indicating the presence, type, or extent of coating defects. Embodiments of methods for generating and training predictive models for the defect identification program are described below. The testing and training phases can be performed on the same or different data processing systems. For example, model M1 can be trained on a first data processing system, the trained model M1 integrated into the defect identification program, which may include additional functions or modules for interacting with users and / or equipment used to produce or test coatings, and the defect identification program transferred to a second data processing system.

[0129] Training phase of the defect identification program model (M1)

[0130] According to an embodiment, the method includes a training step based on training data, which includes a set of labeled digital training images of a coated surface, the labels identifying the location / position and / or type of defects in the training images. A prediction model is trained to identify patterns using backpropagation through the labeled training images.

[0131] For example, training images may each have an assigned label indicating whether the image depicts one or more defect types for which the location and / or extent of unspecified coating defects are not indicated. Preferably, training images include images of various coated surfaces without any defects, training images include defects of a single defect type at different degrees, and / or training images include a mixture of two or more different types of defects.

[0132] The labels for the training data may include image category labels, which indicate the type of defect depicted in the image (if any), but without defect location information. Additionally or alternatively, the labels may include semantic defect type labels, object recognition labels, and / or defect type instance labels, which provide information about the location of the defect type or defect type instance at the pixel or bounding box level.

[0133] Additionally or alternatively, the training images according to embodiments of the invention have been assigned quantitative measures of one or more defects depicted in the training images, such as the size and / or severity of the defects or the number of bubbles. These parameters, along with the assigned labels and quantitative measures, are processed during the training step to enable the prediction model to correlate the parameters with the defect pattern and the measure of coating defects on the coating surface.

[0134] According to an embodiment, the defect identification procedure may include multiple prediction models M1.1, M1.2, and M1.3, each having learned to associate defect-related image annotations with intensity or color patterns in a digital image. For example, prediction model M1.1 might be a region proposal network of a Mask-RCNN program, which has learned to associate labeled defects with pixel patterns, while model M1.2 might be another neural network that has learned to associate defect metrics with other parameters stored in relation to the respective digital image with labeled defects. The defect identification procedure includes multiple prediction models M1.1, M1.2, and M1.3 and integrates and / or synthesizes these models and their results to perform predictions.

[0135] Providing a defect identification procedure during machine learning has the following advantages: the resulting predictive model M1 will learn many highly complex interrelationships between image patterns and various defect types. The applicant has observed that integrating these complex and mostly implicit interrelationships into a predictive model through one or more explicitly defined rules is extremely difficult, and sometimes even impossible.

[0136] According to an embodiment, at least some of the training images also have assignment parameters related to the coating composition, which is used to generate the coated surface drawn in the training images.

[0137] These additional optional parameters assigned to the training images used to train prediction model M1 (or one of several other prediction models M1.2 to be used by the defect identification program) can include one or more of the following:

[0138] - An indication of one or more components of the coating composition used to generate a coating surface plotted in a training image summary.

[0139] - An indication of the absolute or relative amount of one or more components of the coating composition, for example, this parameter may include a detailed description of the type and / or amount of one or more defoamers used to generate the coating composition of the surface being painted, and / or

[0140] - One or more production process parameters, which characterize the process of generating the coating composition, including, for example, the mixing rate, mixing temperature, and / or mixing duration of the coating composition, and / or

[0141] - One or more application process parameters, which characterize the process of applying the coating composition to a substrate. These parameters particularly include the amount of coating composition applied per unit surface area, the substrate type, and / or the type of application equipment (such as a machine or apparatus). The application process parameters may also include parameters indicating the pretreatment method of the substrate on which the coating composition is applied, such as drying, heating, cleaning, or other substrate preparation methods.

[0142] - System parameters of the imaging system used to acquire training images, selected from the following group: type of light source used to illuminate the coating surface, light source brightness, illumination angle, light source wavelength, type of one or more cameras used to acquire digital images of the coating surface, image acquisition angle, and position of one or more cameras.

[0143] Training the predictive model to be integrated into the defect identification procedure based on the aforementioned contextual data may be advantageous, as the applicant has observed that the aforementioned contextual parameters can affect the number and type of defects to be observed on the coating surface, thereby affecting the coating surface quality. Labeling training images with the aforementioned contextual data and / or defect quantitative measures ensures that the trained predictive model M1 can consider any factors that may affect the type and extent of coating defects and the characterization of coating surface quality.

[0144] Other embodiments

[0145] In another aspect, the present invention relates to a computer-implemented method for providing coating composition-related prediction programs, such as composition quality prediction programs and / or coating composition specification prediction programs. The method includes:

[0146] - Provide a database that includes qualitative and / or quantitative characterization of the coated surface and its association with one or more parameters selected from the group consisting of one or more components of the coating composition used to produce the respective coated surface, the relative and / or absolute amounts of one or more said components, the production process parameters of the coating composition, and / or the application process parameters used to produce the coated surface.

[0147] - A machine learning model is trained based on the association between the coating surface characterization and one or more parameters in a database to provide a predictive model (e.g., a predictive model referred to in this paper as model "M2" or "M3") that has learned to associate the qualitative and / or quantitative characterization of one or more coating surfaces with one or more parameters stored in relation to the respective coating surface characterization.

[0148] - Provides a composition quality prediction program including a prediction model (M2), configured to use the prediction model (M2) to predict the properties of the coating surface to be produced from one or more input parameters selected from the group consisting of one or more components of the coating composition to be used to produce the coating surface, relative and / or absolute amounts of the one or more said components, production process parameters to be used to prepare the coating composition, and / or application process parameters to be used to produce the coating surface; and / or

[0149] - Provides a composition specification prediction program including a prediction model (M3) configured to use the prediction model (M3) to predict and output one or more parameters related to the predicted coating composition that produces a coating surface having the input surface characterization, based on inputs that at least explicitly specify the desired coating surface characteristics. The one or more output parameters are selected from the group consisting of one or more components of the coating composition, relative and / or absolute amounts of the one or more components, production process parameters to be used to prepare the coating composition, and / or application process parameters to produce the coating surface. Optionally, the composition specification prediction program is configured to receive an incomplete coating composition specification and use the specification to limit the solution space of the predicted output parameters.

[0150] Embodiments of the present invention offer the advantage of providing a composition quality prediction procedure that can automatically predict the quality characterization of a particular coating composition in relation to the type and / or amount of the composition components, and optionally in relation to other parameters. This can significantly accelerate the process of testing and identifying coating compositions capable of providing a coated surface with desired quality characteristics. Unlike prior art methods that rely on human experience and involve the production and testing of typical large quantities of coating compositions on a workbench, embodiments of the present invention can allow for accurate prediction of whether a particular coating composition will have the desired coating surface characteristics. This can significantly accelerate the process of identifying suitable coating compositions and reduce the costs associated with the reagents, machinery, and consumables required to perform this identification process.

[0151] While the predictive model M1 for the defect identification procedure is best obtained by performing machine learning steps on manually annotated training images and has learned to correlate pixel patterns with defect type characterization metrics and coating surface characterization, the predictive model M2 for the composition quality prediction procedure can be trained on training data, which may include, but preferably do not include, digital images. The purpose of predictive model M2 is to predict coating surface characterization, particularly quality-related characterization, in relation to one or more components of the coating material composition and optional contextual parameters.

[0152] According to an embodiment, the method includes providing multiple images, each depicting a coated surface. The depicted coated surfaces are produced by applying multiple different coating compositions to respective surface samples. At least some of the depicted coated surfaces each have one or more coating defects of one or more different defect types. The method includes applying a defect identification procedure to the images to identify patterns in the images, obtaining a measure of the coating defects represented by the identified patterns, and calculating a qualitative and / or quantitative characterization of the coated surfaces depicted in the images. The method also includes storing the qualitative and / or quantitative characterization of the coated surfaces in a database in association with one or more parameters associated with the coating compositions used to produce coated surfaces including these defects, to provide training data to predictive models (M2, M3). These parameters may, for example, be selected from the group consisting of: one or more components of the coating composition, the relative and / or absolute amounts of one or more said components, production process parameters for preparing the coating composition, and / or application process parameters for producing the coated surfaces.

[0153] This can be advantageous because automated defect identification and coating surface characterization allow for the automatic annotation of large numbers of images in a reproducible, similar manner. Providing a large, unbiased training dataset ensures that the model M2 trained on the training dataset can accurately predict the properties of the coating composition, especially the quality of the coating surface obtained by applying the composition to the substrate.

[0154] According to the embodiments, the defect identification procedure is a defect identification procedure that is described in detail in one of the embodiments or examples described herein.

[0155] According to an embodiment, qualitative and / or quantitative measures have assigned parameters that are considered in the machine learning step for generating predictive models (M2, M3) to be used by the composition quality prediction procedure or composition specification prediction procedure. The parameters include:

[0156] - An indication of one or more components of the coating composition, for example, the indication of one component may specify a particular chemical compound such as a particular pigment or a particular defoamer or a category of substances such as "solvent-based coating medium";

[0157] - An indication of the absolute or relative amount of one or more components of the coating composition, for example, the indication may specify the relative amount of two defoamers;

[0158] - One or more production process parameters, which characterize the production process of the coating composition, including, for example, the mixing rate and / or mixing duration of the coating composition; and / or

[0159] - One or more application process parameters, which characterize the process of applying a coating composition to a substrate, including, in particular, the amount of coating composition applied per substrate area, substrate type and / or type of application equipment.

[0160] Data files or records containing the above parameters are also referred to as the “specification” of the coating composition. A specification is complete if it clearly specifies the properties and quantities of each component and clearly specifies all further information required in their respective application environments to prepare the coating composition and the corresponding coated surface. Typically, specifications are incomplete: for example, the properties and / or quantities of at least some components may not be clearly specified, or certain coating composition manufacturing process parameters and / or coating composition application parameters may not be clearly specified.

[0161] According to an embodiment, the method includes using a composition quality prediction procedure to predict the properties of the coating composition. The coating composition may be a paint, varnish, printing ink, abrasive resin, pigment concentrate, or other mixture for coating a substrate.

[0162] According to an embodiment, the method includes using a composition specification prediction procedure to predict one or more of the above parameters (the properties and / or amount of the coating composition, production process parameters, application process parameters) based on the desired coating surface characterization provided as input. These parameters can be used to refine and / or optimize the specifications of the coating composition.

[0163] According to a preferred embodiment, a combination of two models, M2 and M3, is used to supplement and / or optimize the coating composition specification. For example, a composition quality prediction program using prediction model M2 can receive the coating composition specification as input, which also includes certain production and / or application process parameters. Based on the received specification, the composition quality prediction program can predict that the coating surface may include numerous bubble defects. The user can then input the coating composition specification predicted to minimize bubble defects into the composition specification prediction program using model M3, where the coating composition specification can be provided to the composition specification prediction program as additional input and constraints for prediction. Model M3 will determine that changing the ratio of the two defoamers will result in a reduction of bubble defects and can output a modified version of the input specification, which includes the changed ratio of the two defoamers.

[0164] Coating compositions are complex mixtures of raw materials. Common compositions, formulas, or formulations of coating compositions contain about 20 raw materials, hereinafter also referred to as "components." For example, these compositions consist of raw materials selected from solids, such as pigments and / or fillers, other binders, solvents, resins, hardeners, and various additives such as thickeners, dispersants, wetting agents, adhesion promoters, defoamers, surface modifiers, leveling agents, catalytically active additives such as desiccants and catalysts, and specialized effective additives such as pesticides, photoinitiators, and corrosion inhibitors.

[0165] To date, new compositions, formulations, and reformulations with certain desired properties have been specified based on empirical values, then prepared and tested. The composition of new compositions that meet certain expectations in terms of chemical, physical, optical, tactile, and other measurable properties, particularly in terms of the quality / defects of the coated surface obtained by coating a substrate with the composition, is difficult to predict even for experts due to the complexity of interactions and the lack of high-quality data suitable for machine learning training steps. This process is both time-consuming and costly due to the diversity of interactions between raw materials and process parameters and the associated large number of failed experiments.

[0166] Therefore, there are currently very narrow limitations on the human and computer-aided evaluation and prediction of the components of compositions having desired properties. This is especially true for complex compositions with many related properties and many components, such as in the cases of paints, varnishes, printing inks, abrasive resins, pigment concentrates, or other coatings, because these components interact in complex ways and determine the properties of the corresponding coating composition. It has been found particularly challenging to identify one or more defoamers, individually or in combination, that can minimize the formation of bubble defects.

[0167] Currently, new coating surfaces must first be produced in a real laboratory environment, and then their properties must be measured to evaluate whether the new coating composition possesses certain desired properties. Although methods for automatically predicting chemical properties exist, creating a training dataset of sufficient size and quality is often even more complex than directly producing and testing the relevant compositions. The development of new compositions in the fields of paints, varnishes, printing inks, abrasive resins, pigment concentrates, or other coatings is particularly complex and time-consuming.

[0168] According to the embodiments, the use of the composition quality prediction procedure includes:

[0169] - Provides each of many specifications of numerous different candidate coating compositions as input to the composition quality prediction procedure;

[0170] - A composition quality prediction procedure is used to predict the quality of the coating surface produced by applying the candidate coating composition to a substrate for each candidate coating composition, and this prediction is performed in relation to the specifications of the candidate coating composition;

[0171] - A candidate coating composition specification is selected based on its respective metrics. This selection can be done manually, such that one of the candidate coating composition specifications whose predicted coating surface properties meet the quality requirements is chosen. According to other embodiments, one or more parameters indicate one or more desired surface characteristics, because quality criteria are provided as input to the composition quality prediction procedure, thereby allowing the composition quality prediction procedure to automatically perform this selection step based on the input quality criteria.

[0172] - As a prediction of the recommended coating composition with the highest quality output, the selected specifications; and / or

[0173] - Input the selected candidate coating composition specifications to the processor, which controls the equipment for producing and / or testing the composition for the coating composition, wherein the processor operates the equipment to produce the input coating composition.

[0174] For example, the prediction is generated by a computer system connected to equipment used for producing and / or testing compositions for coating compositions. This equipment may be an HT device.

[0175] According to an embodiment, the method further includes:

[0176] - Receive incomplete specifications of the coating composition;

[0177] - A set of candidate coating composition specifications is generated manually or automatically. This set of candidate compositions consists of different versions of the received incomplete coating compositions. The generation of these candidate coating composition specifications includes:

[0178] a) Supplement the specifications of the incomplete coating composition with one or more additional components; and / or

[0179] b) Supplement the specifications of the incomplete coating composition with different absolute or relative amounts of one or more of the other components and / or with different absolute or relative amounts of the explicitly specified components; and / or

[0180] c) Compensate for the specifications of incomplete coating compositions using one or more process parameters characterizing the production process of the candidate coating composition; and / or

[0181] d) Supplement the specifications of the incomplete coating composition with one or more application process parameters that characterize the process of applying the candidate coating composition to the substrate.

[0182] For example, an incomplete specification may clearly state that the (desired) coating composition to be produced should contain / be based on an organic solvent, but the incomplete specification does not specify which of the many available solvents should be used. Candidate compositions can thus be generated by using different organic solvents (but not water-based solvents) as solvent components.

[0183] According to another example, the received incomplete specification may explicitly state (expected) that the coating composition should contain two specific defoamers (e.g., TEGO Foamex 810 and TEGO Wet 285, Evonik), but not their absolute or relative amounts. Candidate compositions can be generated by using different ratios of the two defoamers.

[0184] As another example, an incomplete specification may explicitly state that a particular pigment or combination of pigments should be used, but does not mention the amount of pigment or only provides a range of pigment amounts. Candidate compositions may be coating compositions that differ from each other in terms of pigment amounts.

[0185] According to another example, an incomplete specification may explicitly state the amounts of all or at least most of the components of the coating composition, but without mentioning contextual parameters, particularly manufacturing process parameters such as mixing time, mixing duration, mixing temperature, etc., and / or coating application process parameters (spreading, spraying, coating, impregnation, substrate pretreatment, temperature, ventilation, etc.). Candidate compositions may be coating compositions that differ from each other in terms of the parameter values ​​of one or more of the aforementioned contextual parameters.

[0186] According to an embodiment, additional data to supplement the candidate coating composition specifications in each of the following cases (a), (b), (c), and / or (d) are used as input to perform predictions by the coating composition quality prediction program.

[0187] For many reasons, embodiments of the present invention may be advantageous for the production of coating additives and coating compositions: manufacturers of coating compositions can predict the performance of a wide variety of candidate compositions by combining various contextual parameter values ​​without actually producing and testing these coating compositions.

[0188] However, embodiments of the present invention are also useful for manufacturers of individual coating composition components such as solvents, binders, pigments, foaming agents, defoamers, rheology additives, flame retardants, dispersants, etc. For example, a defoamer manufacturer may receive an order for a defoamer in a coating composition for wood materials from a coating composition manufacturer. The received specifications may indicate that the coating will be applied to wood materials using water as a solvent, but the detailed specifications of the coating composition are not disclosed for confidentiality reasons. Nevertheless, the defoamer manufacturer still wants to identify one or more defoamers, ideally the type and amount of a mixture of defoamers, and the amount of solvent that best provides the coating for wood materials with high coating surface quality. To do this, the additive manufacturer's employees or software programs generate multiple candidate coating composition specifications, which differ in the type of defoamer, its mixture and / or amount, and / or the proportion of defoamers used. Each candidate coating composition specification includes an indication that the coating will be used to coat the wood material. The composition quality prediction program then uses each candidate coating composition specification as input to predict the characterization of the respective candidate coating composition, particularly characterizing the quality of the coating surface produced by applying the candidate coating composition to a wood substrate. Because the composition quality prediction program has learned to correlate many component types and / or amounts, as well as optional contextual parameters, with coating surface characterization, it is able to identify suitable defoamer combinations. Defoamer manufacturers can provide defoamer mixtures along with recommendations regarding the relative or absolute amounts of defoamer and solvent, and / or with recommendations for process parameters related to the production, storage, and / or application of the coating composition.

[0189] Its composition quality prediction procedure is configured to generate specifications for recommended coating compositions that explicitly define one or more of the above parameter types a)-d) may be advantageous because the applicant has observed that both composition and process parameters have a significant impact on coating surface properties and quality. However, due to the vast number of possible combinations of components and process parameters, it has been impossible to evaluate all combinations that correspond to high-quality coating surfaces to date.

[0190] The above embodiments are based on providing multiple (hypothetical) candidate coating composition specifications, predicting the surface quality of each coating, and selecting one of the candidate coating compositions whose predicted surface properties appear to be the most desirable. However, an alternative approach may utilize learned relationships between coating surface properties and coating composition-related parameters, which have been more directly incorporated into the prediction model M3 to identify promising coating compositions.

[0191] According to the embodiments, the use of the coating composition specification prediction procedure includes:

[0192] - Provide at least the specifications for the desired coating surface characterization as input to the composition specification prediction procedure;

[0193] - A composition specification prediction procedure is used to predict the specifications of a coating composition suitable for providing a coated surface with desired surface properties, wherein the specifications include parameters selected from the group consisting of one or more coating composition components, absolute or relative amounts of one or more coating composition components, production process parameters, and / or application process parameters.

[0194] According to a preferred embodiment, the method further includes outputting a predicted specification of the coating composition to a person and / or inputting the specification of a selected candidate coating composition to a processor, the processor controlling an apparatus for producing and / or testing a composition for the coating composition, wherein the processor operates the apparatus to produce the input coating composition.

[0195] In addition to the desired surface characteristics, additional constraints can be provided as input. These constraints may be constituted by specifying an incomplete, rough coating composition, indicating certain components or component material levels and certain absolute or relative amounts. Such constraints may include requiring any suggested substitute components to belong to the same material class or ensuring that any substitute amount does not deviate from the amount specified in the constraints by more than a maximum threshold. Production process parameters and / or applied process parameters may also be specified as constraints.

[0196] According to an embodiment, the method is performed on a computer system effectively connected to an Automated Surface Coating and Image Acquisition Unit (ACAIA Unit). The ACAIA Unit includes one or more cameras and optionally one or more light sources. The ACAIA Unit is part of an apparatus for producing and / or testing coating compositions, or is effectively connected to the apparatus via an automated transport means, for example, for automatically transporting samples of coated substrates to and from the ACAIA Unit. The method also includes sending one or more control commands to the apparatus. The control commands cause the apparatus to:

[0197] - The produced coating composition is transported to the ACAIA unit and the transported coating composition is applied to the substrate, or the coating process is carried out in a separate coating unit of the equipment and the equipment is configured to transport coated substrate samples to the ACAIA unit;

[0198] - Position the coated surface and the camera relative to each other within a predetermined distance and / or predetermined angle range. For example, the predetermined distance and / or angle range may be specific to one or more surface defect types to be detected in the digital image to be acquired; indication of the defect type of interest may be provided along with control commands or may be explicitly specified in the configuration of the ACAIA unit; and

[0199] - After positioning, the ACIAA unit's camera acquires a digital image of the coated surface; and

[0200] - The acquired images are returned to the computer system.

[0201] This can be advantageous because it provides a fully automated system for predicting, producing, and testing coating compositions in automated production equipment, especially HT equipment environments. Data obtained in the image analysis-based testing steps can be used as training data for extending the composition quality prediction program and / or composition specification prediction program, as well as for retraining the model (M2, M3) of the composition quality prediction program or composition specification prediction program based on the extended training data to obtain improved model versions M2+, M3+.

[0202] According to an embodiment, the method includes:

[0203] - Apply a defect identification procedure to the returned image to identify the measure of one or more defects in the coating surface drawn in the returned image and calculate the qualitative and / or quantitative characterization of the coating surface. For example, the defect identification procedure can be obtained in advance by training a machine learning model based on manually annotated images of the coating surface.

[0204] - Store qualitative and / or quantitative characterizations in the database to provide an extended database;

[0205] - The prediction model (M2, M3) is retrained based on the correlation between the characterization and coating composition-related parameters (components, absolute and / or relative amounts of components, production process parameters and / or application process parameters) described in the extended data to provide an improved version of the prediction model.

[0206] - Replace the prediction model (M2) of the composition quality prediction procedure and / or the prediction model (M3) of the composition specification prediction procedure with an improved version of the prediction model.

[0207] Therefore, in the context of automated equipment used for the production and testing of coating compositions, composition quality prediction programs can be used to predict numerous promising candidate compositions. This equipment can be used to produce and test the actual performance of these candidate compositions, including coating surface quality, and the obtained test data can be used to improve the prediction model M2 of the composition quality prediction program and / or the prediction model M3 of the composition specification prediction program. Thus, over time, integrated systems including this equipment and the composition quality prediction program (and / or composition specification prediction program) have the potential to mutually reinforce and improve each other.

[0208] According to an embodiment, the training process involves an active learning module. The method also includes:

[0209] a. Based on the correlation between the stated metric and the coating components in the database, a prediction model is trained to provide a trained prediction model (M2, M3), where the loss function is minimized for training purposes.

[0210] b. Testing to determine whether the loss function value obtained for the prediction model meets specified criteria, wherein if the criteria are not met, the following steps are selectively performed:

[0211] i. The active learning module selects one candidate coating composition specification from multiple candidate coating composition specifications. The selected candidate composition specification is determined to be one of the candidate compositions that provides the highest learning performance for the prediction model (M2, M3) with regard to qualitative and / or quantitative coating surface characterization and one or more parameters selected from the group consisting of coating components, component amounts, production process parameters and / or application process parameters of the coating composition.

[0212] ii. The computer system operates the device to automatically produce candidate coating compositions according to selected specifications, automatically apply the produced candidate compositions to the substrate, and automatically acquire images of the surface on which the coating compositions are applied;

[0213] iii. Apply a defect identification program to the images acquired in step ii to calculate and store qualitative or quantitative characterizations of the coating surfaces depicted in the images within the database, thereby expanding the database;

[0214] iv. Retrain the prediction model (M2, M3) based on the expanded database to provide an improved version of the prediction model.

[0215] v. Repeat step b by using an improved version of the prediction model.

[0216] c. Replace the prediction model (M2) of the composition quality prediction procedure with an improved version of the prediction model and / or replace the prediction model (M3) of the composition specification prediction procedure with an improved version of the prediction model.

[0217] The use of the active learning module can have the following advantages, improving and accelerating the learning effect during the training or retraining of model M2: the active learning module is adapted to identify one of the candidate coating compositions, and the testing and storage of its performance will allow for maximum improvement in the quality of the prediction model M2.

[0218] In situations where manufacturing and empirically testing coating compositions is prohibitively expensive, adaptive learning modules can be beneficial. In such scenarios, the active learning module proactively identifies the most promising candidate compositions and actively queries them for production and testing. Because the active learner selects the most promising candidate compositions, the number of compositions produced and tested to learn a concept can be far less than that required for normal supervised learning.

[0219] According to an embodiment, active learning is employed during the training of a prediction model M1 for identifying coating defects in digital images of coated surfaces. In this case, the active learner identifies at least one of the highest learning outcomes by manually annotating multiple unlabeled test images of the coated surface within a small set of images. The active learning module prompts the user to manually annotate (assign labels) each image in the set, where the labels indicate the type and location of the coating defect depicted. These one or more additionally labeled images are added to the training images, thereby expanding the training data. The prediction model M1 is then retrained based on the expanded training data, thereby providing an improved and more accurate prediction model M1. The outdated prediction model in the defect identification procedure is then replaced by the improved version of the model.

[0220] On the other hand, the present invention relates to a system comprising:

[0221] - Equipment for producing and testing compositions for use in paints, varnishes, printing inks, abrasive resins, pigment concentrates, or other coatings, wherein at least two processing stations are connected to each other by a transport system on which self-propelled transport vehicles are capable of moving to transport components of the composition and / or the produced composition between the processing stations.

[0222] - A computer system configured to perform the methods according to any of the embodiments described herein.

[0223] In another aspect, the present invention relates to a computer program configured to perform one of the methods described herein.

[0224] In another aspect, the present invention relates to a defect identification program provided by executing a method for providing and using a defect identification program as described herein with respect to embodiments of the invention.

[0225] In another aspect, the present invention relates to a distributed data processing system comprising a monolithic or distributed data processing system and a camera. The distributed or monolithic data processing system includes a defect identification procedure as described herein with reference to embodiments of the invention, adapted to identify coating defects in digital images of a coated surface captured by a camera. For example, the data processing system may be a monolithic data processing system such as a general-purpose data processing system like a computer or smartphone, or it may be a dedicated device associated with surface quality inspection. Alternatively, the data processing system may be a distributed data processing system such as a cloud system or a client-server system, comprising a server computer and one or more client data processing systems.

[0226] In another aspect, the present invention relates to a composition quality prediction program, which is provided by performing a method for providing and using a composition quality prediction program as described herein with reference to embodiments of the invention.

[0227] In another aspect, the present invention relates to a composition specification prediction program, which is provided by performing a method for providing and using a composition specification prediction program as described herein with respect to embodiments of the invention.

[0228] On the other hand, the present invention relates to a coating composition produced according to a composition specification provided by performing any of the methods described herein for providing the coating composition specification. In particular, the specification can be calculated according to embodiments of the invention by a coating composition specification prediction procedure based on a desired coating surface characterization provided using a training model (M3) as input.

[0229] In another aspect, the present invention relates to a composition specification provided by performing a method for providing any of the embodiments of the coating composition specifications described herein.

[0230] In another aspect, the present invention relates to a volatile or non-volatile data storage medium comprising computer-readable instructions for implementing a defect identification program, a composition quality prediction program, and / or a composition specification prediction program.

[0231] In another aspect, the present invention relates to a volatile or non-volatile data storage medium comprising the above-described specifications, the data storage medium being effectively coupled to a device via an interface, the device being configured to produce a coating composition according to one or more specifications stored in the data storage medium.

[0232] As used herein, “coated surface” refers to a substrate surface that has been coated once or multiple times with a coating composition. For example, the coating composition can be applied by spreading, spraying, or coating the substrate, by immersing at least one surface of the substrate in the coating composition, or by other coating methods.

[0233] As used in this article, “coating defect” or “coating surface defect” refers to any visually detectable deviation of the coated surface from its typical or desired appearance. For example, image defects originating from dust on a camera lens are not coating defects.

[0234] As used in this article, "program" refers to software such as an application or a module or function of an application, or a script or any other type of software code that can be executed by one or more processors such as a CPU or GPU. A program can be, for example, an application. Applications, and especially programs designed to run on portable devices, can be implemented as "apps".

[0235] The “defect identification program” used in this article is a software program or software module configured to analyze digital images to automatically identify one or more coating defects and calculate coating surface features in relation to the identified coating defects in the surface depicted in the image.

[0236] The “composition quality prediction program” used herein is a software program or module configured to receive a (complete or incomplete) specification of a coating composition and to predict one or more properties of the coating composition in relation to the data detailed in that specification. The properties of the coating composition may include coating composition quality indicators, such as indicators of the quality of the coating surface obtained by applying the coating composition to a substrate. Quality indicators may, for example, be the likelihood of certain types of coating defects occurring.

[0237] The “composition specification prediction program” used herein is a software program or module configured to receive a desired coating surface characterization as input parameters. Optionally, the composition specification prediction program may be configured to receive one or more other input parameters, such as an incomplete coating composition specification to limit the solution space of the prediction. The composition specification prediction program is configured to predict, in relation to the received input data, one or more of the following output parameters: one or more components, one or more absolute or relative amounts of components, one or more production process parameters, and / or one or more application process parameters.

[0238] The term "measure" for defects, as used in this article, refers to any qualitative or quantitative parameter value or set of parameter values ​​that describes the nature of a coating defect. Examples of qualitative defect measures are defect type labels such as "bubble defect," "delamination defect," "foam defect," etc. Examples of quantitative measures of coating defects include "percentage of coating surface covered by defects," "number of bubbles in an image," "number of bubbles per unit area of ​​a defect," "average bubble size," etc. A defect measure can be a single parameter value or a set of parameter values, such as an attribute of an individual defect instance or an attribute derived from multiple defect type instances, such as the average diameter of a bubble defect instance or a histogram indicating the size distribution of multiple defect instances, such as bubbles. For example, a defect measure can be obtained by analyzing the size or other attributes of the identified defects (e.g., based on the bounding box and / or based on individual pixels). A defect measure can be a single parameter value or a set of parameter values ​​obtained, for example, by applying arithmetic operations to obtain the average diameter or by complex statistical calculations, such as mathematical formulas, histograms, etc., characterizing the distribution of bubble size and location distances on the coating surface.

[0239] As used herein, “characterization of the coated surface” refers to any qualitative or quantitative parameter value or set of parameter values ​​that describes the quality of the surface coating. The quality of the coated surface depends on the measure of one or more defects contained in the coated surface. For example, the coating surface characterization can be the same as the qualitative and / or quantitative measure of one or more defects contained in the coated surface. According to other examples, the coating surface characterization is obtained by aggregating measures of multiple defects in the coated surface. For example, in the case where the coated surface contains three bubble defects BD1, BD2, BD3, the aggregated coating surface characterization can be a fraction of the plotted surface covered by the total identified bubble defects, such as “5% bubble defect area”. The aggregation function input can be an arithmetic or geometric mean (e.g., used to calculate the average bubble diameter), a sum, a product, or a more complex arithmetic and / or statistical aggregation function. The coating surface characterization can include a combination of two or more qualitative and / or quantitative characterizations. For example, the characterization can be a combination of defect type indication and defect severity, such as “coated surface with severe bubble defects” or “coated surface with moderate bubble defects” or “coated surface with minor bubble defects”. The extent can be provided numerically, such as as a percentage of the area covered by the defect, or as a numerical range indication, histogram, etc. Coating surface characterization can include a combination of qualitative and / or quantitative characterizations obtained for each of two or more different types of defects.

[0240] As used herein, the term "composition" or "coating composition" refers to a composition comprising two or more raw materials (components) formed from said raw materials and to be applied to a substrate to provide a coated surface. When references are made in the context of this application to the production or testing of a composition by automated equipment, it should be understood that the coating composition is produced based on information regarding the properties and / or amounts of the components.

[0241] As used herein, the term "candidate composition" or "candidate coating composition" refers to a coating composition that has not yet been prepared and / or tested, and whose properties are therefore at least partially unknown at the time the candidate coating composition is specified. For example, a candidate composition may be a composition that is artificially or automatically specified but has not yet been prepared and tested. Therefore, the properties of such a composition are unknown.

[0242] As used herein, a “composition specification” is a set of data containing parameters related to a coating composition. These parameters may specify the identity and / or substance class of some or all of the components to be combined to produce the coating composition. Optionally, the specification may include additional parameters, such as relative or absolute amounts or ranges of the components, coating composition manufacturing process parameters, coating application process parameters, etc. These parameters may explicitly specify how the components must be handled and / or mixed to obtain the composition and / or how the composition should be applied to a substrate to obtain a specific coated surface. The composition specification may be complete or incomplete. For example, some specifications may only specify the component type (e.g., a solvator based on an organic solvent) without specifying the exact identity and / or amount of the components. The composition “specification” can be provided in various forms, such as as a printout, a document like an XML file, or an object in an object-oriented programming language like a JSON file.

[0243] As used herein, “coating composition manufacturing process parameters” or “manufacturing process parameters” are parameters that indicate the characteristics of the process of processing and / or combining components to form a coating composition. Examples include mixing duration, mixing speed, mixing temperature, component mixing sequence, and equipment used for mixing or otherwise preparing the coating composition.

[0244] As used herein, “coating composition application process parameters” or “application process parameters” are parameters that indicate the characteristics of the process immediately following the application of the coating composition on the substrate. Examples include application technique instructions (spraying, spreading, coating, immersion, etc.), application duration (e.g., immersion time), number of repetitions, substrate pretreatment steps (drying, priming, cleaning, heating, etc.), and instructions on the equipment used to apply the coating composition and / or the properties of the substrate (wood, plastic, metal, cardboard, etc.).

[0245] As used herein, a “known composition” is a composition whose properties (e.g., coating surface characteristics, rheology, elasticity, shelf life, etc.) are known to the person performing the neural network training. For example, a known composition may have been manufactured for a customer months or years ago, and the performance of that product has been empirically determined. Measurements do not necessarily have to be performed by the laboratory operator who now determines the predictive composition, but may be performed and published by other laboratories; therefore, in this case, the performance is derived from professional literature. Since compositions as defined above also include formulations as a subset, a “known composition” according to embodiments of the invention may also include “known formulations” or “known recipes”.

[0246] The term "defoaming" is commonly used to describe the removal of air bubbles from a composition or coating. However, in some cases, the terms "defoaming" and "air removal" should be distinguished. First, the bubbles need to reach the surface. Removing foam bubbles from the surface is called defoaming (strictly speaking). Defoamers (strictly speaking) only work on the surface, eliminating bubbles located there. In contrast, degassing agents work throughout the entire coating film. In this case, the term "defoamer" is used broadly and encompasses both defoamers in the strict sense and degassing agents.

[0247] As used in this article, "database" refers to any volatile or non-volatile data storage medium that stores data, especially structured data. A database can be one or more text files, spreadsheet files, directories in a directory tree, or a database of a relational database management system (DBMS) such as MySQL or PostgreSQL.

[0248] The "loss function" used in this paper for the prediction problem is the following function, which is used in the training of predictive models (such as neural network models) with the help of machine learning procedures used to train and improve the model. The loss function outputs a value whose magnitude indicates the quality of the predictive model. Here, the loss function is minimized during training because the magnitude of this value indicates that the prediction model's predictions are inaccurate.

[0249] As used herein, "equipment" for producing and testing compositions refers to a device or system comprising multiple laboratory instruments and transport units capable of coordinated joint control of the laboratory instruments and transport units to perform automated or semi-automated workflows. Workflows may be, for example, coating composition preparation workflows (e.g., combination and mixing workflows), analytical workflows, or combinations of two or more of these workflows. Workflows may include automated preparation of coating compositions and / or automated application of compositions to one or more substrates and / or automated acquisition and processing of digital images of the substrate coating surface. The equipment may, for example, be a high-throughput device (HT device), also known as a "high-throughput equipment" (HTE).

[0250] "Testing" or "analyzing" a coating composition using automated production and / or testing equipment is a process of analyzing the chemical, physical, mechanical, optical, or other empirically measurable properties of the composition using one or more analytical modules. For example, testing may include applying the composition to a substrate, acquiring and analyzing digital images of the coating surface, and calculating coating quality metrics in relation to one or more defects detected in the digital images of the coating surface. The analysis may also include measuring other object properties such as opacity, elasticity, rheology, color, etc.

[0251] The “active learning module” used in this paper is a software program or a module of a software program designed to select a (relatively small) small set of candidate compositions from a set of candidates, thereby exhibiting a strong learning effect after the properties of the selected candidate compositions are prepared and empirically measured, because these data are taken into account in training the predictive model.

[0252] As used herein, a “model” or “predictive model” is a data structure or executable software program or program module configured to generate predictions based on input data. For example, the model may be a model obtained during machine learning by training it on manually labeled and / or automatically labeled training data. A predictive model, as used herein, may also comprise a collection of two or more functionally integrated models, such as a set of models used and / or included in an application. Predictive models may be, for example, neural network models, support vector models, random forests, decision trees, etc. According to embodiments of the invention, a predictive model suitable for calculating the surface characterization of a coating based on digital image data regarding the presence, location, and / or extent of one or more coating defects is also referred to as an “M1” model. A predictive model suitable for predicting the properties of a coating composition, particularly the characterization of the coating surface generated from that composition, based on one or more input parameters, such as those relating to components, component amounts, production process parameters, and / or applied process parameters, is also referred to as an “M2” model. A parameter prediction model suitable for predicting one or more parameters involving, for example, the components of the coating composition, the amount of components, production process parameters, and / or applied process parameters, based on input data that specifies the desired performance of the coating composition, especially the desired characterization of the coating surface produced by the composition, is also referred to as the "M3 model". According to an embodiment, the model M3 is configured to take into account incomplete coating composition specifications as additional input data, whereby the incomplete coating composition can be used to limit the solution space of the predictions provided by the model M3. Attached Figures

[0253] The following describes only exemplary embodiments of the present invention in detail, with reference to the accompanying drawings which include these exemplary embodiments, wherein:

[0254] Figure 1 A flowchart illustrating an automated surface characterization method for coatings is shown.

[0255] Figure 2 A more detailed flowchart of the automated surface characterization method for coatings is shown;

[0256] Figure 3 A flowchart illustrating a method for coating a surface and obtaining an image of the coated surface;

[0257] Figure 4 A block diagram of a data processing system for automated coating surface characterization is shown.

[0258] Figure 5A The diagram shows a data processing system in the form of a smartphone, which includes locally installed applications.

[0259] Figure 5B A data processing system in the form of a smartphone including web applications is shown;

[0260] Figure 5C A data processing system in the form of a customized coating surface quality inspection device is shown;

[0261] Figure 5D A data processing system in the form of a computer is shown, connected to equipment for producing coating compositions;

[0262] Figure 6 This shows a view of the training and testing phases used to generate and use the defect identification program;

[0263] Figure 7 This shows a view of the training and testing phases for generating and using a composition quality prediction program;

[0264] Figure 8A Several coating compositions are shown;

[0265] Figure 8B The "pull-down" coating application unit is shown.

[0266] Figure 8C This illustrates a "spraying" type coating application unit;

[0267] Figure 8D An automated conveyor belt used for transporting coating samples in HTE is shown;

[0268] Figure 9A A digital image of the coated surface before defects were manually marked;

[0269] Figure 9B Showing defects including those marked (annotated) Figure 9A Digital images;

[0270] Figure 10 Digital images of coated surfaces with automatically identified and marked defects are displayed;

[0271] Figure 11 The defects in the surface area are measured in the form of histograms and area percentage values;

[0272] Figure 12 The architecture of a neural network used to predict the quality of a coating composition is shown; and

[0273] Figure 13 This view shows the relative positioning of the camera, light source, and coated surface used to acquire digital images;

[0274] Figure 14 A flowchart illustrating a method for identifying candidate coating compositions that best suit the quality of a prediction model for improving composition quality prediction procedures;

[0275] Figure 15 A block diagram of a distributed data processing and coating production system for improving the prediction model of the composition quality prediction procedure is shown; and

[0276] Figure 16 A 2D cross-sectional view of the multidimensional data space is shown for various combinations of defoamer DF1 with optimal antifoaming performance and defoamer DF2 with optimal compatibility in a specific coating composition, from which the "active learning module" selects specific data points. Detailed description

[0277] Figure 1 A flowchart illustrating an automated surface characterization method for a coating is shown. In a first step 102, a defect identification program processes a digital image depicting the coating surface. The defect identification program identifies one or more coating defects and provides a characterization of the identification effect in step 102. For example, the program may determine that the coating surface includes a first bubble defect region containing approximately 100 bubbles and a second bubble defect region containing approximately 400 bubbles. The characterization of the identification effect may include the type, location, and extent of the identified defects. The data obtained in step 104 may be output to a user and / or internally used by the defect identification program to calculate derived data values, such as aggregated coating surface characterizations.

[0278] Figure 2 A flowchart of the automated coating surface characterization method is shown in more detail. Following steps 102 and 104, the defect identification procedure calculates measures of individual defects, such as size, shape, spatial distribution, and size distribution, in step 106. In step 108, the defect identification procedure uses these measures to calculate and provide a qualitative and / or quantitative characterization of the coating surface.

[0279] Figure 3A flowchart illustrating a method for coating a substrate and obtaining an image of the coated surface is provided. In step 110, multiple coating compositions are generated by mixing multiple components together according to a mixing and production protocol. In step 112, the coating compositions are applied to one or more substrate samples according to a coating application protocol. The samples are automatically or manually transported to an image acquisition unit. In steps 114 and 118, a camera, an optional light source, and the coated sample are positioned relative to each other in a prescribed manner to allow image analysis software 124 to correctly analyze the image. Then, in step 118, one or more images depicting the surface of the coated sample are captured.

[0280] Figure 4 A block diagram of a data processing system 120 for automated coating surface characterization is shown. The data processing system includes one or more processors 126 and volatile or non-volatile storage medium 122. The storage medium may include images 125, such as training images 125 for training a model M1 of a defect identification program 124, or test images input to a trained predictive model M1. Additionally or alternatively, the storage medium may include training data for training a model M2 of a composition quality prediction program and / or may include a component quality prediction program that includes a trained predictive model M2.

[0281] Additionally or alternatively, the storage medium may include an untrained version of the composition specification prediction program model M3 and / or may include a composition specification prediction program containing a trained prediction model M3.

[0282] The data processing system 120 can be implemented in many different ways. For example, the data processing system can be a single computer system, such as a desktop computer system, portable telecommunications equipment, smartphone, dedicated coating surface quality control equipment, or a computer system integrated into an automated production and / or testing equipment for coating compositions. Alternatively, the data processing system 120 can be a distributed computer system, such as a client / server computer system, which can be optionally connected to one or more devices for the automated production and / or testing of coating compositions. The components of the distributed computer system can communicate with each other via network communications such as the Internet or an organizational intranet. Figures 5A-5D Some embodiments of the data processing system 120 are shown.

[0283] Figure 5A A data processing system in the form of a smartphone 130 is shown. The smartphone includes a defect detection program in the form of a locally installed application, also referred to as "app" 124.

[0284] For example, the application may include a graphical user interface 132 that allows a user to control a smartphone camera 134 to capture images of the surface of a coated sample at appropriate distances and positions relative to the coated surface. Successful acquisition of the digital image can automatically trigger a defect identification application 124 to analyze the image and identify one or more coating defects depicted in the image. Preferably, the application 124 is configured to generate another GUI 136 that notifies the user of the processing results. For example, GUI 136 may indicate the type of the identified defect (bubble defect) and one or more quantitative measures of the defect (e.g., average bubble diameter, bubble density, etc.). Furthermore, GUI 136 includes a coating surface characterization calculated based on the performance of one or more identified coating defects. For example, the defect identification program may be configured to calculate the total size of the surface area covered by bubbles based on the size and number of identified bubbles.

[0285] Using a smartphone app to acquire images of coated surfaces and automatically identify coating defects may have the following advantages: it eliminates the need to equip company employees with a large number of specialized devices to obtain objective and reproducible measurements of coating surface quality. Simply downloading and installing the app is sufficient.

[0286] Figure 5B A data processing system in the form of a smartphone 130 is shown, which includes a defect identification program 124 in the form of a web application.

[0287] As an example, the defect identification program is implemented as a script, which runs in a smartphone's browser and is downloaded by a user visiting a specific website, such as a company web portal generated by server 144 and provided via the internet or intranet. For example, program 124 could be implemented as a JavaScript program.

[0288] According to another embodiment, the defect identification program is implemented in the form of a program that runs outside the browser, such as a Java program.

[0289] The defect identification program can be implemented as a two-part program, comprising a client part and a server part, which are interoperable and configured to exchange data via network communication 142. For example, the program part installed on portable telecommunications device 130 (“client application”) can be configured to control the image acquisition process and output defect identification results to the user. The program part installed on the server (“server application”) can be configured to receive digital images from the client part via the network, analyze the digital images to detect coating defects, determine the measure of the identified defects, and calculate the qualitative and / or quantitative characterization of the drawn coating surface. The server part returns the characterization to the client part, preferably also including measures and indications of the type and extent of the identified defects.

[0290] Figure 5CA data processing system 150 in the form of a customized coating surface quality inspection device is shown; that is, dedicated hardware designed to control and objectify the coating surface quality, as well as the implicit quality of the coating composition and / or the coating process. The device includes a storage medium with a defect identification program 124, an interface 152 allowing the user to control the testing process and output test results, and several hardware components preferably used to test the surface properties of the coated sample. For example, the device may include a camera 134, which is engaged to the device via a robotic arm 158 or other connectors that allow changing the relative position of the camera and the coated sample. The device may include one or more light sources 160, which are preferably also engaged to the device via movable and / or rotatable connectors such as a robotic arm 156.

[0291] The quality inspection equipment 150 can be implemented as a portable or stationary device. For example, the equipment can be implemented as an integral part of a device for automatically producing and / or testing coating compositions. The equipment includes a conveyor belt 154 for conveying multiple coating samples 162, 164, 166, 168 to the control device 150, thereby allowing fully automated, rapid, and reproducible quality inspection of numerous coated surfaces.

[0292] Figure 5D A data processing system in the form of a computer 170 is shown, which is connected to equipment 244 for producing coating compositions.

[0293] The device 244 includes a main control computer 246 for controlling, monitoring, and / or scheduling various tasks related to the production of the coating composition, the application of the coating composition to various surfaces, and / or the testing of the coated surface or coating composition (e.g., determining rheological, chemical, physical, or other parameters of the coating composition). Each task is performed by several different units included in the device 244. For example, the device may include one or more analyzers 257 for chemical, physical, mechanical, optical, or other forms of testing and analysis of the coating composition or the surface of a substrate coated with the coating composition. The device may include one or more mixing units 256 configured to produce various different coating compositions, for example, by mixing the components of the composition based on a specific production and mixing protocol. The device may include one or more sample coating units 254 configured to automatically coat surface samples. For example, coating unit 254 may include, for example, […]. Figure 8C and 8BThe illustrated "spray" or "pull" type coating application unit is shown. According to some embodiments, the device also includes an image acquisition unit 252, which includes a camera and a light source, as well as tools for positioning the sample relative to each other and the camera and / or light source, so that the acquired digital images can be used as input by the defect identification program 124. One or more transport units 258, such as conveyor belts, connect different units and transport components, mixtures, coating compositions, and coated samples from one unit to another.

[0294] The control computer 246 includes a control unit 248 configured to transmit digital images of the coating sample acquired by the image acquisition unit 252 to a defect identification program 124 of the computer system 170. According to some embodiments, additional parameters may be provided to the defect identification program along with the image data. These parameters may indicate the identity, relative amount, and / or absolute amount of one or more components of the coating composition used to coat the sample, and optionally also indicate manufacturing process parameters and / or application process parameters. These parameters may be provided in complete or incomplete specification form of the coating composition and / or the respective manufacturing process parameters or application process parameters.

[0295] The defect identification program is configured to use a received image, and optionally parameters, as input to automatically identify coating defects depicted in the image, calculate defect measures, and compute coating surface characterization in relation to the defect measures. The results calculated by the defect identification program can be output to the user via a GUI and / or stored in database 204.

[0296] Preferably, some data obtained by other units such as analyzer 257, mixing unit 256, or coating unit 254 may be directly stored in a database in association with the identifier of a specific coating composition and / or the identifier of a coated sample, or may be sent to computer system 170 so that computer system 170 stores the data in a database.

[0297] Using a defect identification program in the 244 equipment environment may be particularly advantageous, as images of the coated surface can be automatically analyzed to identify defects to be inspected. The results obtained can be correlated with formulation data and / or analytical data, and thus can be used to optimize the composition.

[0298] Figure 6 This shows a view of the training and testing phases for generating and using the defect identification program.

[0299] In the first step, training data set 602 is generated. For this purpose, a variety of different coating compositions are prepared. These different coating compositions vary in the properties of their components, the amounts of each component, and / or manufacturing process parameters. The various coating compositions are then applied to a substrate to create multiple coated surfaces. The number of coated samples can be much greater than the number of coating compositions, because the same coating composition can be applied to many different types of materials (wood, plastic, cardboard, metal, etc.) using different types of coating techniques (spraying, painting, dipping, smearing, etc.). Then, one or more digital images 604, 606 are acquired for each coated sample. For example, multiple digital images can be obtained for a specific coated sample by varying the light intensity, the relative position of the light source, the wavelength, etc.

[0300] Depending on the coating composition, sample material, coating process parameters, and many other factors, the coated surface depicted in the image may contain one or more coating defects of different predetermined defect types.

[0301] In the next step, the defects depicted in the acquired digital images are manually marked. For example, this can be done as shown in the reference. Figure 9B Perform the annotation process as described.

[0302] Digital image 604 can be labeled with tag 616, indicating the location, type, and preferably degree of each coating defect depicted in image 604. Tag 616 also includes qualitative and / or quantitative characterization of image 604, such as "overall quality level 7, including level 2 bubble defects and level 8 wrinkling defects". Preferably, image 604 is stored in association with supplementary data 608. Supplementary data 608 may include complete or incomplete specifications of the components used to generate the coating composition of the depicted coating surface, wherein the specifications may further include component names and / or quantities, production process parameters and / or composition application process parameters. For example, the specifications of the composition and the above parameters may be stored in a database in association with an identifier of the coating composition used to generate the depicted coating surface. Image 604 may have an image ID stored in association with an identifier of the coated sample depicted in the image, whereby the identifier of the coated sample is stored in association with an identifier of the coating composition used to coat the sample.

[0303] The digital image 606 can be labeled with tag 618 and similarly stored in association with additional data 610.

[0304] A computer system 120.1 is provided, which includes an untrained version of a prediction model 612 to be trained. The computer system has access to a database 204 containing training data. Training data 602 is used as input during the training process. During training, the prediction model M1 learns the correlation between pixel patterns in labeled images 604, 606 and coating defects / coating surface characterization.

[0305] According to an embodiment, the training data includes additional data 608, 610 in parametric form. The prediction model M1 (or the additional prediction model M1.2 included in the defect identification procedure) will also learn the relationship between pixel patterns, coating defects / coating surface characterization, and additional data such as component and / or composition component amounts, production process parameters, and / or applied process parameters.

[0306] Untrained models can be implemented as neural networks. These neural networks preferably include, for example, region proposal networks provided by the Mask R-CNN program.

[0307] As a result of training, 612 provides a predictive model M1 (which may include one or more additional predictive models M1.2) that has learned the aforementioned correlations. The trained model can be integrated into the defect identification program 124 and used to automatically identify coating defects depicted in digital images. Program 124 may include additional functions, such as a GUI 614, for assisting the user in acquiring images during the training and / or testing phases and / or for displaying prediction results to the user, for example, in numerical and / or segmented image formats.

[0308] Figure 6 The lower half shows the testing phase of the trained model M1. During the testing phase, digital images of multiple coated surfaces, referred to as "test images," are provided. For example, test images 624 and 626 may be stored in the same database 620 or in a different database used to store training images. The test images are initially unlabeled. Optionally, the test images may be stored in association with additional data 628 and 630, particularly complete or incomplete specifications of the properties and / or quantities of the components of the coating composition, various production protocol parameters, coating composition application process parameters, and / or image acquisition system parameters.

[0309] A computer system 120.2 is also provided, in which a copy of the defect identification program 124 is installed and / or instantiated. The computer system 120.1 used for training the model may be the same computer system 120.2 used for applying the trained model to test images, or it may be a different computer system that receives a copy of the defect identification program.

[0310] The defect identification program 124, including the trained prediction model M1 612, receives one or more test images 624, 626, which are used as input to predict qualitative and / or quantitative characterizations 632, 634 of the coating surface depicted in the respective test images. For example, the defect identification program may perform pixel-by-pixel image analysis to identify the location, type, and extent of coating defects depicted in the images. The program may then analyze the acquired data to calculate a measure for each coating defect identified in the images. For example, a pixel score representing a bubble defect and a pixel score representing a delamination defect in the same image may be determined. In a further step, these measures are used to calculate an aggregate feature of the coating surface in relation to the defect measure, such as a label like "bubble defect level 3".

[0311] Optionally, the defect identification program receives and uses additional data 628, 630 assigned to each test image as additional input for performing predictions.

[0312] Preferably, the automatic prediction measurements and characterizations 632, 634 are stored in a database in association with their respective test images 628, 613. Therefore, the provided data includes labels indicating the coating surface characterization, and consequently, the coating quality of a particular coating composition or protocol. This data can be used to augment model M1602 with training data and to retrain the prediction model based on an expanded database to improve accuracy. Additionally or alternatively, this data can also be used to train another prediction model M2 with a different prediction range, such as referenced... Figure 7 As explained.

[0313] Figure 7 The diagram illustrates the training and testing phases for generating and using the composition quality prediction program 714. Unlike the defect identification program, the composition quality prediction program 714 does not require digital images as input. Instead, the prediction model M2 712 used by the composition quality prediction program uses multiple parameters as input. These input parameters are selected from a group including components, component amounts, production process parameters, and / or application process parameters. Quality-related characterizations of the coating surface and, optionally, coating composition quality indices are calculated and output by the coating composition quality prediction program.

[0314] The combined use of a defect identification program and a composition quality prediction program may be beneficial because the defect identification program provides, for the first time, a coating surface quality characterization with sufficient quantity, quality, and objectivity to allow the data to be used to train different machine learning programs 712 to solve different tasks, such as coating composition quality prediction, in which the coating surface quality is also taken into account.

[0315] Training data 702 is provided during the training phase of model M2. Training data 702 includes multiple data records (here, two data records are indicated by circles), where each data record represents a coating composition. Each data record may include complete or incomplete specifications 628, 630 of the properties and / or amounts of the composition components, specifications of production process parameters, and / or specifications of coating composition application process parameters. Furthermore, each data record includes characterizations 632, 634 of the resulting coating surface, obtained by a) coating a sample with the respective coating composition, b) acquiring images of the coating surface, and c) analyzing the images using a defect identification procedure to calculate the coating surface characterizations 632, 634. Additionally, each data record may include one or more other properties 732, 734, particularly rheology, shelf life, density, etc.

[0316] During the training phase, machine learning model 712 learns to correlate coating surface characterizations 632, 634, data contained in rules 628, 630, and additional properties 732, 734 (if any).

[0317] As a training result, the trained prediction model M2 can predict the quality of coating compositions 732 and 734 based on their components and relevant process parameters 728 and 730 (if any). For example, the quality of the coating composition can be predicted through a defect identification procedure and provided as outputs in the form of coating surface quality characteristics 732 and 734. In other embodiments, the quality of the coating composition can be predicted and provided as a combination of coating surface quality characteristics and other attribute values ​​729 and 731, such as indications of shelf life, viscosity, etc.

[0318] The training of the prediction model M3 to be used by the coating specification prediction procedure can be performed similarly to that described for model M2, wherein the same correlations are learned, but the input data of M2 is used as the output data of M3 for learning, and the output data of M2 is used as the input data of M3 for learning. Optionally, M3 uses additional data such as the provided incomplete coating specifications to constrain the solution space that must be evaluated by model M3.

[0319] Figure 8A This illustrates several coating compositions automatically produced in HTE based on their respective coating composition specifications.

[0320] Figure 8B A "pull-down" coating application unit is shown. In a first step, a certain amount of coating composition is applied to the sample surface. Then, a roller passes over the surface at a defined distance, thereby uniformly distributing the amount on the surface and forming a coating.

[0321] Figure 8C This shows a "spraying" coating application unit.

[0322] Figure 8D An automated conveyor belt is shown for a high-throughput apparatus for producing and / or testing coating compositions. The conveyor belt is adapted to automatically transport coated samples to the image acquisition unit of the apparatus. The sample is loaded into the image acquisition unit. The camera, light source, and / or sample are moved and positioned relative to each other, making the images acquired from the sample surface suitable for automated processing using a defect identification procedure.

[0323] Figure 9A A sub-region of a digital image 1202 of the coated surface is shown prior to manual defect labeling (label assignment). For example, this image may have been captured by a camera in the image acquisition unit of an apparatus used for automated production and / or testing of coating compositions. Alternatively, the image may have been obtained by referencing, for example... Figure 4 The data is obtained from the camera of the data processing system described in Figure 5.

[0324] Image acquisition conditions (lighting, camera settings, image acquisition angle, illumination angle, etc.) are selected to show the defect to be inspected as well as possible in the image. According to embodiments, different image acquisition conditions are set and employed for different types of defects. For example, defects related to surface bulges or depressions can be analyzed based on images obtained using a small illumination angle (i.e., a small angle of light incidence) to ensure that the defect causes shadows of sufficient size and contrast. Other defects, such as color defects, can be analyzed based on digital images obtained using a large illumination angle (approximately 80-100°).

[0325] Digital image 1202 shows a coated surface containing foam defects. The coated surface includes two main defects (pores) 1204 and 1206 and several smaller defects (small pores). These defects are clearly identifiable when the substrate is illuminated from the side due to the formation of shadows. Shadow formation allows for the identification and differentiation of bumps and depressions in the coating on the substrate. Furthermore, shadow formation can be used to determine, for example, whether there are sharp edges in the case of depressions or whether the coating thickness is slowly decreasing. This can distinguish pit defects from bubble defects.

[0326] In addition to defects, digital images also include artifacts 1208 that are not coating defects. For example, artifacts may be caused by dust spots on one of the lenses of the image acquisition system or on the substrate.

[0327] To generate a training dataset of sufficient size, digital images of many different coated surfaces, including many different types of coating defects, were acquired. For foam defects, a small light incident angle was selected. For other types of defects, different image acquisition settings and conditions could be selected. Preferably, the surface of each coated sample was illuminated under many different conditions, and their respective digital images were acquired to generate a defect identification program capable of recognizing many different types of defects that may overlap.

[0328] Preferably, a large number of digital images showing different defect types and thousands of defects on different types of coated substrates are acquired and manually annotated (marked).

[0329] Figure 9B Show Figure 9A The digital image 1202 includes manually marked (annotated) defects. The annotated digital image may include storing the digital image in association with information about the location and type of coating defects depicted in the image. Optionally, the label may include additional data such as image resolution, which may allow determination of quantitative defect measurements such as diameter, circumference, etc.

[0330] Manually annotating many images consumes a significant amount of time and effort. To alleviate this problem, images are cut into many smaller parts. For some of these parts, foam defects are marked using the VIA-VGG image annotator software (Abhishek Dutta and Andrew Zisserman, 2019, “VIA Annotation Software for Images, Audio and Video,” Proceedings of the 27th ACM International Conference on Multimedia (MM'19), October 21-25, 2019, Nice, France; ACM, New York, USA, 4 pages, https: / / doi.org / 10.1145 / 3343031.3350535.).

[0331] Defects 1206 and 1208 were manually marked using VIA software by manually drawing circles 1212 around each defect. VIA software was then used to output the marked information in a structured format, associated with an image or image identifier. The structured format could be, for example, an XML file, a JSON file, a comma-separated file, or data records from a relational database.

[0332] According to some embodiments, additional data is stored in association with an image or image identifier. The additional data may include a complete or incomplete description of the coating composition used to produce the coated surface, wherein the description may include indications of the identity and / or amount of components, manufacturing process parameters indicating aspects of the composition production process, application process parameters specifying the process of applying the composition to the substrate, and / or image acquisition system parameters. This allows the predictive model M1 (or the additional predictive model M1.2 used by the coating quality prediction program) to learn, on the one hand, the relationship between defect types and coating surface characterization, and on the other hand, the relationship with one or more of the aforementioned parameters.

[0333] It can generate the prediction model M1 to be used in the defect identification program, for example, as referenced. Figure 6 As stated above.

[0334] The trained model M1 can then be used to detect bubble defects (as well as other types of defects covered by the training data) in new unlabeled images (called “test images”) that have not yet been used in the training step.

[0335] Figure 10 A digital image 1302 showing the coated surface is included. This image includes a label 1304, which is automatically created by a defect identification program 124 according to an embodiment of the invention. This label marks defects 1306 and 1308 that have been automatically detected by the defect identification program.

[0336] In addition to a visual representation of detected defects (e.g., by overlaying an image segment of the identified defect, or, as in this case, by using edges and circles around the identified defect), the defect identification program is configured to also temporarily or permanently store the type and location of the identified defects in a structured form. For example, the location can be stored as pixel coordinates representing the pixels of the defect. Storing the identity and location of defects in a structured form allows the defect identification program to process the structured data to calculate an aggregate characterization of the coated surface.

[0337] For example, aggregate characterization of the entire surface can be a quantitative characterization of the coated surface, involving scale values ​​with a quality scale greater than 5, such as more than 10 possible scale values, where the scale can represent the overall coated surface quality. The surface quality score is negatively correlated with the size and number of defects identified in the coated surface.

[0338] Another example of using qualitative defect measurement and / or quantitative characterization for coated surfaces would be histograms of defects of different sizes, such as... Figure 11 As shown.

[0339] Figure 11The defects in the surface area are measured in the form of a histogram 1402 and area percentage values. The histogram and / or percentage values ​​can be used as a quantitative characterization of the coated surface or can be used to calculate and derive such a characterization.

[0340] Histogram 1402 depicts the distribution of foam bubbles of different sizes, where bubble size is defined as bubble area measured in pixels. Bubble sizes are grouped into 10 different bars, and the number of bubbles whose size falls within the size range of a bar is plotted. Therefore, the histogram provides a rough estimate of the bubble size distribution, which can allow for the identification of problems in the coating composition or coating process.

[0341] The percentage of surface covered by bubble defects is also calculated (5.88% in this case). This value can be used as a quality characterization of the coated surface in the processed digital image. By evaluating the number and size of defects, the coated surface can be objectively and reproducibly classified into a predetermined quality grade or level.

[0342] Figure 12 The architecture of a neural network 400 that encodes the prediction model M2 and is used to predict the quality of the coating composition is shown.

[0343] Network 400 is configured and trained to receive input vector 402 and compute and output an output vector 406 based on the input vector.

[0344] For example, the input vector can encode complete or incomplete coating composition component specifications (and optionally, the concentration or amount of each component). Optionally, the input vector may also include process parameter values ​​for the coating composition production process and / or the coating application process.

[0345] Output vector 406 explicitly specifies one or more properties of the composition or the surface produced by coating a substrate with the composition. These properties preferably include one or more properties that represent the quality of the coating composition and / or the quality of the coated surface formed by the coating composition.

[0346] The network consists of several layers of 404 neurons, which are linked to neurons in other layers through weighted mathematical functions. This allows the network to compute, based on the information encoded in the input vector, the properties and quality characterization of the corresponding composition and the coating surface produced by the composition, and to output the predicted properties and quality characterization in the form of an output vector of 406.

[0347] Before training, the neurons of the neural network are first initialized with predetermined or random weights. During training, the network receives the specifications of a coating composition (which may include the type and optional components, as well as the amount of components and optional production or application process parameters) along with empirically measured characteristics of the composition, including defect measures and coating surface characterization calculated and output by defect identification procedure 124. The network calculates an output vector containing predicted performance and quality measures of the composition and is penalized by a loss function for deviations of the predicted performance and quality measures from known, empirically determined performance and coating surface characterization. The determined prediction error is distributed back to the respective neurons that caused it through a process called backpropagation, causing changes in the weights of some neurons such that the prediction error (and the loss function value) is reduced. Mathematically, this is done by determining the slope of the loss function, thus allowing for targeted changes in neuron weights to minimize the loss function output value. Once the prediction error or loss function value falls below a predetermined threshold, the trained neural network is considered sufficiently accurate and therefore requires no further training.

[0348] After successful training, the trained prediction model M2 can be used to predict performance, particularly the quality characterization of the coated surface for new, unknown coating compositions. When the exact composition and / or optimal manufacturing and / or coating process parameters are unknown, a human user or an auxiliary software program generates several candidate coating composition specifications that represent and explicitly define variations of the new coating composition of interest. A trained neural network is used to automatically predict the properties of each new (candidate) coating composition. When predictions are made for multiple candidate coating compositions, one or more of these compositions are selected and / or actually manufactured and tested in the equipment, exhibiting the performance or quality characterization best suited to their respective application scenarios.

[0349] The predicted properties of each candidate coating composition are output as an output vector 406 of the neural network to the user for manual evaluation and / or stored in a database, for example, for further evaluation and comparison with empirically derived performance values ​​of compositions that may be obtained later. The input vector may contain, for example, 20 components of the coating composition, some production process parameters, and some application process parameters. The output vector 406 may contain various properties whose nature depends on the training data used to train the prediction model M2 encoded in the network. For example, the output vector may contain indications of the type and extent of coating defects that may occur if the coating is applied to a substrate.

[0350] Figure 13This illustration shows the relative positioning of camera 134, light source 160, and coated surface 164 for acquiring digital images. For example, relative positioning can be performed manually, such as when the camera is a smartphone camera. In other embodiments, relative positioning can be performed automatically, for example within the image acquisition unit of apparatus 244 for producing and / or testing coating compositions. Relative positioning ensures that the image acquisition angle 1504 and illumination angle 1502 are selected such that the digital image of the coated surface will allow the defect identification program 124 to automatically identify coating defects. For example, angles 1502 and 1504 should generally be the same as or similar to the angles used to obtain the digital image used to train the predictive model M1 of the defect identification program. For example, an angle “similar” to angle X can be an angle within the range of X up to ±40%, particularly up to ±20% of X. According to embodiments, multiple digital images are acquired for each coated sample, wherein the image acquisition angle 1504 and / or illumination angle 1502 are different from each other. This ensures that for many different types of defects, the acquired image set includes one or more images that allow for accurate defect identification and characterization.

[0351] Figure 14 A flowchart is shown illustrating a method for identifying the quality of a coating composition for a prediction model M2 (or M3) that is best suited to improve the composition quality prediction procedure.

[0352] For example, this process can be achieved through, as Figure 7 The computer system 120.2 shown or as... Figure 5D The computer system 170 shown is executing.

[0353] In the first step 802a), the known composition and its properties, along with quality characteristics including the surface characterization of the coating produced by the composition, are used as an "initial training data set" to train model M2 (or M3), such as a neural network, support vector machine, decision tree, random forest, etc. The trained model M2 / M3 can be integrated into a component quality prediction program designed to predict the properties of the coating composition, such as the quality of the coating surface produced by the coating composition, or integrated into a component specification prediction program.

[0354] In the next step 804(b), a check is performed to determine whether the loss function value meets a predetermined criterion. Meeting the criterion indicates that the prediction accuracy of the trained neural network is considered sufficient. Steps 806-812 below are selectively performed for cases where the criterion is not met. Otherwise, training is terminated (step 814) and the trained neural network is deprecated.

[0355] In step 806, the active learning module automatically selects a candidate coating composition specification from a plurality of candidate coating composition specifications provided manually or calculated automatically. Several different active learning methods exist that can be used according to embodiments of the present invention.

[0356] According to one embodiment, the active learning module follows the “expected model change” approach and selects the candidate coating composition specification that will change the current prediction model to the greatest extent (the candidate coating composition and its measured performance are taken into account when the network is retrained).

[0357] According to another variant embodiment, the active learning module follows the “expected error reduction” approach and selects candidate compositions that will most strongly reduce the error of the current prediction model of the trained neural network.

[0358] According to another variant embodiment, the active learning module follows a “minimum edge hyperplane” approach and selects the test composition closest to a dividing line or dividing plane defined in the multidimensional data space by the current predictive model of the trained model. This dividing line or dividing plane is the interface within the multidimensional data space where the predictive model makes a classification decision; that is, assigning data points on one side of the dividing line or dividing plane to a different class or category than data points on the other side. The degree to which data points are close to the dividing plane is interpreted as the predictive model’s uncertainty about the classification decision, and it is highly beneficial if additional measurements are taken from a set of measured data near the dividing plane (consisting of the components of the coating composition produced according to the composition of the component, and optionally their concentration and measurement properties) to further train the model.

[0359] After retrieving the specifications of the selected candidate coating compositions from the database, in step 808, the computer system controls the composition production and testing equipment 244 to automatically produce and test the product according to the retrieved specifications. This testing is understood as a metrological record of one or more properties of the product, such as pH value, color value, viscosity, calculations 809 of the characterization of the coating surface produced by coating the sample with the composition, image capture, and image analysis through a defect identification procedure, etc.

[0360] The measured performance obtained in step 108 and the calculated values ​​from the image data obtained in step 809 are used to supplement the selected candidate coating composition to obtain another complete data point consisting of known compositions and known properties, which is used to expand the training data set used in the current or previous iteration a).

[0361] In step 810, models M2 / M3 are retrained based on the extended training data set. According to a variant implementation, this can be done by performing the training entirely again based on the extended training data set, or by incrementally increasing the training in step 810, so that what has been learned is retained and modified only by considering new training data points.

[0362] In step 812, an iterative check of the prediction quality of the trained models M2 / M3 is initiated, and steps 804-812 are repeated until the model has sufficient prediction quality, indicated by the loss function meeting a criterion, such as the "error value" calculated by the loss function being lower than a predetermined maximum value.

[0363] The fully trained model can now be used to predict the performance of coating compositions very quickly and reliably, including the quality characterization of the coated surface obtained from the coating composition. To do this, a retrained model M2 is integrated into the quality prediction procedure, where it replaces the less accurate older version of the model.

[0364] Because the models have learned the statistical relationships between various parameters related to the coating composition (especially its components, absolute or relative amounts, production process parameters, and / or application process parameters) and the resulting product performance (including the characterization of the coating surface produced by the composition), the trained model M2 can now predict the performance and quality of the corresponding coating surface, even for compositions for which no empirical data is available, providing a description of the coating composition. Similarly, the trained model M3 can predict one or more of the aforementioned parameters related to the coating composition, assuming the desired coating surface characterization and an optional incomplete coating composition. Both the M2 and M3 models rely on learned relationships based on empirically measurable or derivable coating composition performance, such as coating surface characterization, and the aforementioned various parameters related to the coating composition. The difference between models M2 and M3 lies only in which of the above two aspects is expected as input and which is provided as output.

[0365] The "loss function" (also called the "objective function") used for prediction problems can, in its simplest case, simply count the correctly identified predictions from the prediction set. The higher the proportion of correct predictions, the higher the quality of the predictive model used in the machine learning process (such as a model implemented in a neural network). For example, the question of whether rheological properties such as viscosity and / or quality properties such as the type and extent of coating defects within the coating surface are within a predetermined acceptable range can be understood as a classification problem.

[0366] However, many alternative loss functions and corresponding criteria for evaluating the prediction accuracy of trained models are also feasible.

[0367] Figure 15A block diagram of a data processing and coating process distributed system 900 used to improve the composition quality prediction process is shown.

[0368] The system includes a database 904 containing known coating compositions 906 and candidate compositions 908. The known composition 906 may be, for example, a set of data records, each containing detailed descriptions of the component types and / or quantities, production process parameters, and / or coating composition application process parameters, whether complete or incomplete. Furthermore, each data record for a known coating composition includes empirically determined physical, chemical, tactile, optical, and / or other measurably ascertainable properties of the coating composition and / or the resulting coated surface, wherein "empirically determined" includes measurements and characterizations calculated in relation to empirical data, such as quality characterizations calculated from image data.

[0369] On the other hand, candidate composition 908 is a composition whose physical, chemical, tactile, optical and / or other metrologically ascertainable properties are unknown.

[0370] For example, composition 206 is known to contain a coating composition specification that has been produced and tested using HTE equipment.

[0371] The coating composition description 908 may include coating composition descriptions provided by the buyer of the coating components, or may be provided in a calculation step that automatically generates different coating composition descriptions based on a single, complete or incomplete coating description.

[0372] For example, the description of coating composition variants can be created by increasing and / or decreasing the amount of one or more components of the composition by 10%. If only a single component is changed each time by increasing or decreasing the amount of that component by 10%, two variants are formed for each component. In the case of 20 components, this method produces 40 candidate compositions. The number of automatically generated candidate compositions is preferably further increased by simultaneously increasing or decreasing the concentration of two or more components by 10% compared to their concentration in known compositions and by modifying process parameters.

[0373] Generally speaking, in terms of cost and profitability, the number of candidate compositions that can be automatically calculated is considerably larger than the number of coating compositions that can actually be prepared and tested in the laboratory.

[0374] The distributed system 900 includes a computer system 924 comprising a neural network or another type of machine learning model and an active learning module 922. The active learning module 922 has at least read access to read one or more selected candidate compositions and their respective assigned parameters, such as their components, from a database 904. According to some embodiments, the active learning module and / or a device 944 for preparing and analyzing coating compositions according to the specifications of the selected candidate compositions, and optionally, the resulting coating surfaces, has write access to the database 904 to store, in the database 904, empirically obtained properties of the selected candidate coating compositions or their respective coating surfaces. For example, obtaining and storing coating surface characterization and / or coating defect measurements of newly prepared candidate compositions may result in such candidate compositions becoming known compositions and thus stored in different locations in the database 904 and / or equipped with different metadata (tags).

[0375] Figure 16 A 2D section 1000 of the multidimensional data space is shown, illustrating various combinations of defoamer DF1 with optimal antifoaming performance and defoamer DF2 with optimal compatibility in a specific coating composition. The "active learning module" selects specific data points 1008 from the multidimensional data space to expand the training data set and improve the accuracy of the prediction model M3 used in the composition specification prediction procedure. The model improvement is described below for model M3, but it can also be used to improve model M2 used in the coating composition quality prediction procedure.

[0376] In training the predictive model M3, which can be implemented as a neural network or other type of machine learning model, the predictive model learns to compute an output vector, which may include one or more of the following: the type of component, the absolute and / or relative amounts of the component, coating composition production parameters, and / or coating composition application process parameters. Input data is preferably provided as an input vector and includes one or more desired surface characteristics, such as quality characteristics provided as input. Optionally, the input may include incomplete descriptions of one or more components, absolute or relative component amounts, production process parameters, and / or application process parameters, or their respective amounts or effective ranges. Incomplete descriptions can be used by the model to limit the solution space for predicting the coating composition. For example, if the incomplete description specifies a water-based coating medium, the predicted composition suggested by the model will not be based on an organic coating medium. If the incomplete description indicates that two defoamers, DF1 and DF2, should be used, the model can predict only the relative amounts of these defoamers. If only one defoamer, DF1, is provided as input, model M3 may suggest only another defoamer, DF2, which is compatible with defoamer DF1 according to the prediction.

[0377] The input vector of model M3 may include, in particular, one or more of the following properties: coating surface quality measures, coating defect types (especially bubble defects and pit defects), coating defect measures, storage stability, pH value, rheological parameters and especially viscosity, density, relative mass, colorimetry and especially color intensity, and / or production cost reduction. Production cost reduction can be automatically recorded, for example, during composition preparation by automated production equipment 244 and may involve, for example, given reference values. However, costs can also be recorded manually. Similarly, coating defects and surface quality measures can be automatically determined by equipment 244.

[0378] In the described example, the input received by the model may specify that the occurrence of both bubble defects and pit defects should be minimized. The input may also specify one or more components of the coating composition, specifically, in the (incomplete) specified details of the coating composition, that defoamer DF1 with optimal antifoaming properties and defoamer DF2 with optimal coating medium compatibility should be used as components, but the absolute or relative amounts of the defoamers are not provided. For example, DF1 could be Evonik's Tego Foamex 810, and DF2 could be Evonik's Tego Wet 285.

[0379] Based on available data, the M3 model has learned that the foam volume produced by a mixture of DF1 and DF2 defoamers falls between the foam volumes produced by either defoamer alone. The foam volume-defoamer ratio relationship is almost linear, where the foam volume and associated bubble defects decrease as the DF1:DF2 ratio increases, thus improving the surface quality associated with bubble defects. Furthermore, the model has learned that the compatibility of the mixture of these two defoamers with the coating medium decreases as the DF1:DF2 ratio increases, i.e., reducing the surface quality associated with pitting defects.

[0380] After some initial training steps, the neural network model M3 has "learned" certain relationships between the components or process parameters of the coating composition and certain properties of the resulting coating composition or coating surface. Based on the knowledge stored in model M3, the model can predict that if the DF1:DF2 ratio in the composition provided as input is too high, a large number of pits will appear, and if the DF1:DF2 ratio is too low, many bubble defects will appear.

[0381] These learned relationships are Figure 16The diagram is shown by dividing the data space by contour line 1016, which divides the data space into a data space 118 with acceptable coating surface quality performance and a data space 1020 with unacceptable coating surface quality performance in relation to the performance "coating quality". This plot shows that both the absolute and relative amounts of defoamers can be relevant: if the total amount of both defoamers is too low, bubble defects may occur. If the DF1:DF2 ratio is too high, pitting defects will occur. If the ratio is too low, bubble defects will occur. If the total amount of both defoamers is too high, other surface defects may occur, or production costs (not shown) may be predicted to be unacceptably high.

[0382] Figure 16 Only a portion of the data space can be presented, as the plot can only display two dimensions ("Concentration DF1" and "Concentration DF2"), but other coating components and their absolute and / or relative amounts also affect the coating surface quality. Coating compositions typically have more than 10, generally around 20, components, and their respective dimensions can also be presented as production process parameters and coating application process parameters. The data space 1000 includes... Figure 16 The diagram shows many more dimensions. Each of the many subspaces formed by these 20 dimensions contains its own dividing lines and areas of acceptable or unacceptable coating surface quality. The sum of the dividing lines 1016 in the multidimensional space learned by model M3 during training is also called the dividing plane (hyperplane).

[0383] exist Figure 16 The data points circled in the diagram represent candidate coating compositions for which their respective properties have not yet been empirically determined. Model M3 can be fairly certain that the coating compositions represented by, for example, data points 1008 and 1009 are within the acceptable coating quality range, and that the coating compositions represented by data points 1004, 1012, and 1010 have unacceptable coating quality. However, the model cannot confirm the surface quality of the coating composition represented by data point 1002. Therefore, it can be assumed that preparing the coating composition represented by data point 1002, empirically determining the various properties of the coating composition (including coating surface defects), and using the obtained empirical data to expand the training dataset and retraining model M3 based on the expanded training dataset will provide the model with the highest learning performance.

[0384] For example, data points 1002 to be used to further expand the training data with further experience can be selected, for example, according to the so-called "minimum edge hyperplane" method. For example, the active learning module can be designed as a support vector machine or another algorithm that can divide the data space defined by all data points into multiple subspaces based on the knowledge that the predictive model has learned from the subgroup of data points regarding one or more performance parameters. Thus, the current "knowledge" acquired during the initial training of model M3 is represented by the dividing line or dividing plane 1016. The "minimum edge hyperplane" method assumes that the data points 1002 with the smallest distance from the dividing line 1016 are those data points for which the learned predictive model represented by the dividing line 1016 is most uncertain, and therefore the experimental composition belonging to this data point should be selected, prepared, and analyzed to empirically determine the current performance, such as coating surface characterization and / or viscosity. In the example presented here, the active learning module then selects the candidate coating composition represented by data point 1002, considering only the performance "coating surface quality," and causes device 244 to prepare and analyze this composition to expand the training data with the specifications of the components of composition 1002 and empirically determined performance of the coating composition and the corresponding coating surface. Model M3 is then subsequently retrained based on the expanded training data set.

[0385] For example, empirical measurements of the composition represented by data point 1002 may indicate that its predicted coating surface quality falls within the unacceptable coating quality zone 1020. The result of retraining based on the extended training data set will therefore be that the predictive model of the neural network, plotted here by the dividing line 1016, will be adjusted so that, for the composition represented by data point 1002, the future prediction will be that its coating surface quality falls within region 1020. Thus, through retraining based on the extended training data set, the line / plane 1016 will be altered such that it acquires a "bulge," so that the improved model will now identify and predict that the composition represented by point 1002 is within the unacceptable coating quality zone 1020. In practice, when selecting the data point or the corresponding test composition, the distance of the corresponding data point from the dividing lines of several properties is preferably taken into account, for example, by selecting the data point with the minimum average distance from all dividing lines / dividing planes of the complete multidimensional data space.

[0386] A similar improvement can be made to the prediction model M2, which is intended to predict the quality of a coating composition based on complete or incomplete coating composition specifications and / or relevant process parameters, the only difference being that the information used as inputs or outputs of the respective models is interchanged.

[0387] On the other hand, this document discloses computer-implemented methods and corresponding systems for qualitative and / or quantitative characterization of coated surfaces. These methods and systems include steps and features as described in the following paragraphs.

[0388] 1. Paragraph: A method for qualitative and / or quantitative characterization of a coated surface, the method comprising:

[0389] - A defect identification procedure (124) processes (102) digital images (604, 606, 1202) of the coating surface (162-168), the defect identification procedure being configured to identify patterns, each pattern representing a type of coating surface defect (1204, 1206, 1306, 1308); and

[0390] The characterization of the coating surface is output (104) by the defect identification program, which is calculated in relation to the coating surface identified by the defect identification program during processing.

[0391] 2. The method according to paragraph 1 includes: calculating (106) a measure (632, 634, 1402) for the identified defects by means of a defect identification procedure, wherein the characterization of the coating surface is calculated in relation to the qualitative and / or quantitative characterization of the measure.

[0392] 3. According to the method in paragraph 2,

[0393] - This measure is a quantitative measure selected from the following group, which includes: defect area, the number of bubbles or depressions observed in digital images (604, 606, 1202), the maximum size, minimum size, and / or average size of bubbles or depressions in digital images (604, 606, 1202); and / or

[0394] - This measurement is a qualitative measurement, specifically of defect types selected from the following group: pitting defects, scratches, adhesion failure defects, alligator cracks, bleeding defects, blistering defects, efflorescence defects, runner defects, bubble defects, cathodic disbondment defects, fine cracks, shrinkage defects, wire drawing defects, cracking defects, crazing defects, claw-shaped wrinkles, delamination defects, fading defects, peeling defects, exposed substrate defects, heat loss defects, impact damage defects, interlayer fouling defects, mud cracking defects, orange peel defects, flaking defects, pinhole defects, wavy coating defects, sagging defects, sheet rust defects, pitting rust defects, rust spot defects, dent defects, settling defects, skinning defects, solvent undercut defects, solvent bubble defects, stress cracking defects, underfilm corrosion defects, and wrinkling defects.

[0395] 4. The method according to one of the preceding paragraphs also includes:

[0396] - Identify at least one type of coating surface defect to be identified;

[0397] - Automatically determine one or more illumination angles and / or one or more image acquisition angles, which allow the acquisition of a digital image that enables a defect identification program to calculate the characterization of the coating surface depicted in the image in relation to at least one identified defect type;

[0398] - Position one or more light sources (160) at one or more illumination angles (1502) relative to the coating surface; and / or

[0399] - Position one or more cameras, preferably one camera (134), at one or more image acquisition angles (1504) determined relative to the coating surface;

[0400] - After positioning the light source, camera, and / or coating surface relative to each other, use the camera to acquire digital images of the coating surface.

[0401] 5. According to one of the methods in the preceding paragraphs, digital image processing also includes:

[0402] - Digital image classification related to the type and / or quantity of surface defects depicted in the image, and / or image semantic segmentation based on one or more surface defect types depicted in the image, and / or object detection and / or image instance segmentation of defect instances within the image, thereby automatically assigning one or more labels to the entire digital image, multiple image regions, and / or individual pixels, each label representing the defect type identified in the digital image; and

[0403] - Output one or more of the assigned tags.

[0404] 6. According to one of the methods in the preceding paragraphs,

[0405] - The defect detection program is effectively integrated with the camera and designed for...

[0406] o Determine whether the camera is adjusted within a predetermined distance range and / or a predetermined image acquisition angle range relative to the coating surface to allow images to be acquired from a similar relative position to that used to acquire training images to generate a predictive model for a defect identification procedure.

[0407] Based on the determined results,

[0408] ■ Generate feedback signals for the user and / or camera regarding whether camera position adjustment is required; and / or

[0409] ■ Automatically adjust the relative position of the camera and the coated surface; and / or

[0410] ■ Allows the camera to selectively acquire images when it is located within a predetermined distance and / or image acquisition angle range.

[0411] 7. According to the method of one of the preceding paragraphs, the defect identification procedure is selected from the group consisting of:

[0412] - An application that is installed on portable data processing systems (130), especially portable telecommunications devices such as smartphones.

[0413] - An application that is installed on a portable or stationary device (150) specifically designed for the quality inspection of coated surfaces;

[0414] - An application that is installed on a high-throughput device (244) for automated or semi-automated production and / or testing of coatings;

[0415] - Web applications downloaded and instantiated via the network;

[0416] - Programs that execute within a browser, such as JavaScript programs;

[0417] - A server program instantiated on a server computer, which connects effectively to a client program installed on a client data processing system via network communication. The client program is specifically designed to acquire digital images and provide the images to the server program via the network and / or display the results provided by the server program.

[0418] 8. According to one of the preceding paragraphs, the defect identification procedure includes a predictive model (M1) learned from training data (602) in a training step to identify a predetermined pattern, the training step being performed by a machine learning procedure, in particular a neural network.

[0419] 9. According to the method in paragraph 8, the machine learning procedure is a neural network or a set of neural networks including a region proposal network, which is designed to scan anchors of an input image to make a suggestion as to whether the anchors may contain one of the defect patterns, the anchors being sub-regions of the input image whose anchor size matches the expected size of the defect pattern.

[0420] 10. The method according to one of paragraphs 8 to 9 includes: performing a training step based on training data (602), the training data including a set of labeled digital training images (604, 606) of a coated surface, labels (616, 618) marking the location / position and / or type of defects in the training images, and a prediction model being trained to recognize patterns through the labeled training images using backpropagation.

[0421] 11. According to the method of paragraph 10, wherein each training image has assigned supplementary data (608, 610), the supplementary data is processed in the training step to allow the prediction model (M1) to associate the supplementary data with the defect pattern, the supplementary data including:

[0422] - A quantitative measure of one or more defects plotted in the training image, such as the size and / or severity of the defect or the number of bubbles;

[0423] -Optionally, parameters may also be selected from the following group, which includes:

[0424] o is an indication of one or more components of the coating used to generate the coating surface drawn in the training image;

[0425] Indication of the absolute or relative amounts of one or more components of the coating composition; and / or

[0426] o One or more production process parameters characterizing the process of generating the coating composition, such process parameters including, for example, the mixing rate and / or mixing duration of the coating composition; and / or

[0427] o One or more application process parameters characterizing the process of applying a coating composition to a substrate, the application process parameters including, in particular, the amount of coating composition applied per unit coating surface area, the substrate type and / or the type of application device; and / or

[0428] o System parameters of the imaging system used to acquire training images, selected from the following group: type of light source used to illuminate the coated surface, light source brightness, illumination angle, light source wavelength, type of one or more cameras used to acquire digital images of the coated surface, image acquisition angle, and position of one or more cameras.

[0429] On the other hand, a computer-based method and corresponding system for providing a prediction procedure related to coating compositions are disclosed herein. The method and system include the steps and features described in the following paragraphs.

[0430] paragraph:

[0431] 12. A computer-implemented method for providing a prediction program related to a coating composition, the method comprising:

[0432] - Provides a database (204,904) that includes qualitative and / or quantitative characterization of the coated surface associated with one or more parameters selected from the group consisting of one or more components of the coating composition used to produce the respective coated surface, the relative and / or absolute amounts of one or more of the components, the production process parameters of the coating composition, and / or the application process parameters used to produce the coated surface.

[0433] - A machine learning model is trained based on the association between the coating surface characterization and one or more parameters in a database to provide a predictive model (M2, M3), which learns to associate qualitative and / or quantitative characterizations of one or more coating surfaces with one or more stored parameters associated with their respective coating surface characterizations; and

[0434] - Provides a composition quality prediction program, which includes a prediction model (M2). The composition quality prediction program is designed to use the prediction model (M2) to predict the properties of the coating surface to be produced from one or more input parameters selected from the group consisting of one or more components of the coating composition to be used to produce the coating surface, the relative and / or absolute amounts of the one or more said components, the production process parameters to be used to prepare the coating composition, and / or the application process parameters to be used to produce the coating surface; and / or

[0435] - Provides a composition specification prediction program, which includes a prediction model (M3) designed to use the prediction model (M3) to predict and output one or more parameters related to the prediction of a coating composition for producing a coating surface having the input surface characterization, based on an input that at least details the desired coating surface characterization. The one or more output parameters are selected from the group consisting of one or more components of the coating composition, relative and / or absolute amounts of one or more of the components, production process parameters to be used to prepare the coating composition, and / or application process parameters for producing the coating surface. Optionally, the composition specification prediction program is designed to receive incomplete coating composition specifications and to use the specifications to limit the solution space of the predicted output parameters.

[0436] 13. According to the method in paragraph 12, the method includes:

[0437] - Provide multiple images depicting coated surfaces produced from multiple different coating compositions, wherein at least some of the coated surfaces selectively have one or more coating defects of one or more different defect types;

[0438] - Apply a defect identification procedure to an image to identify patterns in the image in order to obtain a measure of coating defects presented by the identified patterns in the image and calculate a qualitative and / or quantitative characterization of the coating surface plotted by the image, wherein the defect identification procedure is preferably a defect identification procedure as detailed in one of paragraphs 1 to 11.

[0439] - Store in the database qualitative and / or quantitative characterizations of the coated surface associated with one or more parameters of the coating composition used to produce a coated surface that includes these defects.

[0440] 14. The method according to one of paragraphs 12 to 13 above further includes using a composition specification prediction procedure to predict coating composition specifications that meet the desired coating surface characterization provided as input, the use of which includes:

[0441] - Provide at least the specifications of the desired coating surface characterization to the composition specification prediction program as input;

[0442] - Predict the specifications of a coating composition suitable for providing a coated surface with desired surface properties through a composition specification prediction procedure, wherein the specifications include one or more parameters selected from the group consisting of one or more coating composition components, absolute and relative amounts of one or more coating composition components, production process parameters and / or application process parameters;

[0443] - Preferably, it also includes outputting the predicted coating composition specifications to the person and / or inputting the specifications of the selected candidate coating compositions to the processor, the processor controlling the equipment (244) for the production and / or testing of the coating compositions, wherein the processor causes the equipment to produce the input coating composition.

[0444] 15. Based on one of the methods in paragraphs 12 to 14 above, where the training process involves an active learning module, this method also includes:

[0445] a. Based on the measurements and coating composition in the database, the preferred machine learning model is the training of a neural network to provide a predictive model (M2, M3), where the loss function is minimized for training purposes.

[0446] b. Test to determine whether the loss function value obtained for the prediction model meets the prescribed criteria, thereby selectively performing the following steps iv if (meaning only if) the criteria are not met:

[0447] i. The active learning module selects a candidate coating composition specification (1002) from multiple candidate coating composition specifications. The selected candidate composition specification is determined to be one of the candidate compositions that provides the highest learning effect for the prediction model (M2, M3) with respect to qualitative and / or quantitative coating surface characterization and one or more parameters selected from the group consisting of coating components, component amounts, production process parameters and / or application process parameters of the coating composition.

[0448] ii. The computer system operates the device to automatically produce candidate coating compositions according to selected specifications, automatically apply the produced candidate compositions to the substrate, and automatically acquire images of the surface on which the coating compositions are applied;

[0449] iii. Apply a defect identification program to the images acquired in step ii to calculate and store qualitative or quantitative characterizations of the coating surface drawn in the images within a database, thereby expanding the database;

[0450] iv. Retrain the prediction models (M2, M3) based on the expanded database to provide an improved version of the prediction model.

[0451] v. Repeat step b by using an improved version of the prediction model.

[0452] c. Replace the prediction model (M2) of the composition quality prediction procedure with an improved version of the prediction model and / or replace the prediction model (M3) of the composition specification prediction procedure with an improved version of the prediction model.

[0453] 16. Based on one of the methods in paragraphs 12 to 14 above, where the training process involves an active learning module, the method further includes:

[0454] - Use the active learning module to interactively query the prediction model (M2) of the composition quality prediction procedure to perform multiple predictions of the quality of the surface coating to be produced by the coating composition. Each prediction is based on the specifications of the candidate composition, which include one or more components of the coating composition, the relative and / or absolute amounts of one or more of the components, and / or the production process parameters to be used to prepare the candidate coating composition and / or the application process parameters to be used to produce the candidate coating surface.

[0455] - The uncertainty of each prediction result is determined through an active learning module (e.g., through the minimum edge hyperplane method, loss function, etc.);

[0456] - Identify one or more candidate composition specifications whose resulting predictions have an uncertainty exceeding a predetermined uncertainty threshold;

[0457] - Selectively produce candidate coating compositions (but not another candidate coating composition whose prediction has a sufficiently high degree of certainty, i.e., an uncertainty level below a threshold) based on the identified candidate coating composition specifications, thereby automatically generating new coating compositions;

[0458] - Apply a new coating composition to obtain a new coated surface;

[0459] - Acquire images of the newly coated surface;

[0460] - Image analysis of acquired images to identify and / or characterize surface defects on the surface of a new coating;

[0461] - Retrain the composition quality prediction program, thereby also taking into account the identified candidate coating specifications and their respective obtained surface coating defect characteristics to improve the accuracy of the composition quality prediction program.

[0462] For example, the generation and selection of new coating compositions, as well as the generation and selection of coating surfaces, and image acquisition and analysis, can be achieved by a computer system running a device (244) to automatically produce candidate coating compositions based on identified candidate coating specifications (but not other candidate coating compositions whose predictions have sufficiently high certainty, i.e., uncertainty levels below a threshold). The device automatically applies the produced candidate coating compositions to their respective substrates. The device may include or be effectively connected to an image acquisition system. The image acquisition system is used to acquire images of the surfaces on which the coating compositions are applied. The image acquisition system may apply a defect identification procedure to the acquired images of the surface coatings, as described herein with respect to embodiments of the invention, that identify candidate coating compositions. This yields qualitative or quantitative characteristics of the new candidate coating surfaces plotted in each image. These characteristics can be used to retrain and improve the prediction model (M2). Predictions with high uncertainty are interpreted by the active learning module as indicating that the candidate coating composition specification represents a coating composition that the prediction model is extremely unfamiliar with, thus enabling the prediction model to achieve strong learning effects based on real, empirical, and therefore reliable data obtained for such candidate coating compositions. The prediction model (M3) of the composition specification prediction procedure can be similarly improved using the active learning module. The only difference is that the input to the prediction model (M3) of the composition specification prediction procedure corresponds to the output of the prediction model (M2) of the composition quality prediction procedure, and vice versa. Therefore, the two models are challenged based on different candidate input data. In the case of model (M3), the candidate input data is a detailed description of the desired physical or chemical properties of the coating composition, particularly the qualitative and / or quantitative characteristics of the coating surface produced by the coating composition. The other steps and aspects are the same.

[0463] 17. A system comprising:

[0464] - An apparatus (244) for producing and testing compositions for paints, varnishes, printing inks, abrasive resins, pigment concentrates, or other coatings, wherein the apparatus comprises at least two processing stations interconnected by a transport system on which self-propelled transport vehicles are movable to transport components of the composition and / or the resulting composition between the processing stations, and

[0465] - A computer system (224) designed to perform the methods according to one of paragraphs 1 to 16.

[0466] The features of the methods and systems described and the systems described above can be freely combined with the embodiments and examples of the present invention, as long as they are not mutually exclusive.

[0467] List of reference numerals

[0468] Steps 102-118

[0469] 120 Data Processing System

[0470] 122 Data storage media

[0471] 124 Defect Identification Program

[0472] 125 digital images

[0473] 126 processor

[0474] 130 Portable Telecommunications Equipment

[0475] 132 GUI of the Defect Identification Program

[0476] 134 cameras

[0477] 136 Defect Identification Program GUI

[0478] 140 Browser

[0479] 142 Network

[0480] 144 Server Computer

[0481] 146 Network Server

[0482] 150 Coating Quality Control Device

[0483] 152 Control Panel

[0484] 154 Carrier / Conveyor Belt

[0485] 156 robotic arms

[0486] 158 robotic arm

[0487] 160 light source

[0488] 162-168 Coated Samples

[0489] 170 Computer System

[0490] 204 Database

[0491] 244 Equipment for producing and / or testing coating compositions

[0492] 246 Main Control Computer

[0493] 248 Control Unit

[0494] 252 Image Acquisition Units

[0495] 254 Sample Coating Units

[0496] 246 Hybrid Units

[0497] 257 Analyzer

[0498] 258 transport units

[0499] 400-neuron network structure

[0500] 402 Input data (e.g., vector)

[0501] 404 Layers of a Neuron Network

[0502] 406 Output of the prediction model

[0503] 602 Training data used for prediction model M1

[0504] 604 Training images used for prediction model M1

[0505] 606 training images used for prediction model M1

[0506] 608 Marked with context parameters

[0507] 610 Marked with context parameters

[0508] 612 Prediction Model M1

[0509] 614 Defect Identification Program GUI

[0510] 615 Camera Control Module

[0511] 616 Artificial labels with defect measurement and quality characterization

[0512] 618 Artificial labels with defect measurement and quality characterization

[0513] 620 Database or image acquisition system

[0514] 622 Test data used for predictive model M1

[0515] 624 test images used to predict model M1

[0516] 626 test images used for predicting model M1

[0517] 628 Marked with context parameters

[0518] 630 Marked with context parameters

[0519] 632 Calculated labels with defect measurement and quality characterization

[0520] 634 Calculated labels with defect measurement and quality characterization

[0521] 711 Untrained prediction model M2

[0522] 712 trained prediction model M2

[0523] 714 Composition Quality Prediction Procedure

[0524] 715 GUI for Composition Quality Prediction Program

[0525] 720 Database

[0526] 721 Active Learning Module

[0527] 722 M2 test data

[0528] 728 Context Parameters

[0529] 729 Calculated rheological properties

[0530] 730 Context Parameters

[0531] 731 Calculated rheological properties

[0532] 732 Calculation of coating quality / performance

[0533] 734 Calculated coating quality / performance

[0534] 736 Output of the Composition Quality Prediction Program

[0535] Steps 802-816

[0536] 900 System

[0537] 902 DBMS

[0538] 904 Database

[0539] 906 Known Compositions

[0540] 908 Candidate Composition

[0541] 910 Known compositions and properties

[0542] 912 Selected candidate compositions

[0543] 914 choices

[0544] 916 Selected Compositions

[0545] 918 Performance

[0546] 944 equipment

[0547] 946 Control Computer

[0548] 948 Control Software

[0549] 952 Analysis Unit

[0550] 954 Preparation Unit

[0551] 956 Preparation Unit

[0552] 958 Transportation Unit

[0553] 1000 Hyperplane Drawing

[0554] Data points 1002-1009

[0555] 1016 dividing line

[0556] 1018 Acceptable coating quality area

[0557] 1020 Unacceptable coating quality area

[0558] Digital image of the 1202 coated surface

[0559] 1204 Bubbles

[0560] 1206 bubbles

[0561] 1208 Dirt

[0562] 1210 Artificially labeled image 1202

[0563] 1212 Manually added tags

[0564] Digital image of the 1302 coating surface

[0565] 1306 Foam Pores

[0566] 1308 Foam Pores

[0567] 1304 Automatically generate labels / calculate segmentation boundaries for identifying holes and defects.

[0568] 1402 Quantitative Measurement of Bubble Defects: Histogram

[0569] 1502 lighting angle

[0570] 1504 Image Acquisition Angle

Claims

1. A method for qualitatively and / or quantitatively characterizing a coating surface, the method comprising: A defect identification program (124) is provided, which is designed to identify coating surface defect types (1204, 1206, 1306, 1308). The defect identification program includes a prediction model that has been trained based on training images (604, 606) acquired within a predetermined distance range relative to the coating surface depicted in the training images and / or within a predetermined image acquisition angle range. A data storage medium is provided, wherein multiple coating surface defect types are stored in association with a predetermined distance range between at least one camera and the coating surface and / or in association with one or more predetermined image acquisition angle ranges, wherein the predetermined image acquisition angle ranges and / or distance ranges stored in association with a specific defect type are angle ranges and / or distance ranges that allow the acquisition of digital images of the coating surface, which allows the defect identification program to identify the defect type in the digital images of the coating surface; Identify at least one type of coating surface defect to be identified; For each of the at least one type of coating surface defect to be identified: Automatically identify one of a predetermined distance range and / or one of a predetermined image acquisition angle range that are stored in association with the determined defect type; The defect identification program determines whether at least one camera (134) effectively connected to the defect identification program is located within the predetermined distance range and / or the predetermined image acquisition angle range relative to the currently presented coating surface. The defect identification program uses the identified predetermined distance range and / or the identified predetermined image acquisition angle range to determine whether the camera is located within the predetermined distance range or the image acquisition angle range. Based on the determined results Generate a feedback signal indicating whether adjustment of the position of the at least one camera is required so that the camera is within the predetermined distance range and / or the predetermined image acquisition angle range; and / or The relative distance between the at least one camera and the coating surface is automatically adjusted so that the camera position is within a predetermined distance range from the coating surface, and / or the angle of the at least one camera is automatically adjusted so that the camera position is within a predetermined image acquisition angle range; The camera is allowed to capture digital images of the coated surface (604, 606, 1202) only when the camera is within a predetermined distance range and / or image acquisition angle range; The digital images (604, 606, 1202) of the coating surface (162-168) are processed (102) by the defect identification procedure used to identify defects on the coating surface; The defect identification program outputs (104) a characterization of the coating surface, which is then calculated in relation to the identified defects on the coating surface.

2. The method according to claim 1, comprising: The defect identification procedure calculates (106) the measure (632, 634, 1402) for the identified defect, wherein the characterization of the coating surface is calculated in relation to the qualitative and / or quantitative characterization of the measure.

3. The method according to claim 2, This measure is a quantitative measure selected from the following group, which includes: Defect area; number of bubbles or dents observed in digital images (604, 606, 1202); maximum, minimum, and / or average size of bubbles or dents in digital images (604, 606, 1202); and / or This measurement is a qualitative measurement, which is a selection of defect types from the following group: pitting defects, scratches, adhesion failure defects, alligator cracks, bleeding defects, blistering defects, efflorescence defects, run-through defects, bubble defects, cathodic disbondment defects, fine cracks, shrinkage defects, wire drawing defects, cracking defects, crazing defects, claw-shaped wrinkles, delamination defects, fading defects, peeling defects, exposed substrate defects, heat loss defects, impact damage defects, interlayer fouling defects, mud cracking defects, orange peel defects, flaking defects, pinhole defects, wavy coating defects, sagging defects, sheet rust defects, pitting rust defects, rust spot defects, depression defects, settling defects, skinning defects, solvent undercut defects, solvent bubble defects, stress cracking defects, underfilm corrosion defects, and wrinkling defects.

4. The method according to any one of claims 1 to 3, wherein, The predetermined distance range is the distance range between the presented coated surface and the at least one camera, which allows the camera to capture images with a resolution of at least a predetermined minimum resolution.

5. The method according to claim 4, wherein, This predetermined minimum resolution is a minimum resolution specific to the coating defect type.

6. The method according to any one of claims 1 to 3, The automatic adjustment of the relative distance between the at least one camera (134) and the coating surface is performed by automatically changing the position of the at least one camera and / or automatically changing the position of the support including the sample with the coating surface, so that the distance between the at least one camera and the coating surface is within a predetermined distance range; and / or Automatic adjustment of the image acquisition angle (1504) relative to the coating surface includes changing the orientation of the at least one camera so that it is oriented relative to the coating surface at an image acquisition angle within a predetermined image acquisition angle range.

7. The method according to any one of claims 1 to 3, wherein, At least one of one or more surface defect types is stored in association with a predetermined illumination angle range, the predetermined illumination angle range stored in association with a specific defect type being the range of illumination angles of the light source relative to the coating surface, which allows the acquisition of digital images of the coating surface, which allows a defect identification program to identify the defect type in the digital images of the coating surface; For each of the at least one type of coating surface defect to be identified: Automatically identify one of the predefined lighting angle ranges associated with the determined defect type; The defect identification procedure uses the identified predetermined illumination angle range to determine whether one or more light sources are positioned relative to the coating surface with illumination angles within the identified predetermined illumination angle range. and If the illumination angle of the one or more light sources is outside the identified predetermined illumination angle range, The one or more light sources (160) and the coating surface are positioned relative to each other such that the illumination angle is within the identified predetermined illumination angle range; and / or A feedback signal is generated, which indicates and / or how the relative positioning of the one or more light sources (160) and the coating surface can be adjusted such that the illumination angle is within the identified predetermined illumination angle range.

8. The method according to any one of claims 1 to 3, further comprising: After the coating surface, at least one camera, and / or one or more light sources are positioned relative to each other, the at least one camera is used to acquire a digital image of the coating surface.

9. The method according to any one of claims 1 to 3, wherein the processing of the digital image further comprises: The defect identification program classifies digital images in relation to the type and / or number of surface defects depicted therein and / or performs image semantic segmentation based on one or more surface defect types depicted therein and / or performs object detection and / or image instance segmentation of defect instances within the image, thereby automatically assigning one or more labels to the entire digital image, multiple image regions and / or individual pixels, each label representing the type of defect identified in the digital image. and Output the one or more labels assigned.

10. The method according to any one of claims 1 to 3, wherein the defect identification procedure is selected from the group consisting of: The application is installed on a fixed or portable data processing system (130); The application is installed on a portable or stationary device (150) specifically designed for the quality inspection of coated surfaces; The application is installed on high-throughput equipment (244) for automated or semi-automated production and / or testing of coatings; Web applications that are downloaded and / or instantiated over a network; Programs that run in a browser; A server program is instantiated on a server computer and is effectively connected via a network link to a client program instantiated on a client data processing system. The client program is designed to acquire the digital image and provide the image to the server program via the network and / or display the results provided by the server program.

11. The method according to any one of claims 1 to 3, wherein the prediction model (M1) learns from training data (602) including training images (604, 606) in a training step performed by a machine learning program designed to recognize patterns in digital images, the machine learning program being a neural network.

12. The method according to any one of claims 1 to 3, further comprising: The prediction model is generated by performing a training step based on training data (602) containing training images, which include multiple labels (616, 618) indicating the location and / or type of defects within the coating surface drawn in the training images. The prediction model is trained to identify defect types using backpropagation through the labeled training images.

13. The method according to claim 11, wherein, Each training image has assigned supplementary data (608, 610), which is processed in this training step to allow the prediction model (M1) to associate the supplementary data with the defect type. This supplementary data includes: A quantitative measure of one or more defects plotted in training images; Optionally, parameters may also be selected from the following group, which includes: Indicators for generating one or more components of the coating on the coated surface drawn in the training image; Indication of the absolute or relative amounts of one or more components of the coating composition; and / or One or more production process parameters characterizing the process of generating the coating composition, including the mixing rate and / or mixing duration of the coating composition; and / or One or more application process parameters characterize the process of applying a coating composition to a substrate, including the amount of coating composition applied per unit coating surface area, substrate type, and / or application device type; and / or System parameters of the imaging system used to acquire training images, which are selected from the following group: the type of light source used to illuminate the coated surface, the light source brightness, the illumination angle, the light source wavelength, the type of one or more cameras used to acquire digital images of the coated surface, the image acquisition angle, and the position of one or more cameras.

14. The method according to claim 13, wherein, The system parameters include at least the illumination angle, the image acquisition angle of the at least one camera, and / or the relative distance between the at least one camera and the coated surface.

15. The method according to claim 10, wherein, The portable data processing system is a portable telecommunications device.

16. The method according to claim 15, wherein, The portable telecommunications device is a smartphone.

17. The method according to claim 10, wherein, The program that executes in the browser is a JavaScript program.

18. The method according to claim 13, wherein, The quantitative measure is the size and / or severity of the defect or the number of bubbles.

19. A computer system (224) for qualitative and / or quantitative characterization of a coated surface, the computer system comprising: A defect identification program (124) is effectively connected to at least one camera (134) and is designed to identify coating surface defect types (1204, 1206, 1306, 1308). The defect identification program includes a prediction model trained based on training images (604, 606) acquired within a predetermined distance range and / or a predetermined image acquisition angle range relative to the coating surface drawn within the training images. A data storage medium wherein multiple coating surface defect types are stored in association with a predetermined distance range between at least one camera and the coating surface and / or in association with one or more predetermined image acquisition angle ranges, wherein the predetermined image acquisition angle ranges and / or distance ranges stored in association with a specific defect type are angle ranges and / or distance ranges that allow the acquisition of digital images of the coating surface, which allows the defect identification program to identify the defect type in the digital images of the coating surface; This defect identification program is designed to: Identify at least one type of coating surface defect to be identified; For each of the at least one type of coating surface defect to be identified: Automatically identify one of a predetermined distance range and / or one of a predetermined image acquisition angle range that are stored in association with the determined defect type; The defect identification procedure determines whether the at least one camera is within a predetermined distance range and / or a predetermined image acquisition angle range relative to the currently presented coating surface. The defect identification procedure uses the identified predetermined distance range and / or the identified predetermined image acquisition angle range to determine whether the camera is within the predetermined distance range or the image acquisition angle range. Based on the determined results: Generate a feedback signal indicating whether adjustment of the position of the at least one camera is required so that the camera is within the predetermined distance range and / or the predetermined image acquisition angle range; and / or The relative distance between the at least one camera and the coating surface is automatically adjusted so that the camera position is within a predetermined distance range from the coating surface, and / or the angle of the at least one camera is automatically adjusted so that the camera position is within a predetermined image acquisition angle range; The camera is allowed to capture digital images of the coated surface only when it is within a predetermined distance range and / or image acquisition angle range; Process (102) digital images (604, 606, 1202) acquired from at least one enabled camera to identify one or more defect types; Output (104) a characterization of the coating surface, which is calculated in relation to coating surface defects identified by a defect identification procedure during processing.

20. A system comprising: The computer system of claim 19; The at least one camera; and / or An apparatus (244) for testing components of paints, varnishes, printing inks, abrasive resins, pigment concentrates, or other coatings, comprising: At least one processing station designed to apply one or more coating components to at least one surface of multiple objects. An automated transport system for transporting coated objects to an image acquisition and analysis system. The system is designed to use an image acquisition and analysis system to automatically acquire images of the coated surfaces of multiple objects and output a characterization of the coated surfaces.

Images