Automated segmentation method for quality inspection

Abstract

An automated segmentation method for quality inspection is proposed. A method for measuring defects in components using quality inspection methods is presented. The method includes: receiving a digital image of a component to be inspected, receiving a first reference image of the component to be inspected, and obtaining a second reference image from a combination of the received digital image and the first reference image. Furthermore, the method includes: activating a machine learning-based trained classifier system, which has been trained with training data to build a model, wherein the model serves as the basis for semantic segmentation of voxels in the received image for defect classification; and classifying the voxels of the received digital image by the activated classifier system, wherein the voxels of the received digital image and the voxels of the second reference image are used as input data to the classifier system.

Description

Technical Field

[0001] This invention relates to the automated segmentation of defects in image data during inspection processes that inspect components of the same type (inline and at-line); and more particularly to a computer-implemented method, a corresponding system, and a corresponding computer program product for measuring defects in components using quality inspection methods. Background Technology

[0002] To maintain a consistently high level of quality in manufactured components, permanent quality control during the ongoing production process is crucial. For this purpose, optical systems that can identify defects on or within manufactured components using image recognition methods are now commonly used. Microscopic systems are also increasingly used for such quality inspection measures, where smaller and even minute defects in components can be identified through magnification. This is typically accomplished using image segmentation.

[0003] Especially in in-line and near-line processes, fully automated defect segmentation is required to automatically determine the size of defects, such as their volume, perimeter, diameter, or local frequency of occurrence, in downstream steps without user interaction. Only in this way can cost-effective and complete component inspection be achieved to ensure quality.

[0004] Automated segmentation is achieved in different ways here. During the inspection process (which typically must react to process variations and handle numerous variations in defect manifestations), a model based on a training set of machine-learning models composed of components of the same type (e.g., specimens) is generally preferred. One approach involves building a model that primarily selects error-free components as the training set and determines the mathematical mean and / or standard deviation of image brightness values ​​pixel-by-pixel or voxel-by-voxel. However, this presupposes that the training set consists of image / volume data of the same components. The main drawback of this method is that, in addition to defects, scanning artifacts and slight displacements of specimen components (e.g., walls) due to manufacturing tolerances are also segmented, leading to erroneous results. Contrast effects also contribute to the low robustness of this method.

[0005] The second approach to building the model involves using both faulty and fault-free sample parts in the training set for the machine learning model. Here, defects in faulty sample parts are annotated / labeled pixel-by-pixel / voxel-by-voxel with the defect location. In principle, it is not necessary to assume that the training set consists of images / volume data of identical components; rather, the prerequisite is the existence of locally similar defects with similar appearances. A static classification method is trained based on the training set, where only the image brightness information of the sample is used to segment defects. A major drawback of this method is that a sufficiently large (annotated) training set is required to achieve robust defect segmentation. This typically requires a very long training time. Furthermore, this method tends to segment structures that are similar to defects in the local context, but are actually target structures, as defects.

[0006] Based on the shortcomings of known methods, the basic purpose of the concept proposed here is to overcome the aforementioned shortcomings of known methods, and in particular to propose a method that relies less on the contrast of defects in the surrounding environment of an image and does not require a large amount of training data, thereby also reducing the training time for creating machine learning models. Summary of the Invention

[0007] This objective is achieved by the method, the corresponding system, and the associated computer program product described herein, corresponding to the independent claim. Further design options are described in the relevant dependent claims.

[0008] Corresponding to the first aspect of the invention, a computer-implemented method for measuring defects in a component using a quality inspection method is proposed. The method may include: receiving a digital image of a component to be inspected, receiving a first reference image of the component to be inspected, and obtaining a second reference image from a combination of the received digital image and the first reference image.

[0009] Furthermore, the method may include: activating a machine learning-based trained classifier system, which has been trained with training data to build a model, wherein the model serves as the basis for semantic segmentation of voxels in a received image in defect classification; and classifying the voxels of the received digital image using the activated classifier system. Here, the voxels of the received digital image and the voxels of the second reference image can be used as input data for the classifier system.

[0010] Corresponding to the second aspect of the invention, a system for measuring defects in a component using a quality inspection method is proposed. The system may include: a first receiving unit adapted to receive a digital image of a component to be inspected; a second receiving unit adapted to receive a first reference image of the component to be inspected; and a determining module adapted to obtain a second reference image from a combination of the received digital image and the first reference image.

[0011] Furthermore, the system can have a machine learning-based trained classifier system, which has been trained with training data to build a model that serves as the basis for semantic segmentation of voxels in the received image during defect classification. Here, the classifier system can be adapted to classify voxels of the received digital image by an activated classifier system. The voxels of the received digital image and the voxels of the second reference image can be used as input data for the classifier system.

[0012] Furthermore, embodiments may involve computer program products accessible by a computer-usable or computer-readable medium having program code for use by, or in conjunction with a computer or other instruction processing system. In the context of this specification, a computer-usable or computer-readable medium can be any device suitable for storing, communicating, transmitting, or transporting program code.

[0013] The computer-implemented method for measuring defects in components using quality inspection methods has several advantages and technical effects, which can also be applied to related systems:

[0014] The proposed method or system for measuring (and thus identifying) defects in components using quality inspection methods (especially in-line or near-line quality inspection methods) overcomes the shortcomings of previously known discrete segmentation methods by using a classification system based on differential signals and segmentation of recorded specimen images. In addition to the recorded digital image of the specimen, the proposed concept uses other information to accurately and robustly segment defects. This information is generated from the recorded digital image of the specimen and a reference image. The data for the reference image can be generated in different ways (e.g., averaging, derivation of abstract construction models, etc.). That is, this additional information consists, for example, of the differential signal of the specimen relative to a reference set, which can be derived relatively easily. Specifically, the classifier system uses not only image brightness / volume brightness information derived from the specimen, but also, for example, an image / volume (or information about the image / volume) of similar size describing the local deviation from the reference set. Such differential signals are always available when inspecting similar specimens of the same type in large numbers (e.g., > 20). For this reason, the concept presented here is highly suitable for segmenting defects in workpieces or components during in-line or near-line inspection processes. Therefore, fully automated defect segmentation is particularly needed in in-line or near-line processes to automatically determine defect dimensions, such as volume, perimeter, diameter, or local accumulation, in downstream steps without user interaction. In this regard, the method proposed in this paper makes a significant contribution to improving in-line or near-line inspection processes, and thus produces positive cost-effectiveness in both production and quality control.

[0015] In principle, multiple different classifier systems can be used. Examples are mentioned in other parts of this document. In particular, when using a deep neural network (DNN) as a classifier system, the information channels of the input data (first reference image and second reference image) can be correlated and introduced into the input layer of the DNN, so that the classifier system can determine the weighting of information based on the training set.

[0016] Another noteworthy advantage of the concept proposed in this paper is the improved generalization, higher robustness, and reduced annotation costs. Regarding defects that are only weakly represented in the training set, the proposed method achieves better segmentation accuracy. Furthermore, fewer false positives occur in imaging artifacts, thus improving the accuracy of the method. Additionally, it correctly segments defect-similar target structures with a higher probability.

[0017] Furthermore, the information in the channels representing the differences from the reference set is relatively convincing, resulting in lower complexity for training the classifier system. This also allows for relatively accurate and robust results using less annotated data compared to cases without additional channels. This significantly reduces training time, allowing the proposed method to be applied more quickly to new components or specimens, while also helping to reduce required costs and debugging time.

[0018] In addition to the primary applications for in-line or near-line quality inspection (where many example data are available for the reference set), methods can also be provided for use in laboratories or measurement rooms, where the reference set is typically obtained from fewer examples, and in extreme cases from a single good example. Another possibility is to calculate the reference model using image simulation (e.g., with the aid of CAD models).

[0019] Other embodiments of the inventive concept for this method are presented below, which can be applied in the same and corresponding manner to the corresponding systems:

[0020] According to an advantageous implementation of this method, the second reference image can be obtained by voxel-by-voxel subtraction of the brightness values ​​of the received digital image and the first reference image. Weighting factors for the image or for sub-regions of these images can also be used where appropriate. Subtraction requires relatively few computational resources and can be performed quickly even with a large number of voxels.

[0021] In one possible implementation of this method, the first reference image and the second reference image can be the same. That is, the difference calculation may be completely omitted, which could further accelerate the method. Thus, the additional interpretation work could be handled by the classifier system.

[0022] According to another advantageous embodiment of the method, the received reference image may contain voxel-by-voxel distribution parameter values ​​of the brightness values ​​(especially with respect to the training dataset), and the second reference image can be normalized with respect to these distribution parameter values. In this way, test results that can be concisely compared can be derived.

[0023] According to another advantageous implementation of the method, for voxel classification, the spatial context of the voxel (especially the number of surrounding pixels in a predefined direction) or the temporal context (especially a sliding window of the last N volumes) can be included during classification. In this way, drift effects can be eliminated in particular.

[0024] According to a further development of this method, the first reference image (e.g., using a moving average) can be adapted to the time curve. This also allows for proactive mitigation of unavoidable drift effects.

[0025] This method can be implemented using classifier systems such as support vector machines, random forests, boosting, gradient boosting, Gaussian processing, nearest neighbor, logistic regression, linear regression, and neural networks. Alternative classifier systems are also possible.

[0026] According to a particularly advantageous implementation of this method, the neural network can be a deep neural network (DNN). Such neural networks are particularly suitable for processing digital images. Especially in its implementation as a convolutional neural network (CNN), fast and efficient processing of digital image information can be achieved.

[0027] According to a further embodiment of the method, the input layer of the deep neural network is extended relative to the number of input channels (i.e., the artificial neurons required for processing digital images at the input of the DNN), such that voxel data of the first reference image and / or the second reference image are directed at additional input channels of the classifier system. Thus, a cascading of pixel information from the input data (of the recorded digital image and one or more reference images) can be performed.

[0028] According to an optional embodiment of the method, the image resolution of the received digital image and the image resolution of the first reference image and / or the second reference image may be different. Therefore, the method is also independent of the recording system used for the received digital image and the reference image. The recorded digital image and the reference image may be recorded by different recording systems. Furthermore, the reference image can be preprocessed in virtually any way.

[0029] According to another embodiment of the method, the classifier system can have a first classifier system and a second classifier system. Here, the first classifier system can operate with a first number of parameter values ​​(especially, for example, the weight parameters of a DNN or a lower resolution); and the second classifier system can operate with a second number of parameter values ​​(especially, for example, a higher resolution), wherein the second number of parameter values ​​is greater than the first number. This flexibility allows different requirements to be considered during different stages of the quality control process. In particular, the smaller number of parameter values ​​results in faster processing.

[0030] According to another implementation of the above-described embodiments, the second classifier system is activated only when the first classifier system has already classified an associated set of voxels (whose values ​​undergo threshold comparisons so that only the minimum value is considered) as belonging to the defect category. In short, the more complex second classifier system is activated only when the simpler and faster first classifier system generates a signal indicating a suspected quality problem, and then a more detailed study is conducted using the more accurate second classifier system (especially with higher resolution) to determine whether a quality problem actually exists in the component.

[0031] According to possible implementations of this method, the received digital image and elements of the training dataset for the classifier system can be derived from the image recording method. This image recording method can be selected from the group consisting of: electron microscopy, fluorescence microscopy, optical microscopy, optical coherence tomography, interferometer, spectrometer, surgical microscope, and computed tomography. That is, in principle, all commonly used quality control schemes can be extended using the method proposed herein.

[0032] According to another embodiment of the method, the first reference image is formed by averaging multiple sample components of the component (especially “good” ones, i.e., samples without defects or with reduced defects), or derived from a CAD model of the component, or generated from one or more object models by simulating an image recording method (e.g., from a CAD system, such as one with voxel metadata “0”, “1”, or “gray”), or derived from one or more object models based on rules.

[0033] In this method, the distribution parameter value can also be obtained by determining the standard deviation relative to the average brightness value, which is higher than the average of multiple "good" sample components of the component. Therefore, there are multiple possibilities for obtaining the reference image; only a few are exemplarily listed here. Attached Figure Description

[0034] It should be noted that embodiments of the present invention may be described with reference to different implementation types. In particular, several embodiments relating to the method are described, while other embodiments may be described in the context of the corresponding device. In any case, unless otherwise stated, those skilled in the art can identify and combine possible combinations of features of the method and possible combinations of features of the corresponding system from the description above and below, even if they belong to different categories of claims.

[0035] The aspects and other aspects of the present invention described above are derived, in particular, from the described embodiments and from other additional specific designs described with reference to the accompanying drawings.

[0036] Preferred embodiments of the present invention are described exemplarily and with reference to the accompanying drawings:

[0037] Figure 1 A block diagram illustrating an embodiment of a computer-implemented method according to the present invention for measuring defects in a component using a quality inspection method.

[0038] Figure 2 A simplified diagram illustrating the working principle underlying the concepts presented in this paper is shown.

[0039] Figure 3 A block diagram illustrating an embodiment of a system for measuring defects in components using quality inspection methods.

[0040] Figure 4 This diagram shows a block diagram of a computer system, which may additionally have, wholly or partially, based on... Figure 3 The system. Detailed Implementation

[0041] In the context of this specification, conventions, terms and / or expressions should be understood as follows:

[0042] The term "workpiece" describes a component or sample that is undergoing quality inspection.

[0043] The term "defect" describes an anomaly on the surface or in the volume of a workpiece. Pores, dents, inhomogeneities, inclusions, etc., can all cause defects. Defects on the surface or in the volume of a component can typically be detected optically.

[0044] The term "digital image" describes the result of mapping or generating a dataset in the form of pixel or voxel data representing a real-world object (e.g., a component in a line-in-line or near-line inspection process). Typically, a "digital image" can be a signal of different dimensions (1-D, 2-D, 3-D, ..., nD), as long as local parts (spatial points on the object) from different "digital images" can be explicitly assigned to each other. In principle, this method can also be applied to the audio spectrum, provided a 1:1 assignment of spatial points to signals can be established.

[0045] The term "first reference image" (especially "first digital reference image") describes a dataset that depicts an image used as a reference for the recorded digital images. This first reference image generally corresponds to the ideal case of a fault-free specimen.

[0046] The term "second reference image" (especially a second digital reference image) describes a dataset derived from a combination of a recorded digital image and the first reference image. For this combination, different mathematical methods are considered for associating the corresponding pixel or voxel data. Examples include deviations from the mean of the reference image, standard deviations, etc.

[0047] The term "classification" describes the process of assigning a received image or a portion of that image (in particular, individual pixels of a recorded image) to a pixel category. One pixel category might describe, for example, that the corresponding pixel belongs to a defect-free image segment, while another pixel category might describe that pixel as a defect on / in the specimen. Furthermore, a probability value for this conclusion can be given. In the context of this paper, the instrument used to perform the classification is a classifier system.

[0048] The term "classifier system" (also known as a classifier or classification system in the context of machine learning) describes a machine learning-based system that, through training on training data, is able to assign features of input data (in particular, image data of recorded digital images) to a certain category (e.g., defective / defect-free) of images.

[0049] It should also be noted that classifiers typically classify data into a predetermined number of categories. This is usually achieved by determining the classification value of the input data for each category and then using a WTA (winner takes it all) filter to select the category with the highest classification value as the classified category. In the case of classifiers, the deviation from the 100% classification value is typically used as a quality parameter of the classification or the probability of correct classification.

[0050] Examples of classifier systems applicable to the objects of the inventive concepts presented herein are systems based on the following principles: Support Vector Machines, Random Forests, Boosting, Gradient Boosting, Gaussian Processing, Nearest Neighbors, Logistic Regression, Linear Regression, and Neural Networks. Other algorithms are, of course, also possible.

[0051] The term "machine learning" (or "maschinelles Lernen") is a fundamental term or basic function of artificial intelligence, where statistical methods are used, for example, to endow computer systems with the ability to "learn." For example, in this context, it refers to optimizing certain behavioral patterns within a specific task. The methods used enable trained machine learning systems to analyze data without requiring explicit procedural programming. For example, NN (Neural Network) or CNN (Convolutional Neural Network) are typical examples of machine learning systems used to form networks that act as nodes, or artificial neurons, and artificial connections (so-called links) between these artificial neurons, where parameters (such as the weight parameters of the connection) can be assigned to these artificial connections. During the training of the neural network, the weight parameter values ​​of these connections are automatically adapted to each other based on the input signals to produce the desired result. In monitored learning, the desired output data (annotations) supplements the image (generally, the (input) data) provided as input values ​​(training data) to generate the desired output value (the desired category). Very generally, learning a mapping from input data to output data is crucial.

[0052] The term "training a classification system" here means, for example, that a machine learning system is calibrated by partially and repeatedly analyzing multiple sets of example data (i.e., reference data) in, for example, a neural network, so that, after the training phase, unknown images or individual pixels or voxels can be assigned to one or more categories (with which the learning system has been trained). The example data is typically labeled (i.e., metadata is set) to generate the desired result based on the input image; for example, a defect category, a defect-free category.

[0053] In the context of this paper, the term "semantic segmentation" describes the segmentation of data portions describing a digital image into regions of the same category of the digital image using a classifier system.

[0054] The term "convolutional neural network" (CNN) (as an example of a classifier / classifier system) describes a category of artificial neural networks based on feed-forward techniques. These techniques are commonly used for image analysis, taking images or their pixels as input data. Here, the main component of a CNN is the convolutional layer (hence the name), which enables efficient analysis through parameter sharing. Typically, each pixel of the recorded image is assigned as an input value to an artificial neuron in the neural network.

[0055] It should also be mentioned that deep neural networks consist of multiple layers with different functions (e.g., input layer, output layer, and one or more layers in between, such as those used for convolution operations, applications of nonlinear functions, dimensionality reduction, normalization functions, etc.). These functions can be "executed in software," or special hardware components can take over the computation of the corresponding function values. Furthermore, combinations of hardware and software components are also known.

[0056] The term "input channel" (especially for classifier systems and more precisely, for DNNs or CNNs) describes the input of a classifier system, for example, in the form of an ANN (artificial neural network), where artificial neurons represent input nodes. Typically, each artificial neuron in the input layer of an ANN is designed as an input node. In the concept proposed in this paper, not only can pixel or voxel information of the recorded digital image be fed into this input layer, but also pixel or voxel information of a second reference image, which typically exists as matrix data.

[0057] Detailed description of the accompanying drawings is given below. It should be understood that all details and descriptions in the drawings are schematic. First, a block diagram of an embodiment of a computer-implemented method according to the present invention for measuring defects in a component using a quality inspection method is shown. Embodiments of other embodiments or corresponding systems are then described:

[0058] Figure 1 A block diagram illustrating an embodiment of a computer-implemented method 100 according to the present invention for measuring (and in particular identifying) defects in a component using quality inspection methods. Method 100 includes receiving 102 digital images of a component to be inspected. The component to be inspected typically involves a sample (i.e., a sample of a workpiece whose quality characteristics are to be tested).

[0059] Method 100 also includes receiving a first reference image of 104 components to be inspected. This reference image may, for example, involve the average value of a calculated "good" (i.e., fault-free or with acceptable faults) component. Alternatively, it may involve an idealized digital model of the component, for example, derived from a CAD (Computer-Aided Design) model. Furthermore, it should be noted that the pixel / voxel digital information may be derived from the received digital image of the component to be inspected, and the reference image may consist of data associated with multiple channels.

[0060] Furthermore, method 100 includes: obtaining 106 second reference images from a combination of the received digital image and the first reference image; and activating a machine learning-based trained classifier system, which has been trained with training data to build a model. Here, the model serves as the basis for semantically segmenting the voxels of the received image into defect categories, thereby grouping other adjacent voxels of the same category into related image segments.

[0061] Finally, method 100 includes classifying voxels of the received digital image by an activated classifier system 108, wherein the voxels of the received digital image and the voxels of the second reference image are used as input data for the classifier system.

[0062] It should not be overlooked that the information or data in digital images (whether received or acquired) is often presented in matrix form. Mathematical methods can be used to concisely process matrix-form data.

[0063] Figure 2 A simplified diagram 200 illustrates the working principle that forms the basis of the concept presented herein. Image brightness values ​​202, measured from a sample (i.e., a digital record of the workpiece to be inspected regarding its manufacturing quality), are used twice. On one hand, a differential signal matrix 204 (corresponding to a second reference image) is formed, which has a matrix containing voxel information 206 of the reference workpiece as input data for computation; on the other hand, voxel data 202 of the image brightness values ​​measured from the sample is used.

[0064] Then, classifier 208 obtains the differential signal matrix 204 and image brightness values ​​202 from the sample measurements as input data at the input layer. Then, classifier 206 segments the input image (i.e., the image brightness values ​​202 from the sample measurements) to obtain a segmentation result 210. Optionally, data describing the reference image can also be directly used as input data for this classifier system (see [link to documentation]). Figure 2 (The dashed lines in the middle connect).

[0065] In other words, the concept proposed here combines direct segmentation of pixels / voxels via differential signals with segmentation via simple classification, and utilizes this concept to make the training dataset for in-line or near-line inspection processes consist of images or volume data of the same component or workpiece.

[0066] Figure 3A block diagram illustrating an embodiment of a system 300 for measuring (and particularly identifying) component defects using quality inspection methods is shown. System 300 includes: a first receiving unit 302 adapted to receive a digital image of a component to be inspected; a second receiving unit 304 adapted to receive a first reference image of the component to be inspected; and a determining module 306 adapted to obtain a second reference image from a combination of the received digital image and the first reference image. Furthermore, system 300 includes a machine learning-based trained classifier system 308, trained with training data to build a model that serves as the basis for semantic segmentation of voxels in the received image during defect classification. Here, the activated classifier system 308 is adapted to classify the voxels of the received digital image. The voxels of the received digital image and the voxels of the second reference image serve as input data for the classifier system 308.

[0067] Figure 4 A block diagram of a computer system is shown, which may include at least a portion of a system for measuring component defects using quality inspection methods. Implementations of the concepts presented herein can, in principle, be used with virtually any type of computer, regardless of the platform in which program code is stored and / or executed. Figure 4 A computer system 400 is illustrated as an example, which is adapted to implement program code corresponding to the methods presented herein. Computer systems already existing in microscope systems (with appropriate extensions where suitable) can also be used as computer systems for implementing the concepts presented herein.

[0068] Computer system 400 has a variety of general-purpose functions. Here, the computer system can be a tablet computer, a laptop / notebook computer, other portable or mobile electronic devices, a microprocessor system, a microprocessor-based system, a smartphone, or a computer system with special functions and special configurations. Computer system 400 can be configured to execute instructions (e.g., program modules) that can be executed to implement the functions of the concepts proposed herein. For this purpose, program modules can have routines, programs, objects, components, logic, data structures, etc., to implement specific tasks or specific abstract data types.

[0069] The components of a computer system may include one or more processors or processing units 402, a storage system 404, and a bus system 406 that connects different system components, including the storage system 404, to the processor 402. The computer system 400 typically has multiple volatile or non-volatile storage media accessible to the computer system 400. In the storage system 404, data and / or instructions (commands) from the storage media may be stored in volatile form (e.g., in RAM (Random Access Memory) 408) for execution by the processor 402. These data and instructions implement one or more functions or steps of the concepts presented herein. Other components of the memory system 404 may be permanent memory (ROM) 410 and long-term memory 412, in which program modules and data (reference numeral 516) and workflows may be stored.

[0070] The computer system has several dedicated devices for communication (keyboard 418, mouse / pointing device (not shown), display screen 420, etc.). These dedicated devices can also be integrated into a touch-sensitive display. A separately configured I / O controller 414 is responsible for smooth data exchange with external devices. A network adapter 522 can be used for communication via a local area network (LAN, WAN, e.g., via the Internet). This network adapter can be accessed by other components of the computer system 400 via bus system 506. It should be understood that (although not shown) other devices can also be connected to the computer system 400.

[0071] In addition, at least a portion of the system 300 used for measuring defects in components (see...) Figure 3 It can be connected to bus system 306.

[0072] For better understanding, various embodiments of the invention have been described, but are not intended to directly limit the concept of the invention to these embodiments. Other modifications and variations will be readily apparent to those skilled in the art. The terminology used herein has been chosen for the purpose of best describing the basic principles of the exemplary embodiments and for ease of understanding by those skilled in the art.

[0073] The principles presented herein can be embodied as a system, method, combination thereof, and / or as a computer program product. Here, the computer program product may include one or more computer-readable storage media having computer-readable program instructions to enable a processor or control system to implement various aspects of the invention.

[0074] Electronic, magnetic, optical, electromagnetic, infrared media, or semiconductor systems used as transmission media can be used as the medium; for example, SSDs (Solid State Devices / Drives as Solid State Storage), RAM (Random Access Memory) and / or ROM (Read-Only Memory), EEPROM (Electrically Erasable ROM), or any combination thereof. Diffused electromagnetic waves, electromagnetic waves in waveguides or other transmission media (e.g., optical pulses in optical cables), or electrical signals transmitted by wires can also be considered as transmission media.

[0075] Computer-readable storage media can be physical devices that pre-store or store instructions for use by instruction execution devices. The computer-readable program instructions described herein can also be downloaded to the corresponding computer system as an application from a service provider, for example, via a cable-based connection or mobile wireless network.

[0076] Computer-readable program instructions for performing the operations of the invention described herein can be machine-dependent or machine-independent instructions, microcode, firmware, state-defined data, or any source code or object code, such as in C++, Java, or similar languages, or in conventional procedural programming languages, such as programming language "C" or similar programming languages. The computer-readable program instructions can be implemented entirely by a computer system. In several embodiments, they can also be electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), which use state information of the computer-readable program instructions to execute the computer-readable program instructions in order to configure or personalize the electronic circuitry in a manner corresponding to aspects of the invention.

[0077] Furthermore, the invention presented herein is illustrated with reference to flowchart illustrations and / or block diagrams corresponding to methods, apparatus (systems), and computer program products in accordance with embodiments of the present invention. It should be noted that each block of the flowcharts and / or block diagrams may, in practice, be designed as computer-readable program instructions.

[0078] Computer-readable program instructions may be provided to a general-purpose computer, special-purpose computer, or other programmable data processing system to create a machine, such that the instructions, which are executable by a processor or computer or other programmable data processing device, create means for implementing the functions or processes shown in the flowchart and / or block diagram. These computer-readable program instructions may also be correspondingly stored in a computer-readable storage medium.

[0079] In this sense, each box in the flowchart or block diagram may represent a module, section, or part of an instruction, which represents a number of executable instructions for implementing a specific logical function. In several embodiments, the functions shown in the various boxes may be executed in different orders (and in parallel where appropriate).

[0080] The structures, materials, processes, and equivalents shown for all devices and / or steps having the relevant functions in the following claims are intended to be applied to all structures, materials, or processes expressed in the claims.

[0081] List of reference numerals

[0082] 100 Methods for measuring defects in components

[0083] 102 100 Method Steps

[0084] 104 100 Method and Steps

[0085] 106 100 Method Steps

[0086] 108 100 Method Steps

[0087] 200 The process of proposing the basic concepts in this paper

[0088] Image data of sample 202

[0089] 204 Differential Signal

[0090] 206 Reference Data

[0091] 208 Classifier / Classification System

[0092] 210 segmentation results

[0093] 300 Systems for measuring defects in components

[0094] 302 First Receiving Unit

[0095] 304 Second Receiving Unit

[0096] 306 Determine Module

[0097] 308 Classifier / Classification System

[0098] 400 Computer System

[0099] 402 processor

[0100] 404 storage system

[0101] 406 bus system

[0102] 408 RAM

[0103] 410 ROM

[0104] 412 Long-term memory

[0105] 414 I / O Controller

[0106] 416 Program Module, Potential Data

[0107] 418 Keyboard

[0108] 420 display screen

[0109] 422 Network Adapter

Claims

1. A computer-implemented method for measuring defects in components using quality inspection methods during production and quality inspection processes, the method comprising: - Receive digital images of the components to be inspected. - Receive the first reference image of the component to be inspected. -A second reference image is obtained by combining the received digital image and the first reference image. - Activate a machine learning-based trained classifier system, which has been trained on training data to build a model that serves as the basis for semantically segmenting voxels of a received image into defect categories, thereby classifying other neighboring voxels of the same category into related image segments, and - The voxels of the received digital image are classified by the activated classifier system, wherein the voxels of the received digital image and the voxels of the second reference image are used as input data for the classifier system.

2. The method of claim 1, wherein the second reference image is obtained by voxel-by-voxel difference of the brightness values ​​of the received digital image and the first reference image.

3. The method of claim 1, wherein the first reference image and the second reference image are identical.

4. The method according to any one of the preceding claims, wherein the received reference image comprises voxel-by-voxel distribution parameter values ​​of brightness values, and the second reference image is normalized with respect to these distribution parameter values.

5. The method according to any one of claims 1 to 3, wherein, for the classification of voxels, the spatial context or temporal context of the voxels is included during classification.

6. The method according to any one of claims 1 to 3, wherein the first reference image is adapted in the time curve.

7. The method according to any one of claims 1 to 3, wherein the classifier system is selected from the group consisting of: support vector machine, random forest, boosting, gradient boosting, Gaussian processing, nearest neighbor, logistic regression, linear regression and neural network.

8. The method of claim 7, wherein the neural network is a deep neural network.

9. The method of claim 8, wherein the input layer of the deep neural network is extended relative to the amount of input channels required to process the digital image, such that voxel data of the first reference image and / or the second reference image are directed at additional input channels of the classifier system.

10. The method according to any one of claims 1 to 3, wherein the image resolution of the received digital image is different from the image resolution of the first reference image and / or the second reference image.

11. The method according to any one of claims 1 to 3, wherein the classifier system comprises a first classifier system and a second classifier system. The first classifier system operates with a first number of parameter values. The second classifier system operates with a second number of parameter values, wherein the second number is greater than the first number.

12. The method of claim 11, wherein the second classifier system is activated only when the first classifier system has already classified an associated set of voxels as belonging to a defect category.

13. The method according to any one of claims 1 to 3, wherein the received digital image and elements of the training dataset for the classifier system are derived by the image recording method, wherein the image recording method is selected from the group consisting of: electron microscope, fluorescence microscope, optical microscope, optical coherence tomography, interferometer, spectrometer, surgical microscope and computed tomography.

14. The method according to any one of claims 1 to 3, wherein the first reference image is formed by averaging a plurality of sample components of the component, or derived from a CAD model of the component, or generated from one or more object models by simulating an image recording method, or derived from one or more object models based on rules.

15. A system for measuring defects in components using quality inspection methods during production and quality inspection processes, the system comprising: - A first receiving unit, adapted to receive a digital image of the component to be inspected. - A second receiving unit, adapted to receive a first reference image of the component to be inspected. - A determining module adapted to obtain a second reference image from a combination of the received digital image and the first reference image. - A machine learning-based trained classifier system has been trained on training data to build a model that serves as the basis for semantically segmenting voxels of received images into defect categories, thereby grouping other neighboring voxels of the same category into related image segments, and The classifier system is adapted to classify voxels of a received digital image by the activated classifier system, wherein the voxels of the received digital image and the voxels of the second reference image are used as input data for the classifier system.

16. A computer program product for measuring defects in a component using a quality inspection method, wherein the computer program product has a computer-readable storage medium having program instructions stored thereon, wherein the program instructions are executable by one or more computers or control units and cause the one or more computers or control units to perform the method corresponding to any one of claims 1 to 14.

Images