Method and apparatus for detecting anomalies in two-dimensional digital images of products
The method addresses the complexity of anomaly detection in digital images by using statistical distributions to determine threshold values and detection rates, enhancing efficiency and reliability in anomaly detection.
Patent Information
- Application Number
- EP2024169881
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-04
- Filing Date
- 2024-04-12
- Publication Date
- 2024-11-06
AI Technical Summary
Current methods for detecting anomalies in digital images of products require a complex learning process to set threshold values, necessitating a large number of good products, which is time-consuming and difficult to predict the required number for reliable false ejection rate maintenance.
A method that determines threshold values for detecting anomalies by assuming a statistical distribution of extreme values using a probability density function, allowing for the use of a small number of digital images and enabling the determination of detection rates for previously known anomalies, thereby simplifying the process and improving efficiency.
This approach allows for the efficient determination of threshold values and detection rates, reducing the need for extensive learning processes and improving the reliability of anomaly detection in digital images of products.
Smart Images

Figure IMGAF001_ABST
Abstract
Description
[0001] The invention relates to a method for detecting anomalies in digital images of products having the features of the preamble of patent claim 1. Furthermore, the invention relates to a device and a computer program product for carrying out the method.
[0002] During product manufacturing, there is often a desire or need to subsequently inspect the manufactured products to determine whether they contain any anomalies, particularly defects, foreign bodies, or the like. For example, in the food industry during the production of yogurt or cheese, the task may arise to examine the finished product to determine whether there are any unwanted foreign bodies or other undesirable material areas within the product. To solve this problem, inspection devices are used in practice that irradiate the product with electromagnetic radiation, particularly radiation in the X-ray spectrum. This creates a digital image of the product that contains information not only about the external geometric dimensions or the surface of the product, but also about the interior of the product.Depending on their attenuation, anomalies to be detected can, when irradiated, result in image areas that have a higher or lower "gray value," i.e., pixel value, than the good areas, compared to the image areas that do not exhibit anomalies (hereinafter referred to as "good areas"). In this description, the term "gray value" is used to refer to the information generated by the detector depending on the radiation power incident on the individual pixels or the corresponding radiation energy detected during the relevant exposure time, regardless of the color or manner in which the pixel values can be represented in the digital image in question.
[0003] For such product inspections, devices or methods are also known in which the product to be inspected is not completely irradiated (i.e., the radiation source and detector are located on opposite sides of the product), but in which the radiation penetrates deeply enough into the product to be inspected and is "reflected" in the product. This "reflection" is physically caused by scattering the radiation penetrating a volume area or by generating fluorescent radiation in the volume area. In such inspection devices, the radiation source and detector can also be located on the same side of the product to be inspected.
[0004] The product to be examined can be in the form of a piece of any kind or as bulk material, which is conveyed through the inspection device by means of a conveyor device.
[0005] Line detectors, which have one or more detector rows, each with a predetermined number of pixels, are often used as radiation detectors in production lines where the products to be examined are moved along a conveyor path. The products are moved through such a scanning device, usually at a constant speed, with a digital image being composed of a plurality of detected rows. However, it is of course also possible to use an area scanner instead of a line scanner. In this case, the digital image of the product to be examined can be captured using a single detection process (i.e., a single "exposure process").
[0006] Digital images generated in this way are typically automatically examined to determine whether any anomalies are present within the product in question. The image generated directly from the scanning process can be processed or enhanced prior to such an examination. For example, the original image can be digitally filtered, with the filter used for this purpose potentially improving contrast.
[0007] It is also possible to take measures to improve contrast during image generation, particularly with regard to anomaly detection. For example, a dual-energy method could be used, with the two spectra selected so that superimposing the relevant partial images results in an improved contrast.
[0008] It is also possible to use spectrally resolving detectors that generate multiple images, each composed of pixels whose gray value corresponds to the radiation energy in a specific spectral section of the detected radiation. Anomalies can then be detected using an image generated from all or selected partial images of such a spectrally resolving detector, for example, by weighted addition of the relevant pixel values. However, each partial image can also be examined separately to determine whether an anomaly is present.
[0009] For the automated examination of the digital image of the product, methods are known in which a threshold is set, whereby an anomaly exists if the gray value of at least one pixel is greater than the threshold. For this purpose, a predetermined number of good products (i.e., products that do not exhibit an anomaly) of the same product type are usually scanned in a learning process in order to obtain information about the maximum gray values typically found in such good products. Depending on this, the threshold is then set such that a predetermined value for the false rejection rate (e.g., one percent or one per mille) is maintained. The false rejection rate is the probability with which a good product is identified as a "bad product." This (theoretical) false rejection rate can be verified by determining the empirical frequency with which a good product is identified as a bad product.For this purpose, a sufficient number of digital images of good products can be generated and checked using the specified threshold value, whereby the empirical misrejection rate is determined as the quotient of the good products identified as bad products by the total number of good products.
[0010] Determining such a threshold therefore requires a complex learning process, in which a relatively large number of digital images of good products must be generated. However, this is disadvantageous because such a large number of good products must first be generated on the relevant production line, resulting in a corresponding loss of production time. Furthermore, it is difficult to predict how high the number of good products required for the learning process will be in order to reliably set the threshold so that the desired false rejection rate is maintained.
[0011] The unpublished European patent application with application number 23166616 describes a method for detecting anomalies in digital images, according to which an automated process for setting a threshold value can be carried out even using a small number of digital images. According to this method, a threshold value for detecting anomalies in digital images of products is determined with good reliability in a learning process. An assumption is made regarding the statistical distribution of the largest and smallest values (hereinafter referred to as extreme values) of a specific variable used to detect anomalies in the digital images under investigation. This distribution of extreme values is sufficiently well described by a predefined probability density function that is to be parameterized (the term "parameterizing" in this description refers to the determination orDetermining values for parameters of a probability density function that have not yet been determined. In other words, the assumption is made that the elements of the sample (i.e., the set of respective maxima or minima) satisfy the respective specified probability density function or that their probability of occurrence can be well described by this probability density function. Using the elements of the sample, estimates are determined for all parameters to be determined (i.e., for all parameters of the probability density function that may not already have been specified).According to this method, the generalized extreme value distribution, for example a given variant thereof, i.e. a Weibull distribution, a Fréchet distribution or a Gumbel distribution, is used as the probability density function to be parameterized in order to describe the distribution of the maximum or minimum values of the specific quantity.
[0012] This method is not limited to the case explained at the beginning, in which the pixel values, in particular each individual pixel value, in conjunction with a threshold value for the pixel values are used to decide whether the digital image to be examined contains an anomaly.
[0013] Rather, the invention generalizes the principle underlying this procedure. The image to be examined is divided into one or more regions, each region being assigned at least the same one property or the same several properties. Each region comprises either exactly one or more pixels, with multiple pixels only being assigned to the same region if they are adjacent (i.e., each pixel in the region has at least one immediately adjacent pixel from the same region). Any pixel that is adjacent to an edge of the pixel under consideration (i.e., the pixels above, below, left, and right of the pixel under consideration) or (additionally) any pixel that is adjacent to a corner of the pixel under consideration (i.e., the pixels in the extension of the diagonal of the pixel under consideration) can be regarded as adjacent to a pixel under consideration.For each property of the previously defined regions, a value is determined that describes the respective property. If multiple properties are assigned to the regions, separate values can be determined for each property, or the values of two or more properties can be combined into a single value, for example, using an arithmetic operation (e.g., multiplication, division, or weighted addition).
[0014] In this generalization, anomalies are detected by determining a threshold for each property or for each combination of several properties.
[0015] In an extreme case, the entire image can be viewed or defined as a single region. If, as a specific parameter for detecting anomalies, the pixel value of each pixel is examined to determine whether the value exceeds or falls below a predefined threshold for the pixel value, this leads to the same result as if each pixel were viewed or defined as a region with exactly one pixel. In this case, the generalization explained above, using predefined regions and assigned arbitrary properties, leads back to the previously common procedure, whereby the individual pixels are examined with regard to their pixel values to determine whether they exceed or fall below a threshold for the pixel values.
[0016] According to the invention, the threshold value for a property or the threshold value for a combination of properties is determined in a learning process. The images required to determine the threshold value can be generated in advance in a sufficient, predetermined number or as needed, ie, one or more new digital images are successively generated until a sufficient number is reached.
[0017] According to this known method, the learning process or the automated process for determining the threshold value for at least one property can be carried out using digital images of good products that exhibit no anomalies. As already mentioned above, the images, which may be the direct result of a scanning process, can first be processed or prepared for the detection process. In this process, a suitable section of a determined overall image can also be generated, which includes the product as a whole or specified sub-areas. The section thus generated or the entire captured digital image can also be subjected to digital filtering, for example, to enhance contrast with the aim of highlighting anomalies even more clearly.If a continuously produced product or a bulk product is to be examined for anomalies, digital images of sections of such a product can be created and processed and examined in the same way as images of individual products (or sections of such products).
[0018] It should be noted at this point that it is not mandatory to carry out the automated process exclusively with good products. Rather, the automated process can also be carried out using images of products as they are produced in a production line, without ensuring that none of the products are defective. Instead of good products, products referred to as "good process products" can also be used, whereby a number of good process products consists predominantly of good products and only a small proportion of defective products. In practice, it can be assumed that the proportion of defective products in a number of good process products is low, in particular less than 25%, preferably less than 10%, and most preferably less than 5%.
[0019] Within the framework of the automated process, either a fixed number of digital images or a number of images of good products or good process products to be determined during the course of the automated process are generated or used.
[0020] For each of the digital images, the one or more regions are defined, and for each region, the value of at least one property or the combined value for the multiple properties is determined. The maximum value of these values is determined as the maximum sample value of a maximum value sample, and / or the minimum value of these values is determined as the minimum sample value of a minimum value sample.
[0021] This allows for the exclusion of extreme values that are implausible or clearly indicate an error. For example, when determining extreme values, pixel values that exhibit an absolute maximum or absolute minimum value of the grayscale used can be excluded, since a corresponding minimum value, such as a value of 0, may indicate a defective detector pixel, and a maximum value may indicate an overdriven detector pixel.
[0022] The extreme values determined in this way represent the sample mentioned above and can be stored, for example, in a list (if necessary separately for the minimum and maximum values).
[0023] Subsequently, estimates are determined for all free, non-specified parameters of the probability density function specified to describe the maximum value sample using the maximum sample values, and / or estimates are determined for all free, non-specified parameters of a probability density function specified to describe the minimum value sample using the minimum sample values. A statistical estimation method is used for this purpose.
[0024] Since a meaningful result of a statistical estimation procedure, especially using an estimator (also called a statistical estimator), can only be expected if the sample includes a minimum number of values, such a minimum number is usually specified, which in turn requires a corresponding minimum number of digital images.
[0025] A key aspect of this known method is that a rate is specified at which a maximum anomaly is incorrectly detected in images to be examined (when performing the method using the threshold value), or a rate at which a maximum anomaly is correctly not detected in images to be examined (when performing the method using the threshold value), and / or a rate is specified at which a minimum anomaly is incorrectly detected in images to be examined (when performing the method using the threshold value), or a rate at which a minimum anomaly is correctly not detected in images to be examined (when performing the method using the threshold value). The maximum or minimum threshold to be determined can therefore be determined such that the specified rate is maintained.
[0026] The term "rate" refers to the quotient of the number of good products that meet the respective rate criterion when applying the test and a specified total number of good products or process products. The rate at which a maximum or minimum anomaly is falsely detected in images under examination thus corresponds to the term "false rejection rate" often used in practice. The rate at which a maximum or minimum anomaly is falsely detected in images under examination and the rate at which no maximum or minimum anomaly is correctly detected in an image under examination each add up to 1. During plant operation, these rates can also be determined as a sliding value, for example, based on the number N of most recently inspected products.
[0027] If the specified probability density function is parameterized using the estimation method, the maximum threshold can be determined using the previously parameterized probability density function or a distribution function corresponding to it in such a way that the probability of a maximum value occurring that is greater than or equal to the maximum threshold corresponds to the respective specified rate at which a maximum anomaly is incorrectly detected in images to be examined, or that the probability of a maximum value occurring that is less than or equal to the maximum threshold corresponds to the specified rate at which no maximum anomaly is correctly detected in an image to be examined. In other words, the maximum threshold can be determined in such a way that the area under the probability density function lies above the maximum threshold orthe area under the probability density function below the maximum threshold corresponds to the given rate.
[0028] Analogously, the minimum threshold can be determined using the previously parameterized probability density function or a distribution function corresponding to it such that the probability of a minimum value occurring that is less than or equal to the minimum threshold corresponds to a predetermined rate at which a minimum anomaly is incorrectly detected in images to be examined, or such that the probability of a minimum value occurring that is greater than or equal to the minimum threshold corresponds to a predetermined rate at which no minimum anomaly is correctly detected in a good product. In other words, the minimum threshold can be determined such that the area under the relevant probability density function lies below the minimum threshold orthe area under the probability density function above the minimum threshold corresponds to the given rate.
[0029] To determine the area, the integral over the probability density function can of course be used, whereby the integral is to be formed from the relevant threshold value up to an upper limit of the definition interval of the probability density function (e.g. infinity or a predetermined upper limit above which the value of the integral, ie the area, changes only less than a predetermined error bound) or from a lower limit of the definition interval (e.g. minus infinity or a predetermined lower limit below which the value of the integral, ie the area, changes only less than a predetermined error bound) up to the threshold value.
[0030] Of course, instead of calculating the integral, as explained above, it is also possible to use the distribution function of the relevant probability density function, since the distribution function represents the value of the integral over the probability density function from minus infinity or the lower limit of the definition interval of the probability density function up to the relevant threshold value. The relevant area under the probability density function, i.e. the value of the integral or the value of the distribution function, thus corresponds to the (theoretical) probability with which a maximum or minimum value of a property of the regions of an image of a good product is less than or equal to the relevant threshold value. If the probability is to be calculated with which a maximum or minimum value of a property of the regions of an image of a good product is less than or equal to the relevant threshold value,minimum value is greater than or equal to the threshold, then the probability calculated in this way only needs to be subtracted from 1.
[0031] Since the value of the area, integral, or distribution function is specified for these calculations, the corresponding inverse function must be used. These calculations can be performed either analytically or using a numerical method.
[0032] According to this known method, the regions can be defined using a base threshold, whereby neighboring pixels whose pixel value is greater than (or equal to) the base threshold form a first group of regions, and neighboring pixels whose pixel value is less than the base threshold form a second group of regions. The first and second groups of regions can also be combined into a single group. Another possibility for defining the regions is to use a predefined (geometric) mask. For example, a matrix-like mask can be used which superimposes a square grid (i.e., a checkerboard-like grid) over the digital image, whereby all pixels within a square form a region. Of course, the mask can also cause other subdivisions, whereby the entire image does not have to be divided into regions.
[0033] The regions are assigned a property that can be described by a value. These can be geometric properties, such as the area, perimeter, or diameter of the regions (if they are at least approximately circular), or pixel value properties, which are described by a value resulting from the pixel values of the region in question, such as the maximum or minimum value of a region, the average value, or the variance of the pixel values.
[0034] Multiple properties can also be combined and described by a combined value. For example, the mean and standard deviation can be added together, with this information representing a kind of confidence interval for the pixel values. The difference between the maxima and minima can also be used as a combined value, which describes the brightness difference of the region. Since this measure is sensitive to outliers, quantiles can be used instead, for example the quantiles at 10% and 90% as an alternative to maxima and minima. Furthermore, the quotient of circumference and area can be used to determine how circular the region is or how far it deviates from circularity. Geometric and pixel value properties can also be combined.
[0035] This known method thus offers the possibility of specifying a rate, in particular a false ejection rate (or the relevant complementary rate, i.e. the rate at which images are correctly recognized as error-free) and, depending on this, of determining a threshold value for the detection of a minimum or maximum anomaly.
[0036] In practice, however, it is often desirable or necessary to know not only the false rejection rate but also the actual detection rate, or at least a predicted value for it. While the actual false rejection rate can be determined relatively easily, albeit laboriously, by manually inspecting individual products that have actually been rejected, this is hardly possible for the detection rate. For example, it is known to examine images of actual defective products to determine whether the anomalies contained in them can be detected with sufficient certainty or probability, i.e., a sufficiently high detection rate, using a previously known threshold.
[0037] Based on this prior art, the invention is based on the object of creating a method for detecting anomalies in digital images of products, which enables the determination of at least one estimated value for the detection rate for anomaly detection in a simple manner and with minimal effort. Furthermore, the invention is based on the object of creating a device and a computer program product for implementing the method.
[0038] The invention solves this problem with the features of patent claim 1 or 14 and 15. Further embodiments of the invention emerge from the dependent claims.
[0039] The invention is based on the finding that by creating digital images of similar products (or of a similar product if it is a continuously produced product, for example a bulk material), which contain at least one previously known anomaly that is generated by a previously known (physical or virtual) interfering object, a statement can be made about a detection rate to be expected during active operation of an inspection device or an estimated value can be determined for this.
[0040] Similar products are understood to mean products that are manufactured in series production, for example, yogurt filled in cups or biscuits packaged in bags or rigid boxes, or the like. When inspecting such products, it must be determined whether they contain unwanted interfering substances, such as contaminants, metal fragments, small stones, bone fragments, or the like. The inspection device typically creates digital images of the products, which are then examined using suitable automated analysis for the presence of anomalies caused by unwanted interfering substances.
[0041] For this purpose, the digital image is viewed as a single region or divided into multiple regions, each consisting of one or more neighboring pixels. For each such region, a value is determined for at least one property of the region or a combined value for several properties of the region. For example, an average brightness value (gray value) can be determined for the region. The presence of an anomaly is detected if the respective value of the property is greater than a specified maximum threshold or a specified minimum threshold.
[0042] As already mentioned above, the maximum threshold value or the minimum threshold value can be specified directly or determined from a specified value for the false ejection rate (or the rate complementary thereto) according to the method according to the European patent application with the application number 23166616.
[0043] According to the invention, the following steps are carried out in a testing process: ▪ Generating a plurality of digital defect images of real or fictitious defect products, each having at least one previously known anomaly; ▪ For each defect image, defining the region or regions and determining the value of the at least one property or the combined value for several properties of each region and determining the maximum value of these values as the maximum sample value of a maximum value sample or determining the minimum value of these values as the minimum sample value of a minimum value sample;▪ Determining a detection rate for the at least one previously known anomaly o by determining estimated values for all free, non-specified parameters of a probability density function specified to describe the maximum value sample or the minimum value sample (parameterization) using the maximum sample values or the minimum sample values and using a statistical estimation method, and ∘ by integrating the parameterized probability density function using the specified maximum threshold value or minimum threshold value as the integration limit, or ▪ Determining a detection rate as the ratio of the number of values of the maximum value sample or the minimum value sample that are greater than or equal to the specified maximum threshold value or less than or equal to the specified minimum threshold value, and the total number of values of the maximum value sample or the minimum value sample;and ▪ Assigning the detection rate to the at least one previously known anomaly. ;
[0044] By assigning the detection rate thus determined to the at least one previously known anomaly, the operator of an inspection device, for example an X-ray inspection device in which this method is implemented, has the advantage that the effect of the preceding threshold value or the false ejection rate underlying this threshold value (as a predetermined value) on the detection rate is immediately recognizable.
[0045] In this way, the threshold or false rejection rate can be selected such that the detection rate for previously known anomalies in images of good products, which are actually to be examined with the inspection device in practical operation for the presence of previously unknown anomalies, satisfies the economic requirements and the safety requirements as far as possible. This is because these two parameters often have to be weighed against each other. Of course, a false rejection rate of 0% and a detection rate of 100% are desirable, but this is hardly or not attainable in practice. From an economic perspective, the lowest possible false rejection rate is desirable. This can be achieved for the detection of maximum anomalies by setting the highest possible threshold, but this in turn reduces the detection rate or the probability of correctly detecting a maximum anomaly.If the interfering objects to be detected are hazardous items, such as metal or bone fragments in food, the detection rate must be correspondingly high, for example, 99% or higher, to minimize claims for recourse from end consumers who are injured when consuming such contaminated food. On the other hand, an excessively high false rejection rate can be extremely economically disadvantageous for the product manufacturer if the effort required for manual follow-up inspection of erroneously rejected inspected products is time-consuming and therefore costly.
[0046] The invention makes it possible to determine the false ejection rate on the one hand and the detection rate on the other hand by specifying a single parameter, namely either the respective maximum threshold value or minimum threshold value or the false ejection rate (or the rate complementary thereto) so simply and quickly that this meets the requirements of economic efficiency and / or safety.
[0047] In practice, this method will of course provide more accurate results the better the previously known anomalies match the actually occurring anomalies or the previously known (physical or virtual) interfering objects match the actually occurring interfering objects.
[0048] According to one embodiment of the invention, the maximum threshold value or the minimum threshold value is determined in a learning process, wherein the following steps are carried out: Generating or using a number of digital images of good products that do not contain an anomaly, or of good process products that predominantly do not contain an anomaly, wherein the number of images is predetermined or determined during the learning process; for each of the digital images, defining the region or regions and determining the value of at least one property or the combined value for several properties of each region and determining the maximum value of these values as the maximum sample value of a maximum value sample and / or determining the minimum value of these values as the minimum sample value of a minimum value sample; using a statistical estimation method, determining estimated values for all free,non-specified parameters of a probability density function specified for describing the maximum value sample using the maximum sample values and / or determining estimated values for all free, non-specified parameters of a probability density function specified for describing the minimum value sample using the minimum sample values; specifying a first rate at which a maximum anomaly is falsely detected in images to be examined, or a second rate at which no maximum anomaly is correctly detected in images to be examined, and / or specifying a third rate at which a minimum anomaly is falsely detected in images to be examined, or a fourth rate at which no minimum anomaly is correctly detected in images to be examined,Determining the maximum threshold using the parameterized probability density function or a distribution function corresponding thereto such that the probability of the occurrence of a maximum value that is greater than or equal to the maximum threshold corresponds to the predetermined first rate or that the probability of the occurrence of a maximum value that is less than or equal to the maximum threshold corresponds to the predetermined second rate, and / or determining the minimum threshold using the parameterized probability density function or a distribution function corresponding thereto such that the probability of the occurrence of a minimum value that is less than or equal to the minimum threshold corresponds to the predetermined third rate or that the probability of the occurrence of a minimum value that is greater than or equal to the minimum threshold corresponds to the predetermined fourth rate.
[0049] This is essentially the method described in European patent application number 23166616, where the false ejection rate or the complementary rate is specified. The advantage of this method over directly specifying the threshold value is, of course, that one of the relevant variables in practice, namely the false ejection rate, is specified, rather than an initially arbitrary threshold value whose effect on both the false ejection rate and the detection rate cannot be easily predicted.
[0050] According to one embodiment of the invention, a geometric property determined from the location information of the pixels in the region, in particular the area, perimeter, or diameter, is used as a property of a region. Furthermore, a pixel value property determined from the values of the pixels in the region, in particular the maximum or minimum value of all pixels in the region, the mean, the variance, or the standard deviation, can be used as a property of a region.
[0051] In practice, a size will be used as a property of a region which allows the (unknown) interfering objects to be detected during the practical operation of an inspection device or during the practical application of the method according to the invention or the (unknown) anomalies caused thereby to be detected most reliably.
[0052] If the maximum or minimum threshold is not directly specified but is determined from a given false rejection rate, the generalized extreme value distribution can be selected or used as the required probability distribution, particularly its special cases, the Gumbel distribution, the Weibull distribution, or the Fréchet distribution. The generalized extreme value distribution is ideally suited to describing probability distributions of practically occurring extreme values.
[0053] According to one embodiment of the invention, the digital defect images required for the inspection process are generated such that the at least one predetermined anomaly is located at a predetermined position within the defect image. This information, and optionally also the geometry of the interfering object causing at least one predetermined anomaly, can be used to determine the maximum sample value or minimum sample value representing this predetermined anomaly.
[0054] For example, the interfering object generating the at least one predetermined anomaly can be positioned on or in the good product to be inspected (which, after the attachment or introduction of the interfering object, represents a defective product) such that it is located – in an image plane perpendicular to the direction of radiation – in the upper right quarter of the relevant digital image. The detection of the respective maximum or minimum sample value can then be limited, for example, to this upper right quarter of the digital image. This reduces the probability that a sample value is determined in the digital image in a region outside the predetermined anomaly that is larger or smaller than the maximum or minimum sample value actually caused by the anomaly or the relevant interfering object.
[0055] The region or image section actually used for image analysis can also be determined using a base threshold. This can be used, for example, to identify all regions of an image whose pixel values are greater than or equal to the specified base threshold. It can then be checked whether there is a sufficiently large overlap between a region thus determined and the previously known geometric external dimensions of the interfering object in question (or its geometric external dimensions in a projection onto the image plane). For example, it can be checked whether the interfering object lies with its entire projection area within a region thus determined, or with a proportion greater than a predetermined overlap threshold.
[0056] As already mentioned above, according to one embodiment of the invention, a good product that contains no anomalies can be used to generate a bad image, with at least one interfering object being provided on or in the good product, so that the good product becomes a bad product with at least one previously known interfering object. Furthermore, the at least one interfering object can also be provided on or in a carrier that is attached to the product or arranged within it.
[0057] According to another embodiment of the invention, digital image data of a good product can be used to generate a defective image. The digital image data of the good product are digitally transformed using known material and geometric properties of at least one predetermined interfering object to include the at least one interfering object. In other words, a previously known anomaly is "included" or "blended" into the digital image of a good product. This variant is particularly suitable when an interfering object consists of a single material and has a simple geometric structure, for example, a sphere, a cuboid, or a cube.For an inspection device that radiates through the products to be inspected, this "calculation" can be done by using a previously known material-dependent specific attenuation for the radiation used for inspection, determining the additional attenuation and taking this into account in addition to the attenuation represented by the image data of the acceptable image. Using a logarithmic scale for the attenuation, this can be done by simple addition.
[0058] The carrier for the at least one interfering object can be plate-shaped or card-shaped. Several interfering objects with identical material properties, in particular made of a single material, with similar geometry, but of different sizes, can also be arranged in or on the carrier. In this way, defective images with several previously known anomalies can be generated. The evaluation can then be carried out in such a way that only one specific region or section of the defective image (as described above) is used to determine a corresponding maximum or minimum threshold value.
[0059] According to one embodiment of the invention, the detection rate can be output as a numerical value or a related quantity or label on a display device.
[0060] The detection rate can also be displayed associated with a graphical representation of at least one interfering object and / or associated with the corresponding bad image. For example, when using previously known anomalies caused by spheres of different sizes made of the same material (regardless of whether these anomalies were generated with physical interfering objects or by calculation), the corresponding detection rate value can be displayed for each representation of a sphere with a corresponding material designation. Instead of or in addition to the representation of the interfering object, the good image or an image section containing the anomaly can also be displayed.
[0061] According to a further embodiment of the invention, the maximum threshold or minimum threshold or the false ejection rate (or the complementary rate) for detecting a maximum or minimum anomaly can be changed using an input device, for example, a keyboard, or a setting device, for example, a (physical or digitally implemented) slider or rotary knob. Furthermore, a current value for the detection rate can be displayed on the display device for each current value for the maximum threshold or minimum threshold or the respective rate.
[0062] Through these measures, the operator can develop a feeling for how and with what reliability the detection of previously known interfering objects or previously known anomalies takes place.
[0063] Instead of specifying the detection rate as a numerical value, it can also be evaluated using a classifier, for example, using a color code. For example, red can be chosen for a detection rate that is too poor (depending on the specific circumstances, e.g., less than 50%), yellow for a detection rate that is still acceptable (e.g., greater than or equal to 50% and less than or equal to 90%), and green for a high detection rate that ensures sufficiently reliable detection of anomalies (e.g., >90%).
[0064] A data processing device suitable for a device according to the invention can, in the usual way, comprise a processor with suitable input and output interfaces. The processor can, for example, be designed as a special processor for industrial image processing. Of course, the processor can also be implemented by combining a conventional processor with a special image processing processor. The entire data processing device can also be implemented as a standalone CPU unit with corresponding interfaces, for example, as a slot CPU.
[0065] It should be noted at this point that all of the above-described features of the method and device according to European patent application No. 23166616 can also be used in conjunction with the present invention.
[0066] The invention is explained in more detail below with reference to exemplary embodiments illustrated in the drawings. In the drawings: Fig. 1 shows a schematic representation of an X-ray inspection device with a device for carrying out the method according to the invention; Fig. 2 shows a diagram showing the empirical frequency distribution of a sample of maximum values of a property of previously determined regions in digital images of good products, as well as a probability density function fitted to this; Fig. 3 shows a diagram showing a probability density function for minimum and maximum values of a property determined using digital images of good products (or good process products); Fig.Fig. 4: a diagram showing one probability density function (curve (a)) for maximum values of a property determined using digital images of good products (or good process products), and a second probability density function (curve (b)) for maximum values of a property determined using digital images of bad products, each with a previously known anomaly; Fig. 5: a diagram similar to Fig. 5. Fig. 4 , wherein the curves (a) and (b) have a greater distance on the abscissa, so that a higher detection rate with a simultaneously lower false ejection rate is possible; Fig. 6 a representation of a display of a display device, wherein for three different materials, six similar interfering objects of different sizes are shown and each interfering object is assigned a numerical value of the detection rate; Fig. 7 a representation of a display of a display device similar Fig. 6 , where each interfering object is assigned a smiley symbol for the detection rate; Fig. 8 a representation of a display of a display device similar Fig. 6 , where for the three different materials only the interfering object size is shown which can be detected with a detection rate greater than 95%; Fig. 9 a representation of a display of a display device similar Fig. 6 , whereby an additional setting means for the misejection rate is provided in the form of a digital slider.
[0067] In Fig. 1 An X-ray inspection device 100 with a device 102 for detecting anomalies in digital images is schematically shown, which is designed to carry out the method described below. The X-ray inspection device 100 is merely one possible example of how digital images that may contain anomalies can be generated. It is possible to apply the method according to the invention to any digital images that are to be examined for the presence of anomalies.
[0068] As already mentioned above, the digital images will usually represent products that in turn exhibit the anomalies to be detected. In the case of the Fig. 1 The X-ray inspection device shown here examines products 104 in the form of piece goods as an example. However, it is also possible to generate digital images of any other products, for example, bulk products. In this case, it is advisable to generate digital images that each represent a section of the bulk product.
[0069] At the Fig. 1 In the X-ray inspection device 100 shown, the products 104 to be examined are transported by means of a conveyor device 106 along a predetermined conveyor path (indicated by the arrow F). The conveyor device 106 has a plurality of conveyor belts 108, 110, 112, 114. The conveyor belt 108 is used to feed the products 104, and the conveyor belt 114 is used to remove the products. The inspection device 100 has a shielding housing 116, in which the conveyor belts 110 and 112 are arranged, as well as an X-ray source 118 and an X-ray detector 120. The X-ray source 118 generates an X-ray beam 121, which has a fan-like shape perpendicular to the plane of the drawing and has a small width in the conveying direction. The X-ray beam 121 passes through a gap orFree space between the mutually facing end faces of the conveyor belts 110 and 112 and then strikes the X-ray detector 120, which in the case of the in . Fig. 1 In the illustrated embodiment of the X-ray inspection device 100, it is arranged below the conveyor belts 110, 112. The X-ray detector 120 has a width which, in the direction perpendicular to the plane of the drawing, corresponds to the maximum width of the products to be examined. Typically, the width of the X-ray detector is selected to be approximately the same as the width of the conveyor belts 110, 112. The X-ray detector 120 can be designed as a line detector having one or more detector lines in the direction perpendicular to the plane of the drawing, each detector line having a predetermined number of pixels.
[0070] The device 102 for detecting anomalies has an image processing unit 122, to which the signal from the X-ray detector 120 is fed. The image processing unit 122 can be embodied as a conventional computer unit with one or more processors, a working memory, and optionally a hard disk or SSD storage, as well as suitable interfaces for supplying the signal from the X-ray detector 120 and for supplying or receiving and discharging or transmitting further data. Furthermore, the device 102 can have a display unit 124, on which information generated by or supplied to the image processing unit 122 can be displayed.
[0071] As in Fig. 1 As shown schematically, the products 104 to be examined are moved through the fan-shaped X-ray beam 121, whereby a corresponding digital image signal is generated by the X-ray detector 120, which is designed as a line detector. This digital image signal is fed to the image processing unit 122. The image processing unit 122 can be configured such that a digital image is first generated from the image signal, which, in the case of a piece of goods, contains the entire product 104 or at least a predetermined section. The section can be selected using conventional image processing or pattern recognition methods.The image processing unit 122 can also subject the final digital image, which is to be examined for the presence of anomalies using the method described below, to further image processing from the thus generated digital image, for example, a digital filtering selected so that anomalies to be detected are more easily recognizable, in particular, stand out more clearly compared to the remaining image. Such image preprocessing can also include any desired further image processing steps, for example, noise reduction or the like. At the end of such (not mandatory) image preprocessing, the digital image is obtained, which is then to be examined for the presence of anomalies.
[0072] As mentioned above, it is common practice to use a threshold for pixel values to detect anomalies. In this case, the presence of an anomaly is detected when a pixel value or a group of neighboring pixels with a specified minimum number of pixels exceeds a threshold. If one or more anomalies are detected in a digital image, this information can be used to trigger an action related to the associated product, for example, removing the product from a stream of products. Instead or in addition, the product in question can also be physically or virtually marked, i.e., by assigning corresponding data.
[0073] The display unit 124 can be used to display desired information, such as the digital images to be examined with any anomalies detected therein, the threshold value used in each case, the type of a selected probability density function and the associated parameters, the quality of the fit of the probability density function to the sample, confidence intervals for the threshold values, the parameters or the rates (in particular the false ejection rate), and the like. The output can, of course, be in the form of data (numerical values) and / or graphics. When performing the detection in the normal operating mode of the device, a diagram can also be displayed which represents the course of the current ejection rate during the (normal) operating mode of the device 102 (but also during the learning process) over time, wherein the predetermined rate (e.g.The false rejection rate (FRP) and the confidence interval for the given rate can be displayed. Furthermore, a list of sample values and / or a graph similar to . Fig. 2 Furthermore, representations can be displayed as they are shown in the Fig. 6 bis 9 This information is about the detection rate of specified anomalies generated by physical or virtual interference objects. The data in question can, of course, also be output to a higher-level unit and / or stored.
[0074] The image processing unit 122 can also be supplied with one or more starting threshold values that have already been determined using the method described below for a product type to be examined, or with further information that is required to carry out the method described below, for example information regarding the type of predetermined probability density function(s) to be used or information regarding the manner in which regions are defined in a digital image to be examined (see below).
[0075] As stated above, the method described below for detecting anomalies in digital images is not limited to checking whether one or more pixel values exceed a predefined maximum threshold or fall below a predefined minimum threshold. Rather, the method described below for setting or determining a maximum or minimum threshold can be generalized to determine a maximum threshold and / or a minimum threshold for arbitrary properties of predefined regions in the digital images under investigation.
[0076] To do this, the regions to be examined must first be defined in a digital image to be examined, each of which can be assigned the value of a given property. The regions can be defined, for example, by using a base threshold, with all pixel values that are equal to or exceed the threshold forming a first group of regions and the remaining pixel values forming a second group of regions. Depending on the property to be examined, it may be sufficient for only the first or only the second group to be further processed. This is the case, for example, if only the maximum or minimum pixel value or the average pixel value of the regions is evaluated as a property of the regions for detecting an anomaly.
[0077] However, it is also possible to assign one or more properties to the regions that cannot be described solely by individual pixel values. For example, geometric properties such as area, circumference, diameter (in the case of at least approximately circular regions), or deviation from circularity can be assigned to a region. In such a case, an anomaly is detected if the value of the property in question exceeds a specified maximum threshold or falls below a specified minimum threshold. Even in this general case, the required maximum and minimum thresholds can be determined using the method explained below.
[0078] Another way to determine the regions of interest in a digital image is to use a predefined geometric mask overlaid on the image. This could, for example, be a mask consisting of equally sized adjacent squares of a given size.
[0079] Here, too, one or more properties can be assigned to each region defined in this way, for example the variance or standard deviation of the pixel values encompassed by the regions, the relevant average value or the maximum or minimum value contained therein.
[0080] Combined values can also be determined for each region, which are calculated from two or more values for different properties. For example, a value for the variance or standard deviation of the pixel values and an average value can be combined into a combined value, particularly through an arithmetic operation such as (weighted) addition, which in this particular case represents a type of confidence interval if the pixel values in this region are reasonably normally distributed. Another example of a combined value or combination of properties of the regions is the use of a difference between the maximum and minimum pixel values contained within each region. To be more stable against outliers, quantiles can also be used, for example 90% and 10% quantiles instead of the maximum and minimum values.
[0081] The following describes how a corresponding maximum or minimum threshold can be determined using a small number of digital images. As mentioned above, such a teach-in process can be performed using good products or good process products. The teach-in process can be carried out, for example, during commissioning of a system for manufacturing or processing products that includes such an inspection device. In this case, it will generally be necessary to determine the threshold values for each product type.
[0082] Of course, the threshold values can be saved for specific product types so that the learning process does not have to be repeated every time the type of products to be tested is changed.
[0083] It is still possible to perform the learning process during ongoing plant operation, at least if it can be assumed that the plant produces good process products, i.e., products that are predominantly good products (which do not contain any anomalies). In this case, as explained below, the bad products contained in the good process products can be eliminated as outliers when creating the required sample.
[0084] A key feature of the threshold determination method described below is that, when using good products (with unknown detailed information), a rate is specified at which a maximum anomaly is to be incorrectly detected in an image under examination (hereinafter also referred to as the false positive rate or false rejection rate) or at which a maximum anomaly is correctly not detected in an image under examination (hereinafter also referred to as the true negative rate). This eliminates the need to determine a threshold in a first step and then to check in a second step whether using the respective threshold results in a corresponding acceptable rate.However, for the determination of the detection rate for previously known anomalies described below, it is also possible to directly specify a false positive rate or the complementary true negative rate. However, in this case, a significantly larger number of digital images of acceptable products is required to select the threshold value so that the desired false positive rate is achieved.
[0085] The required number of digital images, i.e. the number of good products or good process products, can be specified in advance, although this number must then be high enough to determine a threshold value with the desired reliability. However, it is also possible to specify the required number of digital images during the training process (regardless of whether the images are generated during the training process or are already available before the start of the training process). It is advisable to start with a minimum number of digital images and gradually increase this minimum number by one or more digital images until the respective threshold value is sufficiently reliable, i.e. the specified rate is sufficiently reliably maintained. This can be checked by determining a confidence interval.
[0086] In the next step, as with the detection of anomalies in normal operating mode (i.e. outside of the learning process), the regions are defined for each of the digital images and the value of at least one property assigned to the region is determined for each region. For each digital image, the maximum value and / or the minimum value of the property or properties in question or the combined value for several properties is then determined and assigned to a corresponding sample. This can be done, for example, by storing all minimum values in a minimum value list and all maximum values in a maximum value list. It should be noted that, of course, both alternatives of the method do not have to be used simultaneously. If only maximum anomalies are to be detected in normal operating mode, i.e.For anomalies that are detected as such when a maximum threshold is exceeded, only a maximum threshold needs to be determined. The same applies if only minimal anomalies are to be detected.
[0087] Once a sufficient number of sample elements (maximum or minimum values for at least one property or combined property) have been determined, a further step can be used to parameterize a given probability density function for the maximum value list or minimum value list using a statistical estimation method. This corresponds to fitting the probability density function to the empirical frequency distribution of the sample in question.
[0088] For this purpose, the sample can be binned, meaning that the sample elements are assigned to equally wide, adjacent intervals of the corresponding value range. The given probability density function can then be fitted to this empirical (relative) frequency distribution, for example, using the least squares method. Typically, however, a more advantageous statistical estimation function is used for this purpose, such as the maximum likelihood method or the method of moments.
[0089] The probability density function that must be specified for the procedure should be of a type that can be assumed to describe the sample well when parameterized. Since in this case, extreme values for the property in question are selected and form the respective sample, one will often choose a type of generalized extreme value distribution or the generalized extreme value distribution (with its three parameters), which combines the Gumbel distribution, the Weibull distribution, and the Fréchet distribution. The frequently used Gumbel distribution has the form: f x = exp − exp − x − μ β
[0090] Here, f denotes the value of the probability density as a function of the random variable x. The value x of the respective random variable denotes in this case the value of the respective property or the combined value. The parameters µ and βare determined by the chosen statistical method.
[0091] Fig. 2 shows a diagram schematically illustrating the fitting of a probability density function f(x) to a relative empirical frequency distribution of maximum pixel values, where x denotes the pixel value. The pixel values are plotted on the abscissa, and the values for the probability density and relative frequency, respectively, are plotted on the ordinate.
[0092] Using the parameterized probability density function, the desired threshold (for the specified rate at which a maximum anomaly is falsely detected in images under investigation, or at which no maximum anomaly is correctly detected in images under investigation) can be determined in a subsequent step. For this purpose, the threshold is set such that the area under the parameterized probability density function above the threshold is equal to the specified rate at which a maximum anomaly is falsely detected in images under investigation, or the area under the parameterized probability density function below the threshold is equal to the specified rate at which no maximum anomaly is correctly detected in images under investigation.This area corresponds to the probability of a maximum value of the property in question occurring in a digital image that is greater than or equal to the threshold. The threshold can be determined by rearranging the equation from the inverse function of the distribution function associated with the parameterized probability density function, provided a closed-form solution for the inverse function exists. Otherwise, the threshold can be calculated using known and suitable numerical methods.
[0093] In Fig. 2 Threshold values X so,1 , X so,2 and X so,3 are shown, which correspond to a false rejection rate (false positive rate) of 1.0%, 0.6% and 0.1%, ie the areas under the probability density function to the right of these threshold values result in the values 0.01, 0.006 and 0.001.
[0094] If the true negative rate is specified instead of the desired false positive rate, the area under the probability density function below the upper threshold X must be used accordingly to calculate the maximum threshold.
[0095] In Fig. 3 is a similar diagram as in Fig. 2 shown, although only the course of the already fitted probability density function is shown. In this example, both a maximum threshold X so and a minimum threshold X su are determined using the parameterized probability density function. Here, in both cases, i.e. for determining the maximum threshold as well as for determining the minimum threshold, the same false positive rate R fp is specified, whereby for determining the minimum threshold X su the false positive rate R fp is equal to the area under the probability density function below the minimum threshold X su . Accordingly, the true negative rate R rn is equal to the area under the probability density function above the minimum threshold X su . This is because a minimal anomaly is detected when the value of the property in question orthe combined value for several properties is less than the relevant minimum threshold X su .
[0096] In the case of parameterized probability density functions according to Fig. 3 A digital image would be detected as containing an anomaly if, for at least one region, a value of the property under consideration or a combined value for the properties in question lies outside the good range between the minimum threshold and the maximum threshold (or in other words, if the value in question is less than or equal to the minimum threshold X su or greater than or equal to the maximum threshold X so).
[0097] Furthermore, Fig. 3 a confidence interval for the maximum threshold value X so is plotted, whereby for the sake of simplicity a symmetrical confidence interval with the limits X so - ΔX so and X so + ΔX so was assumed. In order to reduce the false rejection rate to the desired specified value with even greater certainty, a secondary maximum threshold value can be used when detecting anomalies which is ΔX so greater than the determined threshold value X so . In the same way, a secondary minimum threshold value can be used which is smaller than the determined threshold value X su by the width of a confidence interval ΔX su for the lower threshold value X su . In other words, the secondary maximum threshold value corresponds to the upper limit of the confidence interval for the maximum threshold value and the secondary minimum threshold value corresponds to the lower limit of the confidence interval for the minimum threshold value.
[0098] As explained above, the number of digital images used to determine the threshold(s) (maximum or minimum threshold) can be fixed. However, this number should be large enough to determine a sufficiently reliable threshold.
[0099] The method is suitable for detecting anomalies in product images, which can be generated in any way and represent any product properties. The method is particularly suitable for analyzing images obtained using inspection devices that operate with X-rays or terahertz radiation and can thus generate information about the interior of a product. The method can be implemented, in particular, via software in an inspection device. The result of the detection method can be used to control other devices, such as a sorting device.
[0100] Once the desired maximum or minimum threshold value has been determined as described above, the further method described below follows, which is carried out in a separate inspection process. The inspection process can be carried out, in particular, before the start of productive operation of a system with an X-ray inspection device 100 (or any other inspection device).
[0101] It is expressly emphasized again that the determination of the desired threshold(s) in the manner described above is not a mandatory requirement for carrying out the testing process described below. Rather, the desired threshold can also be specified directly, for example, by an operator or by taking the threshold from a memory in which the desired maximum and / or minimum threshold values assigned to a product type are stored.
[0102] The testing process comprises the following steps: First, a plurality of digital defective images are generated, each of which has at least one previously known anomaly. The digital defective images can be generated using one or more physical defective products, each of which contains one or more previously known interfering objects that generate a corresponding previously known anomaly in a digital image (rejection image) of the defective product in question. Such defective products are manufactured from good products, into each of which one or more previously known interfering objects are introduced, preferably at a specific position within the good product. This makes it possible to determine the maximum extreme value or minimum extreme value in a defective image produced using a defective product created in this way, which is generated by the respective previously known anomaly and which serves as the maximum sample value or minimum extreme value.The minimum sample value used to determine the detection rate using the method explained below is to search only in a suitably defined area of the digital image which includes the previously known anomaly or in which the interfering object in question is located (regardless of whether the interfering object creates a detectable anomaly in the bad digital image).
[0103] The area of the digital image to be considered can be defined in a variety of ways. For example, the geometric shape of the area (square, circular, or similar) can be specified and the size can then be selected so that the interfering object or the anomaly caused by it is located entirely within the area.
[0104] According to another variant, a base threshold can be used, whereby (in the manner already explained above) all regions of the defective image are determined whose pixel values are greater (or greater or equal to) the respective base threshold for determining a maximum anomaly or smaller (or smaller or equal to) the respective base threshold for determining a minimum anomaly. Of these regions, the region that includes the interfering object or the previously known anomaly caused by it can then be used. Using the information about the position within the defective product or within the respective defective image and the information about the geometry of the respective interfering object, it can also be checked whether the interfering object or the previously known anomaly is sufficiently encompassed by the region thus determined.This requirement can be affirmed, for example, if the geometric area of the interfering object within the defective image or the area of the previously known anomaly caused thereby sufficiently overlap, for example, with an overlap degree of more than 50 or more than 80%. The overlap degree can be used as the ratio of the area of the interfering object or the previously known anomaly caused thereby in the digital defective image to the area of the relevant previously determined region.
[0105] However, the generation of bad images can also be carried out using at least one previously generated good image, i.e., using at least one digital image of a good product, which preferably corresponds to the same type of products to be inspected during productive operation of the respective inspection device 100. In practice, instead of a single good image, a plurality of good images will preferably be used in order to take into account information about the statistical distribution of the properties of the good images. Such good images can already be generated during the above-described learning process (in which a threshold value is determined based on the misrejection rate) and used to create "artificial" bad images. Of course, one or more good images can be stored in a memory for this purpose.
[0106] To generate such an "artificial" negative image, a virtual interfering object can first be defined, specifying a predetermined material and the geometry of the virtual interfering object. Simple shapes, such as a sphere, cube, or cuboid, can be used as the geometry. The material of the interfering object can be taken into account by considering the specific attenuation of the material, particularly in inspection procedures that involve X-raying the product in question. For X-ray inspection, databases or tables containing the specific attenuation of a variety of materials can be used for this purpose. Specific attenuation is understood to be the attenuation experienced by the radiation in question per unit length (seen in the direction of radiation) when penetrating the material.This additional attenuation can thus be easily incorporated into the existing image data of the good image, which is used to create the "artificial" bad image, at the relevant position. If the image data of the good image is available on a logarithmic scale, the additional attenuation, which is also available on a logarithmic scale, can be incorporated by adding it.
[0107] Regardless of whether physical or virtual interfering objects are used to generate digital defective images, with the exception of spherical interfering objects, the position of the interfering object relative to the direction of radiation through the product to be inspected (or more generally: relative to the direction of observation of the product) must also be determined. To calculate the additional attenuation generated by the respective interfering object, the interfering object (more precisely: its projection in a plane perpendicular to the direction of radiation) is divided into pixels (where the pixel division preferably corresponds to the existing pixel division of the good image). In this way, the additional attenuation can be calculated for each individual pixel (i.e. pixel by pixel).The "artificial" bad image created in this way can then be processed in the same way as a bad image created using a physical interference object.
[0108] As already indicated above, for a sufficient number of such generated bad images, a corresponding maximum sample value or minimum sample value (depending on whether it is a maximum or minimum anomaly) is determined for each given anomaly contained in a bad image. This can be done in the same way as previously explained for determining a maximum or minimum threshold.
[0109] A detection rate can then be determined from a maximum value sample or minimum value sample generated in this way. According to a first variant, a probability density function can be specified for this purpose, which has at least one free, non-specified parameter. This parameter can then be determined in a suitable manner, in particular using a statistical estimation method, so that the parameterized probability density function optimally describes the respective sample. This parameterization can be performed in the same way as previously described in connection with the determination of a maximum threshold or minimum threshold.
[0110] Fig. 4 shows a diagram with two curves representing probability density functions parameterized using a generalized extreme value function as described above. Curve (a) shows a probability density function that was selected and parameterized to determine a maximum threshold using good products (or good process products) in the manner described above. From this curve, the corresponding threshold X is determined by specifying a value for the false rejection rate or false positive rate R fp.
[0111] The threshold value X so thus determined can then be used to determine from the second curve (b) an estimate for the detection rate that will occur during productive operation of the inspection device 100 when inspecting similar products. As already mentioned above, this estimate is more accurate the better the interfering objects occurring in practice match the previously known interfering objects that were used to determine the respective maximum value sample or minimum value sample.
[0112] The curve (b) in Fig. 4 shows a probability density function selected and parameterized for a maximum value sample for a previously known anomaly in the manner described above. Consequently, from this probability density function, a detection rate for the respective previously known anomaly that will occur during the inspection of products of the same type can be determined. This simply requires integrating the probability density function from the previously determined maximum threshold X 50 to infinity (or a very high value, for example, the maximum possible pixel value provided by the radiation detector).
[0113] This integral value corresponds to the true positive rate for detecting the known anomaly in question, i.e. the detection rate.
[0114] This now known detection rate can be evaluated by an operator or automatically to determine whether the respective value is acceptable in practice, particularly taking into account the consequences that arise if an interfering object is not detected. For example, if a food product such as yogurt, cheese, chocolate, biscuits, or the like is inspected and a metal or bone fragment is found that is not detected, this can lead to potentially life-threatening injuries to the end consumer who consumes such contaminated food. In such a case, the detection rate must meet correspondingly strict specifications, for example, be greater than 99%. In other cases where non-detection leads to less serious consequences, the detection rate can be set to a significantly lower value of 80% or 90%.
[0115] Since the detection rate and the false ejection rate, or the specified maximum or minimum threshold, are interdependent, it will often not be possible to simultaneously achieve a desired low false ejection rate and a desired high detection rate. Such a case is Fig. 4 As can be seen from this diagram, a high maximum threshold X so or a low false ejection rate R fp inevitably leads to a drastically reduced detection rate R rp.
[0116] Fig. 5 shows a different case in which the previously known maximum anomaly is clearly visible in a poor image. Here, curve (b) is compared to curve (b) in Fig. 4 significantly further to the right, ie in the direction of larger pixel values. Consequently, the same maximum threshold X so or the same false ejection rate R fp as in Fig. 4 in the case according to Fig. 5 to an extremely high detection rate R fp .
[0117] The two cases according to the Figuren 4 and 5 For example, these can occur for previously known anomalies caused by interfering objects of the same size but made of different materials. Similarly, these cases can arise when the interfering objects are made of the same material, but the interfering object in the case of the Fig. 4 is significantly smaller than the disturbing object in the case of
[0118] Fig. 5 .
[0119] Instead of using a probability density function to describe the maximum or minimum value sample to determine the detection rate, the respective sample values can also be used as such. In this case, the detection rate can be determined by determining the number of sample values that are greater than or equal to the previously determined threshold. The detection rate is then calculated by dividing the thus determined number of sample values by the total number of sample values.
[0120] In this case, too, it is not necessary to determine the relevant threshold from a previously specified false ejection rate. Rather, it is also possible to specify the threshold directly.
[0121] In Fig. 6 a display of the display unit 124 of the X-ray inspection device 100 is shown, which shows the result of an inspection process that was carried out as explained above.
[0122] As can be seen from this figure, the detection rate for spherical interfering objects with different diameters, namely 0.5 mm, 1.0 mm, 1.5 mm, 2.0 mm, 2.5 mm, and 3.0 mm, was determined for three different materials: aluminum, ceramic, and glass. The numerical values for the detection rate are shown associated with a corresponding schematic representation of the respective interfering object. This display provides the operator with immediate information about the detection rate for the various materials and sizes of the interfering objects. Fig. 6 As can be seen, for example, spherical interfering objects made of aluminum with a diameter of 0.5 mm are still detected with a detection rate of 13%, while correspondingly large interfering objects made of ceramic and glass are no longer detected. Interfering objects made of aluminum or ceramic with a diameter of 2.5 or 3.0 mm are still detected with 100% accuracy, while corresponding interfering objects made of glass are only detected with a detection rate of 76% or 93%, respectively.
[0123] Fig. 7 shows another variant of an ad that is largely similar to the display in Fig. 6 Instead of numerical values for the detection rate, however, a classifier is used here, with a detection rate below 80% being classified as insufficient, a detection rate between 80% and 90% (including the limits) as still acceptable, and a detection rate >90% as good. The three different classifier values are represented by smileys. A color code could also be used in the same way, for example, green for "good," yellow for "acceptable," and red for "inadequate." Such a classifier enables an operator to interpret the display or the result of the inspection process even more quickly.
[0124] Fig. 8 shows a further variant of a display of the display unit 124, wherein the size of spherical interfering objects made of different materials is displayed here, for which a detection rate classified as good is still achieved. In the case shown, good detectability is still achieved for spherical interfering objects made of aluminum for sphere diameters greater than or equal to 2.5 mm, for ceramics for sphere diameters greater than or equal to 2.0 mm, and for glass for sphere diameters greater than or equal to 3.0 mm, whereby a gradation in steps of 0.5 mm was also assumed for the size of the interfering objects made of one and the same material when carrying out the testing process.
[0125] Fig. 9 shows a further variant of a display of the display unit 124, which largely corresponds to the display according to Fig. 6In addition, a setting means for the threshold value X is specified here. In the illustrated embodiment, the setting means is designed as a slider 124a, which can be easily implemented using a touch display for the display unit 124.
[0126] In an initial state, a sliding element 124b of the slider 124a can be in an initial position in which an initial threshold value X s0,0 is preset. In this initial state, the detection rate for each size and material of the interfering objects can be determined and displayed. If an operator determines that the detection rate is too low for certain sizes and / or materials, the operator can move the slider toward higher threshold values. New detection rates can be calculated and displayed immediately after changing the position of the sliding element 124b. In this way, the operator can easily and quickly change the threshold value so that the detection rate is optimal for the respective case.
[0127] Of course, instead of the threshold value, a false ejection rate can also be set or changed using the adjustment tool. In this case, the operator receives immediate information about any strong relationship between the false ejection rate and the detection rate and can, for example, adjust the false ejection rate so that a satisfactory detection rate is (still) achieved. List of reference symbols
[0128] 100X-ray inspection device 102Anomaly detection device 104Product 106Conveyor 108Conveyor belt 110Conveyor belt 112Conveyor belt 114Conveyor belt 116Shielding housing 118X-ray source 120X-ray detector 121X-ray beam 122Image processing unit 124Display unit 124aSlider 124bSliding element F Conveying direction X su lower threshold (determination of the threshold using good products or good process products) Y su lower threshold (determination of the threshold using bad products or bad process products) X so upper threshold (determination of the threshold using good products or good process products) Y so upper threshold (determination of the threshold using bad products or bad process products) ΔX so width of the confidence interval for the maximum threshold ΔX su width of the confidence interval for the minimum threshold R fp false positive rate R rn true negative rate R rp true positive rate R fn false negative rate
Claims
1. A method for detecting anomalies in digital images of products, wherein each digital image is formed by a plurality of pixels represented by image data, and wherein each pixel represents an associated location within the respective image and has a value characterizing the respective location, (a) wherein each image to be examined is understood as a region or divided into two or more regions, each consisting of one or more adjacent pixels, (b) and wherein for each region a value for at least one property or a combined value for several properties of the region is determined, (c) wherein a region is detected as a maximum anomaly if the value of the at least one property or a combined value for several properties of the region is greater than a predetermined maximum threshold, or wherein a region is detected as a minimum anomaly,if the value of at least one property or a combined value for several properties of the region is less than a specified minimum threshold, characterized by (d) thatin a testing process, the following steps are carried out: (i) generating a plurality of digital defective images of real or fictitious defective products, each having at least one previously known anomaly; (ii) for each defective image, defining the region or regions and determining the value of the at least one property or the combined value for several properties of each region and determining the maximum value of these values as the maximum sample value of a maximum value sample or determining the minimum value of these values as the minimum sample value of a minimum value sample; (iii) determining a detection rate for the at least one previously known anomaly (1) by determining estimated values for all free,unspecified parameters of a probability density function specified to describe the maximum value sample or the minimum value sample (parameterization) using the maximum sample values or the minimum sample values and using a statistical estimation method, and (2) by integrating the parameterized probability density function using the specified maximum threshold or minimum threshold as the integration limit, or (iv) determining a detection rate for the at least one previously known anomaly as the ratio of the number of values of the maximum value sample or the minimum value sample that are greater than or equal to the specified maximum threshold or less than or equal to the specified minimum threshold and the total number of values of the maximum value sample or the minimum value sample; and (v) assigning the detection rate to the at least one previously known anomaly.
2. Method according to claim 1, characterized in thatthe maximum threshold value and / or the minimum threshold value are determined in a training process in which the following steps are carried out: (a) generating or using a number of digital images of good products which do not contain an anomaly, or of good process products which predominantly do not contain an anomaly, wherein the number of images is predetermined or is determined during the training process; (b) for each of the digital images, defining the region or regions and determining the value of at least one property or the combined value for several properties of each region and determining the maximum value of these values as the maximum sample value of a maximum value sample and / or determining the minimum value of these values as the minimum sample value of a minimum value sample; (c) using a statistical estimation method, determining estimated values for all free,non-specified parameters of a probability density function specified to describe the maximum value sample using the maximum sample values and / or determining estimated values for all free, non-specified parameters of a probability density function specified to describe the minimum value sample using the minimum sample values; (d) specifying a first rate at which a maximum anomaly is falsely detected in images to be examined, or a second rate at which no maximum anomaly is correctly detected in images to be examined, and / or specifying a third rate at which a minimum anomaly is falsely detected in images to be examined, or a fourth rate at which no minimum anomaly is correctly detected in images to be examined,(e) determining the maximum threshold using the probability density function parameterized according to feature (c) or a distribution function corresponding thereto such that the probability of the occurrence of a maximum value that is greater than or equal to the maximum threshold corresponds to the predetermined first rate or that the probability of the occurrence of a maximum value that is less than or equal to the maximum threshold corresponds to the predetermined second rate, and / or (f) determining the minimum threshold using the probability density function parameterized according to feature (c) or a distribution function corresponding thereto such that the probability of the occurrence of a minimum value that is less than or equal to the minimum threshold corresponds to the predetermined third rate or that the probability of the occurrence of a minimum value that is greater than or equal to the minimum threshold,corresponds to the specified fourth installment.
3. Method according to claim 1 or 2, characterized by (a) that the region or regions are determined using a base threshold, wherein each isolated pixel and each group of neighboring pixels whose pixel value is greater than the base threshold are each assigned to a region of a first group of regions, and / or wherein each isolated pixel and each group of neighboring pixels whose pixel value is less than or equal to the base threshold are each assigned to a region of a second group of regions, or (b) that the region or regions are defined using a geometric mask, in particular a fixed mask or a mask generated from the respective image by means of image processing.
4. Method according to one of the preceding claims, characterized in thatas a property of a region, a geometric property is used which is determined from the location information of the pixels of the region, in particular the area, the circumference or the diameter, or that as a property of a region, a pixel value property is used which is determined from the values of the pixels of the region, in particular the maximum value or the minimum value of all pixels of the region, the mean value, the variance or standard deviation.
5. Method according to one of the preceding claims, characterized in that the given probability distribution is the generalized extreme value distribution, in particular its special cases, the Gumbel distribution, the Weibull distribution or the Fréchet distribution.
6. Method according to one of the preceding claims, characterized in thatthe at least one predetermined anomaly is located at a predetermined position in the bad image and that this information and optionally also the geometry of an interfering object causing this predetermined anomaly is used to determine the maximum sample value or minimum sample value representing this predetermined anomaly.
7. Method according to one of the preceding claims, characterized by (a) that to produce a bad image, a good product is used which does not contain any anomaly, wherein at least one interfering object or a carrier with at least one interfering object arranged thereon is provided on or in the good product, which produces the at least one predetermined anomaly in the bad image, or (b) thatDigital image data of a good product are used to generate a bad image, wherein the digital image data of the good product are digitally transformed using known material and geometric properties of at least one predetermined interfering object in order to take the at least one interfering object into account.
8. Method according to claim 7, characterized in that the carrier for the at least one interfering object is plate-shaped or card-shaped and that in or on the carrier several interfering objects with identical material properties, in particular made of a single material, with similar geometry but different sizes are arranged.
9. Method according to claim 7 or 8, characterized in that the at least one disruptive object is a ball made of a predetermined material.
10. Method according to one of the preceding claims, characterized in thatthe detection rate is displayed as a numerical value or a related quantity or marking on a display device.
11. Method according to claim 10, characterized in that the detection rate is shown associated with a graphical representation of the at least one interfering object and / or the associated bad image.
12. Method according to claim 10, characterized in that the maximum threshold value or minimum threshold value or the first, second, third or fourth rate can be changed by means of an input means or setting means, for example a slider or rotary knob, and that a current value for the detection rate is shown on the display device for each current value for the maximum threshold value or minimum threshold value or the first, second, third or fourth rate.
13. Method according to claim 11, characterized in that the detection rate is evaluated using a classifier, for example with a color code.
14. Device for detecting anomalies in digital images of products, in particular an X-ray inspection device, with a data processing device which is designed to receive and process digital image data from digital images of products, characterized in that the data processing device is designed to carry out the method according to one of the preceding claims.
15. A computer program product for detecting anomalies in digital images of products, in particular for an X-ray inspection device, comprising instructions which, when executed by a data processing device, cause the device to carry out the method according to one of the claims.
Citation Information
Patent Citations
Method and apparatus for detecting anomalies in two-dimensional digital images of products
EP4287116A2
Surface anomaly detection system and method
US20070098245A1
