Method and apparatus for detecting abnormality in two-dimensional digital image of product
The method addresses the inefficiency of existing anomaly detection by determining threshold values statistically, ensuring reliable anomaly detection with reduced false discard rates and enabling quick threshold adjustments.
Patent Information
- Application Number
- JP2024065986
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-04
- Filing Date
- 2024-04-16
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-04-16
AI Technical Summary
Existing methods for detecting anomalies in digital images of products require a large number of digital images of acceptable products to determine threshold values, leading to significant production time loss and uncertainty in defining the false discard rate.
A method that determines threshold values using a statistical estimation of pixel characteristics in a learning process, allowing for the determination of detection rates for anomalies with a smaller number of images, by parameterizing probability density functions for maximum and minimum thresholds.
Enables efficient and reliable anomaly detection with reduced false discard rates, providing operators with direct insight into detection certainty and allowing for quick adjustment of thresholds to meet economic and reliability requirements.
Smart Images

Figure 2024160948000002 
Figure 2024160948000003 
Figure 2024160948000004
Abstract
Description
[Technical field]
[0001] The present invention relates to a method for detecting anomalies in two-dimensional digital images of a product, comprising the features of the preamble of claim 1. Furthermore, the invention relates to an apparatus for implementing this method as well as to a computer program product. [Background technology]
[0002] During the production of products, it is often desirable or necessary to subsequently check the produced products for defects, in particular defects, foreign bodies, and other anomalies. For example, in the food industry, during the production of yogurt or cheese, the task can be to continuously check the finished products for unwanted foreign bodies or other undesirable material areas in the product. To solve this problem, in practice, inspection devices are used which transilluminate the product with electromagnetic radiation, in particular with radiation in the X-ray spectrum. In this way, a digital image of the product is generated which contains information not only about the external geometric dimensions or the surface of the product, but also about the product's interior. Depending on the attenuation, anomalies to be detected can then result in image areas which, when transilluminated, have a "grey value" or pixel value which is higher or lower than anomaly-free image areas (hereinafter referred to as "pass areas"). In the present description, the term "grey value" is used to refer to information generated by a detector depending on the radiation power or corresponding radiation energy falling on each individual pixel detected during a reference exposure time, regardless of the color or manner in which the pixel values can be displayed in the corresponding digital image.
[0003] For the inspection of these types of products, devices or methods are also known in which the product to be inspected is not completely transilluminated (i.e. the radiation source and detector are located on opposite sides of the product), but rather the radiation penetrates deep enough into the product to be inspected that it is "reflected" within the product, such "reflection" being caused physically by scattering of the radiation penetrating the volume or by the generation of fluorescent radiation in the volume. In these types of inspection devices the radiation source and the detector may be on the same side of the product to be inspected.
[0004] The products to be inspected can then be present in the form of any type of individual item or as bulk goods which are conveyed through the inspection device by a conveying device.
[0005] As detectors for radiation, line detectors are often used in the production line, where the products to be inspected move along a transport path, with one or more detector lines, each with a predetermined number of pixels. The products then move, usually at a constant speed, through a scanning device of this kind, and a number of detected lines are combined to form a digital image. However, it is of course also possible to use area scanners instead of line scanners. In this case, the digital image of the product to be inspected can be detected in a single detection process (i.e. in a single "exposure process").
[0006] The digital image thus generated is then inspected, usually automatically, for defects in the interior of the product in question. The image generated by the scanning process carried out can then be directly processed or pre-processed prior to such inspection. For this purpose, the original image can be digitally filtered, for example, and the filters used for this purpose can cause contrast enhancement.
[0007] It is also possible to take measures to improve the contrast when generating the images, particularly with a view to detecting anomalies: for example, a dual energy method can be employed, in which both spectra are selected such that the overlap of the corresponding partial images leads to an improved contrast.
[0008] Furthermore, it is possible to use a spectrally resolved detector which generates several images, each of which is composed of pixels whose grey values correspond to the radiation energy of a particular spectral portion of the detected radiation. For the detection of anomalies, images can be used which are generated from all or selected partial images of such a spectrally resolved detector, for example by weighted addition of the corresponding pixel values. However, each partial image can also be examined separately for the presence or absence of anomalies.
[0009] Methods are known in which a threshold value is defined for the automated inspection of digital images of products, and an anomaly is present if the grey value of at least one pixel is greater than the threshold value. For this purpose, in a learning process, a predefined number of acceptable products (i.e. products without anomalies) of the same product type are usually scanned to obtain information on the maximum grey value that usually occurs in such acceptable products. Depending on this, a threshold value is then defined such that a defined value of the mis-rejection rate (for example in percentages or per thousand) is respected. The mis-rejection rate here is the probability that an acceptable product is recognized as a "rejected product". Such a (theoretical) mis-rejection rate can be checked by determining the empirical frequency with which acceptable products are identified as rejected products. For this purpose, a sufficient number of digital images of acceptable products can be generated and checked with the aid of a defined threshold value, and the empirical mis-rejection rate is determined as the quotient of the acceptable products recognized as rejected products divided by the total number of acceptable products.
[0010] That is, the determination of such a threshold value requires a costly learning process, during which a relatively large number of digital images of the acceptable products must be generated. However, this is a disadvantage, since such a large number of acceptable products must first be produced by the corresponding production line, i.e., a corresponding amount of production time is lost. Moreover, it is hardly possible to predict how large the number of acceptable products required for the learning process must be in order to be able to define the threshold value with a high degree of confidence so that the desired rate of false rejection is respected.
[0011] Unpublished US Pat. No. 5,399,366 describes a method for detecting anomalies in digital images, which allows an automated process for defining thresholds to be carried out even with the use of a small number of digital images. In this method, thresholds for detecting anomalies in product digital images are determined with good reliability in a learning process. In this case, assumptions are made about the statistical distribution of maximum or minimum values (hereinafter called extreme values) of characteristic quantities used for the recognition of anomalies in the digital image to be inspected. The distribution of such extreme values is described sufficiently well by a given probability density function to be parameterized (the concept of "parameterizing" in this specification means determining or defining values of parameters of a probability density function that are not already defined). In other words, the assumption is made that elements of a random sample of the respectively set probability density function (i.e. the set of the respective maximum or minimum values) are sufficient or that the probability of occurrence can be well described by this probability density function. Using the elements of the random sample, estimates are determined for all parameters of the probability density function to be determined (i.e. all parameters that are not already set, as the case may be). As a probability density function to be parameterized, the method makes use of in particular the generalized extreme value distribution, for example certain of its variants, namely the Weibull, Frechet or Gumbel distribution, in order to describe the distribution of maximum or minimum values of a characteristic quantity.
[0012] This method is not limited to the case described at the beginning, in which pixel values, in particular each individual pixel value, in relation to a threshold value for pixel values, are used to determine whether the digital image to be examined contains an anomaly.
[0013] Rather, the invention generalizes the principle underlying such a scheme. The image to be examined is divided into one or more areas, and each area is assigned at least one identical property or multiple identical properties. Each area then contains exactly one or multiple pixels, and multiple pixels are assigned to the same area only if they are adjacent (i.e. each pixel of this area has at least one pixel of the same area directly adjacent). In this case, each pixel that is adjacent to the pixel of interest with one side (i.e. the pixels above, below, left and right of the pixel of interest) or each pixel that is adjacent to the pixel of interest with one corner (i.e. the pixels in the diagonal extension of the pixel of interest) can be considered as adjacent to the pixel of interest. For each property of the predefined areas, a value that represents the respective property is determined. Where multiple characteristics are assigned to each area, a separate value may be defined for each characteristic, or the values of two or more characteristics may be combined into a single value, for example by a mathematical operation (e.g. multiplication or division or weighted addition).
[0014] Under such generalization, anomalies are detected by determining a threshold for each characteristic or each combination of characteristics.
[0015] In an extreme case, the entire image can be defined as a region, but if the pixel value of each pixel is checked as an inherent quantity for detecting anomalies, whether it is above or below a threshold value set for the pixel value, this leads to the same results as if each pixel were defined as a region containing just one pixel. In this case, the generalization described above, using predefined regions and any assigned properties, leads to the conventional approach where each pixel is checked for its pixel value, whether it is above or below a threshold value for the pixel value.
[0016] The determination of the threshold value for a characteristic or for a combination of characteristics is carried out according to the invention in a learning process, where the images required for the threshold determination can already be generated in a sufficient predetermined number or can be created on demand, i.e. one or more new digital images are generated successively until a sufficient number is reached.
[0017] In this known method, a learning process or an automated process for determining the threshold value for at least one characteristic is carried out with the aid of a digital image of a passable product without anomalies. As already mentioned above, an image, which may be the direct result of a scanning process, can first be processed or preprocessed for the detection method. In this case, it is also possible to generate a suitable part of the entire determined image, which contains the product as a whole or contains a set partial area. The part thus generated or the entire detected digital image can be subjected to digital filtering, for example in order to cause a contrast improvement in order to make the anomalies stand out more strongly. If endlessly produced products or bulk products are to be checked for anomalies, digital images of fragments of such products can be generated and processed and checked in the same way as is possible with images of single products (or parts of such type of products).
[0018] It should be noted here that it is not essential that the automated process is carried out using only pass products. Instead, the automated process can be carried out using images of products produced on the production line, without guaranteeing that none of the products are pass products. Instead of pass products, products called "pass process products" can be used, with the majority of the pass process products being pass products and only a minority being pass products, since in practice the proportion of pass products in the pass process products is expected to be low, in particular less than 25%, preferably less than 10% and most preferably less than 5%.
[0019] Within the framework of the automated process, a fixed number of digital images, or a number of images to be determined in the course of the automated process, are generated or utilized for a pass product or a pass process product.
[0020] One or more areas are defined for each digital image and for each area a value of at least one property or a combined value of a number of properties is determined, the maximum of these values being determined as the maximum random sample value of the maximum random sample and / or the minimum of these values being determined as the minimum random sample value of the minimum random sample.
[0021] In this case, it is possible to exclude extreme values that are not valid or that clearly indicate an error. For example, when determining the extreme values, pixel values that have an absolute maximum or minimum value of the gray value scale used can be excluded, since a corresponding minimum value, for example a value of 0, may indicate a pixel with a detector defect, and a maximum value may indicate a pixel with a detector overshoot.
[0022] The extremums thus determined are the random samples mentioned above and can be stored, for example, as lists (possibly separately for minimum and maximum values).
[0023] Subsequently, estimates can be determined for all pending parameters of the probability density function set to represent the maximum random sample using the maximum random sample value and / or for all pending parameters of the probability density function set to represent the minimum random sample using the minimum random sample value, whereby a statistical estimation method is applied.
[0024] In particular, since statistical estimation methods utilizing estimation functions (also called statistical estimators) can only expect meaningful results if the random sample contains a minimum number of values, such a minimum number is usually set, which in turn results in a corresponding minimum number of digital images.
[0025] One key feature of this known method is that the rate at which the largest anomaly is erroneously detected in the image to be examined (when the method using a threshold value is carried out) is set, or the rate at which the largest anomaly is not correctly detected in the image to be examined (when the method using a threshold value is carried out) is set, and / or the rate at which the smallest anomaly is erroneously detected in the image to be examined (when the method using a threshold value is carried out) is set, or the rate at which the smallest anomaly is not correctly detected in the image to be examined (when the method using a threshold value is carried out), i.e. the maximum or minimum threshold value to be determined can be determined such that the set rates are respected.
[0026] The concept of rate is understood here as the quotient of the number of acceptable products which meet the corresponding rate criterion when the test is applied and the set total number of acceptable or process products. The rate of erroneous detection of the largest or smallest anomaly in the image to be inspected corresponds accordingly to the concept of "false reject rate" which is often used in practice. The rate of erroneous detection of the largest or smallest anomaly in the image to be inspected and the rate of incorrect recognition of the largest or smallest anomaly in the image to be inspected each time sum to 1. These rates can also be determined during the operation of the system as floating values, i.e. for example via the last number N of checked products.
[0027] If the set probability density function is parameterized by an estimation method, the maximum threshold value can be determined with the aid of the previously parameterized probability density function or a corresponding distribution function such that the probability of occurrence of the maximum value greater than or equal to the maximum threshold value corresponds to the corresponding set rate of erroneous detection of the largest anomaly in the image to be examined, or the probability of occurrence of the maximum value less than or equal to the maximum threshold value corresponds to the corresponding set rate of incorrect recognition of the largest anomaly in the image to be examined, in other words such that the area under the probability density function above the maximum threshold value or below the maximum threshold value corresponds to the corresponding set rate.
[0028] Accordingly, the minimum threshold value can be determined using a previously parameterized probability density function or a corresponding distribution function such that the probability of occurrence of the smallest value smaller than or equal to the minimum threshold value corresponds to a corresponding set rate of erroneous detection of the smallest anomaly in the image to be examined, or the probability of occurrence of the smallest value larger than or equal to the minimum threshold value corresponds to a corresponding set rate of incorrect recognition of the smallest anomaly in a non-compliant product. In other words, the minimum threshold value can be determined such that the area under the corresponding probability density function below the minimum threshold value or the area under the probability density function below the minimum threshold value corresponds to a corresponding set rate.
[0029] To determine these areas, it is of course also possible to use the integral of the probability density function, which is formed from a corresponding threshold value to the upper limit of the definition interval of the probability density function (e.g. infinity or a set upper limit value above which the value of the integral, i.e. the area, varies less than the set error boundary) or from the lower limit of the definition interval (e.g. minus infinity or a set lower limit value below which the value of the integral, i.e. the area, varies less than the set error boundary) to the threshold value.
[0030] Of course, instead of calculating the integral as explained above, the distribution function of the corresponding probability density function can be used, since the distribution function represents the value of the integral of the probability density function from minus infinity or from the lower limit of the defined interval of the probability density function to the corresponding threshold value. Thus, the main area under the probability density function, i.e. the value of the integral or the value of the distribution function, corresponds to the (theoretical) probability that the maximum or minimum value of the characteristic of each area of the image of the acceptable product is less than or equal to the corresponding threshold value. If one wishes to calculate the probability that the maximum or minimum value is greater than or equal to the threshold value, then for this purpose it is sufficient to subtract the probability calculated in this way from 1.
[0031] For this calculation, the values of the areas or integrals or distribution functions are set, and the corresponding inverse functions must be used. Such calculations can be carried out analytically or by means of numerical methods.
[0032] In this known method, each area can be defined by a base threshold value, adjacent pixels whose pixel value is greater than (or equal to) the base threshold value form a first group of areas, and adjacent pixels whose pixel value is less than the base threshold value form a second group of areas. The first and second groups of areas can also be combined into a single group. Yet another way to define each area is to use a predefined (geometric) mask. For example, a matrix mask can be used, which for example places a square grid (i.e. a chessboard grid) on the digital image, with all pixels forming an area inside the square. Naturally, the mask can also give rise to other subdivisions, and the entire image does not have to be divided into areas.
[0033] Each area is assigned a characteristic that can be represented by a value, which may in particular be a geometric characteristic such as the area, circumference or diameter of each area (if each area is at least approximately circular), or a pixel value characteristic represented by a value resulting from the pixel values of the area, for example the maximum or minimum value, the average value or the variance of the pixel values of a area.
[0034] Several characteristics can also be combined and these characteristics can be represented by a combined value. For example, the mean and the standard deviation can be added, and this information becomes a kind of confidence interval for the pixel value. The difference between the maximum and minimum values can also be used as a combined value that represents the brightness difference of the area. Since this measure is sensitive to outliers, quantiles can be used instead, for example the 10% and 90% quantiles as an alternative to the maximum and minimum. Furthermore, the quotient of the circumference and the area can be used to estimate how well a region is circular or how much it deviates from circularity. Alternatively, geometrical characteristics can be combined with pixel characteristics.
[0035] Thus, the known method offers the possibility of setting a rate, in particular the false rejection rate (or the corresponding complementary rate, i.e. the rate at which the image is correctly recognized as error-free), and depending on this determining a threshold value for the recognition of the minimum or maximum anomaly.
[0036] However, in practice, it is often desirable or necessary to know the actual detection rate, or at least its predicted value, in addition to the rejection rate. The actual rejection rate can be relatively easily determined in practice, even if it entails high costs, by a human individual inspection of the products that are actually rejected, whereas such an individual inspection is almost impossible for the detection rate. For this purpose, it is known, for example, to check images of actual rejected products as to whether the anomalies contained therein are recognized with sufficient certainty or probability by a known threshold value, i.e. with a sufficiently high detection rate. Summary of the Invention
[0037] Based on the prior art, it is an object of the present invention to provide a method for detecting anomalies in digital images of a product, which allows for a simple and cost-effective determination of at least one threshold for the detection rate of anomaly recognition, and further to provide an apparatus and a computer program product for implementing the method.
[0038] The invention solves this problem by means of the features of claims 1 to 14 and 15. Further preferred embodiments of the invention become apparent from the dependent claims.
[0039] The present invention is premised on the discovery that by producing digital images of multiple products of the same type (or a single product of the same type in the case of products produced in series, e.g. bulk products) containing at least one known anomaly caused by a known (physical or virtual) contaminant, it is possible to obtain information about, or determine an estimate of, the expected detection rate under active operation of the inspection equipment.
[0040] Homogeneous products are understood here to mean products that are produced within the scope of series production, such as, for example, yogurt portioned into cups or cheese packaged in bags or rigid cartons. When inspecting products of this type, it must be determined whether they contain undesirable impurities, such as dirt, metal particles, stones, bone chips, etc. In this case, an inspection device usually produces a digital image of the product, which is checked by suitable automated evaluation for the presence of anomalies caused by undesirable impurities.
[0041] For this purpose, the digital image is perceived as a single area or is subdivided into several areas, each area consisting of one or more adjacent pixels. For each such area, a value for at least one characteristic or a combined value for several characteristics of the area is determined. For example, an average brightness value (gray value) can be determined for the area. The presence of an anomaly is recognized if the value of the corresponding characteristic is greater than a set maximum threshold or greater than a set minimum threshold.
[0042] As already mentioned above, the maximum or minimum threshold value can be set directly or determined from a set value for the false discard rate (or a complementary rate) based on the method described in US Pat. No. 5,999,363.
[0043] According to the present invention, the following steps are performed in the inspection process:
[0044] ■ A plurality of digital reject images of real or virtual rejected products are generated, each having at least one known anomaly;
[0045] For each failed image, an area or areas are defined, and a value of at least one characteristic or a combined value of a plurality of characteristics of each area is determined, and the maximum of these values is determined as the maximum random sample value of the maximum random sample or the minimum of these values is determined as the minimum random sample value of the minimum random sample;
[0046] ■ The detection rate is determined for at least one known anomaly, which is
[0047] all open parameters of the probability density function to describe the maximum or minimum random sample are estimated (parameterized) using the maximum or minimum random sample value and using statistical estimation methods; and
[0048] by integrating a parameterized probability density function using set maximum or minimum thresholds as integration limits, or
[0049] ■ the detection rate is determined as the ratio of the number of values in the maximum or minimum random samples that are greater than or equal to a set maximum threshold or less than or equal to a set minimum threshold to the total number of values in the maximum or minimum random samples; and
[0050] ■ At least one known anomaly is assigned a detection rate.
[0051] The assignment of the detection rate determined in this manner to at least one known anomaly has the advantage that an operator of the inspection device in which the method is implemented, e.g. an X-ray inspection device, can immediately see the effect that a pre-created threshold value, or the false rejection rate (as a set quantity) on which the threshold value is based, has on the detection rate.
[0052] In this way, the threshold value or the false rejection rate can be selected in such a way that the detection rate of known anomalies in the images of acceptable products that should actually be inspected by the inspection device for the presence of unknown anomalies during practical operation meets as far as possible the economic and reliability requirements, since both parameters must often be taken into account reciprocally. Naturally, a false rejection rate of 0% and a detection rate of 100% is desirable, but this is hardly or not feasible in practice. From an economic point of view, the lowest possible false rejection rate is desirable. This is achieved by a threshold value as high as possible for the recognition of the largest anomaly, which in turn reduces the detection rate or the probability of a valid detection of the largest anomaly. If the impurities to be detected are dangerous objects, for example metal or bone fragments in food, the detection rate must be correspondingly high, for example 99% or more, in order to avoid as far as possible compensation claims of end consumers who are injured when eating food containing such impurities. On the other hand, a too high rejection rate can be economically very disadvantageous to the product manufacturer if the cost of manually re-inspecting the incorrectly rejected test products becomes high and therefore cost intensive.
[0053] The invention makes it possible to easily and quickly ascertain the false rejection rate on the one hand and the detection rate on the other hand by setting only one parameter, i.e. directly by the respective maximum or minimum threshold value or false rejection rate (or a complementary rate) such that it meets the requirements related to economy and / or reliability.
[0054] Naturally, in practice this method will produce more valid results the better the match between known anomalies and actual anomalies, or the better the match between known (physical or virtual) contaminants and actual contaminants.
[0055] In an embodiment of the present invention, the maximum and minimum thresholds are determined in a learning process, and the following steps are performed:
[0056] Multiple digital images of a passing product that does not contain anomalies, or a passing process product that is substantially free of anomalies, are generated or utilized, the number of images being set or determined during the learning process;
[0057] ● for each digital image an area or areas are defined, a value of at least one characteristic or a combined value of several characteristics of each area is determined, and the maximum of these values is determined as the maximum random sample value of the maximum random sample and / or the minimum of these values is determined as the minimum random sample value of the minimum random sample,
[0058] Using a statistical estimation method, estimates of all pending parameters that do not have a probability density function set to represent the maximum random sample value are determined using the maximum random sample value and / or estimates of all pending parameters that do not have a probability density function set to represent the minimum random sample value are determined using the minimum random sample value;
[0059] a first rate at which the largest anomaly is erroneously detected in the image to be examined or a second rate at which the largest anomaly is not correctly recognized in the image to be examined is set, and / or a third rate at which the smallest anomaly is erroneously detected in the image to be examined or a fourth rate at which the smallest anomaly is not correctly recognized in the image to be examined is set,
[0060] the maximum threshold is determined using a parameterized probability density function or a corresponding distribution function, whereby the probability of occurrence of a maximum value greater than or equal to the maximum threshold corresponds to a set first rate, or the probability of occurrence of a maximum value less than or equal to the maximum threshold corresponds to a set second rate, and / or
[0061] The minimum threshold is determined using a parameterized probability density function or a corresponding distribution function, whereby the probability of occurrence of a minimum value less than or equal to the minimum threshold corresponds to a set third rate, or the probability of occurrence of a minimum value greater than or equal to the minimum threshold corresponds to a set fourth rate.
[0062] This is essentially the method described in US Pat. No. 5,399,633, where a false rejection rate or a complementary rate is set. The advantage of this method compared to the direct setting of a threshold can of course be seen in the fact that here one of the quantities that is important in practical work, namely the false rejection rate, is set and not an arbitrary threshold value, the influence of which on both the false rejection rate and the detection rate cannot be easily predicted, is set first.
[0063] In an embodiment of the invention, the characteristics of the area are determined from the position information of the pixels of the area, in particular the area, circumference or diameter, and can also be determined from the values of the pixels of the area, in particular the maximum or minimum value, the average value, the variance or the standard deviation of all the pixels of the area.
[0064] In practical operation, the quantity used as the characteristic of the area is the quantity which, when used during practical operation of the inspection device or during practical application of the method according to the present invention, allows the (unknown) impurity to be detected or the (unknown) anomaly caused by it to be detected with the highest degree of reliability.
[0065] If the maximum or minimum threshold is not set directly but is determined from the set mis-rejection rate, the required probability distribution to be set for this purpose can be selected or used as the generalized extreme value distribution, in particular its special cases, the Gumbel distribution, the Weibull distribution or the Fréchet distribution, since the generalized extreme value distribution is particularly suitable for representing the probability distribution of extreme values that occur in reality.
[0066] In an embodiment of the invention, a digital fail image required for the inspection process is generated such that at least one set anomaly is present at a set location within the fail image, and this information, and optionally also the geometry of the impurity causing the at least one set anomaly, can then be utilized for the determination of a maximum or minimum random sample value representative of the set anomaly.
[0067] For example, the impurity generating the at least one set anomaly can be positioned on or in the acceptable product to be inspected (which becomes an unacceptable product after the impurity is applied on or in) such that it is located - in an image plane perpendicular to the transmission illumination direction - in the upper right quarter of the corresponding digital image. The detection of the respective maximum or minimum random sample value can then be limited, for example, to this upper right quarter only. This reduces the probability of determining random sample values in areas outside the set anomaly in the digital image that are greater or smaller than the maximum or minimum random sample value actually caused by the anomaly or the corresponding impurity.
[0068] The area actually used for the image evaluation or the image portion actually used can also be determined using a base threshold value. This can determine, for example, all areas of the image with a pixel value greater than or equal to a set base threshold value. It can then be checked whether there is a sufficiently large overlap between the area thus determined and the known geometrical outer dimensions of the corresponding foreign object (or its geometrical outer dimensions in the projection onto the image plane). For example, it can be checked whether the foreign object is located within the area thus determined with its entire projection surface or with a percentage that exceeds a predefined overlap threshold value.
[0069] As already mentioned above, in embodiments of the present invention, a pass product containing no anomalies can be utilized to generate a fail image, and at least one contaminant is provided on or within the pass product, thereby resulting in a fail product having at least one known contaminant from the pass product. Additionally, the at least one contaminant may be provided on or within a support attached to or disposed within the product.
[0070] In another embodiment of the invention, the digital image data of the accepted product can be used to generate the rejected image, and the digital image data of the accepted product is digitally converted for the inclusion of at least one impurity using known material and geometric properties of at least one defined impurity. In other words, the known anomaly is "incorporated" or "faded in" into the digital image of the accepted product. This is particularly suitable if the impurity consists of a single material and has a simple geometric structure, for example a sphere, a cuboid, a cube, etc. The "inclusion" can be performed in such a way that in an inspection device which transmits the product to be inspected, an additional attenuation is determined using the known material-dependent intrinsic attenuation of the radiation used for the inspection, and this is taken into account in addition to the attenuation represented by the image data of the accepted image. This can be performed by a simple addition when a logarithmic scale for the attenuation is used.
[0071] The support for at least one impurity may be in the form of a plate or a card. A number of impurities may be arranged in or on the support, which have the same material properties, in particular consist of a single material, have the same type of geometry, but have different sizes. In this way, a reject image with a number of known anomalies can be generated. The evaluation can then be carried out in such a way that only one determined area (in the manner described above) or only a specific part of the reject image is used to determine the corresponding maximum or minimum threshold value.
[0072] In an embodiment of the present invention, the detection rate may be output to a display device as a numerical value or as a quantity or identifier associated therewith.
[0073] The detection rate can then be assigned to at least one graphic representation of the contaminant and / or assigned to the corresponding failed image and displayed. For example, if known anomalies caused by spheres of different sizes of the same material are used (whether these anomalies are generated by physical contaminants or by "counting"), the corresponding value of the detection rate can be displayed for each representation of the spheres with the corresponding material designation. Alternatively or in addition to the display of the contaminant, a pass image or an image part containing the anomaly can also be displayed.
[0074] In another embodiment of the invention, the maximum or minimum threshold for recognition of the maximum or minimum anomaly or the false rejection rate (or the complementary rate) may be modifiable by input means, e.g. a keyboard, or by adjustment means, e.g. a slider or a rotary knob (embodied physically or digitally). Furthermore, the latest value of the detection rate can be displayed on the display device for each latest value of the maximum or minimum threshold or of the rate, as applicable.
[0075] Such an approach allows the operator to develop a feel for how and with what certainty a known contaminant or anomaly can be recognized.
[0076] Instead of displaying the detection rate as a numerical value, it is also possible to use, for example, a color code, which is then evaluated by a classifier: for example, red can be selected for a detection rate that is too low (e.g. below 50%, depending on the respective situation), yellow for a detection rate that is still acceptable (e.g. above 50% and below 90%) and green for a high detection rate that ensures a sufficiently reliable detection of anomalies (e.g. >90%).
[0077] A data processing device suitable for the apparatus according to the invention can have a processor with suitable input / output interfaces in the usual way. The processor can be configured as a dedicated processor for industrial image processing, for example. Of course, the processor can also be embodied by a combination of a conventional processor and a dedicated image processor. The entire data processing device can be embodied as an independent CPU unit with corresponding interfaces or as a slot CPU.
[0078] It should be pointed out here that all the features of the method or device according to the above-described US Pat. No. 5,399,633 can also be applied in the context of the present invention.
[0079] The invention will now be described in more detail with reference to an embodiment shown in the drawings, in which: [Brief description of the drawings]
[0080] [Figure 1] 1 shows a schematic diagram of an X-ray examination apparatus comprising an apparatus for carrying out the method according to the invention; [Diagram 2] 1 is a graph showing an empirical frequency distribution of a random sample of maximum values of a property of a predetermined area in a digital image of a pass product, as well as a probability density function fitted thereto. [Diagram 3] 11 is a graph showing probability density functions for minimum and maximum values of a characteristic determined using digital images of a passing product (or a passing process product), respectively. [Figure 4] 1 is a graph showing a probability density function (curve (a)) for the maximum value of a characteristic determined using digital images of a passed product (or a passed process product) and a second probability density function (curve (b)) for the maximum value of the characteristic determined using digital images of failed products, each having a known anomaly. [Diagram 5]Graphs similar to FIG. 4, but with curves (a) and (b) having a larger spacing on the horizontal axis, allowing for a higher detection rate with a simultaneously lower false reject rate. [Figure 6] FIG. 1 shows the display of a display device, showing six identical impurities of different sizes for three different materials, with each impurity assigned a detection rate value. [Figure 7] FIG. 7 shows a display similar to that of FIG. 6, in which each contaminant is assigned a smiley symbol representing its detection rate. [Figure 8] FIG. 7 shows a display similar to that of FIG. 6, in which only the contaminant sizes that are detectable with a detection rate of greater than 95% are displayed for three different materials, respectively. [Figure 9] FIG. 7 shows a display similar to that of FIG. 6, in which an adjustment means for the false rejection rate is additionally provided in the form of a digital slider. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0081] 1 shows a schematic representation of an X-ray inspection apparatus 100 having a device 102 for detecting anomalies in a digital image, which is arranged for carrying out the method described below. The X-ray inspection apparatus 100 is only one possible example of how a digital image may arise which may contain anomalies. The method according to the invention can be applied to any digital image which is to be inspected for the presence of anomalies.
[0082] As already explained above, the digital image typically represents a product having an anomaly to be detected. In the case of the X-ray inspection device shown in Fig. 1, a product 104 in the form of an individual item is inspected by way of example. However, it is equally possible to generate digital images of any other product, for example a bulk product. In such a case, it is preferred that digital images are generated which each represent an area of the bulk product.
[0083] In the X-ray inspection apparatus 100 shown in Fig. 1, the product 104 to be inspected is transported by a transport device 106 along a set transport path (indicated by the arrow F). The transport device 106 here comprises a number of conveyor belts 108, 110, 112, 114. The conveyor belt 108 serves for the entrance of the product 104 and the conveyor belt 114 serves for the exit of the product. The inspection apparatus 100 comprises a shielded housing 116 in which the conveyor belts 110 and 112 are arranged, as well as an X-ray radiation source 118 and an X-ray radiation detector 120. The X-ray radiation source 118 generates an X-ray beam 121 which has a sector shape perpendicular to the plane of the paper and a small width in the transport direction. The X-ray beam 121 passes through the gap or free space between the facing end faces of the conveyor belts 110 and 112 and then impinges on an X-ray radiation detector 120 arranged below the conveyor belts 110, 112 in the embodiment of the X-ray inspection apparatus 100 shown in FIG. 1. The X-ray radiation detector 120 has a width in a direction perpendicular to the plane of the paper that corresponds to the maximum width of the products to be inspected. Typically, the width of the X-ray radiation detector is selected to be approximately as large as the width of the conveyor belts 110, 112. The X-ray radiation detector 120 may be configured as a line detector with one or more detector lines in a direction perpendicular to the plane of the paper, each detector line having a set number of pixels.
[0084] The device 102 for detecting anomalies comprises an image processing unit 122 to which the signals of the X-ray radiation detector 120 are supplied. The image processing unit 122 may be configured as a conventional computer unit with one or more processors, a working memory, possibly a hard disk memory or a SSD memory, and suitable interfaces for the supply of the signals of the X-ray radiation detector 120 and for the supply and / or reception and / or transmission of other data. Furthermore, the device 102 comprises a display unit 124 on which information generated by or supplied to the image processing unit 122 can be displayed.
[0085] As shown diagrammatically in FIG. 1, the product 104 to be inspected moves through a fan-shaped X-ray beam 121, whereby the X-ray radiation detector 120 configured as a line detector generates corresponding digital image signals, which are fed to an image processing unit 122. The image processing unit 122 can be configured in such a way that a digital image is first generated from the image signal, which in the case of individual items comprises the entire product 104 or at least one set portion. This portion can be selected by the usual methods of image processing or pattern recognition. From the digital image thus generated, the image processing unit 122 can also subject the final digital image to be inspected for the presence of anomalies in the manner described below to further image processing, such as, for example, digital filtering, which is selected in such a way that the anomalies to be detected are better recognizable, in particular stand out better with respect to the remaining image. Such an image preprocessing can also include any other image processing steps, such as, for example, noise suppression. At the end of such an image preprocessing type (not necessarily), a digital image is generated which is then to be inspected for the presence of anomalies.
[0086] As explained at the beginning, it is known to use a threshold value for pixel values to detect anomalies. In this case, the presence of an anomaly is detected if a pixel value or a group of adjacent pixels containing a set minimum number of pixels is above the threshold value. If one or more anomalies are detected in a digital image, this information can be used to take action regarding the corresponding products, for example to derive them from the product stream. Alternatively or additionally, the products in question can be marked physically or virtually, i.e. by assigning corresponding data.
[0087] The display unit 124 can be used to display desired information, for example the digital image to be examined and possibly the anomalies detected therein, the thresholds applied in each case, the type of probability density function selected, the corresponding parameters, the quality of the fitting of the probability density function to the random samples, the confidence interval for the thresholds, the parameters or rates (in particular the false rejection rate). The output can of course be in the form of data (numerical values) and / or graphics. If the detection is performed in the normal working mode of the device, a graph can also be displayed which represents the evolution of the current rejection rate over time during the (normal) working operation of the device 102 (or during the learning process), and the set rate (for example the false rejection rate) and the confidence interval for the set rate. Furthermore, a list with random sample values and / or a graphic similar to that of FIG. 2 can also be displayed. Furthermore, displays such as those shown in FIGS. 6 to 9 can also be displayed, i.e. information on the set detection rate of anomalies generated by physical or virtual impurities. The corresponding data can of course also be output to a higher-level unit and / or saved.
[0088] The image processing unit 122 can be supplied with one or more start thresholds that have already been determined for the product type to be subsequently inspected by the method described below, or it can be supplied with other information that is necessary for the implementation of the method described below, such as information regarding the type of probability density function that is set to be used and information regarding how each area is defined in the digital image to be inspected (see below).
[0089] As explained above, the below described method for detecting anomalies in a digital image is not limited to checking whether one or more pixel values are above a set maximum threshold or below a set minimum threshold. Rather, the below described method can be generalized such that a maximum and / or minimum threshold is determined for any characteristic of each predefined area in the digital image to be inspected in order to define or determine a maximum or minimum threshold.
[0090] For this purpose, in the digital image to be examined, firstly areas must be defined, to which the values of the set characteristics can respectively be assigned. The areas can be defined, for example, in such a way that a base threshold value is used, in which case all pixel values that are equal to or exceed the threshold value form a first group of areas, and the remaining pixel values form a second group of areas. Depending on the characteristic to be examined, it may be sufficient to process only the first or second group further in each case. This is the case, for example, if, as a characteristic of the area, only the maximum or minimum pixel value or the average pixel value of the area is evaluated for the recognition of anomalies.
[0091] However, it is also possible to assign to each zone one or more properties that are not only represented by individual pixel values. For example, a zone can be assigned a geometric property, such as area, circumference, diameter (at least in the case of approximately circular zones) or deviation from a circular shape. In such cases, an anomaly is recognized if the value of the property in question is above a maximum threshold value set for it or below a minimum threshold value set for it. In such a general case, the required maximum or minimum threshold value can be determined by the method described below.
[0092] Another option for determining the area in the digital image to be examined consists in making use of a set geometric mask which is superimposed on the image, which can for example be a mask of set dimensions consisting of adjacent squares of the same size.
[0093] Again, each area thus defined can be assigned one or more characteristics, such as the variance or standard deviation of the pixel values contained within the area, a corresponding average value, or the maximum or minimum values contained therein.
[0094] It is also possible to determine a combined value for each area, determined from two or more values for different characteristics. For example, in particular by arithmetic operations, for example by (weighted) addition, values for the variance and standard deviation of the pixel values and the mean value can be combined to form a combined value, which in this special case represents a kind of confidence interval, if the pixel values of the area are to some extent normally distributed. Another example for a combined value or a combination of characteristics of each area is the difference between the maximum and minimum pixel values contained therein. In order to be more robust against outliers, quantiles can also be used as an alternative to the maximum and minimum values, for example the 10% and 90% quantiles.
[0095] In the following, it will be explained how the corresponding maximum or minimum threshold value can be determined with a smaller number of digital images. As mentioned above, such a learning process can be carried out with a qualified product or a qualified process product. The learning process can be carried out, for example, at the start-up of a facility for manufacturing or processing products, which includes an inspection device of this kind. In this case, it is generally necessary to determine the threshold value for each product type.
[0096] Of course, the threshold values for a particular product type can be saved, so that the learning process does not have to be carried out anew each time, unless a change in the type of product to be inspected has been made.
[0097] Moreover, it is possible to carry out the learning process during ongoing equipment operation, at least if it can be assumed that this equipment produces acceptable process products, i.e. products that are predominantly acceptable (without anomalies), since in such a case, as explained above, the unacceptable products included in the acceptable process products are excluded as outliers when creating the required random samples.
[0098] The main requirement of the below described method for determining the threshold value is that the rate at which the maximum anomaly is falsely detected in the image to be inspected (hereinafter also referred to as the false positive rate or the false reject rate) or the rate at which the maximum anomaly is not correctly recognized in the image to be inspected (hereinafter also referred to as the true negative rate) is set in the case where a pass product (containing unknown anomaly information) is used. That is to say, it is not necessary to determine the threshold value in a first step and check in a second step whether the use of the threshold value leads to a correspondingly acceptable rate. However, for the determination of the below described detection rate for known anomalies, it is also possible in principle to directly specify the false reject rate (false positive rate) or the complementary true negative rate. However, in this case, a significantly larger number of digital images of pass products is required to select the threshold value so that the desired false reject rate is achieved.
[0099] The required number of digital images, i.e. the number of successful products or successful process products, can be predefined, in which case this number must be selected so large that the threshold is determined with the desired degree of confidence. However, it is also possible to determine the required number of digital images during the learning process (whether the images are generated during the learning process or are already available before the start of the learning process). In this case, it is preferable to start with a minimum number of digital images and increase this minimum number stepwise by one or more digital images each time until the determined threshold is sufficiently reliable, i.e. until the set rate is sufficiently reliably adhered to. This can be checked by determining a confidence interval.
[0100] In the next step, as in the detection of anomalies in normal operating mode (i.e. outside the learning process), zones are defined for each digital image and for each zone the value of at least one characteristic is determined, which is respectively assigned to the zone. The maximum and / or minimum value of the respective characteristic or of the corresponding characteristics is then determined for each digital image and assigned to the corresponding random sample. This can be done, for example, by storing all minimum values in a minimum list and all maximum values in a maximum list. It should be noted here that it is of course not necessary that both options of the method are applied at the same time. When maximum anomalies are detected in normal working operation, i.e. when anomalies that are recognized as anomalies when the maximum threshold is exceeded, it is of course necessary to determine only the maximum threshold. The same also applies in the case where only minimum anomalies are to be detected.
[0101] Once a sufficient number of random sample elements (maximum or minimum values for at least one characteristic or combination of characteristics) has been determined, the next step is to parameterize the probability density function for the maximum or minimum list, respectively, using statistical estimation methods, which corresponds to fitting the probability density function to the empirical frequency distribution of the random sample in question.
[0102] For this purpose, the random samples can be binned, i.e. the random sample elements are assigned to adjacent intervals of equal width in the corresponding value range. A corresponding set probability density function can then be fitted to this empirical (relative) frequency distribution, for example by using the least squares method. However, usually more suitable statistical estimation functions can be used for this purpose, for example the maximum likelihood method or the method of moments.
[0103] A type of probability density function that must be set for the implementation of the method should be selected that is assumed to be able to well represent the random samples when the parameterization is performed. In the present case, since extreme values for the characteristic in question are selected to form the respective random samples, a type of generalized extreme value distribution or a generalized extreme value distribution (with its three parameters) that brings together the Gumbel, Weibull and Frechet distributions is often selected. The frequently used Gumbel distribution has the following form:
number
[0104] where f represents the value of the probability density as a function of the random quantity x, each value of which, in this case, represents the value of the corresponding characteristic or a combination of values, and the parameters μ and β are determined by the selected statistical method.
[0105] Figure 2 shows a graph that shows a schematic representation of the fitting of a probability density function f(x) to the relative empirical frequency distribution of maximum pixel values, where x represents the pixel value. On the horizontal axis, pixel values are plotted, and on the vertical axis, probability density values or relative frequencies are plotted.
[0106] The parameterized probability density function thus allows the desired threshold value (for the rate of erroneous detection of the largest anomaly in the image to be examined or for the rate of incorrect recognition of the largest anomaly in the image to be examined) to be determined in a further step. For this purpose, the threshold value is defined such that the area under the parameterized probability density function above the threshold value is equal to the set rate of erroneous detection of the largest anomaly in the image to be examined or the area under the parameterized probability density function below the threshold value is equal to the set rate of incorrect recognition of the largest anomaly in the image to be examined, since this area corresponds to the probability that a maximum value of the corresponding characteristic occurs in the digital image that is greater than or equal to the threshold value. The threshold value can be determined by converting the formula from the inverse of the distribution function belonging to the probability density function to be parameterized in cases where a closed-form solution exists for the inverse function. Otherwise, the calculation of the threshold value can be performed by suitable known numerical methods.
[0107] Figure 2 shows the X ratios corresponding to 1.0%, 0.6%, and 0.1% false-positive rates. so,1 ,X so,2 and X so,3 is plotted, i.e., the area under the probability density function to the right of these thresholds yields thresholds of 0.01, 0.006, and 0.001.
[0108] When the true negative rate is set instead of the desired false positive rate, the upper threshold X so The area under the probability density function below should be utilized for the calculation of the maximum threshold in a similar manner.
[0109] In Fig. 3, a graph similar to Fig. 2 is shown, but only the evolution of the already fitted probability density function is shown. In this example, the maximum threshold X so and the minimum threshold X su are both determined with reference to a parameterized probability density function. Then, in both cases, i.e., for the maximum and minimum threshold determination, the same false positive rate R fp is set, and the minimum threshold X su The false positive rate R fp is the minimum threshold X su It is equal to the area under the probability density function below the true negative rate R rn is the minimum threshold X su The area under the probability density function above the threshold X is the area under which the smallest anomaly is detected. su This is because it is smaller than
[0110] In the case of the parameterized probability density function shown in FIG. 3, a digital image is detected as containing an anomaly if, for at least one zone, a value of the property of interest, or a combination of values of the properties in question, occurs that is outside the pass region between the minimum and maximum thresholds (or in other words, if the value in question is greater than or equal to the minimum threshold value X su is less than or equal to the maximum threshold X so is greater than or equal to.
[0111] Furthermore, in Figure 3, the maximum threshold X so The confidence intervals for are plotted, where for simplicity we use the limit X so -ΔX so and X so +ΔX so In order to reduce the false rejection rate to a desired set value with a higher degree of certainty, the determined threshold value X so From ΔX soIn a similar manner, a secondary maximum threshold value that is larger by X can be used. su Therefore, the lower threshold X su The width of the confidence interval for ΔX su In other words, the secondary maximum threshold corresponds to the upper limit of the confidence interval for the maximum threshold, and the secondary minimum threshold corresponds to the lower limit of the confidence interval for the minimum threshold.
[0112] As explained above, the number of digital images for the determination of one or a threshold (maximum or minimum threshold) may be fixed, but this number will have to be selected to be large enough to determine a sufficiently reliable threshold.
[0113] The method is suitable for detecting anomalies in images of a product, which may be generated in any manner and which represent any characteristic of the product. In particular, the method is suitable for analysing images obtained by an inspection device, which are produced using X-ray or terahertz radiation and which are thus able to generate information about the interior of the product. The method can be implemented in the inspection device, in particular through software. The results of the detection method can be used to control other devices, for example sorting devices.
[0114] Once the desired maximum or minimum threshold value has been determined in the manner described above, another method is then followed, described below, which is carried out in a separate inspection process, which may in particular be carried out prior to the start of production operation of a facility that includes X-ray inspection apparatus 100 (or any configured inspection apparatus).
[0115] It should be pointed out again explicitly that the determination of one or more desired threshold values in the manner described above is not a necessary prerequisite for the implementation of the inspection process described below, but rather the desired threshold values can also be set directly, for example by an operator or by retrieving the threshold values from a memory device in which the desired maximum and / or minimum threshold values assigned to the product type are stored.
[0116] The inspection process includes the following steps:
[0117] First, a number of digital reject images each having at least one known anomaly are generated, which can be generated using one or more physical rejected products each containing one or more known contaminants, which generate corresponding known anomalies in the digital image of the corresponding rejected product (reject image). Reject products of this kind are generated from the accepted products, into which one or more known contaminants are inserted, preferably at specific positions of the accepted products. In the reject images generated by the rejected products thus generated, it is possible to search for maximum or minimum extreme values caused by the corresponding known anomalies, which are used as maximum or minimum random sample values for determining the detection rate using the method described below, only in appropriately defined areas of the digital image containing the known anomalies or where the corresponding contaminants are located (regardless of whether the contaminants generate detectable anomalies in the digital rejected image or not).
[0118] The definition of the area to be considered of the digital image can be done in various ways: for example, the geometric shape of the area can be defined (square, circle, etc.) and then the extent can be selected so that the contamination or the anomaly caused by it is completely inside the area.
[0119] Alternatively, a base threshold value can be used, and all areas of the rejected image are determined (in the manner already described above) in which the pixel value for determining the maximum anomaly is greater than (or greater than or equal to) the corresponding base threshold value, or the pixel value for determining the minimum anomaly is less than (or less than or equal to) this. Of these areas, the areas containing the impurity or the anomaly caused by this can be used. Using information about the location in the rejected product or in the corresponding rejected image and information about the geometry of the corresponding impurity, it can be additionally checked whether the impurity or the known anomaly is covered to a sufficient extent by the area thus determined. Compliance with this requirement can be confirmed if the geometric area of the impurity in the rejected image or the respective faces of the known anomaly caused by this sufficiently overlap with an overlap degree of, for example, more than 50% or more than 80%, where the overlap degree can be the ratio between the area of the impurity or the known anomaly caused by this in the digital rejected image and the area of the corresponding area previously determined.
[0120] However, the generation of the rejected images can also be performed using at least one previously generated accepted image, i.e. at least one digital image of a accepted product, preferably of the same type as the product to be inspected in the production area of the respective inspection device 100. In practice, instead of a single accepted image, multiple accepted images are preferably used in order to take into account information about the statistical distribution of the properties of the accepted images. Such accepted images can already be generated during the learning process described above (wherein the threshold is determined under the setting of the false rejection rate) and used to create the "artificial" rejected images. For this purpose, one or more accepted images can of course be stored in a storage device.
[0121] To generate such an "artificial" reject image, firstly a virtual impurity can be defined, and a predetermined material and geometry of the virtual impurity are specified. As geometry, in particular simple shapes can be used, such as the shape of a sphere, a cube or a rectangular prism. The material of the impurity can be taken into account by taking into account the intrinsic attenuation of the material, in particular in the case of inspection methods in which the product in question is through-irradiated. For X-ray inspection, data banks or tables are available for this purpose, which contain the intrinsic attenuation of a large number of materials. Intrinsic attenuation is understood here to mean the attenuation that the material undergoes per length unit (seen in the direction of through-irradiation) when the corresponding radiation is through-irradiated. In this way, such additional attenuation can be included in the existing image data of the pass image, which is used to generate the "artificial" reject image, at the corresponding position in a simple manner. If the image data of the pass image is present on a logarithmic scale, the inclusion of the additional attenuation, which is also present on a logarithmic scale, can be carried out by addition.
[0122] Regardless of whether physical or virtual impurities are used to generate the digital rejection images, their location relative to the direction of transmitted illumination of the product to be inspected (or generally relative to the direction of observation of the product), with the exception of the spherical case, must also be specified. To take into account the additional attenuation generated by each impurity, the impurity (or more precisely its projection in a plane perpendicular to the transmitted illumination direction) is divided into pixels (where the pixel division preferably corresponds to the existing pixel division of the pass image). In this way, the additional attenuation can be taken into account for each individual pixel (i.e. pixel by pixel). The "artificial" rejection images thus generated can then be processed in the same way as rejection images generated by physical impurities.
[0123] As already alluded to above, for a sufficient number of failed images thus generated, a corresponding maximum or minimum random sample value is determined for each set anomaly contained in the failed images (depending on whether it is the maximum or minimum anomaly), which can be done in a similar manner as described above for determining the maximum or minimum threshold value.
[0124] From the thus generated maximum and minimum random samples, the detection rate can then be determined. For this purpose, in a first embodiment, a probability density function can be set, which has at least one undetermined parameter that is not set. This parameter can then be determined in an appropriate manner, in particular using statistical estimation methods, so that the parameterized probability density function best describes the respective random sample. This parameterization can be performed in a similar manner as explained above in connection with the determination of the maximum and minimum thresholds.
[0125] Figure 4 shows a graph with two curves representing the probability density function parameterized as described above using the generalized extreme value function. Curve (a) shows the probability density function parameterized selected to determine the maximum threshold using the accepted product (or accepted process product) in the manner described above. From this curve, the false reject rate or false positive rate R fp Under the setting of the value of, the corresponding threshold X so is determined.
[0126] And the threshold value X thus determined so Using this, an estimate of the detection rate that would occur under inspection of the same type of product when the inspection apparatus 100 is operated in production can be determined from the second curve (b). As already mentioned above, this estimate will be more valid the closer the actual contaminants match the known contaminants used to determine the corresponding maximum and minimum random samples.
[0127] Curve (b) of FIG. 4 shows the probability density function of the maximum random sample of known anomalies, selected and parameterized in the manner described above. From this probability density function, the probability of detection of the corresponding known anomalies that will appear during the inspection of the same type of product can be determined. To do this, the probability density function is set to a predetermined maximum threshold value X so One need only integrate from 0 to infinity (or up to a very large value, e.g., up to the maximum possible pixel value provided by the radiation detector). This integral corresponds to the true positive rate, or detection rate, for detection of the known anomaly in question.
[0128] The detection rate thus known can be evaluated manually or automatically as to whether the corresponding value is acceptable for practical use, especially taking into account the consequences that would result if the contaminants were not detected. For example, if foodstuffs such as yogurt, cheese, chocolate, cookies, etc. are inspected and undetected metal or bone fragments are found in them, this could lead to life-threatening injuries for the end consumer who consumes such contaminated food. In such cases, the detection rate must meet correspondingly strict settings, for example higher than 99%. In other cases, where non-detection has less serious consequences, the detection rate can be adjusted to clearly lower values of 80% or 90%.
[0129] Since the detection rate and the false rejection rate or the set maximum or minimum threshold are mutually dependent, in many cases it is not possible to simultaneously achieve a desired low false rejection rate and an equally desired high detection rate. Such a case is shown in Figure 4. As is clear from this graph, a high maximum threshold X so or low false rejection rate R fp This inevitably leads to a dramatically reduced detection rate R rp leads to.
[0130] In contrast, FIG. 5 shows another case in which the largest known anomaly is clearly better revealed in the failed image. Here, curve (b) is clearly displaced to the right, i.e., towards larger pixel values, compared to curve (b) in FIG. 4. Accordingly, the same maximum threshold X so or the same discard error rate R fp In the case shown in Figure 5, the detection rate R rp leads to.
[0131] Both cases shown in Figures 4 and 5 can occur for known anomalies caused by contaminants of the same size but different materials, for example, as well as when the contaminants are of the same material, but the contaminant in the Figure 4 case is significantly smaller than the contaminant in the Figure 5 case.
[0132] Instead of using a probability density function describing the maximum or minimum random sample to determine the detection rate, the respective random sample values themselves can also be used. In this case, the detection rate can be determined by determining the number of random sample values that are greater than or equal to a predefined threshold. The detection rate is then obtained by dividing the number of random sample values thus determined by the total number of random sample values.
[0133] Even in such a case, it is not necessary to determine the relevant threshold from a pre-set false discard rate, but rather it is equally possible to set the threshold directly.
[0134] FIG. 6 shows a display of the display unit 124 of the X-ray inspection apparatus 100 showing the results of the inspection process carried out as described above.
[0135] As is evident from this figure, the detection rates for spherical impurities with different diameters, namely 0.5 mm, 1.0 mm, 1.5 mm, 2.0 mm, 2.5 mm and 3.0 mm, have been determined for three different materials, namely aluminum, ceramic and glass. The numerical values of the detection rates are assigned to the corresponding schematic representations of the respective impurities and displayed. Such a representation gives the operator a direct estimate of the detection rates for the various materials and sizes of the impurities. As is evident from FIG. 6, a spherical impurity made of aluminum with a diameter of, for example, 0.5 mm is still recognized with a detection rate of 13%, whereas a correspondingly sized impurity made of ceramic or glass is no longer recognized. An impurity made of aluminum or ceramic with a diameter of 2.5 to 3.0 mm is still recognized with 100%, whereas a corresponding impurity made of glass can only be detected with a detection rate of 76% to 93%.
[0136] Figure 7 shows another embodiment of the display which corresponds substantially to the diagram of Figure 6, except that now instead of numerical values of the detection rate, classifiers are used, where detection rates below 80% are classified as poor, detection rates between 80% and 90% (including the borders) are still acceptable, and detection rates >90% are classified as good. The three different classifier values are represented by smiley faces. In a similar manner, a color code could be used, for example green for "good" and yellow for "acceptable" and red for "poor". Such classifiers allow the operator to more quickly interpret the display or results of the inspection process.
[0137] 8 shows yet another embodiment of the display unit 124, in which the sizes of spherical contaminants of different materials are displayed, for which the good classification detection rate is further evident. In the illustrated case, still good detectability is achieved for spherical contaminants of aluminum when the sphere diameter is greater than or equal to 2.5 mm, for ceramics greater than or equal to 2.0 mm, and for glass greater than or equal to 3.0 mm, again assuming a gradation of the sizes of contaminants of the same material in steps of 0.5 mm when the inspection process is carried out.
[0138] 9 shows yet another embodiment of a display of the display unit 124, which corresponds substantially to the display shown in FIG. so The adjustment means is configured as a slider 124a in the illustrated embodiment, which can be realized in a simple manner by utilizing a touch display for the display unit 124.
[0139] In the initial state, the slide member 124b of the slider 124a is in an initial position, where the initial threshold value X so,0 are preset. In this initial state, the detection rate can be determined and displayed for each size and each material of the impurities. When the operator determines that the detection rate is too low for a particular size and / or a particular material, he can slide the slider in the direction of a higher threshold value. A new detection rate can then be calculated and displayed immediately after changing the position of the sliding member 124b. In this way, the operator can easily and quickly change the threshold value so that the detection rate is optimal for the respective case.
[0140] Of course, instead of the threshold value the false rejection rate can also be adjusted or changed by an adjustment means, in which case the operator has direct information about the possibly strong dependency between the false rejection rate and the detection rate and can adjust the false rejection rate, for example, so that a (still) satisfactory detection rate results. [Explanation of symbols]
[0141] 100 X-ray inspection equipment 102 Device for detecting abnormalities 104 products 106 Transport Device 108 Conveyor Belt 110 Conveyor Belt 112 Conveyor Belt 114 Conveyor Belt 116 Shielded Housing 118 X-ray radiation source 120 X-ray radiation detector 121 X-ray beam 122 Image Processing Unit 124 display units 124a Slider 124b Slide member F Conveying direction X su Lower threshold (determining threshold using a qualified product or a qualified process product) Y su Lower threshold (determining threshold using failed product or failed process product) X so Upper threshold (determining the threshold using a qualified product or a qualified process product) Y so Upper threshold (determining threshold using failed product or failed process product) ΔX so Width of the confidence interval for the maximum threshold ΔX su Width of the confidence interval for the minimum threshold R fp false positive rate R rn true negative rate R rpTrue positive rate R fn False negative rate
Claims
1. 1. A method of detecting anomalies in digital images of a product, each digital image being formed by a number of pixels represented by image data, each pixel representing an assigned location of a corresponding image and having a value characterizing the corresponding location; (a) each image to be inspected is perceived as a region or is subdivided into two or more regions, each of which is comprised of one or more adjacent pixels; (b) determining a value for at least one characteristic for each area, or a combined value for a plurality of characteristics for the areas; (c) an area is detected as most abnormal when the value of at least one characteristic or the combined value of multiple characteristics of the area is greater than a set maximum threshold, or an area is detected as least abnormal when the value of at least one characteristic or the combined value of multiple characteristics of the area is less than a set minimum threshold; In the method, (d) the inspection process performs the following steps: (i) generating a plurality of digital reject images of real or virtual rejected products, each having at least one known anomaly; (ii) for each failed image, a zone or zones are defined and a value of at least one property or a combined value of properties of each zone is determined, and the maximum of these values is determined as the maximum random sample value of the maximum random sample or the minimum of these values is determined as the minimum random sample value of the minimum random sample; (iii) a detection rate for at least one known anomaly is determined, which (1) all open parameters of the probability density function to describe the maximum or minimum random sample are estimated (parameterized) using the maximum or minimum random sample value and using statistical estimation methods; and (2) by integrating a parameterized probability density function using set maximum or minimum thresholds as integration limits; or (iv) determining a detection rate for at least one known anomaly as a ratio of the number of values in the maximum or minimum random samples that are greater than or equal to a set maximum threshold or less than or equal to a set minimum threshold to a total number of values in the maximum or minimum random samples; and (v) assigning a detection rate to the at least one known anomaly.
2. The maximum and / or minimum thresholds are determined by a learning process, in which the following steps are performed: (a) a plurality of digital images of a passing product that is free of anomalies, or a passing process product that is substantially free of anomalies, are generated or utilized, the number of images being set or determined during a learning process; (b) for each digital image an area or areas are defined, a value of at least one characteristic or a combined value of a plurality of characteristics of each area is determined, and the maximum of these values is determined as the maximum random sample value of the maximum random sample and / or the minimum of these values is determined as the minimum random sample value of the minimum random sample; (c) using a statistical estimation method, estimates of all pending parameters that do not have a probability density function set to represent a maximum random sample are determined using the maximum random sample value, and / or estimates of all pending parameters that do not have a probability density function set to represent a minimum random sample are determined using the minimum random sample value. (d) a first rate at which the largest anomaly is erroneously detected in the image to be inspected or a second rate at which the largest anomaly is not correctly recognized in the image to be inspected is set, and / or a third rate at which the smallest anomaly is erroneously detected in the image to be inspected or a fourth rate at which the smallest anomaly is not correctly recognized in the image to be inspected is set; (e) the maximum threshold is determined using a probability density function or a corresponding distribution function parameterized according to (c), whereby the probability of occurrence of a maximum value greater than or equal to the maximum threshold corresponds to the set first rate, or the probability of occurrence of a maximum value less than or equal to the maximum threshold corresponds to the set second rate, and / or (f) determining the minimum threshold value using a probability density function or a corresponding distribution function parameterized according to (c), such that the probability of occurrence of a minimum value smaller than or equal to the minimum threshold value corresponds to the set third rate, or the probability of occurrence of a minimum value larger than or equal to the minimum threshold value corresponds to the set fourth rate; 2. The method according to claim 1 .
3. (a) the definition of one or more areas is performed using a base threshold value, and each isolated pixel and each group of adjacent pixels, whose pixel values are greater than the base threshold value, are assigned to an area of a first set of areas, and / or each isolated pixel and each group of adjacent pixels, whose pixel values are less than or equal to the base threshold value, are assigned to an area of a second set of areas, or (b) the definition of the region or regions is carried out with the aid of a geometric mask, in particular a fixed mask or a mask generated from the respective image by image processing; 3. The method according to claim 1 or 2.
4. A method according to any one of the preceding claims, characterized in that as the characteristic of the area a geometric characteristic, in particular an area, circumference or diameter, determined from position information of the pixels of the area is used, or as the characteristic of the area a pixel value characteristic, in particular a maximum or minimum value, an average value, a variance or a standard deviation of all pixels of the area is used, determined from the values of the pixels of the area.
5. 10. The method according to claim 9, wherein the probability distribution set is the generalized extreme value distribution, in particular its special cases, the Gumbel distribution, the Weibull distribution or the Fréchet distribution.
6. 10. The method according to claim 9, characterized in that at least one set anomaly is present at a set location in the failed image, and this information, and optionally also the geometry of the impurity causing this set anomaly, is utilized for determining a maximum or minimum random sample value representative of this set anomaly.
7. (a) a pass product containing no anomalies is utilized to generate a reject image, and the pass product is provided with at least one contaminant on or within it, or a support having at least one contaminant disposed thereon, that generates at least one configured anomaly in the reject image, or 13. The method according to claim 12, wherein (b) digital image data of the pass product is utilized to generate a fail image, and the digital image data of the pass product is digitally converted for the inclusion of at least one determined impurity using known material and geometric characteristics of the at least one determined impurity.
8. 8. The method according to claim 7, characterized in that the support for at least one impurity is configured in the form of a plate or card, and in the interior or on the surface of the support a number of impurities are arranged which have identical material properties, in particular consist of a single material, have the same type of geometry but have different sizes.
9. 9. The method according to claim 7 or 8, characterized in that at least one contaminant is a sphere of a predetermined material.
10. 10. A method according to any one of the preceding claims, characterised in that the detection rate is output on a display device as a numerical value or as a quantity or identifier associated therewith.
11. 11. The method according to claim 10, characterized in that the detection rate is assigned to a graphical representation of the at least one contaminant and / or assigned to a corresponding rejected image and displayed.
12. 11. The method according to claim 10, characterized in that the maximum or minimum threshold or the first, second, third or fourth rate is changeable by an adjustment means, for example a slider or a rotary knob, and that for each latest value for the maximum or minimum threshold or the first, second, third or fourth rate, the latest value for the detection rate is displayed on the display device.
13. 12. The method according to claim 11, characterized in that the detection rate is evaluated by a classifier, for example by a color code.
14. 13. An apparatus for detecting anomalies in digital images of products, in particular an X-ray inspection apparatus, comprising a data processing device configured for acquiring and processing digital image data of digital images of products, characterized in that the data processing device is configured for performing the method according to any one of the preceding claims.
15. 13. A computer program product, in particular for an X-ray inspection apparatus, for detecting anomalies in digital images of a product, comprising commands which, when executed by a data processing device, instruct the data processing device to perform a method as claimed in any one of the preceding claims.
Citation Information
Patent Citations
Program, method for creating learned model, information processing method and information processing device
JP2021140739A
Surface anomaly detection system and method
US20070098245A1
