Computer-implemented method and system for automated optical inspection

Abstract

The invention relates to a computer-implemented method for automated optical inspection, comprising: acquiring (S1) a time series of image data (BD) of at least one single image (10) of a multitude of products (12) produced on a production line (24); providing (S2) a graphical user interface (14) for displaying the at least one single image (10) of the multitude of products (12) produced on the production line (24), wherein the graphical user interface (14) has a regulator (16), in particular a sliding regulator, which, in operation, is configured for scrolling through the time series of image data (BD); applying (S3) a machine learning algorithm (A) for automated optical inspection of the image data (BD) of the at least one single image (10) of the multitude of products (12) produced on the production line (24); and upon detection of an anomaly, outputting (S4) on the graphical user interface (14) at least one single image (10) having the anomaly and positioning the regulator (16) along the time series of image data (BD) at the position of the single image (10) having the anomaly. The invention furthermore relates to a system (1) for automated optical inspection.

Description

Technical Field

[0001] This invention relates to a computer-implemented method for automated optical inspection.

[0002] Furthermore, the present invention relates to a system for automated optical inspection, a production line for manufacturing workpieces, and a workpiece or product prepared using the production line. Background Technology

[0003] Automated optical inspection is mainly used for the identification of anomalies in industrial manufacturing processes and in defective products obtained from them.

[0004] Manufacturing processes or industrial processes are generally understood as standardized workflows in which products are manufactured by mechanical and / or manual processing and handling of raw materials or semi-finished products using pre-defined manufacturing methods, tools, and equipment.

[0005] Anomaly identification is a data analysis method in which specific instances or patterns (musters) of behavior that deviate from norms or are statistically expected. The goal here is to determine, through data analysis, whether data points deviate from normal or established patterns or behaviors. For this to be effective, a reference must first exist, defining what behaviors can be considered normal.

[0006] Various methods are used for anomaly detection, all of which share the common characteristic of being able to identify outlier measurements from large datasets. These methods typically utilize historical data from the machine or the entire facility.

[0007] There are currently two approaches to identify anomalies in repeatable time-series data, such as that collected during the production or processing of industrial workpieces.

[0008] In rule-based schemes, experts manually define thresholds, either for the time series itself or for the extracted features, where observations above the threshold are classified as anomalous.

[0009] In the case of tabular data, tabular features are extracted from the time series. Algorithms for identifying anomalies in tabular data are then applied to the obtained features to identify anomalies or state changes. Machine learning algorithms are frequently used for this purpose.

[0010] However, domain knowledge from process experts is always required to monitor the performance of the algorithms used.

[0011] Therefore, there is a need to prepare results for process experts within the scope of automated optical inspection, so that the results can be evaluated efficiently.

[0012] Therefore, the objective of this invention is to find a scheme that enables the efficient evaluation of the results of automated optical inspection. Summary of the Invention

[0013] This task is solved according to the present invention by a computer-implemented method for automated optical inspection having the features of claim 1.

[0014] Furthermore, this task is accomplished according to the present invention by means of a computer program having the features of claim 10 and a computer-readable data carrier having the features of claim 11.

[0015] This task is further solved according to the invention by a system for automated optical inspection having the features of claim 12.

[0016] Furthermore, this task is accomplished by a production line for manufacturing workpieces having the features of claim 13 and workpieces prepared using the production line having the features of claim 14.

[0017] This invention relates to a computer-implemented method for automated optical inspection.

[0018] The method includes: detecting a time series of image data of at least one single image of a large number of products produced on a production line.

[0019] Furthermore, the method includes: providing a graphical user interface for displaying at least one single image of a large number of products produced on a production line, wherein the graphical user interface has an adjuster, particularly a sliding adjuster, which is configured to scroll through a time series of the image data during operation.

[0020] The method further includes: applying a machine learning algorithm to perform automatic optical inspection on image data of at least one single image of a large number of products produced on a production line; and, upon detecting an anomaly, outputting at least one single image with the anomaly on the graphical user interface, and positioning the regulator along the time series of the image data at the location of the single image with the anomaly.

[0021] The present invention further relates to a computer program having program code for performing the method according to the invention when the computer program is executed on a computer.

[0022] Furthermore, the present invention relates to a computer-readable data carrier having program code of a computer program for executing the method according to the invention when the computer program is executed on a computer.

[0023] The present invention also relates to a system for automated optical inspection, wherein the system is configured to perform the method according to the invention.

[0024] Furthermore, the present invention relates to a production line for manufacturing workpieces, wherein the workpieces are manufactured using an industrial process, and wherein the production line is configured to monitor the industrial process using a method according to the present invention.

[0025] The present invention also relates to a workpiece manufactured using a production line, wherein the production line manufactures the workpiece using the method according to the present invention.

[0026] The idea behind this invention is to enable efficient evaluation of the results of automated optical inspection by positioning the regulator along the time series of image data at the location of a single image with anomalies.

[0027] Here, the regulator is set or positioned at the location of the first single image with the anomaly. Furthermore, each anomaly along the time series of the image data can be equipped with a label. This allows for easy switching between detected anomalies along the time series of the image data. For example, switching can be performed via dedicated forward and / or backward buttons in the graphical user interface.

[0028] Other embodiments of the invention are the subject of the following description with reference to other dependent claims and figures.

[0029] According to a preferred improvement of the present invention, the machine learning algorithm classifies at least one single image of a large number of products produced on a production line into a first normal category and a second abnormal category.

[0030] In this case, the normal category indicates that the produced product is in a normal state, while the abnormal category indicates that the produced product is in a defective state.

[0031] According to another preferred embodiment of the present invention, the time series of image data comprises multiple single images of spatial sub-regions of the product.

[0032] The spatial sub-regions of a product can be, for example, the joint connection points of components. Therefore, all spatial sub-regions of the product can be output in a single representation.

[0033] According to another preferred embodiment of the invention, the step of outputting at least one single image with an anomaly on the graphical user interface includes automatically annotating the spatial sub-regions of the product with an anomaly.

[0034] Therefore, without further testing, users can see which of the product's multiple spatial sub-regions is abnormal.

[0035] According to another preferred embodiment of the invention, before applying the machine learning algorithm to perform automatic optical inspection on the image data, a time sub-range of the time series of the image data is selected, wherein the selected time sub-range defines the normal range of the produced product.

[0036] Therefore, to define the data foundation for machine learning algorithms within the normal range of the produced products, it is advantageous to select the desired time subrange through process experts. The selection of the time subrange can also be automated according to predefined rules. These rules may include, for example, that specific image attributes, such as the orientation or orientation of an object, move within a pre-defined range. Other image attributes include, for example, brightness, pattern (muster), etc.

[0037] According to another preferred embodiment of the invention, the machine learning algorithm determines a deviation score from a defined normal range for each at least one individual image. The determined deviation score, along with the correspondingly classified anomalous individual images, can then be output to the user in combination.

[0038] According to another preferred embodiment of the invention, a feature vector is generated for each spatial sub-region of the product, wherein for each spatial sub-region of the product, a deviation score is determined from the corresponding feature vector using the machine learning algorithm. The use of the feature vectors enables the measurement of the vector distance to a pre-given normal range or centroid.

[0039] According to another preferred embodiment of the invention, in response to the detection of the anomaly, a directional action is triggered in the technical system, particularly the production line. Therefore, it is advantageous to react to the anomaly, for example, by adapting the production line accordingly.

[0040] According to another preferred embodiment of the present invention, the time series of image data of at least one single image of a large number of products produced on a production line is acquired by a camera image sensor.

[0041] The features of the computer-implemented method for automated optical inspection described herein can also be applied to systems used for automated optical inspection, and vice versa. Attached Figure Description

[0042] To better understand the invention and its advantages, reference is now made to the following description in conjunction with the accompanying drawings.

[0043] The invention will now be explained in more detail with reference to exemplary embodiments illustrated in the schematic depiction in the accompanying drawings.

[0044] Figure 1A flowchart illustrating a computer-implemented method for automated optical inspection according to a preferred embodiment of the present invention; and Figure 2 A schematic diagram of a system for automated optical inspection according to a preferred embodiment of the present invention is shown.

[0045] Unless otherwise specified, the same reference numerals denote the same elements in the figures. Detailed Implementation

[0046] Figure 1 The computer-implemented method for automated optical inspection shown includes: acquiring time series image data BD of at least one single image 10 of a large number of products 12 produced on production line 24 by S1.

[0047] Furthermore, the method includes: providing a graphical user interface 14 for displaying at least one single image 10 of a large number of products 12 produced on a production line 24, wherein the graphical user interface 14 has an adjuster 16, particularly a sliding adjuster, which is configured in operation to scroll through a time series of image data BD.

[0048] The method further includes: applying S3 machine learning algorithm A to perform automatic optical inspection on image data BD of at least one single image 10 of a large number of products 12 produced on production line 24.

[0049] Furthermore, the method includes: detecting anomalies, outputting at least one single image 10 with anomalies on the graphical user interface 14, and positioning the regulator 16 along the time series of image data BD at the location of the single image 10 with anomalies.

[0050] Machine learning algorithm A will classify at least one single image 10 of a large number of products 12 produced on production line 24 into a first normal category K1 and a second abnormal category K2.

[0051] The time series of image data BD has multiple single images of the spatial sub-region 18 of product 12. Here, the step of outputting at least one single image 10 with anomalies on the graphical user interface 14 includes automatically annotating the spatial sub-region 18 of product 12 with anomalies.

[0052] Before applying machine learning algorithm A to perform automatic optical inspection on image data BD, a time sub-range 20 of the time series of image data BD is selected, wherein the selected time sub-range 20 defines the normal range of the produced product 12.

[0053] Machine learning algorithm A determines a deviation score 22 from the defined normal range for each at least one single image 10. Furthermore, a feature vector is generated for each spatial sub-region 18 of the product 12, and for each spatial sub-region 18 of the product 12, the deviation score 22 is determined from the corresponding feature vector using machine learning algorithm A.

[0054] Furthermore, in response to the detection of anomalies, the technical system 1, particularly the production line 24, is triggered to perform directional actions. Additionally, time-series image data BD of at least one single image 10 of the large number of products 12 produced on the production line 24 is acquired via camera image sensor 25.

[0055] Figure 2 A schematic diagram of a system 1 for automated optical inspection according to a preferred embodiment of the present invention is shown.

[0056] exist Figure 2 The system 1 for automated optical inspection shown includes a first device 24 for acquiring time series image data BD of at least one single image 10 of a large number of products 12 produced on production line 26.

[0057] In addition, system 1 includes a second means 28 for providing a graphical user interface 14 for displaying at least one single image 10 of a large number of products 12 produced on production line 24, wherein the graphical user interface 14 has an adjuster 16, particularly a sliding adjuster, which is configured in operation to scroll through a time series of image data BD.

[0058] System 1 further includes a third device 30 for automatically optically inspecting image data BD of at least one single image 10 of a large number of products 12 produced on production line 24 by applying machine learning algorithm A.

[0059] Furthermore, upon detecting an anomaly, a fourth device 32 is used to output at least one single image 10 with the anomaly on the graphical user interface 14 and to position the regulator 16 at the location of the single image 10 with the anomaly along the time series of the image data BD.

Claims

1. A computer-implemented method for automated optical inspection, wherein the method comprises the following steps: Time series of image data (BD) of at least one single image (10) of a large number of products (12) produced on the production line (24) is collected (S1); Provide (S2) a graphical user interface (14) for displaying at least one single image (10) of a large number of products (12) produced on the production line (24), wherein the graphical user interface (14) has an adjuster (16), in particular a sliding adjuster, which is configured to scroll through a time series of image data (BD) during operation; The application (S3) machine learning algorithm (A) is used to perform automatic optical inspection on image data (BD) of at least one single image (10) of a large number of products (12) produced on the production line (24); as well as When an anomaly is detected, at least one single image (10) with the anomaly is output (S4) on the graphical user interface (14), and the regulator (16) is positioned along the time series of the image data (BD) at the location of the single image (10) with the anomaly.

2. The computer-implemented method according to claim 1, wherein the machine learning algorithm (A) classifies at least one single image (10) of a large number of products (12) produced on the production line (24) into a first normal category (K1) and a second abnormal category (K2).

3. The computer-implemented method according to claim 1 or 2, wherein the time series of image data (BD) has multiple single images of a spatial sub-region (18) of the product (12).

4. The computer-implemented method according to claim 3, wherein the step of outputting the at least one single image (10) with anomalies on the graphical user interface (14) includes automatically annotating the spatial sub-regions (18) of the product (12) with anomalies.

5. The computer-implemented method according to claim 3 or 4, wherein before applying (S3) the machine learning algorithm (A) to perform automatic optical inspection on the image data (BD), a time sub-range (20) of the time series of the image data (BD) is selected, wherein the selected time sub-range (20) defines the normal range of the produced product (12).

6. The computer-implemented method according to claim 5, wherein the machine learning algorithm (A) determines a deviation score (22) from a defined normal range for each at least one single image (10).

7. The computer-implemented method according to claim 6, wherein a feature vector is generated for each spatial sub-region (18) of the product (12), and wherein for each spatial sub-region (18) of the product (12), the deviation score (22) is determined from the corresponding feature vector using the machine learning algorithm (A).

8. The computer-implemented method according to any one of the preceding claims, wherein in response to the detection of the anomaly, a directional action of the technical system (1), particularly the production line (24), is triggered.

9. The computer-implemented method according to any one of the preceding claims, wherein a time series of image data (BD) of at least one single image (10) of a large number of products (12) produced on the production line (24) is acquired by a camera image sensor (25).

10. A computer program having program code for performing the method according to any one of claims 1 to 9 when the computer program is executed on a computer.

11. A computer-readable data carrier having program code of a computer program for performing the method according to any one of claims 1 to 9 when the computer program is executed on a computer.

12. A system (1) for automated optical inspection, wherein the system (1) is configured to perform the method according to any one of claims 1 to 9.

13. A production line (24) for manufacturing workpieces, wherein the workpieces are manufactured using an industrial process, wherein the production line (24) is configured to monitor the industrial process using the method according to any one of claims 1 to 9.

14. A workpiece prepared using a production line (24) according to claim 13, wherein the production line (24) prepares the workpiece using a computer-implemented method according to any one of claims 1 to 9.

Images