Method for producing a component, and camera arrangement

Abstract

The invention relates to a method for producing a component (11), in which the following steps are carried out in order to optically identify the component (11): a) creating at least three, preferably four, images (1) of the component (11) using a camera (10) while the component (11) is illuminated by at least one light source (12) and the illumination direction (14) is varied with each image (1); b) reading out two-dimensional and / or three-dimensional identification features from the created images (1) of the component (11) and entering them into a database; c) performing step a) again in the further course of the production of the component (11) and reading out two-dimensional and / or three-dimensional identification features from the component (11) again; and d) assigning the component (11) to its associated database entry by comparing the identification features with the database. The invention further relates to a camera arrangement (15) for performing the method.

Description

[0001] Description

[0002] Title:

[0003] Method for producing a component, camera arrangement

[0004] The present invention relates to a method for manufacturing a component, wherein the component is tracked during the manufacturing process. Furthermore, the invention relates to a camera arrangement.

[0005] State of the art

[0006] With the concept of Industry 4.0, industrial manufacturing processes are becoming increasingly digitalized. A particularly advantageous embodiment of this lies in the individualization of similar components through rework and quality management. Many Industry 4.0 approaches require complete information about the location and movement of all workpieces. Particularly essential is constant, real-time knowledge of the location of a specific component or product and the associated work progress. The prerequisite for this is that the product or component is individually identifiable. Various tracking and identification concepts exist to meet this requirement, such as the assignment and reading of individual serial numbers, the integration of RFID chips in components, and even the non-invasive detection of component characteristics using imaging techniques.

[0007] The latter typically work in such a way that, at a first point in time, an image of a component is created, usually with a camera. This image is pre-processed using computer-aided image processing and recognition and then registered in whole or in part in a database. At a later point in time during production, when the component is to be identified, another image is created and pre-processed using computer-aided image processing and recognition. The image is then compared with the database entries for consistency. As a rule, the corresponding component is assigned to the database entry with which the respective image matches completely or at least to a significant extent.

[0008] However, this tracking method assumes that the component does not undergo any significant visual changes between registration in the database and subsequent identification. Visual changes include, for example, changes in color, changes in surface grain, and other characteristics. Such visual changes occur particularly during heat treatment processes. The result is that the correspondence between the image created for identification and its corresponding database entry continues to decrease, to the point where essential identifying features for differentiation from other database entries are no longer present or can no longer be recognized. As a result, the tracking process loses considerable reliability.

[0009] The invention is therefore concerned with the task of ensuring reliable tracking of the component even if the surface quality has been altered, for example, by heat treatment, to such an extent that reliable tracking is not possible. To achieve this task, the method with the features of claim 1 is proposed. Advantageous further developments of the invention can be found in the subclaims. Furthermore, a camera arrangement for implementing the method according to the invention is specified.

[0010] Disclosure of the invention

[0011] A method for producing a component is proposed, wherein the following steps are carried out for the optical identification of the component: a) taking at least three, preferably four images of the component with a camera, while the component is illuminated by at least one light source and the direction of illumination is varied with each image, b) reading out two-dimensional and / or three-dimensional identification features from the created images of the component and entries in a database, c) repeating step a) in the further course of production of the component (11) and again reading out two-dimensional and / or three-dimensional identification features from the component and d) assigning the component to its associated database entry by comparing the identification features with the database.

[0012] The proposed method enables the identification of a component within a group of similar components and thus enables tracking during the manufacturing process. Identification and tracking are preferably based on both two- and three-dimensional identification features. This enables reliable identification even across different processing steps that change the appearance of the component. Two-dimensional features such as surface shading can still be reliably used even if the three-dimensional nature of the component is altered, for example by removing small surface scratches through polishing, and thus some three-dimensional identification features are lost. Conversely, three-dimensional identification features can still be reliably used if, for example, a characteristic color pattern on the surface is changed by heat treatment.Illumination from different angles creates regional brightness differences in the captured images. These differences contain three-dimensional features that can be inferred using various methods.

[0013] The law of Lambertian reflection is used as the basis. The surface brightness B recorded by a camera depends on the surface color C and the irradiation intensity / as well as the scalar product of the surface normal vector n and the irradiation direction vector L, the latter two generally being unit vectors: B = CI (n ■ L)

[0014] The scalar product is determined by the angle a between the surface normal vector n and the irradiation direction vector L according to: n ■ L = cos a

[0015] From this relationship, it follows that surface brightness B is maximum when illuminated perpendicularly and decreases with increasing tangential illumination. From the recorded surface brightness at various known illumination directions, conclusions about a three-dimensional condition, in particular the surface orientation, can be drawn. By varying the illumination direction, the possible orientation of the surface normal is narrowed down. In particular, the orientation of the surface normal can be unambiguously determined by superimposing several images with different illumination directions. This allows essential three-dimensional features of the component to be captured.

[0016] It is further proposed that in step a) the at least one light source and / or the component is / are moved to vary the illumination direction. This preferred embodiment has the advantage that the illumination direction can be dynamically adapted to the component to be identified. For this purpose, the light source can be moved accordingly. However, this variant is relatively time-consuming. In contrast, there is the second variant, in which the illumination direction is varied by rotating the component. This allows the required images to be taken in rapid succession, which saves time. However, both variants rely on a mechanism that may be prone to failure.

[0017] Therefore, it is proposed that in step a) several light sources be used to illuminate the component. These light sources are arranged around an image axis of the camera, preferably at the same angular distance from each other, and are switched on one after the other. This preferred embodiment retains the advantages of time savings compared to moving a light source or moving the component, while at the same time eliminating the need for complex and potentially error-prone mechanical components.

[0018] It is further proposed that in step a) the at least one light source is positioned in relation to the camera such that the direction of illumination forms an acute angle with an image axis of the camera, wherein the angle is preferably 30° to 60°. This preferred embodiment ensures that the component is not illuminated parallel to the image axis. If the direction of illumination is not parallel to the image axis, three-dimensional features on the component facing the camera, for example mountains and valleys, will cast a more pronounced shadow which a direction of illumination parallel to the image axis does not produce. The evaluation of the respective shadow from different directions of illumination allows conclusions to be drawn about the three-dimensional features that caused the shadow.

[0019] In a further development of the invention, it is proposed that in step a), the images are taken with a telecentric camera and the component is illuminated by a telecentric light source, with three of the different illumination directions of the component being selected linearly independently. This preferred embodiment allows the photometric stereo technique to be applied to the component, with which the three-dimensional nature of the component is measured photometrically. The corresponding surface normal vector is estimated for each pixel.

[0020] The evaluation of the various images is based on the previously explained principles of Lambertian reflection, but is a tensor algebraic problem and is carried out pixel by pixel in practice. A light intensity vector / contains all light intensities measured in the different images at the pixel under consideration. The normalized illumination direction vectors are recorded in an illumination direction matrix L, where the illumination direction matrix has the dimensions 3 x m and m represents the number of images with different illumination directions. The two-dimensional surface properties, such as color, matte, etc., are accounted for using an unknown scalar k, the so-called albedo. The light intensity vector / , the illumination direction matrix L, and the albedo k are linearly related to the surface normal vector n applicable to the pixel under consideration and sought as follows:

[0021] I = k (L ri)

[0022] The tensor operation within the brackets is a linear mapping. The equation is solved for the product of the albedo k and the surface normal n, where the length of the vector result corresponds to the albedo k and the direction of the surface normal at the pixel under consideration. Applying the solution to each pixel yields an albedo and a surface normal for each pixel.

[0023] It is further proposed that, prior to reading out the identification features in step a) and / or step c), a texture image and a gradient image be created based on the recordings. The texture image contains two-dimensional properties of the component that correspond to the reflectivity of the surface, while the gradient image contains the three-dimensional properties that are based on the spatial orientation of the surface. This preferred embodiment makes the two-dimensional and three-dimensional properties of the component available as images for the application of further known image processing methods. With the help of these, methods can be used, in particular, to highlight or isolate features considered relevant and suitable as identification features. At the same time, insignificant or interfering features can be filtered out.In the preferred application of photometric stereo as described above, the texture image is generated by assigning each pixel the value of its corresponding albedo. The result is a grayscale image, for example, in 8-bit format. The gradient image contains information about the curvature of the surface at the respective pixel. It is created by first interpreting the surface normal vectors determined pixel by pixel using photometric stereo as a vector field. This vector field is amenable to various mathematical operations. Preferably, the mean curvature of the vector field can be determined. Furthermore, it can be convolved with a derivative of the Gaussian function, thereby enhancing the expression of three-dimensional features such as scratches or dents. The result is the gradient image, an image that graphically encodes the expression of these three-dimensional properties.

[0024] Furthermore, it is proposed that in step a) and / or step c), the identification features are read from the texture image and the gradient image by applying a feature extraction algorithm, preferably a Förstner algorithm. Not all of the two-dimensional properties contained in the texture image and the three-dimensional properties contained in the gradient image can be meaningfully used as identification features for identifying the component. With this preferred embodiment, those two- and three-dimensional properties that can be used as identification features are extracted. They are available as a collection of points or pixels for further operations. Many different feature extraction algorithms are known and can be considered.

[0025] They generally have in common that they isolate those points that differ significantly from their neighboring regions. In the case of Förstner's algorithm, the inhomogeneity and isotropy, purely mathematical properties, are determined for each point. A point is classified as significant if both its inhomogeneity and its isotropy exceed a previously defined threshold. How precisely these thresholds are to be selected depends on the requirements of the specific application area of ​​the invention and must be determined experimentally or iteratively.

[0026] In a further development of the invention, it is proposed that in step a) and / or step c), identification features are read from the texture image and the gradient image via edge detection, preferably by applying a Sobel operator. This preferred embodiment adds further relevant points to the previously extracted identification features. This is particularly advantageous if the surface texture, for example the color, has been significantly altered by steps in the manufacturing process, such as heat treatment. Various algorithms for edge detection are known and can be used. When applying a Sobel operator, a specific edge detection filter, the first derivative of the respective pixel brightness values ​​is determined using convolution, with simultaneous smoothing orthogonal to the derivation direction.The result is a strong emphasis on the image areas with the most pronounced edges. The relevant points are extracted by applying a threshold function. The parameters of this threshold function depend on the requirements of the specific application area of ​​the invention and must be determined experimentally or iteratively.

[0027] It is further proposed that in step d) the identification features read out in step a) and / or step c) are compared with the database by point-by-point determination of a correspondence, preferably by applying a resampling algorithm, for example the Random Sample Consensus, wherein the number of matching identification features is preferably determined for each database entry. When the component is identified during the manufacturing process, the component has often already been modified to such an extent that it no longer matches its associated database entry in all identification features. This preferred embodiment eliminates the need for complete matching. Rather, the assignment is based on the entry in the database that has the greatest number of matches with the component to be identified.Since the component was explicitly added to the database in step a), it is ensured from the outset that a corresponding database entry exists.

[0028] Furthermore, a camera arrangement is proposed. According to the invention, this comprises at least one camera with an image axis and at least one light source with an illumination direction, wherein the image axis and the illumination direction enclose an acute angle, and the angle is preferably 30° to 60°. With the proposed camera arrangement, images can be taken of a component during its manufacturing process with different illumination directions. The effect of Lambertian reflection can be exploited via the different illumination directions to draw conclusions about the three-dimensional nature of the component. By using individual three-dimensional identification features that can be detected in this way in addition to two-dimensional identification features obtainable with conventional image recognition methods, a component can be reliably tracked during its manufacturing process.This is reliably possible even if the two-dimensional properties, for example the surface color, change due to manufacturing steps such as heat treatment and the like.

[0029] Furthermore, it is proposed that the camera have a telecentric lens and that at least one light source be telecentric. With the use of a telecentric camera and telecentric light sources, and with the correct choice of illumination directions, the necessary prerequisites are met for applying the previously explained photometric stereo technique. This technique allows the three-dimensional nature of the component to be measured with high accuracy. This increases the quantity and quality of the extractable three-dimensional identification features, which increases the reliability of component tracking.

[0030] In a further development, it is proposed that the at least one light source be movable, preferably movable, to vary the illumination direction and / or that multiple light sources be arranged around the image axis of the camera. The first of the variants mentioned has the advantage that the illumination direction can be dynamically adapted to the component to be identified. For this purpose, the light source is moved accordingly. However, this variant is time-consuming. In contrast, there is the second variant, in which the light sources can be permanently installed. This allows the required images to be taken in rapid succession, which saves time. Furthermore, there is no need to resort to complicated and potentially failure-prone mechanics.

[0031] A preferred embodiment of the invention is explained in more detail below with reference to the figures. These show:

[0032] Figure 1 is a schematic representation of an exemplary manufacturing process

[0033] Figure 2 is a schematic representation of a preferred procedure for reading identification features. Figure 3 is a schematic representation of the conversion of four images to a texture image and a gradient image.

[0034] Figure 4 is a simplified side view of a preferred embodiment of a camera arrangement according to the invention with several light sources

[0035] Figure 5 is a plan view of a preferred embodiment of a camera arrangement according to the invention with multiple light sources.

[0036] Character description

[0037] Fig. 1 schematically shows an example manufacturing process for a component 11. The manufacturing sequence takes place at several stations. In a first manufacturing step S0, the component 11 is manufactured. In a subsequent process step S1, four images 1 of the component 11 are created using a camera 10. The component 11 is illuminated from different light sources 12, each time from a different illumination direction 14. The images 1 are merged and converted into a texture image 2 and a gradient image 3 using photometric stereo (see also Fig. 3). The texture image 2 contains the two-dimensional nature of the component 11, and the gradient image 3 contains the three-dimensional nature. Two- and three-dimensional identification features are read out from the texture image 2 and the gradient image 3 and registered in a database. In a further process step S2, the component 11 is further processed.For example, its surface is heat-treated. In a subsequent step S3, the component 11 is again imaged by a camera 10 under different illumination directions 14, and its identification features are read out (see also Fig. 2). The identification features are compared with the database, and the component 11 is assigned to its corresponding database entry. The component 11 is thus identified. In a further process step S4, further post-treatments of the component 11 follow. Tracking of the component 11 according to process step S3 can be inserted at any time. The manufacturing process is concluded in a final process step S5. Fig. 2 shows a preferred sequence for tracking a component 11. In a first step E1, the component is imaged by a camera 10. It is illuminated from one side by a light source 12. This light source goes out after image 1 has been taken.In a second step E2, a second light source 12 illuminates the component 11 from a new direction. The camera 10 takes another image 1 of the component 11, and the second light source 12 goes out. In a subsequent step E3, a third light source 12 illuminates the component 11 from yet another new direction. The camera 10 takes another image 1 of the component 11, and the third light source 12 goes out. In a subsequent step E4, a fourth light source 12 illuminates the component 11 from a direction different from all previous ones. The camera 10 takes a fourth image 1 of the component 11, and the fourth light source 12 goes out. In a subsequent method step E5, the four created images 1 are converted into a texture image 2 and a gradient image 3 using photometric stereo. The texture image 2 contains the two-dimensional surface properties of the component 11, for example the surface color.The three-dimensional texture of component 11 is encoded in gradient image 3. In a subsequent step E6, the identifying features of component 11 are extracted from texture image 2 and gradient image 3. This is preferably done using a feature extraction algorithm such as the Förster algorithm. In a subsequent method step E7, the identifying features extracted in E6 are compared with all database entries for consistency. Component 11 is assigned to the database entry with the most matches.

[0038] Fig. 3 shows, by way of example, the conversion of four images 1 of a component 11 under four different illumination directions 14 into a texture image 2 and a gradient image 3 using photometric stereo. The texture image 2 represents the two-dimensional surface texture of the component 11 in grayscale. In particular, this involves the surface's reflection properties, such as its color, gloss, or mattness, and the like. The three-dimensional properties of the component 11 are encoded in the gradient image 3. These are scratches, dents, edges, and similar three-dimensional characteristics. The gradient image 3 is created in photometric stereo by calculating a surface normal vector for each pixel from the images 1, and calculating a surface curvature from the vector field thus generated and encoding it as a pixel value.

[0039] Fig. 4 shows a side view of a preferred embodiment of a camera arrangement 15 according to the invention for creating images 1. A camera 10 is positioned vertically above a component 11 to be imaged. Four light sources 12, preferably telecentric spotlights, are arranged to illuminate the component 11. The light sources 12 are arranged such that the illumination direction 14 and the image axis 13 each enclose an angle a of 30° to 60°.

[0040] Fig. 5 shows a plan view of a preferred camera arrangement 15 for taking pictures 1, corresponding to the arrangement in Fig. 4. The camera 10 is located in the center. The four light sources 12 are arranged around it.

[0041] To create each image 1, a light source 12 is switched on to illuminate the component 11 to be recorded. The light source 12 is switched off again, then the component 11 is illuminated by another light source 12 and recorded again by the camera 10. This process is repeated four times, each time with a different light source 12.

Claims

Claims 1. Method for producing a component (11), wherein the following steps are carried out for the optical identification of the component (11): a) creating at least three, preferably four images (1) of the component (11) with a camera (10), while the component (11) is illuminated by at least one light source (12) and the direction of illumination (14) is varied with each image (1), b) reading out two-dimensional and / or three-dimensional identification features from the created images (1) of the component (11) and entries in a database, c) carrying out step a) in the further course of production of the component (11) and again reading out two-dimensional and / or three-dimensional identification features from the component (11), and d) assigning the component (11) to its associated database entry by comparing the identification features with the database.

2. Method according to claim 1, characterized in that in step a) the at least one light source (12) and / or the component (11) is / are moved in order to vary the direction of illumination (14).

3. Method according to claim 1 or 2, characterized in that in step a) several light sources (12) are used to illuminate the component (11), which light sources are arranged around an image axis (13) of the camera (10), preferably at the same angular distance from one another, and are switched on one after the other.

4. Method according to one of the preceding claims, characterized in that in step a) the at least one light source (12) is placed in relation to the camera (10) such that the direction of illumination (14) is aligned with a Image axis (13) of the camera (10) encloses an acute angle (a), wherein the angle (a) is preferably 30° to 60°.

5. Method according to one of the preceding claims, characterized in that in step a) the images (1) are created with a telecentric camera (10) and the component (11) is illuminated by a telecentric light source (12), wherein three of the different illumination directions (14) of the component (11) are selected linearly independently.

6. Method according to one of the preceding claims, characterized in that before reading out the identification features in step a) and / or in step c), a texture image (2) and a gradient image (3) are created on the basis of the recordings (1).

7. The method according to claim 6, characterized in that in step a) and / or step c) the identification features are read out from the texture image (2) and the gradient image (3) by applying a feature extraction algorithm, preferably a Förstner algorithm.

8. Method according to one of the preceding claims 6 or 7, characterized in that in step a) and / or step c) identification features are read out from the texture image (2) and the gradient image (3) via edge detection, preferably by applying a Sobel operator.

9. Method according to one of the preceding claims, characterized in that in step d) the identification features read out in step a) and / or step c) are compared with the database by point-by-point determination of a correspondence, preferably by application of a resampling algorithm, for example the Random Sample Consensus, wherein the number of matching identification features is preferably determined for each database entry.

10. Camera arrangement (15) for carrying out a method according to one of the preceding claims, characterized by at least one camera (10) with an image axis (13) and at least one light source (12) with an illumination direction (14), wherein the image axis (13) and the illumination direction (14) enclose an acute angle (α) and the angle (α) is preferably 30° to 60° 11. Camera arrangement (15) according to claim 10, characterized in that the camera (10) has a telecentric lens and the at least one light source (12) is telecentric.

12. Camera arrangement (15) according to claim 10 or 11, characterized in that the at least one light source (12) is movable, preferably displaceable, for varying the direction of illumination (14) and / or several light sources (12) are arranged around the image axis (13) of the camera (10).

Images