Method and image recording arrangement for optically imaging an object or a scene, in particular for quality assurance in production

Abstract

The present invention relates to a method for optically imaging an object (1) or a scene, in particular for quality assurance in production, and to an associated image recording arrangement. The object (1) is recorded by means of light field photography using a camera arrangement (2), and an image (6) of the object (1) is synthesised from the recorded light field data. The method is characterised in that at least one camera configuration (7, 8) is specified for the optical imaging and the image (6) of the object (1) is synthesised from the recorded light field data as if it had been recorded with a camera of this camera configuration (7, 8), wherein optimum recording parameters for the camera arrangement (2) are determined in advance by way of simulation. In this way, different test tasks can be carried out with the same image recording arrangement without the need for time-consuming modification of the camera.

Description

[0001] Method and image acquisition setup for optical

[0002] Image of an object or scene, especially for quality assurance in production.

[0003] Technical application area

[0004] The present invention relates to a method for optically imaging an object or scene, particularly for quality assurance in production, in which the object or scene is recorded using light field photography and an image of the object or scene is synthesized from the recorded light field data. The invention also relates to an image acquisition arrangement for carrying out the method.

[0005] Depending on their intended use, manufactured products must meet varying quality requirements. To this end, images of the object are often recorded and analyzed during production to identify defects. While automated optical inspection systems are increasingly used for quality assurance, requiring no user intervention during production, the cameras used typically need to be specifically adapted to the respective inspection task by experts in a time-consuming procedure. In particular, an optimally suited camera configuration (camera including lens) must be found for each object to be inspected or for each parameter to be tested, and then a camera with this configuration must be used.For example, it must be ensured that the test object is completely captured, that the resolution matches a detectable defect size, or that a fixed relationship exists between the pixels of the image and the geometric size in the object space, for example by choosing a telecentric optic or telecentric camera.

[0006] State of the art

[0007] Currently, when using an optical inspection system in quality assurance, a suitable camera for the specific requirements is selected and deployed in advance. An engineer calculates an optimal camera configuration based on the test parameters, implements this configuration in a test setup, and then verifies experimentally whether the inspection task can be accomplished. If not, the setup or configuration is iteratively refined.

[0008] From H. Jang et al., “Automated Visual Inspection of Defects in Transparent Display Layers Using Light-Field 3-D Imaging”, IEEE Trans. Semicond. Manufact., Vol. 36, No. 3, pp. 486-493, Aug. 2023, it is known to use a light field camera for inspecting transparent display glass. This allows images at different layer depths of the glass to be synthesized from the recorded light field data in order to detect any defects at different depths. R. Ng, “Digital light field photography”, Stanford University California, 2006, describes the technique of light field photography. It shows how images can be synthesized from a recorded light field, focusing on different depth levels of the scene under observation.

[0009] C. Kludt et al., “Light field illumination: Problem-specific lighting adjustment”, tm - Technisches Messen, Vol. 88, No. 6, pp. 330-341, June 2021, address the optimal illumination of an object for a given testing task. This publication proposes using light field illumination and optimizing it for the specific object and testing task.

[0010] Selecting a suitable camera configuration for a given inspection task or object has so far been very time-consuming. This poses a particular problem when making rapid product adjustments or changes, or even in Industry 4.0 scenarios involving batch size 1.

[0011] The object of the present invention is to provide a method for optically imaging an object or scene and a suitable image acquisition arrangement for this purpose, which are suitable for optical inspection for quality assurance in production and reduce the time required for adjustment in the case of rapid product changes.

[0012] Description of the invention

[0013] The problem is solved by the method and the image acquisition arrangement according to claims 1 and 7. Advantageous embodiments of the method and the image acquisition arrangement are the subject of the dependent claims or can be found in the following description or the exemplary embodiment.

[0014] In the proposed method, the object or scene is recorded using light field photography with a camera setup, and an image of the object or scene is synthesized from the recorded light field data. The method is characterized by the fact that at least one camera configuration is specified for the optical imaging, and the image of the object or scene is synthesized from the recorded light field data as if it had been recorded with a camera of that configuration. The recording of the light field data is performed with optimized recording parameters, which were determined beforehand through a simulation. In this simulation, the recording parameters are optimized so that the image synthesized from the recorded light field data closely approximates an image recorded with a camera of the specified camera configuration.For example, a conventional entocentric camera, a telecentric camera or a pericentric camera can be specified as the camera configuration for the optical imaging of an object.

[0015] The proposed image acquisition setup for carrying out the method comprises an array of cameras configured for recording light field data of an object or scene using light field photography, and an image synthesis device that synthesizes an image of the object or scene from the recorded light field data. The image synthesis device is configured such that it synthesizes the image of the object or scene from the recorded light field data for at least one predefinable camera configuration as if it had been recorded with a camera of that camera configuration.

[0016] The proposed method eliminates the need for a complex selection process prior to optical testing or inspection, which might require procuring a different camera or modifying an existing camera setup. This is because the camera configuration required for each specific requirement or inspection task is simulated during the evaluation of the light field data. The most suitable recording parameters are automatically determined beforehand using an appropriate optimization algorithm. Depending on the camera setup used, these parameters can include aperture, focal length, image distance, and the recording position(s) of the respective camera(s) within the setup, or they can be just one or more of these parameters.

[0017] With the proposed method and the associated image acquisition setup, virtually any visual inspection task can be performed without hardware changes or additional time spent on reconfiguration before the inspection. Therefore, adapting the cameras used to the respective inspection task is no longer necessary. Previously, this conversion was time-consuming and may have required the procurement of new hardware. With the proposed method and the associated image acquisition setup, a wide variety of inspection scenarios can be covered with just one hardware configuration.

[0018] The proposed method and the associated image acquisition setup are based on the clever application of light field theory. While conventional cameras only capture two-dimensional images, light field cameras or light field camera setups additionally record the direction of the incident light. This additional information allows, for example, the focus plane to be varied subsequently. The inventors of the present method have recognized that, moreover, images from various cameras commonly used in inspection technology, such as telecentric or pericentric cameras, can be synthesized using the acquired light field data. Thus, with just one image acquisition setup, images from different cameras or camera configurations can be synthesized, each best suited to the respective application or requirement.Furthermore, by simulating image recording with the camera setup and synthesizing the data, the recording parameters of the camera setup (aperture, focal length, magnification, etc.) can be optimally determined and adjusted depending on the application. This is particularly advantageous when using an algorithm, especially with a physically accurate, differentiable ray-tracing simulation framework, which optimizes the recording parameters to a predefined quality metric based on the light field data. When using an array of cameras, the recording parameters can also be individually optimized for each camera in the array. In addition to the best possible match with an image captured by the specified camera configuration, other quality metrics such as image sharpness, depth of field, and / or defect detection performance can also be used. The image of the object...The scene is then synthesized from the light field data using the optimized recording parameters after this automatic optimization of the recording parameters.

[0019] For recording the light field data of the object or scene, an array of cameras is preferably used as the camera arrangement. In principle, the recording of the light field data can also be carried out with just one camera as the camera arrangement of the proposed method, which is moved to different positions relative to the object or scene in order to record light field data at each position. The use of a suitably designed light field camera as the camera arrangement in the proposed method is also possible. The individual cameras, or the camera at different positions, capture the different light directions. Motorized zoom lenses are particularly advantageous for the cameras, as they also enable the acquisition of image series with varying lens focal lengths. The resulting light field, densely populated with captured rays, offers optimal conditions for subsequent image synthesis.

[0020] The proposed method and the associated image acquisition setup are particularly suitable for quality assurance in production, but can in principle also be used for other tasks of optical imaging of an object or a scene.

[0021] Brief description of the drawings

[0022] The proposed method and the associated image acquisition setup are briefly explained below using an exemplary embodiment in conjunction with the drawing. This shows:

[0023] Fig. 1 shows an exemplary schematic representation of the optical imaging of an object using the proposed method; and

[0024] Fig. 2 shows an exemplary schematic representation of a simulation for determining the recording parameters.

[0025] Ways to implement the invention

[0026] The proposed method and the associated image acquisition setup are preferably used for the optical inspection of objects, particularly during object production. Figure 1 schematically illustrates, by way of example, the procedure for carrying out the proposed method for inspecting a screw 1. For this purpose, light field data of the screw 1 are recorded using a camera array 2. In this example, the image synthesis device 3 synthesizes both the image from a telecentric camera and the image from a pericentric camera from the recorded light field data.In the upper right part of Figure 1, a virtual telecentric camera 7, consisting of a virtual lens 4 and a virtual sensor 5, is shown schematically. In the lower right part of Figure 1, a virtual pericentric camera 8, also consisting of a virtual lens 4 and a virtual sensor 5, is shown schematically. The image 6 of screw 1 synthesized for each of these two camera types is also shown. Thus, images from different camera configurations, which previously required the use of different cameras, can be synthesized with only one camera arrangement.

[0027] To accelerate light field acquisition, suitable approaches from compressed sensing can also be used. These methods typically allow for high-quality reconstruction of a signal from an under-defined data set.

[0028] To automate the optimization of parameters for acquiring light field data, the image acquisition setup can be modeled in a physically accurate, differentiated ray tracing simulation framework (such as Mitsuba 3, https: / / mitsuba.readthedocs.io / en / latest / ). The synthesis algorithms can then be directly coupled in a differentiated programming language, for example, Python with PyTorch. This allows the acquisition parameters for the optimal synthesis of the desired image to be optimized, for example, also based on a quality measure relevant to the specific application, such as image sharpness, depth of field, defect detection performance, etc., using gradient descent.

[0029] Figure 2 shows an example of optimizing the recording parameters. In a simulation, the recording of an image of an object with the specified camera configuration (target camera) is simulated. The recording parameters of the target camera are either predefined or can be optimized beforehand to achieve a corresponding quality measure as described above. In addition, the recording of light field data of the object with the existing camera setup using initial recording parameters, and the synthesis of the desired image from the light field data, are simulated. The two images are then compared, and the deviation is minimized by varying the recording parameters using gradient descent.

[0030] Reference symbol list

[0031] 1 screw (inspected object) 2 camera array

[0032] 3 Image synthesis unit

[0033] 4 virtual lenses

[0034] 5 virtual sensors

[0035] 6 synthesized image 7 virtual telecentric camera

[0036] 8 virtual pericentric camera

Claims

Patent claims 1. Method for optically imaging an object or scene, in particular for quality assurance in production, in which - the object (1) or scene is drawn using light field photography with a camera arrangement (2) and - from the recorded light field data, an image (6) of the object (1) or the scene is synthesized, characterized in that at least one camera configuration (7, 8) is specified for the optical imaging and the image (6) of the object (1) or the scene is synthesized from the recorded light field data as if it had been recorded with a camera of this camera configuration (7, 8), wherein the light field data of the object (1) or the scene are recorded with recording parameters of the camera arrangement (2) which are determined in advance by a simulation calculation in which the recording parameters are optimized so that the image synthesized from the recorded light field data comes as close as possible to an image recorded with a camera of the specified camera configuration (7, 8).

2. The method according to claim 1, characterized in that, that for the optical imaging of the object (1) a telecentric and / or a pericentric camera is specified as camera configuration (7, 8).

3. Method according to claim 1 or 2, characterized in that for the respective application of optical imaging, recording parameters of the specified camera configuration (7, 8) are determined and the image (6) of the object (1) or the scene is synthesized from the light field data with these recording parameters.

4. Method according to one of claims 1 to 3, characterized in that the optimization is carried out using the gradient descent technique.

5. Method according to one of claims 1 to 4, characterized in that an array of cameras (2) is used for light field photography.

6. Method according to one of claims 1 to 4, characterized in that for light field photography an array of cameras (2) with motorized zoom lenses is used, with which series of images with different lens focal lengths are recorded.

7. Image capture arrangement for the optical imaging of an object or scene, in particular for Quality assurance in production, comprising an array of cameras (2) designed for recording light field data of an object (1) or a scene by means of light field photography, and an image synthesis device (3) that synthesizes an image (6) of the object (1) or the scene from the recorded light field data, characterized in that the image synthesis device (3) is designed such that it synthesizes the image (6) of the object (1) or the scene from the recorded light field data for at least one predefinable camera configuration (7, 8) as if it had been recorded with a camera of this camera configuration (7, 8).

8. Image acquisition arrangement according to claim 7, characterized in that the array of cameras (2) has motorized zoom lenses with which series of images with different lens focal lengths can be recorded as light field data.

9. Image acquisition arrangement according to claim 7 or 8, characterized in that the image synthesis device (3) is configured such that it determines, by means of a simulation calculation, acquisition parameters for the array of cameras (2) with which the image synthesized from the recorded light field data is displayed to a camera of the specified Camera configuration (7, 8) to match the drawn image as closely as possible.

Images