DEVICE AND METHOD FOR PROCEEDING SENSORREALISTIC IMAGES

Description

[0001] The application concerns image processing, and, in particular, a system and a method for generating sensor-realistic images.

[0002] When developing an image processing system, the challenge lies in designing and parameterizing all its components to achieve the best possible result for solving the given task. Due to the multitude of parameters, such optimization usually requires a comprehensive investigation of all steps—image acquisition, processing, evaluation, classification, etc.—by an expert, who typically has to conduct numerous experiments. Even then, there is no guarantee that the optimal configuration and parameterization will actually be achieved.

[0003] However, there are aspects that make such selection and optimization by experts using real-world test setups difficult or impossible. For example, realistically setting up objects in the scene to create an image processing system is costly and time-consuming. In many cases, the objects or the scene are not reproducible or cannot be realistically recreated (e.g., for crash tests, bin picking, natural objects, natural environments, taking into account the time of day, year, and weather conditions). In some cases (e.g., before a production system is completed), the objects and the scene do not yet physically exist.

[0004] The complete consideration and evaluation of all potential components of an image processing system is costly and time-consuming, as a large number of such components with different properties are available on the market.

[0005] Regarding the arrangement of the objects and the hardware components (lighting system and camera system), there are many geometric degrees of freedom, so that optimization is complex and cannot be exhaustively achieved experimentally.

[0006] Regarding the methods and algorithms for image processing, evaluation, and classification, the sheer number of possible components of an image processing system, the degrees of freedom, and the variability of the scene inherent in image processing means that an optimal selection and combination of image processing methods and their parameterization is practically impossible. Furthermore, the robustness of any solution found is difficult to prove.

[0007] The complexities described above in the development of image processing solutions mean that real-world applications of image processing systems are currently limited to relatively simple tasks. However, image processing systems are recognized as having great technical potential and ever-increasing market opportunities. These opportunities, however, often go unrealized because the practical development of these systems, including their automatic parameterization, optimization, and testing, remains inadequate.

[0008] In today's standard approach to optimizing task-oriented image acquisition and analysis, the expert first selects camera systems (comprising, for example, cameras, lenses, and, if necessary, optical filters), lighting systems, and image processing algorithms according to the task at hand and recreates task-specific, real-world scenes in a laboratory setting. Image acquisition and analysis are then performed, and the results are compared with the desired outcomes. If the result of this comparison remains unsatisfactory, the expert uses their experience to vary hardware components and / or the geometric arrangement of these components and / or image analysis parameters in order to obtain better results.

[0009] However, for a wide range of applications in image processing, the time and cost expenditure could be significantly reduced if the images that a real camera system captures of the real world could be simulated.

[0010] In principle, such images can be produced using the means and methods of computer graphics, but these images only approximate the images of a real camera system, since the images in computer graphics are predominantly based on physically correct, mathematically described or mathematically abstracted laws.

[0011] However, for the simulation of images that correspond to those captured by a real camera system of the real world, it makes sense to base the simulation on the properties and characteristics of real existing light sources (lamps or luminaires; in the following, lamps are understood as a type of luminaire), real existing materials and objects, real existing lenses, and real existing camera systems in order to generate so-called sensor-realistic images.

[0012] Synthesizing images of the real world, as perceived / represented by the eye or a camera system, requires, as a first step, the creation of a model of the real world. This model represents all physical objects involved in the images and their creation, as well as their spatial relationships to one another. This is typically achieved through a mathematical description of the geometric shape of the objects and their spatial arrangement within a suitable world coordinate system. The real world thus modeled represents the virtual scene of the objects involved in the images and their creation. In relevant scientific disciplines (IT, physics, mathematics, optics, mechanical engineering, architecture, etc.), this scene is called a CAD model. At least one or more objects in this scene represent light sources (lamps, etc.).(lights) because otherwise the other objects would be invisible, and at least one or more objects represent eyes or camera systems, because otherwise no image would result.

[0013] Two fundamentally different approaches, radiosity and ray tracing, have become established and prevailed for calculating the synthetic images of such a scene.

[0014] The simulation of heat conduction, based on the finite element method (FEM) Radiosity-based methods calculate the light distribution in a scene according to the law of conservation of energy, by reflecting back into the scene all light that falls on a surface and is not absorbed by it. The method assumes that all surfaces are ideally diffuse reflectors and all light sources, represented in the model by self-illuminating surfaces, are ideally diffuse emitters, meaning that unabsorbed light is reflected or emitted uniformly in all directions. To calculate the light distribution, the surfaces of the objects in a scene are first decomposed into small surface elements. Then, a purely geometric form factor is determined for all possible pairs of surface elements. F ij calculated, which indicates, depending on the position of the surface elements relative to each other and their distance from each other, what proportion B si of the emitted light (= specific luminous radiation = radiosity [W / m 2< ]) of a surface elementi at the receiving surface element j arrives. The entire received light radiation B ej The radiative power of a surface element is then calculated as the sum of the radiative powers of all light-emitting surface elements multiplied by the respective form factor, i.e. B ej = ∑ i = 1 n B si F ij

[0015] Will a surface element now be considered? j When considered as a light source, it can be used to calculate the emitted radiant power. B sj the reflectance of this surface element p j The area element is taken into account by multiplying the received radiation power by it. B sj = p j ∑ i = 1 n B si F ij

[0016] In the event that the surface element j Even if it is itself a light source, it can then be used to calculate the emitted radiant power. B sj even the self-radiation E j will be added B sj = E j + p j ∑ i = 1 n B si F ij

[0017] The calculation of the radiosity of all sub-areas then leads to the following system of equations. B 1 B 2 ⋮ B n = E 1 E 2 ⋮ E n + p 1 F 11 p 1 F 12 ⋯ p 1 F 1 n p 2 F 21 p 2 F 22 ⋯ p 2 F 2 n ⋮ ⋮ ⋱ ⋮ p n F n 1 p n F n 2 ⋯ p n F nn ⋅ B 1 B 2 ⋮ B n ⇔ B = E + T ⋅ B

[0018] This system of equations is usually solved iteratively using the gathering method, starting with B 0< = E (self-illuminating surfaces) and then B k< = E + T · B k- 1< , k > 0 up to a termination criterion.

[0019] After calculating the radiosity of a scene, the desired image is rendered taking into account appropriate form factors between the scene and the eye or camera system.

[0020] The advantages of radiosity methods lie in the fact that the calculation of light distribution is initially independent of the viewpoint and viewing angle, and consequently, images from different locations and viewpoints can be calculated very efficiently. Furthermore, the effects of diffuse light reflections (shadow transitions, color reflections, etc.) are naturally enhanced by the method.

[0021] However, a disadvantage is that, due to the division of the scene into surface elements, analytically describable objects (e.g., spheres) cannot be used, and the assumed ideally diffuse light reflections fundamentally preclude the calculation of caustics and reflections. To account for these effects, the radiosity method was extended accordingly. Further disadvantages include the extremely high memory requirements for more complex scenes and the (empirical) complexity of the calculation, which is associated with... O (N · logN ) significantly higher than with ray tracing methods with O ( logN ). With the so-called shooting method, an extension of the radiosity method, and the Southwell iteration and progressive refinement methods based on it, better convergences to "beautiful" images are achieved.

[0022] Unlike radiosity methods, which are based on the law of conservation of energy, ray tracing methods rely on tracking light rays in a scene whose origins lie at the observer's location (eye or camera system) and travel in the direction of their viewing angle. In its most basic form, the method uses a pinhole camera, where the image on the ground glass screen / film is created by light rays from the scene with a negligible cross-sectional area (small hole). Since light rays from different directions all pass through the camera's pinhole, this is referred to as the viewpoint for calculation purposes, and the ground glass screen / film is technically positioned between the scene and the viewpoint, making the image acquisition process completely equivalent to that of a pinhole camera.

[0023] The actual procedure involves constructing a random ray from the viewpoint through the ground glass / film plane into the scene and determining the nearest intersection point with the scene objects. If such a point exists, one or more further rays are constructed from there in a random direction, and again, the nearest intersection point with the scene objects is determined for each ray. This process is repeated until the rays under consideration hit a light source or leave the scene. Subsequently, starting from the light source and taking into account the BRDF (bidirectional radiation distribution function) or BTDF (bidirectional transmittance distribution function), which describe the scattering behavior of light at opaque object surfaces or transparent objects, respectively, and the solid angles of the rays at the object intersection points, the irradiance and color on the ground glass or film are calculated.calculated on the film, whereby this results at the intersection points where several rays unite as the sum of the irradiances of the individual rays (superposition principle).

[0024] This method was first formally published in 1986 by James T. Kajiya in his publication "The rendering equation" under the term "path tracing". The originally published rendering equation is: I x , x ′ = g x , x ′ ⋅ I e x , x ′ + ∫ S b x , x ′ , x " I x ′ , x " dx "

[0025] It describes the light intensity reaching a surface point x from another surface point x', taking into account a further surface point x" whose light first strikes x' and is then reflected back to x. The individual terms have the following meanings: The radiant intensity (English: radiant) intensity ) I ( x,x '), measured in [W / sr], indicates the light intensity reaching point x of point x' (W = watt, sr = solid angle (steradian)). The same applies to the term I ( x',x" ).

[0026] The geometric position of points x and x' relative to each other is given by the "geometric term" g ( x,x' ) described. In general, the term has the value 1 / r 2< , where r is the distance between x and x' (inverse square law of physical quantities). However, if there is another surface between x and x', the term = 0, meaning that no light arrives at x directly from x'. This also applies to completely transparent surfaces, which completely absorb the light from x' and re-emit it on the opposite side.

[0027] If the point x' lies on the surface of a lamp (which is a light source; a lamp is also a luminaire) or on the surface of a luminaire, then the "emission term" describes I e ( x ,x '), also measured in [W / sr], how much light is emitted from x' to x.

[0028] The "dispersion term" b ( x,x',x" This indicates what proportion of the light reaching x' from x" is reflected in the direction of x. This is usually a bidirectional scattering distribution function (BSDF).

[0029] S is the totality of all surfaces in the scene.

[0030] In the equation above, is the light intensity I through the radiance L , measured in [W / m 2< ·sr], replaced, results in the equivalent form frequently used in the literature: L x ω → = L e x ω → + ∫ Ω f r x , ω → , ω → ′ L x , ω → ′ ω → ′ , n → d ω → ′

[0031] Instead of the second and third points x' and x" A point will be made here x and a direction vector ω The values ​​given for the light ray under consideration are: The radiance. L ( x,ω ) indicates how much light from x towards ω is emitted.

[0032] The emission term L e ( x , ω ) indicates how much light from point x in the direction ω is emitted if x is a point of a light source. 0

[0033] The dispersion term f r ( x,ω,ω' ) is a BRDF at position x with angle of incidence ω and angle of incidence ω '.

[0034] The term L ( x,ω ') describes how much light reaches the point x from the direction ω ' reached.

[0035] n is the normal of the surface at the point x .

[0036] Ω is the totality of all angles of the hemisphere above the point x .

[0037] Of the many methods based on path tracing that offer improvements in runtime, light distribution, shadow calculation, and realism (photorealism), the following should be mentioned: "Photon mapping", "Bi-directional path tracing", "Importance-Driven Radiosity Algorithm", and "Metropolis Light Transport".

[0038] Besides the fundamental difference in approach between radiosity (conservation of energy) and ray tracing methods (tracking of light rays), ray tracing methods take the following parameters into account: The location- and spatial direction-dependent light intensity of more or less extended lamps or luminaires (emission characteristics).

[0039] The spatial direction-dependent reflection behavior of light rays striking matter, using BRDF (bidirectional reflectance distribution function).

[0040] The spatial direction-dependent transmission behavior of light rays striking matter, using BTDF (bidirectional transmitance distribution function).

[0041] The spatial direction-dependent internal reflection behavior of light rays hitting matter, using SSS (surface scattering) or SSLT (subsurface light transport).

[0042] The imaging properties of lenses.

[0043] The aperture-dependent depth-of-field effect.

[0044] The properties of the imaging element (eye: retina, camera system: film or (semiconductor) sensor).

[0045] This results in more realistic images.

[0046] The imaging properties of lenses occasionally considered in ray tracing methods for simulating images of a real-world scene represented by a virtual model are limited to imaging models and mathematical calculation rules for thin lenses, geometric or Gaussian optics, in order to account for various geometric-optical (distortion, vignetting, spherical and chromatic aberration), wave-optical (resolution), and other influences (exposure time) during image formation. The aperture-dependent depth-of-field effect is taken into account by considering not just one central ray per pixel in the virtual scene, but several rays. These rays are constructively derived from the consideration that the light intensity of a pixel results from the radiance of all rays from various object points through the aperture of the pixel. Since the object-side imaging of a pixel is generallyIf the beam does not lie on an object in the virtual scene, the nearest intersection points with the scene objects are determined along these rays and their light intensity is accumulated.

[0047] Ray tracing methods also take into account the properties of the imaging elements, using photometric quantities for the human eye (retina) and radiometric quantities for camera systems (film, (semiconductor) sensor). The influence of relevant parameters for film and semiconductor sensors is rarely considered and is essentially limited to heuristically assumed noise, since the goal is usually to produce "beautiful" images.

[0048] The emission characteristics of more or less extended lamps or luminaires, the reflection or transmission behavior of the materials, the imaging properties of lenses, and the properties of the imaging sensor in camera systems have a decisive influence on the realism of the result of the ray tracing methods.

[0049] From LECH SWIRSKI ET AL: "Rendering synthetic ground truth images for eye tracker evaluation", EYE TRACKING RESEARCH AND APPLICATIONS, March 26, 2014, pages 219-222, DOI: 10.1145 / 2578153.2578188, an approach for generating realistic synthetic images is known in which non-ideal properties of the simulated camera system are taken into account. GARGAN DAVID ET AL: "Approximating Reflectance Functions using Neural Networks",

[0050] RENDERING TECHNIQUES '98 : PROCEEDINGS OF THE EUROGRAPHICS WORKSHOP IN VIENNA, AUSTRIA, DOI: 10.1007 / 978-3-7091-6453-2_3 reveals a representation based on a neural network model for the storage and reconstruction of arbitrary reflection functions.

[0051] Patent publication US 7929142 B2 discloses a device for photodiode-based measurement of bidirectional reflection distribution functions (BRDF).

[0052] It would therefore be desirable to obtain concepts for more realistic camera image generation and thus for the generation of sensor-realistic images.

[0053] It would also be desirable to obtain improved concepts for determining reflection and transmission behavior and improved concepts for determining the emission characteristics of light sources.

[0054] A device according to claim 1, a method according to claim 14 and a computer program according to claim 15 are provided.

[0055] Preferred embodiments of the invention are described below with reference to the drawings.

[0056] The drawings depict: Fig. 1 shows a device for generating a result image according to one embodiment. Fig. 2 shows a schematic representation of the functionality of the Mitsuba® rendering framework. Fig. 3 shows a device for generating a result image according to another embodiment. Fig. 4 shows an exemplary procedure for generating a result image from a photorealistic image according to one embodiment. Fig. 5 shows a system for determining the physical behavior of a measurement object, a lighting system, or a luminaire according to one embodiment. Fig. 6 shows a measurement arrangement suitable for determining the bidirectional reflectance distribution function. Fig. 7 shows a holding device and a measurement sample. Fig. 8 shows a measurement arrangement suitable for determining the bidirectional transmission distribution function. Fig. 9 shows a neural network 995 according to one embodiment.Figure 10 shows a device for calculating a luminous radiance for each pixel of a group of one or more pixels of a camera system according to one embodiment. Figure 11 shows an arrangement according to one embodiment, comprising a lighting system, an object, and a camera system. Figure 12 shows a system according to a group of embodiments. Figure 13 shows a system according to a further embodiment.

[0057] The following sections provide concepts for a sensor-realistic simulation framework and for virtual image processing.

[0058] To obtain the resulting image for a test scene concept, the design can either be implemented practically as an experiment or photo- or sensor-realistic simulation tools of computer graphics can be used.

[0059] For example, the physically based rendering framework Mitsuba® could be used. The basic structure of this software library is described in Abb. 1 This is illustrated. Most currently available rendering frameworks are based on such a structure.

[0060] In order to generate a sensor-realistic image, the scene, i.e., all relevant objects, all active light sources and the camera system used, should be modeled with the relevant properties.

[0061] The objects in the scene can be geometrically described, for example, by their CAD models. Their reflectance properties can be modeled using so-called bidirectional reflectance distribution functions (BRDFs). For (partially) transparent objects, the refractive index or refractive index distribution and the volume scattering behavior of the objects can also be specified. Parameterizable models exist for these object properties, which can be used in Mitsuba® or other renderers.

[0062] Similarly, the light sources present in the scene should also be characterized. This includes, in particular, information about the emitted spectrum, the directional distribution of the outgoing rays or radiances, and, if applicable, the geometric dimensions of the light source.

[0063] Furthermore, a model of the camera system used should be provided. For example, a camera system can be modeled in Mitsuba®< using typical parameters such as focal length, image distance, aperture, number, spacing, and dimensions of the pixels.

[0064] Fig. 2 This shows, as an example, a schematic representation of the functionality of the Mitsuba® rendering framework.

[0065] To obtain the most realistic camera system image possible, the rendering framework selects individual continuous pixel coordinates of the camera system one after the other, for example according to a specific random pattern.

[0066] The corresponding line of sights are then generated via the camera system model, and their path through the modeled scene is tracked. The line of sight can be reflected by objects, absorbed, or directed towards a light source. If the light source emits light in the corresponding direction, the emitted radiance is propagated back to the original sensor pixel along the optical path of the tracked line of sight. The properties of the object surfaces or volumes involved in the path are taken into account accordingly.

[0067] The resulting spectrum is then aggregated together with the selected sensor coordinate by a so-called film component to successively generate the complete sensor image or signal. The process accordingly Fig. 2 This process is repeated until sufficient image quality is achieved or until the generated image no longer changes.

[0068] Embodiments of the invention are now based on using this methodology to simulate, in particular, previously unimplemented or unrealized effects of an image recording that deviate from the ideal and to integrate them into the framework in a comparable manner.

[0069] Fig. 1 Figure 1 shows a device for generating a result image according to an embodiment, wherein the device is configured to generate the result image in such a way as to simulate a recording of a scene by a camera system.

[0070] The device is configured to simulate the presence of a lighting system comprising one or more light sources illuminating the scene. Furthermore, the device is configured to simulate the presence of one or more objects in the scene.

[0071] The device comprises a camera system parameter setting unit 110, which is configured to set one or more camera system parameters, wherein each of the one or more camera system parameters defines a property of the camera system.

[0072] Furthermore, the device includes a lighting system parameter setting unit 120, which is designed to set one or more lighting system parameters, wherein each of the one or more lighting system parameters defines a property of the lighting system.

[0073] Furthermore, the device includes an object parameter setting unit 130, which is designed for each of the one or more objects to set one or more object parameters of this object, wherein each of the one or more object parameters of this object defines a property of this object.

[0074] Furthermore, the device includes an image generation unit 140, which is configured to generate the result image depending on the one or more camera system parameters and depending on the one or more lighting system parameters and depending on the one or more object parameters of the one or more objects.

[0075] In addition to the one or more camera system parameters, the one or more lighting system parameters, and the one or more object parameters of the one or more objects, the device is designed to set one or more additional parameters.

[0076] Furthermore, the image generation unit 140 is designed to generate the resulting image depending on one or more additional parameters. At least one of these additional parameters depends on a disturbance effect or an effect caused by a non-ideal property of the camera system or the lighting system.

[0077] For example, in embodiments, a non-ideal property of the camera system can be a deviation of an actual state of this property of the camera system from a predefined target state of this property of the camera system.

[0078] For example, in embodiments, a non-ideal property of the lighting system can be a deviation of an actual state of this property of the lighting system from a predefined target state of this property of the lighting system.

[0079] In embodiments, for example, at least one of the one or more additional parameters can depend on the effect caused by the non-ideal property of the camera system, wherein the non-ideal property of the camera system can be, for example, contamination of an optical component of the camera system, or, for example, a defect in an optical component of the camera system, or, for example, a faulty pixel, or, for example, an imaging error in the optics of the camera system.

[0080] In further embodiments, for example, at least one of the one or more additional parameters can depend on the effect caused by the non-ideal property of the lighting system, wherein the non-ideal property of the lighting system can be, for example, contamination of the lighting system, or, for example, a reduced luminosity of the lighting system compared to a predefined, optimal luminosity of the lighting system.

[0081] In embodiments, for example, at least one of the one or more additional parameters can depend on the effect caused by the non-ideal property of the camera system, which is due to aging of the camera system. Or at least one of the one or more additional parameters can depend on the effect caused by the non-ideal property of the lighting system, which is due to aging of the lighting system.

[0082] In such embodiments, the additional parameter therefore ultimately depends on the aging of the camera system or the aging of the lighting system.

[0083] In various embodiments, previously unimplemented or unrealized effects deviating from the ideal can be taken into account. These effects are simulated in an image capture and integrated into the framework, and are based on phenomena such as the following: Soiling of the object's surface.

[0084] Contamination of the optical components of the camera or lighting system.

[0085] Defects in the optical components of the camera or lighting system (e.g. mirrors, lenses).

[0086] Faulty pixels in the camera system's sensor.

[0087] Non-ideal effects in radiation transmission through air.

[0088] Image defects of the optics of the camera or lighting system.

[0089] Aging of the lighting system.

[0090] Impact of external conditions (e.g. heat).

[0091] Effect of ambient light.

[0092] The effect of an object's material properties.

[0093] According to one embodiment, the object parameter setting unit 130 can be configured, in addition to the one or more object parameters of the one or more objects, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on a contamination of a surface of one of the objects.

[0094] In one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one or more additional parameters, wherein this one or more additional parameters can depend on contamination of an optical component of the camera or the lighting system.

[0095] According to one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on a defect in an optical component of the camera system.

[0096] In one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on a defective pixel.

[0097] According to one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on an (in particular non-ideal) effect of radiation transmission through air.

[0098] In one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on an imaging error of an optic of the camera system.

[0099] According to one embodiment, the lighting system parameter setting unit 120 can be configured, in addition to the one or more lighting system parameters, to set one or more additional parameters, wherein this one of the one or more additional parameters may depend on an aging of the lighting system.

[0100] In one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on an ambient temperature.

[0101] According to one embodiment, the camera system parameter setting unit 110 can be configured, in addition to the one or more camera system parameters, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on extraneous light that is not emitted by the one or more light sources.

[0102] According to one embodiment, the object parameter setting unit 130 can be configured, in addition to the one or more object parameters of the one or more objects, to set one of the one or more additional parameters, wherein this one of the one or more additional parameters can depend on a material property of one of the objects.

[0103] By extending the simulation of the complete image acquisition of a product under both ideal and non-ideal conditions, it is possible to design the image processing system under realistic (non-ideal) conditions. When implementing the algorithms, their real-time capability must be considered for the specific application, for example, through simulations with varying levels of accuracy in different scene areas (pyramid approach). This also impacts the models and algorithms under consideration, as they must be parameterizable with respect to these external requirements.

[0104] By simply varying the camera system parameters, different approaches can be evaluated quickly and cost-effectively using this method. The resulting images can also be used, in particular, for testing or training (e.g., deep learning) downstream image processing methods.

[0105] According to one embodiment, the image generation unit 140 can be configured to additionally generate the image depending on a luminous radiance of each light ray of one or more light rays, wherein each of the one or more light rays originates from one of the one or more light sources, or from an object of one or more objects, by reflecting another light ray from this object or by transmitting the further light ray to this object.

[0106] In one embodiment, the image generation unit 140 can be configured to determine a luminous radiance of one or more light beams using a bidirectional reflectance distribution function or using a bidirectional transmission distribution function.

[0107] The following concepts are provided for correcting a photorealistic image to obtain a sensor-realistic result image.

[0108] These embodiments are based on modifying an image that has been generated with a conventional image generator (renderer) (i.e., a photorealistic image) so that it corresponds to an image that can actually be generated using an image processing system.

[0109] Conventional renderers from computer graphics (e.g., Mitsuba®, Blender®, POV-Ray®) generate synthetic images by abstracting from physical reality during image acquisition. This eliminates influences during image acquisition that would typically lead to image degradation compared to an ideal optical representation of the scene.

[0110] Considering such degradation effects is unnecessary in typical renderer applications, as the primary goal is usually to visualize scenes for better clarity for human viewers. Omitting these effects has several generally positive consequences. First, the images look good. Second, existing renderers cannot describe or model degradation effects; they lack the methods / algorithms to account for them. Third, it saves the computational time that would otherwise be required to consider degradation effects.

[0111] Consequently, while images generated with traditional renderers are well-suited for many synthetic image applications (e.g., assessing product appearance or evaluating the visibility of object features from the camera system's position), they are particularly unsuitable for the realistic design of automated visual inspection systems. This is because such systems require considering all relevant optical properties of the components involved in image acquisition. Examples of tasks that cannot yet be satisfactorily solved include assessing the measurability of geometric scene properties or evaluating the detectability of surface defects.

[0112] Although newer renderers increasingly take optical error influences into account, these are mostly limited to relatively easy-to-model properties of the imaging optics, the camera system, the scene and the medium between the scene and the camera system.

[0113] Influences already taken into account include, for example, imaging using a lens system with limited depth of field instead of the idealized pinhole camera model; fog or smoke in the medium between the scene and the camera system; transparency and translucency of scene objects; and multiple reflections of the light from the lighting system on scene objects.

[0114] Embodiments of the invention now provide to first generate one or more first images in a conventional manner and to use these one or more first images appropriately to take degeneration influences into account.

[0115] Fig. 3 Figure 1 shows a device for generating a result image according to an embodiment, wherein the device is configured to generate the result image by simulating a recording of a scene by a camera system. The device is configured to simulate the presence of a lighting system comprising one or more light sources illuminating the scene, and the device is configured to simulate the presence of one or more objects in the scene.

[0116] The device comprises a camera system parameter setting unit 110, which is configured to set one or more camera system parameters, wherein each of the one or more camera system parameters defines a property of the camera system.

[0117] Furthermore, the device includes a lighting system parameter setting unit 120, which is configured to set one or more lighting system parameters, each of the one or more lighting system parameters defining a property of the lighting system.

[0118] Furthermore, the device includes an object parameter setting unit 130, which is designed for each of the one or more objects to set one or more object parameters of this object, wherein each of the one or more object parameters of this object defines a property of this object.

[0119] Furthermore, the device comprises an image generation unit 140, which is configured to generate one or more first images depending on the one or more camera system parameters, the one or more lighting system parameters, and the one or more object parameters of the one or more objects. The image generation unit 140 is configured to generate the result image using the one or more first images.

[0120] The modification of images generated by conventional renderers (hereinafter referred to as photorealistic images) can now take into account several additional sources of influence.

[0121] For example, in embodiments, a detailed modeling of the emission characteristics of the light source can be carried out.

[0122] Furthermore, some embodiments can allow the simulation of a depth-staggered scene with desired scene objects and interfering objects (e.g., flying insects in the beam path).

[0123] Furthermore, some embodiments can allow a simulation of the dynamics of the lighting system (e.g., flashing, sinusoidal or other modulation) and the scene components (e.g., flying chips, swirling dust).

[0124] Furthermore, some designs can simulate contamination of the imaging optics.

[0125] Furthermore, some embodiments can allow the simulation of individual imaging errors of the imaging optics (e.g., non-rotationally symmetric aberrations, field curvature).

[0126] Furthermore, some embodiments can take into account properties in spectral ranges adjacent to the visible spectrum (i.e., IR and UV).

[0127] Other embodiments can take into account the coherence of the light from the lighting source.

[0128] These influencing factors are taken into account by modifying the photorealistic images using image processing operators so that these operators reproduce the effects of the influencing factors as accurately as possible. Implementations can, for example, perform the following steps: As a first step, the influencing factors to be considered can be selected: Depending on the image processing task to be solved and the appropriate level of detail in the image acquisition, the influencing factors to be considered can be chosen. This selection can take into account the expected influences and the strength of their impact on the resulting realistic image.

[0129] In a second step, the operators can be modeled for all sources of influence to be considered:

[0130] The effects of the influencing factors on the photorealistic images are modeled individually for each influencing factor and represented by operators. The following influencing factors can be modeled using the described operators as examples: In one embodiment (a), a detailed model of the light source's emission characteristics can be created. For this purpose, several ideal images with different light sources are generated such that the actual emission characteristics of the light source result from the superposition of the light sources (summing of the generated light fields). The resulting image is obtained by combining, e.g., summing, the individual photorealistic images.

[0131] In a further embodiment (b), a scene with varying depths can be generated with desired scene objects and distracting objects: Several photorealistic images can be generated in which the depicted objects are rendered as if they were each within the depth of field. The resulting blur, caused by the objects actually being outside the depth of field, is then accounted for by convolution with a depth-dependent aperture function. The resulting image is obtained by combining, for example, summing the photorealistic images, while also taking into account the visibility of the individual scene components.

[0132] According to a further embodiment (c), the dynamics of the lighting system and the scene elements can be taken into account. For this purpose, several photorealistic images are generated, depicting the scene with different positions of the lighting system and the moving scene elements. As in b., a blurred image is taken into account, and the resulting image is obtained by combining, e.g., summing, the photorealistic images, taking visibility conditions into account.

[0133] In a further embodiment (d), contamination of the imaging optics can be simulated. Depending on the position of the contamination in the lens system, such influences result in an additive superposition of the highly blurred contamination (if the contamination is located in front of the lens or on the front lens) or a change in the aperture function to account for the aperture diaphragm (if the contamination is located in the plane of the aperture diaphragm within the lens system). The operator is therefore an additive operator in the first case or a convolution operator in the second case, whereby intermediate stages or superpositions of these two operators are also useful depending on the position of the contamination.

[0134] Furthermore, in another embodiment, individual imaging aberrations (e) of the imaging optics can be simulated. Since the effects of such aberrations depend on the viewing direction and the resulting position in the image, they can only be taken into account by spatially variable operators. The operators cause a smearing effect (e.g., modeled by convolution of the image content with a convolution kernel) that is adjusted depending on the image position. Image field curvature also causes a blurring of the image depending on the image position. To account for individual imaging aberrations, the properties of the imaging optics are determined once for each imaging system.

[0135] For example, in a third step, the parameters for the operators can be defined.

[0136] All sources of influence, for which operators are used in step 2, have parameters. These include, for example, the positions, radiant power, and emission characteristics for the aforementioned source a; the distances of the scene elements for b; the speeds of the illumination and the scene elements for c; the filter functions and superposition operators for d; and the location-dependent blurring operators in case e. Some of the parameters depend on the scene being mapped (e.g., cases a, b, and c), while others depend on the imaging system used (e.g., cases d and e). Accordingly, in the latter case, the parameters can be retrieved from a database for the imaging system, while in the former case, the parameters are determined from the scene model.

[0137] For example, in a fourth step, the operators can be applied to the photorealistic images. The operators selected and parameterized in steps two and three, for instance, to account for the sources of influence, can then be applied to the photorealistic image. The order in which the operators are applied is determined by physical reality: first, those operators that consider scene influences are applied, followed by those that consider influences from the imaging system.

[0138] Fig. 4 The above-described procedure for generating a result image from a photorealistic image is shown as an example flowchart. The order of the steps "Description of the imaging system," "Operators for the sources of influence," and "Consideration of the sources of influence" can be grouped differently and, for example, performed in a single step. The description of the imaging system can also be predefined.

[0139] According to the above descriptions, in one embodiment the device can be configured to simulate the presence of two or more light sources. The image generation unit 140 can be configured to generate two or more first images, wherein the image generation unit 140 is configured to generate each first image of the two or more first images in such a way that the presence of only one of the two or more light sources is simulated for that first image. The image generation unit 140 can be configured to generate the resulting image by summing the two or more first images.

[0140] As described above, the image generation unit 140 can be configured according to one embodiment to generate two or more first images, wherein the image generation unit 140 is configured to generate each first image of the two or more first images such that the one or more objects are located within a depth of field area. The image generation unit 140 can be configured to generate the resulting image by summing the two or more first images.

[0141] Furthermore, the image generation unit 140, as described above, can be configured in an embodiment to generate two or more first images, wherein the image generation unit 140 is configured such that, for at least one of the two or more light sources, it is simulated that this at least one light source is located at a different position in each of the two or more first images. The image generation unit 140 can be configured to generate the result image by summing the two or more first images.

[0142] Furthermore, the image generation unit 140 can be configured, in an embodiment as described above, to generate two or more first images by adding a simulation of blurred dirt to each of the two or more first images. The image generation unit 140 can be configured to generate the resulting image by summing or by convolution of the two or more first images.

[0143] In a further embodiment, the image generation unit 140, as shown above, can be configured to fold an image content from at least one of the one or more first images with a convolution kernel.

[0144] Some of the embodiments of Fig. 3 The devices described above may, for example, include one or more additional parameters. Such additional parameters may, for instance, specify the degeneration effect being simulated. In some embodiments, the device may... Fig. 3 In addition to the one or more camera system parameters, the one or more lighting system parameters of the one or more light sources, and the one or more object parameters of the one or more objects, the image generation unit 140 is configured to set one or more additional parameters. The image generation unit can be configured to generate the result image, or at least one of the first one or more images, using the one or more additional parameters. At least one of the one or more additional parameters can depend on a disturbance effect or an effect caused by non-ideal properties of the camera system or the lighting system, for example, by sensor aging or by aging of the one or more lighting units.

[0145] Further embodiments are described below. These further embodiments provide improved concepts for describing the reflection and transmission behavior of materials.

[0146] The reflection and transmission behavior of materials is determined, for example, using goniometric measurement technology under directed illumination, generating data sets of the location- and spatial direction-dependent reflection and transmission behavior, which corresponds to a function f r (Θ i ,Φ i ,Θ o ,Φ o ) with four parameters (BRDF, BTDF, also collectively called BSDF = bidirectional scattering distribution function) corresponds to a function f r (Θ i ,Φ i ,Θ o ,Φ o ,λ ) with five parameters, if the wavelengths λ the light is taken into account or a function f r (Θ i ,Φ i ,Θ o ,Φ o ,λ,x e ,y e ) with seven parameters (BSSRDF = bidirectional scattering-surface reflectance distribution function), when the wavelength λ and the location of the light emission must be taken into account.

[0147] BRDF denotes a bidirectional reflectance distribution function. BTDF denotes a bidirectional transmission distribution function.

[0148] For example, the bidirectional reflectance distribution function or the bidirectional transmittance distribution function can specify the ratio of outgoing light to incident light. The output of the bidirectional reflectance distribution function or the bidirectional transmittance distribution function can therefore be, for example, a value y = fr(...), where, for example, 0 ≤ y < 1.

[0149] At Θ i This refers to the zenith angle of an incident light ray.

[0150] At Φ i This refers to the azimuth angle of the incident light ray.

[0151] At Θ o This refers to the zenith angle of an outgoing light beam.

[0152] At Φ o This refers to the azimuth angle of the outgoing light beam.

[0153] λ specifies the wavelength of the light.

[0154] x e specifies an x-position on the object from which the outgoing light beam originates.

[0155] y e specifies a y-position on the object from which the outgoing light beam originates.

[0156] Fig. 6 shows a suitable measurement setup that is suitable for determining the bidirectional reflectance distribution function.

[0157] A measurement sample 210 rests on a table 220. The measurement sample is illuminated by a lighting system 230. The lighting system can be rotated along a rail 250 around the table 220 on which the measurement sample 210 is located. Such a rotation around the measurement sample is a rotation of the lighting system 230 around the table 220 with the measurement sample 210 in the horizontal plane. It specifies the azimuth angle Φ. i of an incident light ray 245.

[0158] Additionally, the lighting system 230, located on a first mounting arm 260, can be adjusted in height. The vertical position of the lighting system 230 specifies the zenith angle Θ. i of the incident light ray 245.

[0159] Furthermore, a detector 270 is provided, which can be rotated, for example, along a rail 250 around the table 220 on which the measurement sample 210 is located. Such a rotation around the measurement sample 210 is a rotation of the detector 270 around the table 220 with the measurement sample 210 in the horizontal plane. It specifies the azimuth angle Φ. o of an outgoing light beam 255, which was reflected by the measuring sample 210.

[0160] Additionally, the detector 270, which is located on a second mounting arm 280, can be adjusted in height. The vertical position of the detector 270 specifies the zenith angle Θ. o of the outgoing light beam 255.

[0161] A control device 291 allows the position of the lighting system 230 to be adjusted. That is, by means of the control device 291, the azimuth angle of the lighting system 230, and thus the azimuth angle Φ, can be adjusted. i of the incident light beam 245; and likewise, the zenith angle of the lighting system 230 and thus the zenith angle Θ can be adjusted by means of the control device 291. i The angle of the incoming light beam can be set to 245.

[0162] Furthermore, the control device 291 allows the position of the detector 270 to be adjusted. That is, the azimuth angle of the detector 270, and thus the azimuth angle Φ, can be adjusted using the control device 291. o of the outgoing light beam 255, which is detected by the detector 270. Likewise, the zenith angle of the detector 270 and thus the zenith angle Θ can be adjusted by means of the control device 291. o The azimuth angle Φ of the outgoing light beam 255, which is detected by the detector 270, is set. It is understood that, as a result of the reflection of the incoming light beam 245, a multitude of light beams typically emanate from the object 210 being measured. Therefore, what is set by adjusting the detector 270 is the azimuth angle Φ. o and the zenith angle Θ o of the reflected light beam 255 that is detected by the detector 270.

[0163] In some embodiments, the control device 291 may be designed to control the wavelength λ to adjust the light emitted by the lighting system 230.

[0164] A detector data processor 292 receives the measurement data from the detector 270 (e.g., a camera system). The detector data processor 292 can be configured to store and / or further process the measurement data from the detector 270 and / or output it to a user.

[0165] The detector data processor 292 can be configured to obtain, for example, a measurement of the light intensity of the light beam 255 from the detector 270. The detector data processor 292 can then be configured to determine an output value of the bidirectional reflectance distribution function for the given parameter input values.

[0166] The control device 291 and the detector data processor 292 can be implemented separately. In other embodiments, the control device 291 and the detector data processor 292 can be implemented together in a control and receiver unit 290.

[0167] The first mounting arm 260 and the second mounting arm can be located on the same rail 250 or on two separate rails. In other embodiments, alternative configurations are provided instead of rails and / or mounting arms to adjust the position of the lighting system 230 and the detector 270.

[0168] The measuring setup of the Fig. 6 It can also be used to measure the bidirectional transmission distribution function. For this purpose, the illumination system 230 and the detector 270 simply need to be positioned relative to each other so that the detector detects light rays that have passed through the sample 210. Instead of detecting light rays reflected from the sample 210, the detector 270 or the illumination system 230 must therefore be positioned so that the detector 270 detects light rays that have passed through the sample.

[0169] Instead of using the table 220, on which the measurement sample 210 lies, as a holding device for the measurement sample, another suitable holding device can be used which allows light rays passing through the measurement object 210 from the light source to reach the detector 270 without interference.

[0170] Fig. 7 Figure 221 shows such a holding device 221, in which the measuring sample 210 is clamped between two, for example metallic, jaws 222, 223 of the holding device 221.

[0171] A light beam 245 emitted in front of the lighting system 230 falls on the measuring sample 210, penetrates the measuring sample 210 and exits (among other things) as a light beam 255 emanating from the measuring sample 210, and is finally received by the detector 270.

[0172] Furthermore, embodiments provide improved concepts for determining the emission characteristics of more or less extended lamps.

[0173] The emission characteristics of lamps or luminaires of varying sizes can be determined using goniometric near-field measurement techniques. This generates data sets of the location- and spatial-direction-dependent light intensity distribution, which is a function I r (Θ o ,Φ o , x e , y e ) with four parameters or a function I r (Θ o ,Φ o , x e , y e ,λ ) with five parameters corresponds when the wavelengths λ The light must be taken into account.

[0174] At Θ o This refers to the zenith angle of the light beam emanating from the lamp.

[0175] At Φ o This refers to the azimuth angle of the light beam emanating from the lamp.

[0176] x e specifies an x-position on the luminaire from which the outgoing light beam originates.

[0177] y e specifies a y-position on the luminaire from which the outgoing light beam originates.

[0178] λ specifies the wavelength of the light.

[0179] Fig. 8 shows a measuring setup suitable for determining the bidirectional transmission distribution function.

[0180] In Fig. 8 A luminaire 310 is shown.

[0181] Furthermore, a detector 370 is provided, which can be rotated around the light 310, for example, along a rail 350. Such a rotation around the light 310 is a rotation of the detector 370 in the horizontal plane. The rotation specifies the azimuth angle Φ. o of an outgoing light beam 355, which emanates from the luminaire 310.

[0182] Additionally, the detector 370, which is located on a mounting arm 380, can be adjusted in height. The vertical position of the detector 370 specifies the zenith angle Θ. o of the outgoing light beam 355.

[0183] Furthermore, in Fig. 8 A virtual coordinate system 343 is drawn. The position 347 of the point from which the light beam 355 originates, which is detected by the detector, can be determined by its x-position. x e and its y-position y e Specify 343 in the virtual coordinate system.

[0184] A control device 391 allows the position of the detector 370 to be adjusted. That is, the azimuth angle of the detector 370, and thus the azimuth angle Φ, can be adjusted by means of the control device 391. o of the outgoing light beam 355, which is detected by the detector 370. Likewise, the zenith angle of the detector 370 and thus the zenith angle Θ can be adjusted by means of the control device 391. o The outgoing light beam 355, which is detected by the detector 370, is adjusted. It is understood that a multitude of light beams emanate from the luminaire 310. Therefore, what is adjusted by means of the detector 370 is the azimuth angle Φ o and the zenith angle Θ. o of the reflected light beam 355 that is detected by the detector 370.

[0185] In some embodiments, the control device 391 may be designed to control the wavelength λ to adjust the light emitted by the lamp 310.

[0186] A detector data processor 392 receives the measurement data from the detector 370 (e.g., a camera system). The detector data processor 392 can be configured to store and / or further process the measurement data from the detector 370 and / or output it to a user.

[0187] The detector data processor 392 can be configured to receive, for example, a measurement of the light intensity of the light beam 355 from the detector 370. The detector data processor 392 is designed to determine an output value of the bidirectional transmission distribution function for the given parameter input values.

[0188] The control device 391 and the detector data processor 392 can be implemented separately. In other embodiments, the control device 391 and the detector data processor 392 can be implemented together in a control and receiver unit 390.

[0189] In other embodiments, different configurations of a rail 350 and / or a mounting arm 380 are provided to adjust the position of the detector 370.

[0190] In practice, intensity distributions for lamps or luminaires are usually provided in the form of IES (Illuminating Engineering Society) files. The number of data points depends on the size of the lamps or luminaires and the number of measurement points. n on the lamps or lights and the number of angular positions of the measurements. For a scan of the entire space in 5° increments, this results in 36 x 72 measured values ​​per measuring point, i.e., n Measuring points approx. n 2,600 measured values ​​multiplied by all wavelengths considered λ , i.e., the description of the radiation pattern of more or less extended lamps or luminaires is used for large ncomplex. Therefore, the emission characteristics of more or less extended lamps or luminaires for the ray tracing method are often idealized by mathematical functions or only by geometric specifications and constants (e.g. Lambertian radiator, homogeneous intensity distribution, etc.), which impairs the physical correctness of the image simulation.

[0191] This data can be provided in the form of measurement files. For a half-space scan in 5° steps, this results in approximately 1,300 x 1,300 = 1,690,000 measurements for the BRDF or BTDF, multiplied by all considered wavelengths. λ Multiplied by all considered light exit points (BSSRDF), the BRDF and BTDF are extremely complex. For this reason, the BRDF and BTDF are almost always approximated by more or less physically accurate mathematical functions, which significantly compromises the physical accuracy of the image simulation.

[0192] The emission characteristics of real-world, more or less extended lamps and luminaires, and the reflection and transmission behavior of real materials (BRDF and BTDF, respectively), can be measured goniometrically and provided as extensive measurement data. For use in ray tracing methods, this data is typically approximated / modeled by more or less physically accurate mathematical functions. Material-specific approximations / models are particularly useful for the reflection and transmission behavior of individual materials; that is, no universally valid, parameterizable models for the emission characteristics of extended lamps and luminaires, or for the reflection and transmission behavior of arbitrary materials, are currently known that reproduce the respective behavior with sufficient realism.

[0193] Embodiments of the invention now consist of using trainable systems, specifically artificial neural networks, for modeling the emission characteristics of any real existing, more or less extended lamps and luminaires, or for modeling the reflection or transmission behavior of any real existing materials.

[0194] Artificial neural networks are characterized, among other things, by their suitability for interpolating complex functions and their ability to generalize even with sample-based training data. Furthermore, generating input data using a propagation function in trained networks is relatively efficient (matrix-vector multiplication, computable very efficiently on a GPU). Additionally, their topology is independent of the implementation of the learning and propagation functions and is typically stored in a network configuration file, including the weighting factors. The network configuration file of the trained network therefore represents a data exchange format for a given problem.

[0195] In some embodiments, artificial neural networks are used, wherein the neural networks are data structures consisting of several (for example, at least three) layers of interconnected artificial neurons, wherein, for example, the connections are so-called weight factors and the neurons have, for example, a propagation, activation, and output function, with an input and an output layer.

[0196] Fig. 9 Figure 995 shows a neural network according to one embodiment. Thus, the neural network 995 receives the angles of the Fig. 6 as inputs, namely Θ i : the zenith angle of an incident light ray; Φ i : the azimuth angle of the incident light ray; Θ o : the zenith angle of an outgoing light ray; and Φ o : the azimuth angle of the outgoing light beam.

[0197] The in Fig. 9 The boxes shown, 421, 422, 423, 424, 431, 432, 433, and 441, represent neurons. Each arrow pointing into one of the neurons is assigned a weight. Fig. 9 The weights g11, g12, g13, g14, g2131, g2132, g2133, g2231, g2232, g2233, g2331, g2332, g2333, g2431, g2432, g2433, g3141, g3241, g3341. Outputs from a neuron are always weighted by their respective weight, for example, multiplied, before being fed into the next neuron.

[0198] The respective neuron 421, 422, 423, 424, 431, 432, 433, 441 determines which functions or which (overall) function convert the (weighted) input values ​​of the respective neuron into an output value.

[0199] The neural network 995 learns by adjusting the weights of the neural network 995, in Fig. 9 the weights g 11 , g 12 , g 13 , g 14 , g 2131 , g 2132 , g 2133 , g 2231 , g 2232 , g 2233 , g 2331 , g 2332 , g 2333 , g 2431 , g 2432 , g 2433 , g 3141 , g 3241 , g 3341 . Thus, the neural network 995 has a large number of training data available, with each training data set containing the input parameters (in Fig. 9 : Θ i ; Φ i ; Θ o ; Φ o ) and the output value (in Fig. 9 : y). The weights are adjusted by a neural network (e.g., continuously with each additional training set).

[0200] This allows the neural network 995 to learn, for new input parameters, which output value to output by using the weights previously adjusted with the training data to calculate the output value.

[0201] A neuron often comprises three cascaded functions: a propagation function, an activation function, and an output function. For example, the propagation function links the neuron's weighted inputs together, perhaps by summing them. The activation function can then determine, depending on an input value, whether the result of the propagation function is passed on to the output function or, for example, set to zero. The output function can then be designed to further process and output the input received from the activation function.

[0202] For the (supervised) training of such structures, so-called learning methods exist with which the neural network 995, or more precisely the weighting factors, are modified so that they produce the desired network output data for a set of training input data. A trained neural network is then able to generate the corresponding output data, similar to the desired network output data, from input data that is similar to the training input data using a propagation function.

[0203] The embodiments of the invention now utilize the input parameters and output data provided by the measuring device of one of the Fig. 6 , Fig. 7 and Fig. 8 The data were obtained to train a neural network 995. As input data, the detector data processor 292; 392 provides either the value directly supplied by the detector 270; 370, for example a luminous intensity value, or a value derived from the value supplied by the detector 270; 370, such as the result of the bidirectional reflectance distribution function or the result of the bidirectional transmission distribution function.

[0204] In embodiments where the input parameters are used together with the result of the bidirectional reflectance distribution function or the result of the bidirectional transmission distribution function, the neural network 995 then functions as the bidirectional reflectance distribution function or as the bidirectional transmission distribution function.

[0205] After the neural network 995 has been trained with the data obtained from the respective measuring device, the trained neural network is then used to calculate the respective output value for further, different combinations of input parameters.

[0206] The artificial neural network 995 can be used to determine the emission characteristics of more or less extensive lamps or luminaires based on goniometric measurement data. I r , Θ o ,Φ o ,x e ,y e or I r ,Θ o , Φ o , x e , y e , λ train, whereby Θ o ,Φ o , x e , y e ) or Θ o ,Φ o ,x e , y e , λ the network input data and I r The desired network output is...

[0207] Similarly, the reflection and transmission behavior of materials can be determined using goniometric measurement data. f r (Θ i ,Φ i ,Θ o ,Φ o ) or f r (Θ i ,Φ i ,Θ o ,Φ o ,λ ) or f r (Θ i ,Φ i ,Θ o ,Φ o ,λ,x e ,y e ) train, where Θ i ,Φ i ,Θ o ,Φ o or Θ i ,Φ i ,Θ o ,Φ o ,λ or Θ i ,Φ i ,Θ o ,Φ o , λ , x e , y e the network input data and f r The desired network output is...

[0208] Fig. 5 Figure 1 shows a general embodiment which, based on the above concepts, shows a system for determining the physical behavior of a measurement object or a lighting system or a luminaire according to one embodiment.

[0209] The system includes a detector 170, a control device 191, a detector data processor 192 and a neural network 195.

[0210] The control device 191 is configured to successively set a plurality of parameter sets, wherein each parameter set of the plurality of parameter sets assigns a parameter value to each parameter of a group of two or more parameters, wherein each pair of two parameter sets of the plurality of parameter sets differs in at least one parameter value of a parameter of the group of parameters. Furthermore, the control device 191 is configured, for each parameter set of the plurality of parameter sets, to adjust at least the detector 170 or the illumination system or the lamp (illumination system and lamp are defined in the following). Fig. 5 not shown; but see lighting system 230 of the Fig. 6 and Fig. 7 and the 310 lamp of the Fig. 8 ).

[0211] The detector 170 is configured to detect a light beam for each of the majority of parameter sets set by the control device 191 and to output a light beam detection value, which depends on the light beam, to the detector data processor 192, wherein the light beam depends on the lighting system or the lamp.

[0212] The detector data processor 192 is configured to receive each parameter set of the plurality of parameter sets set by the control device 191, wherein the detector data processor 192 is configured to obtain from the detector 170 the light beam detection value for each parameter set of the plurality of parameter sets, wherein the detector data processor 192 is configured to assign to each parameter set of the plurality of parameter sets a result value which is the light beam detection value of this parameter set or which depends on the light beam detection value of this parameter set.

[0213] Furthermore, the detector data processor 192 is set up to train the neural network 195 using the plurality of parameter sets and using the result value of each parameter set of the plurality of parameter sets.

[0214] Furthermore, the data detector processor 192 is designed to generate and output a result value for a parameter set that is not covered by the majority of parameter sets by using the neural network 195.

[0215] According to one embodiment, the control device 191 can be configured to adjust, for each parameter set of the plurality of parameter sets, at least the detector 170 or the illumination system or the lamp by adjusting the parameter set, by the control device 191 adjusting at least a first azimuth angle or a first zenith angle of the detector 170, or by the control device 191 adjusting at least a second azimuth angle or a second zenith angle of the illumination system or the lamp.

[0216] In one embodiment, the control device 191 can be configured to set the first azimuth angle and the first zenith angle of the detector 170 for each parameter set of the plurality of parameter sets.

[0217] According to one embodiment, the control device 191 can be configured to set the second azimuth angle and the second zenith angle of the illumination system for each parameter set of the plurality of parameter sets.

[0218] In one embodiment, the system can also include a holding device (in Fig. 5 not shown, but see table 220 in Fig. 6 , which is a holding device, and holding device 221 in Fig. 7 ) exhibit on which the object being measured (in Fig. 5 not shown, but see measurement sample 210 in Fig. 6 and Fig. 7 , which is a measurement object) is attachable, and wherein the system further comprises the illumination system, wherein the light beam detected by the detector 170 for each parameter set of the majority of parameter sets set by the control device 191 is a light beam which is created by reflection of a light beam emitted by the illumination system on the measurement object which is attached to the holding device.

[0219] According to the invention, the detector data processor 192 is configured to calculate the result value for each parameter set of the plurality of parameter sets as the result of a bidirectional reflectance distribution function depending on the light beam detection value of the parameter set.

[0220] In one embodiment, the system may further comprise a holding device on which the object to be measured can be mounted, and wherein the system further comprises the illumination system, wherein the light beam detected by the detector 170 for each of the majority of parameter sets set by the control device 191 is a light beam that is created by a light beam emitted by the illumination system transmitting the object to be measured which is mounted on the holding device.

[0221] According to one embodiment, the detector 170 can be configured to detect a light beam emitted by the lamp for each of the majority of parameter sets set by the control device 191.

[0222] In one embodiment, the control device 191 can be configured to set the position on the lamp for each parameter set of the plurality of parameter sets, so that the light beam which is detected by the detector 170 originates from the position on the lamp.

[0223] According to one embodiment, the control device 191 can be configured to set the position on the lamp for each parameter set of the plurality of parameter sets by setting at least one x-coordinate position and one y-coordinate position of that position.

[0224] According to one embodiment, the control device 191 can be configured to set a position in a linear axis system for each parameter set of the plurality of parameter sets, wherein the detector 170 can be a camera system with a matrix sensor.

[0225] In one embodiment, the control device 191 can be configured to set an aperture value for each parameter set of the plurality of parameter sets.

[0226] According to one embodiment, the control device 191 can be configured to set a wavelength value for each parameter set of the plurality of parameter sets, wherein a wavelength of the light beam detected by the detector 170 depends on the wavelength value.

[0227] In addition to the above-mentioned realistic modeling of the emission characteristics of more or less extended lamps or luminaires and the above-mentioned realistic modeling of the reflection and transmission behavior of materials, a realistic modeling of the imaging properties of real-world lenses is desirable for achieving the highest possible realism in the results of ray tracing methods. With regard to these methods, it is particularly relevant to know which light rays from the real world are imaged with what intensity onto which pixels in the image-side depth of field plane.

[0228] In order to determine this relationship metrologically, embodiments provide a measuring device that mechanically consists of an orthogonally arranged x / y-The system consists of a linear axis system and a goniometric tilting system arranged on it for two orthogonal angles Θ, Φ and a point-shaped laser light source with a small beam cross-section that is fixed in position with respect to the linear axis and tilting system.

[0229] One possible and practical arrangement of the linear axis / tilt system relative to the laser light source is such that the light source shines perpendicularly onto the tilt axes of the tilt system when the system is in its central position. The lens to be measured is then mounted on the linear axis / tilt system with an electronically readable camera system featuring a matrix sensor, ideally positioned so that its optical axis is perpendicular to the tilt axes of the tilt system. The camera system then serves as a measuring sensor for determining the image location and the radiance of the laser beam.

[0230] The measurement process now consists of measuring at arbitrary angles Θ o , Φ o of the tilting system at any location x e , y e the image locations of the linear axis system x i , y i and to determine the radiance of the laser beam on the camera system with matrix sensor by reading out the camera images.

[0231] In some embodiments, this can be achieved, for example, by scanning over a set of defined locations and angles, resulting in a data set that can be used for a function. g r ( x i , y i , Θ o , Φ o , x e ,y e ) with six parameters that correspond to at least the imaging properties "distortion" and "vignetting" at a fixed f-number f describes.

[0232] Should different aperture values ​​be used? f The measurement must be taken into account for the aperture values ​​to be considered. f be carried out, resulting in a function g r ( x i ,y i , Θ o , Φ o , x e , y e , f ) with seven parameters.

[0233] Regarding ray tracing methods, this function can be used to determine the light intensities of arbitrarily chosen light rays in object space from the pixel at the location x i , y i can be seen. By considering several such rays, the effect of depth of field is intrinsically taken into account.

[0234] Analogous to the modeling of the emission characteristics of more or less extended lamps or luminaires, or the reflection or transmission behavior of materials, the imaging properties "distortion" and "vignetting" can be determined in embodiments by trainable systems, especially artificial neural networks.

[0235] In various embodiments, the parameters can be x i , y i , Θ o und Φ o or x i , y i , Θ o , Φ o und f the network input data and g r , x e , y e The desired network output should be.

[0236] In some embodiments, the parameters can therefore be x i , y i , Θ o und Φ o be input parameters of a neural network, and g r , x e , y e They can be a network output of the neural network.

[0237] In other embodiments, the parameters can therefore be x i , y i , Θ o , Φ o und f be input parameters of a neural network, and g r ,x e ,y e They can be a network output of the neural network. f This is the frequency of the light.

[0238] An equivalent measuring arrangement according to one embodiment consists of imaging the laser beam onto a screen through the lens to be measured on the camera side and determining its position using a calibrated camera system.

[0239] The modeling of the emission characteristics of real-world lamps or luminaires, the modeling of the reflection and transmission behavior of real materials (BRDF and BTDF, respectively), and the modeling of the imaging properties of distortion and vignetting of real-world lenses using trainable systems, specifically artificial neural networks, offers several advantages: The models can be created from datasets of multiple identical lamps or luminaires, or of identical materials, ensuring that all data is considered. Furthermore, due to their interpolation and generalization capabilities, the networks also provide reliable output values ​​for untrained input variables. Moreover, the determination of output values ​​for given input variables is efficient.

[0240] The optical and qualitative imaging properties "spherical and chromatic aberration", "resolution" of lenses, the "exposure time" as well as the properties of the imaging sensors of camera systems (VDMA 1288) can be taken into account in a sensor-realistic manner using operators from image processing.

[0241] Further embodiments that provide improved concepts for ray tracing are described below.

[0242] Fig. 11 shows an arrangement according to an embodiment comprising a lighting system 430, an object 410 and a camera system 470.

[0243] According to further embodiments or extended embodiments, a more or less extended lamp or luminaire 430 has different (light) radiances in different spatial directions at various points on its surface. For the following, three such points, numbered 1, 2, and 3, on the lamp or luminaire 430 are considered.

[0244] From each of the three points 1, 2, 3 of the lamp or light fixture 430, light rays fall a) into the void, b) onto an object 410 of the scene, or c) into the camera system or sensor 470. For the following, two such points, numbered 4 and 5, on the object 410 are considered.

[0245] From each of the two points 4, 5 of object 410, light rays again fall a) into the void, b) onto another object in the scene (multiple reflection), or c) into the camera system or sensor 470. For the following, two pixels, numbered 6, 7, of the camera system or sensor 470 are considered.

[0246] Now, from each point 1, 2, 3 of the lamp or light fixture, 430 light rays 445 go to points 4 and 5 on the object 410, i.e., the solid angles and (light) radiances (from the Abstrahlcharakteristik ) of the rays at the locations of the lamp or light fixture 430 and the solid angles at the locations of the object 410 are known.

[0247] Furthermore, light rays 455 fall from each point 4 and 5 of the object onto the image points 6 and 7 of the camera system or the sensor 470, i.e., the solid angle and radiance (from which the Reflexions- bzw. Transmissionsverhalten The descriptive BRDF and BTDF of the reflected rays 455 are also known. Thus, the light intensity perceived by the two pixels 6 and 7 can be calculated as follows: The radiance (from the Abstrahlcharakteristik ) of the lamp or light 430 at points 1, 2 and 3 with respect to the solid angles of the light rays 445 between these points and points 4 and 5 on the object 410 is represented by the three vectors: 1 : L 1 , 4 L 1 , 5 2 : L 2 , 4 L 2 , 5 3 : L 3 , 4 L 3 , 5

[0248] By multiplying the radiances of the points of the lamp or luminaire 430 seen by points 4 and 5 of the object 410 with the design-related constant parameters of the BRDF or BTDF and summing the products (superposition principle), the radiances seen by the image points 6 and 7 of the camera system or the sensor 470 are calculated as follows: L 4 , 6 = L 1 , 4 ⋅ f ω 1 , 4 ω 4 , 6 + L 2 , 4 ⋅ f ω 2 , 4 ω 4 , 6 + L 3 , 4 ⋅ f ω 3 , 4 ω 4 , 6 L 4 , 7 = L 1 , 4 ⋅ f ω 1 , 4 ω 4 , 7 + L 2 , 4 ⋅ f ω 2 , 4 ω 4 , 7 + L 3 , 4 ⋅ f ω 3 , 4 ω 4 , 7 L 5 , 6 = L 1 , 5 ⋅ f ω 1 , 5 ω 5 , 6 + L 2 , 5 ⋅ f ω 2 , 5 ω 5 , 6 + L 3 , 5 ⋅ f ω 3 , 5 ω 5 , 6 L 5 , 7 = L 1 , 5 ⋅ f ω 1 , 5 ω 5 , 7 + L 2 , 5 ⋅ f ω 2 , 5 ω 5 , 7 + L 3 , 5 ⋅ f ω 3 , 5 ω 5 , 7 which corresponds to the following matrix-vector multiplications: L 4 , 6 L 4 , 7 = f ω 1 , 4 ω 4 , 6 f ω 2 , 4 ω 4 , 6 f ω 3 , 4 ω 4 , 6 f ω 1 , 4 ω 4 , 7 f ω 2 , 4 ω 4 , 7 f ω 3 , 4 ω 4 , 7 ⋅ L 1 , 4 L 2 , 4 L 3 , 4 L 5 , 6 L 5 , 7 = f ω 1 , 5 ω 5 , 6 f ω 2 , 5 ω 5 , 6 f ω 3 , 5 ω 5 , 6 f ω 1 , 5 ω 5 , 7 f ω 2 , 5 ω 5 , 7 f ω 3 , 5 ω 5 , 7 ⋅ L 1 , 5 L 2 , 5 L 3 , 5

[0249] This approach can also be used when light rays from a source point reach target points of multiple objects in the scene. Therefore, for each point under consideration, two vectors are appropriately allocated: an address vector specifying the storage location of the observed radiances, and a value vector representing the radiances of subsequent points. A matrix containing the constant parameters of the BRDF and BTDF, respectively, is also used. Assuming the parameters are constant is a simplification, as they depend on the angles of incidence and emission.

[0250] In some embodiments, the calculation can proceed according to the following scheme, for example: 1. Calculate the intersection point of the pixels mapped into object space with the nearest objects in the scene, taking into account the geometric mapping properties and the optical aperture. 2. Populate the remaining surfaces of all light sources and objects in a scene with position points of a defined density. 3. Connect all points in the scene that can be reached directly. 4. For each point on the objects in the scene, create the matrices of the BRDF and BTDF parameters that are constant for the combination of source and target points. 5. Initialize all radiance vector values ​​to zero, except for the values ​​representing the radiance of lamps or luminaires. Initialize these values ​​according to the emission characteristics. 6. Iteratively calculate the radiance vectors of all points whose seen radiance vectors have changed by more than a predefined threshold (termination criterion).

[0251] Since the matrices of the constant parameters of the BRDF or BTDF for the combination of source point and target point require considerable storage space, it is proposed to offload them to a mass storage device and only load them when needed, which can be done in parallel with currently running matrix-vector multiplications.

[0252] Since multiple reflections typically lead to marginal changes in the corresponding radiances very quickly due to diffuse scattering and absorption, the iteration depth can be small, and consequently the time required for the calculations is not too great.

[0253] Furthermore, in some embodiments, it is possible to check after initialization which light paths between light sources and camera system have only small or no contributions to the radiances seen by the individual pixels and can be eliminated before the actual calculation, resulting in smaller data volumes.

[0254] Alternatively or additionally, the density distribution can be adapted according to the importance of certain light paths. Alternatively, an approximately sensor-realistic image can first be generated using a classical ray tracing method, and the light paths can be recorded with high light intensities, which then represent the initial values ​​for the sensor-realistic image simulation method described above.

[0255] Overall, it is assumed that the resulting images from such a simulation are significantly more sensor-realistic due to the use of metrologically determined, rather than analytical, highly simplified characteristics (radiation characteristics and reflection or transmission behavior).

[0256] Fig. 10 shows a device for calculating a luminous radiance for each pixel of a group of one or more pixels of a camera system according to a general embodiment.

[0257] The device comprises a provisioning unit 175 and a calculation unit 180.

[0258] The provisioning unit 175 is designed to provide a luminous radiance for each luminaire point-object point pair, which consists of a point on a luminaire from a group of two or more points on the luminaire and a point on an object from a group of two or more points on the object.

[0259] Furthermore, the provisioning unit 175 is designed to provide a light radiance for each pixel-object point pair, which consists of a pixel from the group of one or more pixels of the camera system and a point on the object from the group of two or more points on the object.

[0260] The calculation unit 180 is designed to determine the luminous radiance of each pixel of the group of one or more pixels of the camera system depending on the luminous radiances of those pixel-object-point pairs to which this pixel belongs, and depending on the luminous radiances of the luminaire-object-point pairs provided by the provisioning unit 175.

[0261] According to one embodiment, the computing unit 180 can be configured to calculate the luminous radiance of each pixel of the group of one or more pixels of the camera system based on one or more matrix multiplications of two or more matrices.

[0262] In one embodiment, the matrix coefficients of the two or more matrices of the one or more matrix multiplications can be, or depend on, the radiance of the pixel-object pairs and the radiance of the luminaire-object pairs.

[0263] According to one embodiment, the computing unit 180 can be configured to calculate the luminous radiance of a pixel from the group of one or more pixels of the camera system, to determine the luminous radiance of each pixel of the group of one or more pixels of the camera system such that the computing unit 180 checks for the luminous radiance of each of the pixel-object-point pairs provided by the provisioning unit 175 whether the luminous radiance is greater than a threshold value, and to calculate the luminous radiance of this pixel using those luminous radiances of the pixel-object-point pairs that are greater than the threshold value, and not using those luminous radiances of the pixel-object-point pairs that are not greater than the threshold value.

[0264] In one embodiment, the computing unit 180 can be configured to calculate the luminous radiance of an image point from the group of one or more image points of the camera system, to determine the luminous radiance of each image point of the group of one or more image points of the camera system in such a way that the computing unit 180 checks for the luminous radiance of each of the image point-object point pairs to which this image point belongs whether the luminous radiance is greater than a threshold value, and to calculate the luminous radiance of this image point using those luminous radiances of the image point-object point pairs that are greater than the threshold value, and not using those luminous radiances of the image point-object point pairs that are not greater than the threshold value.

[0265] According to one embodiment, the provisioning unit 175 can be configured to provide the light radiances for the luminaire point-object point pairs and to provide the light radiances for the pixel-object point pairs by loading them from a database.

[0266] Fig. 12 shows a system according to a group of embodiments.

[0267] The system includes a device 1220 according to Fig. 1 or Fig. 3 and a System 1210 according to Fig. 5 .

[0268] In a first embodiment, the detector data processor 192 of the system 1210 is configured to calculate a result value for each parameter set of a plurality of parameter sets as the result of a bidirectional reflectance distribution function depending on a light beam detection value of the parameter set.

[0269] In a second embodiment, the detector data processor 192 of the system 1210 is configured to calculate a result value for each parameter set of a plurality of parameter sets as the result of a bidirectional transmission distribution function depending on a light beam detection value of the parameter set.

[0270] The detector data processor 192 of system 1210 is set up, the neural network 195 of the system of Fig. 5 to train using the plurality of parameter sets and using the result value of each parameter set of the plurality of parameter sets.

[0271] The data detector processor 192 of the system 1210 is designed to generate and output a result value for a parameter set that is not covered by the majority of the parameter sets by using the neural network 195 of the system 1210.

[0272] The device 1220 is designed to generate a result image depending on the result value.

[0273] Fig. 13 shows a system according to a further embodiment.

[0274] The system includes a device 1320 according to Fig. 1 or Fig. 3 and a device 1310 according to Fig. 10 .

[0275] The device 1310 is designed to calculate a luminous radiance for each pixel of a group of one or more pixels of a camera system.

[0276] The device 1320 is designed to generate a result image depending on the group of one or more pixels.

[0277] The following section discusses aspects of embodiments that relate to the automatic selection and optimization of the components of an image processing system.

[0278] These embodiments provide a novel approach to virtual image processing based on sensor-realistically simulated images, thereby creating a systematic methodology for developing image processing systems that significantly improves their quality and efficiency. This approach enables users to overcome the previous limitations on the use of image processing systems imposed by the effort involved in designing and parameterizing image acquisition and processing.

[0279] In some implementations, the image acquisition system and, based on this, the image processing system are created with all relevant components and their parameters. To approach an optimal configuration, test configurations and test sequences are simulated. This can include, for example, the simulation of the lighting system, camera system, and test object / test scene, as well as their arrangement relative to each other and all image processing components that are relevant to achieving the desired result.

[0280] To optimize the image processing components and to select image processing methods or image processing algorithms, influencing factors are varied in some embodiments and evaluated with regard to different criteria (e.g., performance, price, speed, size, inspection approach, precision, accessibility, visibility).

[0281] In special embodiments, extended synthetic pattern catalogs will also be used, allowing for the supplementation of real pattern catalogs, greater variation of the scene, and the simulation of defects. This provides image data for the design of image processing systems that cannot be generated in this diversity in reality, thus enabling an improved evaluation of the image processing system's quality.

[0282] The images generated in this way can be used to test and evaluate different components (i.e., camera, optics, lighting system with lighting optics), constellations, and image processing methods or algorithms before a real setup is created or before production of a product to be tested begins.

[0283] The provided embodiments achieve greater efficiency (in terms of time and cost) in the design and configuration of image processing systems. Improved design and more targeted optimization of these systems result in more powerful and cost-effective inspection solutions, leading to greater customer satisfaction.

[0284] This also makes it possible to open up applications of image processing systems that cannot be approached economically using conventional methods.

[0285] One technical application area for these embodiments is systems for industrial image processing and visual inspection. Such systems are used in a wide variety of designs in all sectors of industrial production.

[0286] Furthermore, embodiments are applicable in numerous other areas, for example in the field of development and testing of drug substances, in the field of cell-based detection of biomarkers, in the field of aesthetic dental prostheses, in facial and object recognition, as an abstraction layer for (inexpensive) camera systems, e.g. in mobile devices, in the field of driver assistance systems, in the field of Ambient Assisted Living (AAL), in metrology, in the field of object and object position recognition for automation, in human-machine cooperation, and in the field of simulation of individually manufactured products (Rapid Manufacturing, Additive Manufacturing).

[0287] Some embodiments therefore provide an automatic selection of the components of an image processing system and an automatic optimization of the parameterization by means of synthetic generation of images using CAD, modeling of the scene, the object, the lighting system, the camera system and the rendering.

[0288] In preferred embodiments, all properties relevant to solving the image processing task are taken into account during image synthesis. Sensor realism is achieved with respect to the modeling.

[0289] In some preferred embodiments, the rendering takes place with the accuracy required by the image processing system. Sensor realism is achieved with respect to the rendering.

[0290] In some preferred embodiments, the synthetic images cover parameter ranges for which no real images exist (e.g., because the object is not yet in production, or there are no defective parts). This results in an expansion of the parameter range.

[0291] In some preferred embodiments, machine learning methods (including deep learning) are used to optimize the parameters of the image processing system.

[0292] In some preferred embodiments, evolutionary algorithms are used to optimize the parameters of the image processing system.

[0293] In the above explanations, the term "image processing system" should not be understood rigidly, but can encompass different components depending on the task and application. For example, depending on the application's requirements, it may be necessary or advantageous to integrate a lighting system or a handling system into the image processing system. In other implementations, this may not be necessary.

[0294] In these explanations, the term "camera system" should not be interpreted rigidly, but can encompass different components depending on the task and application. A camera system can be a combination of a camera and imaging optics; in some embodiments, for example, the imaging optics can be omitted. A camera can comprise a device consisting of an image sensor and an electronic circuit for controlling the image sensor and processing the image information for transmission to an image processing unit. A mechanical housing for the image sensor and electronic circuitry can ensure their protection and the well-defined geometric position of the image sensor relative to a mechanical interface with the imaging optics. In some embodiments, some of these components may not be required.

[0295] Similarly, the term "lighting system" should not be interpreted rigidly in this context, but can encompass different components depending on the task and application. For example, it may include a lamp integrated into a luminaire. Additionally, it may include lighting optics, which determine the beam pattern of the lighting system. In some embodiments, some of these components may not be necessary.

[0296] A lighting system can therefore include (in particular) one or more light sources.

[0297] Regarding the one or more objects to be examined, some embodiments take into account, among other things, the reflectance and / or spectral properties of a surface and / or tolerances and / or actual geometric deviations from the target shape and / or roughness and / or preferred directions and spectrum of a texture and / or manufacturing parameters and / or dynamics (movement).

[0298] Regarding the scene, some embodiments take into account, among other things, the geometric arrangement of objects and / or a lighting system and / or a camera system and / or ambient light and / or vibrations.

[0299] Regarding the lighting system, some embodiments take into account, among other things, a number and / or arrangement of light sources and / or a geometric shape and / or an operating mode (e.g. flashed, continuous, pulsed) and / or a radiation behavior (e.g. diffuse, directional, telecentric, specified in ray files) and / or spectral properties.

[0300] Regarding the sensor of the camera system, in some embodiments, the type and / or size and / or resolution and / or sensitivity and / or dynamic range and / or noise behavior and / or defects and / or shutter and / or contamination of the sensor chip and / or camera characteristics are taken into account in accordance with EMVA 1288.

[0301] Regarding the imaging optics of the camera system, some embodiments take into account, among other things, the focal length and / or the light intensity and / or the aperture and / or distortions and / or the resolving power of the imaging optics.

[0302] In some embodiments, the selection of components for an image processing system and the automatic optimization of their parameters can encompass all components of the system, including mechanical, optical, and information technology components (e.g., lighting systems and luminaires, lighting optics, mounts, camera systems with sensors and imaging optics, image processing algorithms, image processing analysis algorithms, and parameter estimation and classification algorithms). During selection and optimization, some embodiments can also consider boundary conditions for some or all parameters, such as those arising from mechanical, optical, thermal, other physical, procedural, information technology, or logical constraints.

[0303] Although some aspects have been described in connection with a device, it is understood that these aspects also constitute a description of the corresponding process, such that a block or component of a device can also be understood as a corresponding process step or as a feature of a process step. Similarly, aspects described in connection with or as a process step also constitute a description of a corresponding block, detail, or feature of a corresponding device. Some or all of the process steps can be performed by (or using) a hardware apparatus, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or more of the key process steps can be performed by such an apparatus.

[0304] Depending on specific implementation requirements, embodiments of the invention can be implemented in hardware or in software, or at least partially in hardware or at least partially in software. The implementation can be carried out using a digital storage medium, for example, a floppy disk, a DVD, a Blu-ray disc, a CD, a ROM, a PROM, an EPROM, an EEPROM, a FLASH memory, a hard disk, or another magnetic or optical storage medium, on which electronically readable control signals are stored. These control signals can interact with, or interact with, a programmable computer system in such a way as to execute the respective method. Therefore, the digital storage medium can be computer-readable.

[0305] Some embodiments according to the invention therefore include a data carrier which has electronically readable control signals which are able to interact with a programmable computer system in such a way that one of the methods described herein is carried out.

[0306] In general, embodiments of the present invention can be implemented as a computer program product with a program code, wherein the program code is effective in carrying out one of the methods when the computer program product runs on a computer.

[0307] The program code can also be stored on a machine-readable medium, for example.

[0308] Other embodiments include a computer program for carrying out one of the methods described herein, wherein the computer program is stored on a machine-readable medium. In other words, an embodiment of the method according to the invention is thus a computer program that includes program code for carrying out one of the methods described herein when the computer program is executed on a computer.

[0309] Another embodiment of the methods according to the invention is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program for carrying out one of the methods described herein is recorded. The data carrier or the digital storage medium or the computer-readable medium is typically tangible and / or non-volatile.

[0310] Another embodiment of the method according to the invention is thus a data stream or a sequence of signals that represents the computer program for carrying out one of the methods described herein. The data stream or sequence of signals can be configured, for example, to be transferred via a data communication connection, such as the Internet.

[0311] Another embodiment comprises a processing device, for example a computer or a programmable logic device, which is configured or adapted to perform one of the methods described herein.

[0312] Another embodiment comprises a computer on which the computer program for performing one of the procedures described herein is installed.

[0313] Another embodiment of the invention comprises a device or system designed to transmit a computer program for carrying out at least one of the methods described herein to a receiver. The transmission can be, for example, electronic or optical. The receiver can be, for example, a computer, a mobile device, a storage device, or a similar device. The device or system can, for example, include a file server for transmitting the computer program to the receiver.

[0314] In some embodiments, a programmable logic device (for example, a field-programmable gate array, an FPGA) can be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field-programmable gate array can interact with a microprocessor to perform one of the methods described herein. Generally, in some embodiments, the methods are performed by any hardware device. This can be general-purpose hardware such as a computer processor (CPU) or method-specific hardware such as an ASIC.

[0315] The embodiments described above merely illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be obvious to other people skilled in the art. Therefore, it is intended that the invention be limited only by the scope of protection set forth in the following claims and not by the specific details presented herein by way of description and explanation of the embodiments.

Claims

1. A system for generating a result image, the system for generating the result image comprising: a system for determining a physical behavior of a measurement object (210) or an illumination system (230) or a lamp (310), and a device (1220) for generating the result image, wherein the device (1220) is configured to generate the result image such that a picture of a scene by a camera system is simulated, wherein the device (1220) is configured to simulate a presence of the illumination system comprising one or more light sources illuminating the scene, and wherein the device (1220) is configured to simulate that one or more objects are located in the scene, wherein the device (1220) comprises a camera system parameter adjusting unit (110), an illumination system parameter adjusting unit (120), an object parameter adjusting unit (130) and an image generation unit (140), wherein the camera system parameter adjusting unit (110) is configured to adjust one or more camera system parameters, each of the one or more camera system parameters defining a property of the camera system, wherein the illumination system parameter adjusting unit (120) is configured to adjust one or more illumination system parameters, each of the one or more illumination system parameters defining a property of the illumination system, wherein the object parameter adjusting unit (130) is configured, for each object of the one or more objects, to adjust one or more object parameters of this object, each of the one or more object parameters of this object defining a property of this object, and wherein the image generation unit (140) is configured to generate the result image depending on the one or more camera system parameters and depending on the one or more illumination system parameters and depending on the one or more object parameters of the one or more objects; and wherein the device of the device (1220) is configured to adjust, in addition to the one or more camera system parameters and in addition to the one or more illumination system parameters and in addition to the one or more object parameters of the one or more objects, one or more additional parameters; and wherein the image generation unit (140) is configured to generate the result image additionally depending on the one or more additional parameters, wherein at least one of the one or more additional parameters depends on a disturbance effect or on an effect caused by a non-ideal property of the camera system or the illumination system; or the image generation unit (140) is configured to generate one or more first images depending on the one or more camera system parameters and depending on the one or more illumination system parameters and depending on the one or more object parameters of the one or more objects, wherein the image generation unit (140) is configured to generate the result image using the one or more first images; wherein the system for determining the physical behavior comprises a detector (170; 270; 370), a control device (191; 291; 391), a detector data processor (192; 292; 392) and a neural network (195; 995), wherein the control device (191; 291; 391) is configured to successively adjust a plurality of parameter sets, wherein each parameter set of the plurality of parameter sets assigns a parameter value to each parameter of a group of two or more parameters, wherein each pair of two parameter sets of the plurality of parameter sets differs in at least one parameter value of a parameter of the group of parameters; and wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, by adjusting the parameter set, at least the detector (170; 270; 370) or the illumination system (230) or the lamp (310); wherein the detector (170; 270; 370) is configured to detect, for each parameter set of the plurality of parameter sets adjusted by the control device (191; 291; 391), a light beam and to output a light beam detection value depending on the light beam to the detector data processor (192; 292; 392), wherein the light beam depends on the illumination system (230) or on the lamp (310); and wherein the detector data processor (192; 292; 392) is configured to receive each parameter set of the plurality of parameter sets which is adjusted by the control device (191; 291; 391); and wherein the detector data processor (192; 292; 392) is configured to obtain, for each parameter set of the plurality of parameter sets, from the detector (170; 270; 370) the light beam detection value for this parameter set; wherein the detector data processor (192; 292; 392) is configured to assign, to each parameter set of the plurality of parameter sets, a result value which is the light beam detection value of this parameter set or which depends on the light beam detection value of this parameter set; wherein the detector data processor (192; 292; 392) is configured to calculate, for each parameter set of the plurality of parameter sets, the result value as a result of a bidirectional reflectance distribution function depending on the light beam detection value of the parameter set; wherein the detector data processor (192; 292; 392) is configured to train the neural network (195; 995) by means of the plurality of parameter sets and by means of the result value of each parameter set of the plurality of parameter sets; wherein the data detector processor (192; 292; 392) is configured to generate and output, by using the neural network (195; 995), for a parameter set which is not comprised by the plurality of parameter sets, the result value for this parameter set; wherein the device (1220) is configured to generate the result image depending on the result value.

2. The system according to claim 1, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, by adjusting the parameter set, at least the detector (170; 270; 370) or the illumination system (230) or the lamp (310) by the control device (191; 291; 391) adjusting at least a first azimuth angle or at least a first zenith angle of the detector (170; 270; 370) or by the control device (191; 291; 391) adjusting at least a second azimuth angle or at least a second zenith angle of the illumination system (230) or the lamp (310).

3. The system according to claim 2, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, the first azimuth angle and the first zenith angle of the detector (170; 270; 370).

4. The system according to claim 3, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, the second azimuth angle and the second zenith angle of the illumination system (230).

5. The system according to any of the preceding claims, wherein the system further comprises a holding device (220; 221) on which the measurement object (210) is attachable, and wherein the system further comprises the illumination system (230), wherein the light beam detected by the detector (170; 270; 370) for each parameter set of the plurality of parameter sets adjusted by the control device (191; 291; 391) is a light beam which forms by reflection of a light beam emitted by the illumination system (230) on the measurement object (210) which is attached to the holding device (220; 221).

6. The system according to any of claims 1 to 4, wherein the system further comprises a holding device (220; 221) on which the measurement object (210) is attachable, and wherein the system further comprises the illumination system (230), wherein the light beam detected by the detector (170; 270; 370) for each parameter set of the plurality of parameter sets adjusted by the control device (191; 291; 391) is a light beam which forms by a light beam emitted by the illumination system (230) transmitting the measurement object (210) which is attached to the holding device (220; 221).

7. The system according to any of the preceding claims, wherein the detector (170; 270; 370) is configured to detect, for each parameter set of the plurality of parameter sets adjusted by the control device (191; 291; 391), a light beam emitted by the lamp (310).

8. The system according to any of the preceding claims, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, the position on the lamp (310) such that the light beam detected by the detector (170; 270; 370) emanates from the position on the lamp (310).

9. The system according to claim 8, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, the position on the lamp (310) by adjusting at least an x-coordinate position and a y-coordinate position of this position.

10. The system according to any of the preceding claims, wherein the detector data processor (192; 292; 392) is configured to calculate, for each parameter set of the plurality of parameter sets, the result value as a result of the bidirectional transmittance distribution function depending on the light beam detection value of the parameter set.

11. The system according to any of the preceding claims, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, a position in a linear axis system, and wherein the detector (170; 270; 370) is a camera system with a matrix sensor.

12. The system according to claim 11, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, an f-number.

13. The system according to any of the preceding claims, wherein the control device (191; 291; 391) is configured to adjust, for each parameter set of the plurality of parameter sets, a wavelength value, wherein a wavelength of the light beam detected by the detector (170; 270; 370) depends on the wavelength value.

14. A method for generating a result image, wherein the method comprises a device (1220) for generating the result image to generate the result image such that a picture of a scene by a camera system is simulated, wherein the device (1220) simulates a presence of the illumination system comprising one or more light sources illuminating the scene, and wherein the device (1220) simulates that one or more objects are located in the scene, wherein a camera system parameter adjusting unit (110) of the device (1220) adjusts one or more camera system parameters, each of the one or more camera system parameters defining a property of the camera system, wherein an illumination system parameter adjusting unit (120) of the device (1220) adjusts one or more illumination system parameters, each of the one or more illumination system parameters defining a property of the illumination system, wherein the object parameter adjusting unit (130) of the device (1220) adjusts, for each object of the one or more objects, one or more object parameters of this object, each of the one or more object parameters of this object defining a property of this object, and wherein an image generation unit (140) of the device (1220) generates the result image depending on the one or more camera system parameters and depending on the one or more illumination system parameters and depending on the one or more object parameters of the one or more objects; and wherein the device (1220) adjusts, in addition to the one or more camera system parameters and in addition to the one or more illumination system parameters and in addition to the one or more object parameters of the one or more objects, one or more additional parameters; and wherein the image generation unit (140) generates the result image additionally depending on the one or more additional parameters, wherein at least one of the one or more additional parameters depends on a disturbance effect or on an effect caused by a non-ideal property of the camera system or the illumination system; or the image generation unit (140) generates one or more first images depending on the one or more camera system parameters and depending on the one or more illumination system parameters and depending on the one or more object parameters of the one or more objects, wherein the image generation unit (140) generates the result image using the one or more first images; wherein the method further comprises a control device (191; 291; 391) of a system for determining the physical behavior to successively adjust a plurality of parameter sets, wherein each parameter set of the plurality of parameter sets assigns a parameter value to each parameter of a group of two or more parameters, wherein each pair of two parameter sets of the plurality of parameter sets differs in at least one parameter value of a parameter of the group of parameters; and wherein the control device (191; 291; 391) adjusts, for each parameter set of the plurality of parameter sets, by adjusting the parameter set, at least the detector (170; 270; 370) or the illumination system (230) or the lamp (310); wherein a detector (170; 270; 370) of the system for determining the physical behavior detects, for each parameter set of the plurality of parameter sets adjusted by the control device (191; 291; 391), a light beam and outputs a light beam detection value depending on the light beam to a detector data processor (192; 292; 392) of the system for determining the physical behavior, wherein the light beam depends on the illumination system (230) or on the lamp (310); and wherein the detector data processor (192; 292; 392) receives each parameter set of the plurality of parameter sets which is adjusted by the control device (191; 291; 391); and wherein the detector data processor (192; 292; 392) obtains, for each parameter set of the plurality of parameter sets, from the detector (170; 270; 370) the light beam detection value for this parameter set; wherein the detector data processor (192; 292; 392) assigns, to each parameter set of the plurality of parameter sets, a result value which is the light beam detection value of this parameter set or which depends on the light beam detection value of this parameter set; wherein the detector data processor (192; 292; 392) calculates, for each parameter set of the plurality of parameter sets, the result value as a result of a bidirectional reflectance distribution function depending on the light beam detection value of the parameter set; wherein the detector data processor (192; 292; 392) trains the neural network (195; 995) by means of the plurality of parameter sets and by means of the result value of each parameter set of the plurality of parameter sets; wherein the data detector processor (192; 292; 392) generates and outputs, by using the neural network (195; 995), for a parameter set which is not comprised by the plurality of parameter sets, the result value for this parameter set; wherein the device (1220) generates the result image depending on the result value.

15. Computer program comprising a program code which includes instructions in order to cause a system for generating a result image to perform the steps of the method in accordance with claim 14, wherein the system for generating a result image comprises a device for generating the result image and a system for determining a physical behavior of a measurement object (210) or an illumination system (230) or a lamp (310), wherein the system for determining a physical behavior comprises a detector (170; 270; 370), a control device (191; 291; 391), a detector data processor (192; 292; 392) and a neural network (195; 995).