Device and method for measuring three-dimensional virtual images and objects on a head-up display

Abstract

The invention relates to a method for measuring virtual 3D images on a field of view display device, which is designed to superimpose 3D images into the field of view of a user via reflection on a partially transparent reflection pane. The method comprises the steps of: providing image-generation data relating to a virtual 3D test object, wherein the three-dimensional object surface thereof is overlaid with a predetermined visual surface pattern and / or wherein the 3D test object itself or individual non-continuous partial objects thereof represent a predetermined 3D pattern; generating the virtual 3D test object according to the provided image-generation data; capturing the generated virtual 3D test object from two different perspectives using an optical stereo camera system and identifying corresponding object surface points in the two camera images on the basis of the surface patterns and / or 3D patterns captured therein; and determining therefrom at least one actual representation parameter of the generated virtual 3D test object.

Description

[0001] 22-1285 1 Description Device and method for measuring three-dimensional virtual images and objects of a head-up display The invention relates to a method for measuring virtual 3D images of a field of view display device which is designed to display 3D images in the field of view of a user via reflection on a partially transparent reflection plate arranged in the field of view. The field of view display device can be designed in particular for use in a motor vehicle or another land, air or water vehicle, wherein, for example, a front, rear or side window of the vehicle or a combiner plate provided specifically for this purpose in the field of view of a user who is a passenger of the vehicle serves as a reflection plate. The invention is also directed to a corresponding control unit, an associated field of view display device and a vehicle equipped therewith.Head-up display devices for vehicles are also known as head-up displays (HUDs). These allow speed information and other useful navigation and vehicle operating instructions, or even entertainment content, to be superimposed in the form of a virtual image onto the real surroundings in front of or inside the vehicle observed by the driver or another occupant. For this purpose, a conventional HUD has a projection unit housed in the instrument panel. This comprises an imaging unit, such as a display, for generating a beam of light with the desired display content, as well as, if necessary, suitable projection optics for further shaping the beam of light and projecting it onto the partially transparent reflective screen referred to in 22-1285 2.Reflected by the reflective disc, the light beam reaches a spatial area within the vehicle interior designated for the user's eyes (also called the eyebox), from which the user can perceive the display content as a virtual image floating behind the reflective disc. Evaluating the virtual image and improving HUD performance requires objective recording of the displayed virtual image content using measuring devices. For conventional HUDs in vehicles, photos of the virtual image are usually taken using a mono camera in fixed positions relative to the windshield and evaluated with regard to certain important performance parameters, such as the double image distance or any distortion. However, any existing depth information of the virtual image may be lost in this process.The evaluation of HUD design parameters that are particularly important for 3D images, such as the virtual image distance or image curvature, or performance parameters such as local point disparity, is therefore not possible. The relative position of the virtual image to the environment, which is of utmost importance for augmentation (augmented reality) using contact-analog virtual representations, i.e., virtual representations based on real environmental objects, cannot be objectively recorded and corrected either. Head-up display development is increasingly focusing on the generation of three-dimensional virtual image structures and image augmentation using contact-analog virtual representations. Therefore, an objective evaluation of such image content using suitable measuring devices is becoming increasingly important in order to ensure targeted improvement and reliable calibration of corresponding HUD systems.Both a quantitative evaluation of display performance and a calibration of a head-up display intended for 3D image generation require precise recording of the generated three-dimensional virtual structures through measurements. 22-1285 3 In the prior art, stereoscopic measurement using a specially designed stereo camera is an established method for measuring and recording the spatial structure of physical three-dimensional objects, as described, for example, in US 1,871,281 or US 6,430,373. In this case, a photo of the object to be examined is created using at least two calibrated cameras, whose fixed position in space relative to one another is known. By comparing identical object features in the photos from the left and right cameras, a pixel disparity of the identical features in the left and right images can be determined.This in turn allows the calculation of a 3D position of the respective object features relative to the stereo camera in space using triangulation. The camera calibration of the stereo camera is usually carried out by photographing various targets, i.e. real objects whose spatial positioning and extent are known, using cameras that are positioned in relation to one another. From this, extrinsic camera parameters such as the rotation and translation of the coordinate systems of both cameras relative to one another and, if not already known, intrinsic camera parameters such as focal length and optical center can be determined using known mathematical methods. A known difficulty when measuring a real 3D object with the stereo camera is the rapid and precise determination of corresponding features in both images. In general, this can be achieved, for example, by using a suitable similarity measure such as, for example,This can be done using a brute-force search algorithm that compares the left and right camera images. A frequently used step to further simplify the search is the rectification of both stereo images. This involves transforming the images using known extrinsic parameters (such as a rotation matrix and a translation vector of the two camera coordinate systems of the stereo camera relative to each other) as if both cameras were perfectly parallel to each other. As a result, corresponding features in the right and left camera images have the same vertical pixel coordinate, and corresponding features are thus located on a horizontal line.To enable fast, precise, and robust measurement of the surface of a physical 3D object, a proposal is also made in the literature, for example in DE 102006049695 A1, to extend the stereoscopic camera system with a projection unit (e.g., in the form of a slide projector) that projects a visual stripe pattern and, if necessary, a Gray code suitable for identifying individual line pairs onto the 3D object. The projected pattern serves to quickly and robustly locate corresponding features in the left and right camera images, for example, on large, monochrome surfaces of the 3D object, using a suitable similarity measure.In order to also be able to measure transparent or highly reflective 3D objects, for example those with smooth or painted surfaces, DE 102015211954 A1 proposes, as a variation of this projector idea, replacing the optical cameras of the stereo camera with thermal imaging cameras and equipping the projection unit with an infrared radiator. In particular, it is proposed to imprint an irregular, for example an arbitrary (quasi-) statistical thermal pattern (e.g., a speckle pattern) onto the object surface to be examined. These proposals are not applicable to virtual 3D objects. Compared to the 3D measurement of real objects, the following additional problems arise when capturing virtual 3D images from a HUD: While real objects remain stationary, the position and shape of a virtual 3D object depend on the viewing direction from which the virtual image is observed.Consequently, the position of the virtual image in space depends on the actual eye position of the user (such as the driver). Furthermore, the depth perception of the virtual 3D image also changes due to the superimposition of images in the left and right eyes with different interpupillary distances of the respective users. Calibrating a stereo camera for a precise desired interpupillary distance is generally not possible without considerable effort due to manufacturing tolerances of the individual cameras. Furthermore, generating an ideal virtual image according to the HUD's performance specifications is not readily possible, for example due to manufacturing tolerances of the windshield, and requires a precise assessment of the influence of any construction deviations on the virtual image.It is therefore an object of the present invention to provide a technical concept (method and device) for measuring three-dimensional virtual images and objects of a head-up display or other field of view display device, which makes it possible to overcome the described problems and difficulties. In particular, this should enable a fast, precise, and robust evaluation of the performance of a field of view display device designed for 3D representation and thus also its targeted improvement. The field of view display device should be particularly suitable for use in a vehicle. This object is achieved by a method according to claim 1 for measuring virtual 3D images of a field of view display device, as well as by a corresponding control unit, field of view display device, and a vehicle equipped therewith according to the independent claims. Further embodiments are specified in the dependent claims.All further features and effects mentioned in the claims and the following description of the method also apply to the control unit, the field of view display device, and the vehicle, and vice versa. According to a first aspect, a method is provided for measuring virtual 3D images of a field of view display device, for example a head-up display (HUD), which can be designed in particular for use in a vehicle. The field of view display device is designed to display three-dimensional images and objects in the field of view of a user, such as a driver or another occupant of the vehicle, via reflection from a partially transparent reflection panel arranged in their field of view, 22-1285 6 for example a windshield or another vehicle window or a combiner panel provided specifically for this purpose.The vehicle can be a motor vehicle, but also any other land, air, or water vehicle. The representation of a respective 3D object by the field of view display device can, in particular, also be contact-analog, i.e., oriented towards real environmental objects outside the vehicle. The method comprises the following steps: First, image generation data of a virtual 3D test object is provided to an image generation unit of the field of view display device or its control unit. In this case, a predetermined visual surface pattern is overlaid by software on a virtual object surface of the 3D test object that is to be displayed three-dimensionally, so that it exhibits variations corresponding to the surface pattern, for example in brightness and / or color. Both the 3D test object and the surface pattern can vary in position, shape, and extent over time in a predetermined manner.The surface pattern can significantly improve, simplify, and / or accelerate the recognizability of individual surface points or areas of the virtual 3D test object during its subsequent stereoscopic capture and evaluation. In principle, the entire object surface of the virtual 3D test object to be displayed can be overlaid / covered / covered with the surface pattern so completely when its image generation data is provided that a robust, rapid, and at the same time seamless recognizability of individual surface points is enabled. Examples of suitable surface patterns are given below. The virtual 3D test object used for measurement can have any three-dimensional shape and size and can consist of any number of non-connected 3D sub-objects, each of which can be arranged arbitrarily in space.In particular, the non-contiguous virtual 3D sub-objects themselves can be arranged in such a way that they represent or result in a predetermined, for example, regular, irregular, statistical, or quasi-statistical three-dimensional pattern (hereinafter referred to as 3D pattern) in space. A 3D pattern generated (virtually) in this way can be used in the present method as an alternative or in addition to the aforementioned superposition of the three-dimensional virtual object surface with the predetermined surface pattern in order to achieve the effects described below when measuring the 3D test object.Overlaying the object surface with a surface pattern can be more advantageous than generating a virtual 3D test object in the form of a predetermined (virtual) 3D pattern, for example, if the virtual 3D test object is intended to have a predefined three-dimensional object or surface shape that occurs particularly frequently, for example, during normal operation of the field-of-view display device and is therefore preferably also used in the evaluation of its performance described herein. The virtual 3D test object is then generated by the field-of-view display device according to the provided image generation data and captured from at least two different perspectives by an optical stereo camera system. For this purpose, the stereo camera system has at least one movable camera and / or at least two cameras at a definable distance from one another in the beam path of the field-of-view display device downstream of the reflection disk.By moving the cameras of the stereo camera system relative to each other and to the beam path, different eye positions and / or interocular distances of a user can be taken into account or simulated. When capturing the generated virtual 3D test object, the respective camera can, for example, be arranged at a position in space intended for the corresponding eye of the user. However, this is not mandatory. In the camera images of the virtual 3D test object obtained in this way from the two different perspectives, image points are identified based on the surface patterns and / or 3D patterns contained therein, each of which originates from one and the same object surface point. From this, at least one actual display parameter of the generated virtual 3D test object is determined, which can be used to evaluate the 3D display performance of the field of view display device.The at least one actual display parameter can, for example, comprise a projection distance (also called projection depth or image distance) and / or a spatial position and / or a spatial orientation of the generated virtual 3D test object or its surface points. Alternatively or additionally, the at least one actual display parameter of the generated virtual 3D test object can comprise a local vertical point disparity, which indicates a vertical offset of the pairwise corresponding object surface points in the camera images from the two different perspectives. The respective actual display parameter can, for example, be determined with respect to a coordinate system of the stereo camera system, the field of view display device or the vehicle in which it is mounted, or with respect to an eyebox provided for its user.As usual, an eyebox is understood here as a two- or three-dimensional spatial area intended for the user's eyes or for the respective user's eye, from which the generated virtual 3D images are visible to the user in the intended quality. To evaluate the 3D display performance of the field of view display device, the respective actual display parameter can be compared with a predefined corresponding target display parameter of the virtual 3D test object. Based on this, the 3D display performance of the field of view display device can be evaluated and, for example, specifically improved if a predefined deviation tolerance is exceeded.One idea for overcoming the difficulties described above in evaluating the 3D display performance of a 22-1285 9 field of view display device thus consists in a method and a corresponding measuring device in the form of a stereo camera system that can be suitably positioned in the beam path of the field of view display device and is designed to capture the 3D structure of a virtual image with a software-based superposition of the virtual image to be measured with predefined patterns. This metrological capture can, in particular, also be designed to be eye-position-dependent, i.e., correspond to selectable / adjustable or variable eye positions and / or interpupillary distances of a user. The field of view display device, in particular in the form of a head-up display, can in principle be designed for 3D image generation in any technically feasible way.This can therefore involve not only the mere creation of a depth effect by using multiple, inclined, or curved two-dimensional virtual image planes and image surfaces, but also, in particular, "truly" three-dimensional virtual 3D objects, which are generated, for example, using holographic techniques by a specially designed image-generating unit of the field of view display device. With such a 3D HUD in a motor vehicle, objects such as an arrow can be displayed at a great distance while, at the same time, a sign, for example, can be displayed close to the driver. To facilitate measurement, these objects are displayed overlaid with a checkerboard pattern, for example, in order to measure them with a stereo camera pair (depending on the position of the stereo camera pair). Deviations from the target can then be determined.As mentioned at the beginning, a key feature of virtually displayed 3D objects is that their position and shape (both when using two-dimensional image planes and three-dimensional curved image surfaces as well as with "truly" three-dimensionally generated virtual objects) can change with the eye or camera position. In particular, the individual eye distance of the user plays a role, so that users with different eye distances can perceive a virtual 3D object differently. When used in vehicles, the correct position and alignment of a virtual object in space can be of crucial importance, particularly in contact-analogous (i.e., based on real surrounding objects) representations.With the technical concept presented here for 3D measurement of the virtual objects of a field of view display device, these effects and any deviations from the desired result can be recorded and quantified quickly, robustly and precisely in order to evaluate and specifically improve the field of view display device in this regard. For the purpose described here, the surface pattern can, for example, have at least one of the following pattern types or pattern properties, which can also be combined with one another and / or alternately, i.e., next to one another, in the object surface: a regular or periodic two-dimensional pattern (the periodicity can be one- or two-dimensional, i.e.,the two-dimensional pattern can, for example, only change periodically in one direction in the two-dimensional surface, while remaining constant in a direction orthogonal to this, or it can change in two independent directions with the respective periodicity); an irregular or aperiodic two-dimensional pattern (for example statistical or quasi-statistical); a checkerboard pattern; a two-dimensional distribution of circles with one or more predetermined diameters; a speckle pattern; a surface-covering grey-value pattern whose grey values ​​preferably vary in a predetermined manner at each surface point; a black-and-white pattern; a colour pattern that has one or more different colours. In principle, surface-covering and / or irregular surface patterns, such as e.g.aperiodic sine patterns or speckle-like grayscale distributions, if required, more options for fast and / or 22-1285 11 time-resolved feature assignment between the left and right camera images of the generated virtual 3D object. According to one embodiment, when capturing the generated virtual 3D test object, the respective camera of the stereo camera system is arranged at a spatial position intended for the respective eye of the user. In particular, an entire eyebox volume of the field of view display device intended for the respective eye of the user can be scanned by the stereo camera system by the respective camera of the stereo camera system capturing the generated virtual 3D test object from several different positions within the eyebox volume one after the other in order to evaluate the 3D display performance of the field of view display device as comprehensively as possible for the entire eyebox.According to a further or alternative embodiment, the definable relative distance of the at least two cameras can be varied to adapt to different eye distances of the user. In particular, it can also be varied when capturing the generated virtual 3D test object in order to evaluate the 3D representation performance of the field of view display device for different eye distances of the user. According to one embodiment, the method presented herein further comprises a calibration of at least one of the mentioned cameras of the stereo camera system, which can be performed in particular before measuring the actual 3D test object.For this purpose, this camera can, for example, be positioned at one or more discrete calibration positions, and a calibration can be performed in each case by determining a spatial position and / or orientation and / or, if necessary, at least one internal imaging property of this camera with respect to a coordinate system of the stereo camera system or the field of view display device. In this embodiment, a parametric recalibration function is subsequently determined from this, which, depending on at least one (in particular continuously) measurable camera position parameter, specifies the spatial position and / or orientation of this camera at an infinite number of camera positions lying between and / or beyond these positions.Spatial positions and / or orientations of this camera obtained using this parametric recalibration function can be used to calibrate the camera during the actual measurement of the virtual 3D test object, i.e., to determine at least one of its actual display parameters, such as its projection distance, using triangulation. This avoids the time-consuming and costly recalibration of each individual camera in the stereo camera system with each camera movement. Furthermore, such parameterization of the camera calibration opens up the possibility of quickly and precisely taking into account different eye positions and / or interpupillary distances of the users when measuring the virtual 3D test object. Some exemplary embodiments are given below.In particular, said camera can be displaceable along at least one translational rail with linear position feedback, so that said measurable camera position parameter is a linear camera position or a linear camera distance to the second camera of the stereo camera system on this rail according to the currently received position feedback. The position feedback can, for example, be measured or received directly during the capture of the generated virtual 3D test object or immediately before or after, so that, in particular, recalibration in real time is enabled by such a parametric recalibration function. According to a further aspect, a control unit is provided which is designed and configured to automatically carry out the method presented herein.For this purpose, a corresponding computer program 13 can, for example, be installed in the control unit and run during operation of the field of view display device. According to a further aspect, a field of view display device is provided which can be designed in particular for use in a vehicle. The field of view display device comprises a projection unit designed to generate and output a bundle of light rays with a 3D image content transported therein to a partially transparent reflection plate that is to be arranged or is arranged in the field of view of a user. The design and mutual arrangement of the projection unit and the reflection plate are selected such that a desired virtual 3D image is displayed to the user behind them, provided their eyes are located in a designated spatial area (eyebox) opposite the reflection plate.For this purpose, the projection unit can comprise an image-generating unit, for example in the form of a suitable display or projector, as well as, if necessary, further optical elements in the beam path of the light beam emanating from the display / projector for further beam shaping and redirection. The field of view display device further comprises an optical stereo camera system comprising at least one movable camera and / or at least two cameras at a definable distance from one another, which can be positioned in the beam path of the light beam after its reflection from the reflection plate and are designed to capture the displayed virtual 3D image from at least two different perspectives.Furthermore, the field of view display device comprises the above control unit, which is designed and configured to control the projection unit and the stereo camera system during the automatic implementation of the method presented herein. The aforementioned reflective disc can also be manufactured and / or sold as a component of the field of view display device or, alternatively, separately. When used in a vehicle, the reflective disc can be designed, in particular, as a section of a vehicle windshield or another vehicle windshield. However, it can also be designed as a combiner disc specifically designed for the purpose stated herein. According to a further aspect, a vehicle, in particular a motor vehicle or any other land, air, or water vehicle, is provided. The spatial orientation terms used herein, such as "above," "below," "in front," "side," "horizontal," "vertical," etc.can refer in particular to the usual vehicle-fixed Cartesian coordinate system with mutually perpendicular longitudinal, transverse and vertical axes of the vehicle. The vehicle comprises a windshield and an instrument panel arranged underneath it and is equipped with the above-mentioned field of view display device. Its image-generating unit or, if appropriate, its entire projection unit (which, in addition to the image-generating unit, can also comprise suitable projection optics) can be arranged in particular inside the instrument panel or in / on its upper side, for example, installed directly on or below the upper side of the instrument panel, such that the light beam is projected by the projection unit onto the windshield or onto a combiner plate positioned inside the vehicle in front of it in the field of vision of the driver or another occupant, which serves as the above-mentioned partially transparent reflection plate.Alternatively, the field of view display device can also be installed at any other suitable location in the vehicle. The above aspects of the invention and their specific design variants, embodiments, and further features and effects are explained in more detail below with reference to the examples shown in the accompanying drawings. The drawings are to be understood as purely schematic illustrations, i.e., not to scale.Shown are: 22-1285 15 Figure 1 shows a schematic plan view of a field of vision display device in a motor vehicle, which is designed to display 3D images in the field of vision of an occupant via reflection on a vehicle window arranged in his field of vision and to carry out a method according to an embodiment of the invention; Figure 2 shows a flow diagram of an embodiment of a method presented here for measuring virtual 3D images of a field of vision display device; Figure 3 shows an exemplary step of providing image generation data of a virtual 3D test object in the method of Fig.2, which is to be generated by a field of view display device and whose cuboidal three-dimensional object surface is overlaid by software with a checkerboard surface pattern for its 3D measurement; Figure 4 is a schematic representation of a further example of a virtual 3D test object from two different perspectives, which are captured by the stereo camera system in the method of Figure 2, wherein the cuboidal three-dimensional object surface is overlaid by software with a predefined surface pattern in the form of full circles for its 3D measurement; Figure 5 is a schematic representation of a further example of a virtual 3D test object from two different perspectives, which are captured by the stereo camera system in the method of Figure 5.2 are captured by the stereo camera system, wherein the pyramid-shaped three-dimensional object surface is overlaid by software with a predefined surface pattern in the form of a speckle-like grayscale pattern for its 3D measurement. 22-1285 16 All of the various embodiments, variants, and specific design features of the method, the field of view display device, the control unit, and the vehicle according to the above aspects of the invention mentioned above in the description and in the subsequent claims can be implemented in the examples shown in Figures 1 to 5. They will therefore not all be repeated below. The same applies accordingly to the definitions of terms and effects already given above with regard to individual features shown in Figures 1-5. Fig.1 shows, in a highly simplified schematic plan view, an embodiment of a vehicle 1 with a field of view display device 2 according to the aspects of the invention specified above and in the claims. In this example, it is a motor vehicle, which is only indicated by its windshield 3. Opposite the windshield 3, for example below it, in an instrument panel 4 (not shown in detail) of the vehicle 1, a projection unit 5 of the field of view display device 2 is arranged. In this example, the field of view display device 2 is designed as a 3D head-up display (3D HUD). The projection unit 5 is designed to generate a light beam L with a desired 3D image content.The light beam L emanating from the projection unit 5 is projected onto the windscreen 3, which in this example serves as a partially transparent reflection panel of the field of view display device 2, in such a way that, after reflection on the windscreen 3, it reaches an eyebox E of a user 6, who in this example is a driver of the vehicle 1. The eyebox E is a two- or three-dimensionally defined spatial area in the vehicle 1 at a predetermined position opposite the windscreen 3, which is intended for the eyes of the user 6 so that they can see a virtual 3D image O generated by the field of view display device 2 with both eyes. As shown in Fig.1, the user 6 sees a virtual 22-1285 17 image OL with his left eye and a virtual image OR with his right eye, the relative position and shape of which can vary with the eye distance and the eye position of the user 6 relative to the light beam L, so that the resulting perception and spatial position of the generated 3D object can also vary depending on the eye distance and the eye position of the user 6. Furthermore, in the vehicle 1 orAn optical stereo camera system 7 is provided in the field of view display device 2, which can comprise at least one movable camera and / or at least two cameras at a definable distance from one another (not shown individually), which can be positioned in the beam path of the light beam L after its reflection on the windscreen 3 in order to capture the displayed virtual 3D image O from two different perspectives for evaluating the 3D HUD performance (of course without the user 6 being present). For example, the stereo camera system 7 can be positioned directly in the eyebox E of the user 6 or, as indicated in Fig. 1, immediately in front of it. The stereo camera system 7 and its position are indicated schematically and purely by way of example in Fig. 1.To carry out a method according to the above first aspect of the invention, a correspondingly configured control unit 8 is further provided, which can communicate in a suitable manner with the projection unit 5 and the stereo camera system 7 in terms of information and control technology. The control unit 8 can be arranged, for example, in the projection unit 5 or outside it in the vehicle 1, for example in the instrument panel 4. Fig. 2 shows a flow diagram of an embodiment of a method according to the above first aspect of the invention for measuring virtual 3D images O of a field of view display device 2, as shown, for example, in Fig. 1. However, the reference to Fig. 1 is not to be understood as limiting, but serves solely as an exemplary illustration of the possible 22-1285 18 steps.The method can in particular comprise the following basic steps, with further details, variants and optional further steps being specified below: In a step S1, image generation data of a virtual 3D test object O is provided to the projection unit 5 of the field of view display device 2 or its control unit 8. In this case, a virtual object surface 9 of the 3D test object to be displayed three-dimensionally is overlaid by software with a predetermined visual surface pattern so that it has variations corresponding to the surface pattern, for example in brightness and / or color. Fig. 3 illustrates this step using the example of a checkerboard surface pattern. In this example, the entire object surface 9 of the virtual 3D test object O to be displayed was overlaid with the surface pattern when its image generation data was provided.3 shows the virtual 3D object to be generated before the software-based overlay with the surface pattern. On the right in Fig. 3, the virtual 3D object O to be generated is shown in the representation form used for measurement / calibration. Its cuboid-shaped object surface 9, in this example, is overlaid by the software with a known structure, such as a checkerboard pattern, for particularly fast and precise measurement of the 3D HUD with regard to projection distance and local point disparity (see below). This structure, such as a checkerboard pattern, whose corners 10 are particularly easy to locate after capture by the stereo camera system 7 through gradient formation, can also be used as surface patterns. Alternatively, more general (quasi-) statistical patterns can also be used as surface patterns.The overlay with a suitable surface pattern in step S1 serves purely for the measurement and, if necessary, calibration of the field of view display device 2 (see below). In a step S2, the virtual 3D test object O is generated by the field of view display device 2 according to the provided image generation data 22-1285 19 and captured from at least two different perspectives by an optical stereo camera system 7. For this purpose, the stereo camera system 7 has at least one movable camera and / or at least two cameras at a definable distance from one another in the beam path of the field of view display device 2 after the reflection disk. By moving the cameras of the stereo camera system 7 relative to one another and to the beam path of the light beam L, different eye positions and / or interpupillary distances of a user 6 can be taken into account or simulated.In a further step S3, in the camera images of the virtual 3D test object O thus obtained from the two different perspectives (cf. Figs. 4 and 5), image points are identified based on the surface patterns contained therein, each originating from one and the same object surface point or feature (so-called feature matching). From this, at least one actual display parameter of the generated virtual 3D test object O is determined, which can be used to evaluate the 3D display performance of the field of view display device 2. This method and device enable, in particular, fast, robust, and precise measurement of virtual 3D test objects O, which are generated, for example, by a HUD, which, among other things, also enables fast calibration of the HUD.Important measured variables or output parameters in the 3D measurement of the generated virtual test objects O for HUD evaluation with a stereo camera system 7 (also called a 3D camera) are, for example, the projection distance (to answer the question of whether the projection distance corresponds to the specifications) and the vertical local point disparity (local point disparity Y, to answer the question of how much the virtual image OL for the left eye and the virtual image OR for the right eye deviate vertically from each other, see Fig. 1). The projection distance can deviate from the specifications, for example, due to construction tolerances of the field of view display device 2. The vertical local point disparity corresponds to a vertical offset of the features between the left and right eyes and arises because the driver's left and right eyes see a different virtual image in the case of HUD projection. The latter does not occur, for example, when measuring real objects, i.e.there the vertical local point disparity is naturally zero. To determine the projection distance and the local point disparity, the camera images recorded from the two different perspectives in step S2 can first be rectified in a known manner, i.e. converted to a parallel alignment of both cameras. The corresponding pixel coordinates ( ) and ( ) in the (rectified) left and right camera images can then be determined. In the rectified camera images of a real object, it is known that would apply. In the case of a virtually displayed 3D object, however, the right and left eye perceive different virtual images OL and OR (cf. Fig. 1) and in the rectified camera images, accordingly, applies. The projection distance is obtained, for example, by triangulation orfrom the horizontal pixel deviation of the pixel coordinates ( ) of the corresponding features in the (rectified) left and right camera images as well as the camera distance from the ray theorem. In order to improve and accelerate the measurement, the present method proposes overlaying the 3D objects typically displayed in the HUD for the driver (such as an arrow on the road, a sign or pedestrian marking, a map for navigation, and much more) with regular (such as the checkerboard pattern in Fig. 3), irregular, statistical, or quasi-statistical surface patterns in software before or during their generation by the projection unit 5 (such as an augmented reality HUD display) solely for the purpose of measurement and / or calibration. This in turn allows a quick and precise assignment of corresponding features in the left and right camera images and thus serves to measure the image quality or calibrate the HUD.Using a checkerboard pattern, for example, the vertical local point disparity at the corner points 10 of the checkerboard pattern can be determined with high precision. Calibration of the HUD can also be achieved, for example, by measuring the virtual, patterned 3D test object in reference to a known real object in the driver's field of vision. As mentioned above, another problem with virtual objects in a HUD is the dependence of the position of the virtual object on the eye distance and the eye position of the user 6. This plays a role in both the stereo camera calibration and the measurement of the 3D test object O with the stereo camera system 7.The method described herein enables a solution to this problem, which is described below using the example of a stereo camera system with cameras that can be flexibly positioned relative to one another: The integration of the interpupillary distance into the measurement proposed here requires, as a first step, the determination of a rotation matrix and a translation vector, as in a conventional stereo camera calibration. This allows the distortion of the two cameras to be determined (if they exhibit optical distortion) and eliminated algorithmically. Furthermore, the intrinsic camera parameters (focal lengths, optical center) are determined if these are unknown, as well as the extrinsic parameters (rotation matrix, translation vector of the two camera coordinate systems relative to each other or to the vehicle).To avoid recalibration for each shift in the relative position of the cameras of the stereo camera system 7, the two cameras can be mounted, for example, on a translational rail or other movement device with position feedback. Based on a stereo camera calibration performed at a few fixed and known relative positions of the cameras to each other, the stereo camera calibration according to the embodiment of the invention described above is extended to positions in between and / or beyond using mathematical means (such as mathematical modeling or polynomial interpolation or extrapolation, and many more). In other words, based on actual calibration at some known position points of the cameras, a parametric recalibration function can be specified that depends on one or more (for example, continuously) measurable camera position parameters.The parametric recalibration function can, for example, specify a precise relative 3D camera position depending on a measurable distance of the camera on a rail (which can be described by a simple linear model). To incorporate the eye distance and the eye position into the measurement of a virtual 3D object, the eyebox volume of the HUD can, for example, be scanned or sampled using the stereo camera system for variable distances between the left and right cameras. The eye position can be parameterized by the respective center point of the stereo camera system 7. For each camera position and each eye distance, a 3D point model is created, for example, using a stereo photo of the virtual 3D test object (provided with the surface pattern).By describing the 3D point model as a fitting function of the eye position and the eye distance, 3D point cloud models for unknown eye distances and eye positions can be interpolated or extrapolated. The required number of measurement points of the grid in the eyebox volume or the complexity of the fitting function (in the simplest case, this can be described by a linear interpolation function and in the extreme case, 23 by a neural network) generally depends on the complexity of the HUD optics or the reflective screen (such as the windscreen 3). In the simplest case, the eyebox volume can be "sampled", for example, by a single eye position with a predetermined constant eye distance. Furthermore, in the present method for identifying corresponding surface points in both camera images, a suitable similarity measure as well as, for example,A brute-force search algorithm can be used to compare the left and right camera images. The measuring device in the presented method can thus, for example, comprise one or more variably positionable cameras (stereo camera system 7), whose relative positions can be either fixed (only for at least two cameras) or variable. The calibration of the measuring device is ideally achieved using the parametric description presented above with the help of camera-position-dependent rotation matrices and translation vectors or projection matrices composed of them. where λ is the measurable camera position parameter mentioned above. In the photos of the virtual object from two or more different perspectives, the pixel disparities of the identical features can be interpreted as a function of the camera positions thus determined. This allows the 3D structure of the surface 9 of the virtual 3D test object O to be calculated position-dependently using a parametric representation of this functional relationship. Thus, the virtual 3D test object O can be interpolated for different eye positions and interpupillary distances. In this way, the present measurement concept can avoid the problem of very difficult to achieve exact positioning of the two cameras during stereo calibration (e.g., due to camera construction tolerances), which is required to describe / evaluate the 3D HUD performance for a desired interpupillary distance, using rigorous mathematical means.This allows for fast and reliable measurement of the virtual 3D structures generated by the HUD for variable eye distances and eye positions. Figs. 4 and 5 show further examples of the software-based overlay of the virtual 3D test object O to be measured with predefined surface patterns in step S1 of Fig. 2 before it is generated in steps S2 and S3 and photographed from different camera perspectives. If, for example, the virtual object is overlaid with full circles 11 as in Fig. 4, a simple detection of the centers of the circles 11 can ensure fast, reliable, and precise feature matching (identification of corresponding object surface points) between the photos of the individual perspectives (which can correspond to the virtual images OL and OR of Fig. 1 perceived by the user 6). A further implementation option according to Fig.5 consists in overlaying the virtual 3D test object to be measured (here a pyramid with a rectangular base) with a speckle-like surface pattern in software. The feature matching between the photos from the individual camera perspectives can be achieved, for example, by rectifying the two photos and assigning the grayscale values ​​around the corresponding maxima of the speckles 12 in the photos from the individual camera perspectives. Otherwise, the same applies here as in Fig. 4. Another implementation option (cf. Fig. 3) is to overlay the virtual 3D test object O to be measured with a checkerboard pattern in software. The feature matching between the photos from the individual camera perspectives is achieved, for example, by determining the corner points 10 of the checkerboard pattern and assigning the corresponding coordinates in the photos from the individual camera perspectives.In order to enable a clear assignment of the corner points 11 of the chessboard pattern between the left and right camera images, the 22-1285 25 individual quadrilaterals of the chessboard can be encoded, for example, using colors or QR-like patterns. Otherwise, the same applies here as in Fig. 4 or 5. Another implementation option is for the virtual 3D test object O to consist of N virtual sub-objects or partial objects distributed arbitrarily in 3D space (e.g., it can be a regular or grid-like three-dimensional arrangement of three-dimensional spheres), each of which is overlaid with a surface pattern whose feature(s) is / are used for stereoscopic feature matching. Another implementation option would be to overlay the virtual 3D object to be measured with an aperiodic surface pattern, for example, a sine pattern (not shown), using software.The feature matching between the photos of the individual camera perspectives can be implemented here by rectifying the photos and assigning the gray value values ​​around the corresponding maxima of the sine pattern in the photos of the individual camera perspectives.

[0002] 22-1285 26 List of Reference Symbols 1 Vehicle 2 Field of View Display Device 3 Windscreen 4 Instrument Panel 5 Projection Unit 6 User 7 Stereo Camera System 8 Control Unit 9 Virtual Object Surface to be Displayed Three-Dimensionaly 10 Corners of the Checkerboard Pattern 11 Full Circles, Circles 12 Speckles L Light Beam O Virtual 3D (Test) Object or Virtual 3D Image OL Virtual Image Perceivable by the Left Eye OR Virtual Image Perceivable by the Right Eye E Eyebox

Claims

22-1285 27 claims 1. Method for measuring virtual 3D images (O) of a field of view display device (2), in particular for a vehicle (1), which is designed to display 3D images (O) in the field of view of a user (6) via reflection on a partially transparent reflection plate arranged in his field of view, comprising the steps of: - providing image generation data of a virtual 3D test object (O), wherein the object surface (9) of the test object to be displayed three-dimensionally is superimposed in the image generation data with a predetermined visual surface pattern and / or wherein the virtual 3D test object (O) itself or its individual non-connected partial objects represents / represent a predetermined 3D pattern; - generating the virtual 3D test object (O) by the field of view display device (2) according to the provided image generation data;- Capturing the generated virtual 3D test object (O) from two different perspectives using an optical stereo camera system (7) that includes at least one movable camera and / or at least two cameras at a definable distance from each other in the beam path of the field-of-view display device (2); - Identifying pairs of corresponding object surface points in the camera images from the two different perspectives based on the surface patterns and / or 3D patterns captured therein, and determining therefrom at least one actual display parameter of the generated virtual 3D test object (O).

2. The method according to claim 1, wherein; - the at least one actual display parameter comprises a projection distance and / or a spatial position and / or a spatial orientation of the generated virtual 3D test object (O) and / or a local vertical point disparity that indicates a vertical offset of the pairwise corresponding object surface points in the camera images from the two different perspectives.

3. The method according to claim 1 or 2, wherein - the respective actual display parameter is compared with a predetermined associated target display parameter of the virtual 3D test object (O), and the 3D display performance of the field of view display device (2) is evaluated depending thereon.

4. The method according to one of the preceding claims, wherein - substantially the entire object surface (9) of the virtual 3D test object (O) to be displayed is overlaid with the surface pattern when its image generation data is provided. 5.Method according to one of the preceding claims, wherein the surface pattern has at least one of the following pattern types or properties: - a regular or periodic 2D pattern; - an irregular or aperiodic 2D pattern; - a checkerboard pattern; - a planar distribution of circles (11) with one or more predetermined diameters; - a speckle pattern; - a surface-covering grayscale pattern whose grayscale values ​​preferably vary in a predetermined manner at each surface point;. 29 - a black and white pattern; - a color pattern having one or more different colors.

6. The method according to one of the preceding claims, wherein - when capturing the generated virtual 3D test object (O), the respective camera of the stereo camera system (7) is arranged at a spatial position intended for the respective eye of the user (6); and / or - when capturing the generated virtual 3D test object (O), an entire eyebox volume of the field of view display device (2) intended for the respective eye of the user (6) is scanned by the stereo camera system (7) by the respective camera capturing the generated virtual 3D test object (O) one after the other from several different positions within the eyebox volume; and / or - the definable relative distance of the at least two cameras can be varied to adapt to different interocular distances of the user (6) and is preferably also varied when capturing the generated virtual 3D test object (O). 7.Method according to one of the preceding claims, further comprising a calibration of at least one of said cameras of the stereo camera system (7), by - said camera is positioned at one or more discrete calibration positions and in each case a calibration is carried out by determining a spatial position and / or orientation of this camera with respect to a coordinate system of the stereo camera system (7) or the field of view display device (2) or the vehicle (1); and. 22-1285 30 - a parametric recalibration function is determined therefrom, which, depending on at least one measurable camera position parameter, specifies the spatial position and / or orientation of this camera at an infinite number of camera positions lying between and / or beyond it; - wherein said spatial position and / or orientation of this camera is used as its calibration in determining the at least one actual display parameter of the generated virtual 3D test object (O).

8. The method according to claim 7, wherein - said camera is displaceable along at least one translational rail with linear position feedback, so that the associated measurable camera position parameter is a linear camera position on this rail according to the currently received position feedback.

9. A control unit (8) designed and configured to automatically carry out the method according to one of the preceding claims. 10.Field of view display device (2), in particular for use in a vehicle (1), comprising: - a projection unit (5) which is designed to generate and output a light beam (L) with a 3D image content to a partially transparent reflection plate which is to be arranged or is arranged in the field of view of a user (6) in such a way that a virtual 3D image (O) is displayed to the user (6) behind it, provided his eyes are in an eyebox (E) intended for them; - an optical stereo camera system (7) which has at least one movable camera and / or at least two cameras. 22-1285 31 a definable distance from one another, which can be positioned in the beam path of the light beam (L) after the reflection disk and are designed to capture the displayed virtual 3D image (O) from two different perspectives; - a control unit (8) according to claim 9, which is designed and configured to control the projection unit (5) and the stereo camera system (7) during the automatic implementation of the method according to one of claims 1 to 8; and - preferably also said reflection disk, which is designed in particular as a section of a windshield (3) of a vehicle (1). 11.Vehicle (1), in particular a motor vehicle, with mutually perpendicular longitudinal, transverse, and vertical axes of a vehicle-specific Cartesian coordinate system, comprising: - a windscreen (3) and an instrument panel (4) arranged thereunder; and - a field of view display device (2) according to claim 10, the reflection disc of which is designed in particular as a section of the windscreen (3) or as a combiner disc arranged in front of it on the inside of the vehicle.

Images