Method for calibrating smartglasses comprising a retinal scan display, computing unit and smartglasses

Abstract

The invention relates to a method for calibrating smartglasses (12) comprising a retinal scan display (10), wherein a laser projector (14) of the retinal scan display (10) generates scanned images (16) and the latter are projected as beamlets into an eyebox of the smartglasses (12) by way of an optical system (18) of the smartglasses (12), wherein calibration images (20) and / or calibration patterns (22) are output to a wearer of the smartglasses (12) by means of the beamlet, and wherein user feedback given by the wearer of the smartglasses (12) on the basis of the wearer's perception is registered and used to calibrate the smartglasses (12). It is proposed that in order to calibrate the smartglasses (12), an automated mechanical actuator (24) for adjustment of a position of the beamlet is controlled by a computing unit (26) of the smartglasses (12), or an actuation of a manual mechanical actuator (28) for manual adjustment of the position of the beamlet is instigated, or a manual modification of the smartglasses (12) for manual adjustment of the position of the beamlet is instigated, or the laser projector (14) and / or the optical system (18) are / is controlled by the computing unit (26) of the smartglasses (12) in order to adapt a brightness, a sharpness or a hue of the rendered calibration images (20) and / or calibration patterns (22).

Description

[0001] Description

[0002] Procedure for a virtual having data glasses, computing unit and data glasses

[0003] State of the art

[0004] A method for calibrating data glasses having a virtual retinal display (retinal scan display) has already been proposed, wherein scanned images are generated by a laser projector of the virtual retinal display and projected as beamlets into an eyebox of the data glasses via an optical system of the data glasses, wherein calibration images and / or calibration patterns are output to a wearer of the data glasses by means of the beamlet, and wherein user feedback, which is provided by the wearer of the data glasses based on his perception, is recorded and used to calibrate the data glasses.

[0005] Disclosure of the invention

[0006] The invention is based on a method for calibrating data glasses having a virtual retinal display (retinal scan display), wherein scanned images are generated by a laser projector of the virtual retinal display and projected as beamlets into an eyebox of the data glasses via an optical system of the data glasses, wherein calibration images and / or calibration patterns are output to a wearer of the data glasses by means of the beamlet, and wherein user feedback, in particular conscious feedback, which is provided by the wearer of the data glasses based on their, in particular conscious, perception, is recorded and used to calibrate the data glasses. The user feedback is therefore provided in particular as a function of conscious observation and evaluation by the user.

[0007] It is proposed that, for the calibration of the data glasses, an automated mechanical actuator for adjusting a position of the beamlet is controlled by a computing unit of the data glasses, or an actuation of a manual mechanical actuator is initiated for manual adjustment of the position of the beamlet, or a manual conversion of the data glasses is initiated for manual adjustment of the position of the beamlet, or the laser projector and / or the optical system for adjusting a brightness, sharpness, or hue of the reproduced calibration images and / or calibration patterns are controlled by the computing unit of the data glasses. This advantageously allows a calibration, in particular a field calibration, to be optimized. Advantageously, a calibration can be simplified and / or made more precise. Advantageously, a user calibration can be enabled.This can advantageously eliminate a cost factor (visit by a trained specialist to perform a calibration), which can significantly reduce overall costs and significantly increase the overall user-friendliness and / or attractiveness of the data glasses.

[0008] “Data glasses” should be understood in particular as a wearable (head-mounted display) by means of which information can be added to a user’s field of vision. Data glasses preferably enable augmented reality, virtual reality and / or mixed reality applications. Data glasses are also commonly referred to as smart glasses or VR glasses or AR glasses. In particular, the data glasses have the virtual retinal display (also called a retinal scan display or light field display), which is particularly familiar to those skilled in the art. The virtual retinal display is particularly designed to sequentially scan an image content by deflecting at least one visible laser beam from at least one temporally modulated light source, such as one or more (RGB) laser diodes of a laser projector, and to project it directly onto the retina of the user’s eye using optical elements.The virtual retinal display is generally based on the use of at least one combiner. The combiner is an element that transmits light from the environment to the eye (real image) and simultaneously overlays / combines the transmitted ambient light with light from artificially generated image content (augmented image / scanned image) from the laser projector of the virtual retinal display. A well-known type of combiner is the so-called free-space combiner. The free-space combiner can be integrated into the lens of the data glasses. Alternatively, the free-space combiner can also be designed as a separate optical element, separate from the lens.The free-space combiner overlays / combines the augmented image / scanned image of the virtual retinal display with the ambient image by serving as a reflecting or diffracting surface for the image content projected by the laser projector of the virtual retinal display (the scanned image). However, the free-space combiner allows other wavelengths, predominantly contained in the ambient image, to pass through essentially unhindered. This reflection or diffraction can be generated by reflection at a refractive index transition and / or by diffractive structures (such as holograms). For example, a holographic optical element (HOE) can be integrated into the spectacle lens as a free-space combiner. HOEs possess a high angular and wavelength selectivity, which is advantageous for achieving the desired selective reflection effect.

[0009] The laser projector emits, in particular, at least laser light in the red, green, and blue spectral ranges. In addition, the laser projector can also emit an infrared laser beam, which can be used, for example, to track the eye movement of a user of the data glasses. For this purpose, the laser projector could, for example, have a laser feedback interferometry (LFI) sensor. The scanned image is then generated, for example, with the aid of a two-dimensional deflection of the laser light emitted by the laser projector by a MEMS mirror system of the laser projector with a tiltable element (2D MEMS mirror) / with tiltable elements (two 1D MEMS mirrors). This laser light thus writes a rasterized image onto the retina of the user of the data glasses.A laser light beam generated by the laser projector is then preferably aligned via the MEMS mirror system and the free-space combiner so that it has a small beam diameter / circle of confusion at the location of the entrance pupil of the user's eye. This small circle of confusion in the pupil plane of the user's eye is also called a beamlet. A beamlet is preferably a small area in the pupil of the user's eye through which the individual beams of all pixels of the scanned images pass. The beamlet thus contains all the image information (pixels) of each scanned image. If the pupil moves due to eye movement of the user, the user will only see a non-vignetted scanned image / augmented image as long as the beamlet is still within the pupil area of ​​their eye.The area within which the center of the user's pupil can move while still allowing the user to see a complete image is called the eyebox. An "eyebox" is therefore preferably understood to be a spatial area within which all light rays of a scanning projection from the laser projector can pass through the entrance pupil of the user's eye. Using known eye tracking techniques, the smart glasses can obtain information about the current pupil position and other parameters of the user's eye. The beamlet can then be continuously moved by a 2D tilt mirror system of the smart glasses that is different from the MEMS mirror system of the laser projector, thus tracking any movement of the user's pupil (moving beamlet principle).In addition, the use of a lens with variable focal length (varifocal lens) in the beam path of the virtual retinal display can optimize the imaging performance and / or image sharpness of the scanned images / augmented images.

[0010] Calibration images and / or calibration patterns are, in particular, images or patterns that are specifically provided for carrying out a calibration procedure, in particular a field calibration procedure. In particular, the calibration images and / or calibration patterns are generated using the laser light of the laser projector. The user feedback can be provided by the wearer of the data glasses in various ways. For example, by an acoustic signal, by a (hand) gesture, by an eye movement, by an eyelid movement, by a head movement, by actuating an actuating element of the data glasses or another device that is in communication with the data glasses, e.g. a smartphone, or by any other conceivable means of communication. The calibration itself can be a software and / or hardware setting change on the data glasses, in particular one or more components of the data glasses, such asoptical system, laser projector, or head-mounted data glasses positioning elements. "Configured" or "intended" is understood to mean, in particular, specifically programmed, designed, and / or equipped. The fact that an object is intended or configured for a specific function is understood to mean, in particular, that the object fulfills and / or performs this specific function in at least one application and / or operating state.

[0011] The automated mechanical actuator can, for example, be an electrically actuated linear actuator or an electric motor of the data glasses. The manual mechanical actuator can, for example, comprise a manually operable adjustment wheel or a manually operable adjustment slide of the data glasses. The automated and / or manual mechanical actuator can, for example, be configured to adjust elements of the optical system of the data glasses relative to one another or internally. The mechanical actuator can, for example, be configured to adjust elements of the laser projector of the data glasses relative to one another or internally. The mechanical actuator can, for example, be configured to adjust elements for positioning the data glasses relative to the head / eye of the wearer of the data glasses.The manual conversion of the data glasses can in particular comprise a replacement or manual deformation of one or more parts of the data glasses, such as nose pads, temples or lens spacers. In particular, an adjustment of the position of the beamlet is also generated by repositioning the data glasses relative to the head / eye of the wearer of the data glasses. The manual mechanical actuator could, for example, be an adjusting screw for adjusting an interpupillary distance. The manual mechanical actuator could, for example, be an adjusting screw on the glasses frame, e.g. on the ear, for changing a glasses geometry, such as the inclination of a temple to a lens. The manual mechanical actuator could, for example, be an adjusting screw provided for moving optical components, e.g. projection optics, lenses, etc., of the optical system, e.g. within the glasses frame.

[0012] A "computing unit" is understood, in particular, to be a unit with an information input, an information processing unit, and an information output. The computing unit advantageously comprises at least one processor, a memory, input and output means, further electrical components, an operating program, control routines, control routines, and / or calculation routines. The components of the computing unit are preferably arranged on a common circuit board and / or advantageously arranged in a common housing, in particular the data glasses. Alternatively, the computing unit can also be arranged at least partially external to the data glasses (e.g., on a connected mobile device) or distributed (e.g., in a cloud).

[0013] It is further proposed that the data glasses issue instructions to the wearer of the data glasses, in particular via a visual display generated by the virtual retinal display or via an acoustic sound output, as to how the manual mechanical actuator must be operated to calibrate the data glasses or which manual modification of the data glasses must be carried out to calibrate the data glasses. This can advantageously simplify and / or accelerate calibration considerably. A high level of user-friendliness can advantageously be achieved. The risk of incorrect calibration can advantageously be reduced. Field calibration, which can in particular be carried out by the user of the data glasses themselves, can advantageously be enabled. For example, the data glasses can transmit visual and / or acoustic instructions similar to: "Please turn screw number two three turns to the right."

[0014] Furthermore, it is proposed that by actuating the manual mechanical actuator or the automated mechanical actuator, a position and / or inclination of a temple of the data glasses relative to a lens of the data glasses, an adjustment of a hinge of the data glasses, an adjustment of a distance between the lenses of the data glasses, or a relative positioning and / or inclination of optical elements of the optical system of the data glasses to one another can be adjusted. This advantageously allows optimal calibration to be achieved, particularly in terms of adapting the data glasses to different users, especially to different user head geometries.

[0015] It is also proposed that a nose pad of the smart glasses be modified or replaced during the manual conversion of the smart glasses. This advantageously allows for easy adaptation of the smart glasses to different users, in particular to different user head geometries. In particular, the nose pad comprises at least one bridge plate or at least two bridge plates or is formed by one or more bridge plates. However, alternative designs of nose pads for smart glasses without bridge plates are also conceivable.

[0016] Furthermore, it is proposed that by controlling the automated mechanical actuator, a position of at least one tilt mirror, in particular of the 2D tilt mirror system, of the optical system of the data glasses is adjusted, in particular one that modifies a position of the beamlet. This advantageously allows for particularly precise calibration, in particular without the need for specialist personnel. A high level of user-friendliness can advantageously be achieved. In particular, the tilt mirror is different from the MEMS mirrors of the MEMS mirror system of the laser projector. In particular, the main task of the tilt mirror is to move the beamlet, preferably to track the pupil of the beamlet within the framework of the moving beamlet principle.

[0017] It is further proposed that the calibration image and / or the calibration pattern displayed to the wearer of the data glasses comprise a plurality of, preferably geometrically identical, white surfaces, in particular rectangles, circles or other shapes, the brightnesses of which are adjusted to one another, in particular individually, by the wearer of the data glasses for the purpose of calibrating a surface brightness of an image output from the data glasses. This advantageously enables a particularly simple calibration, in particular a field calibration, of the surface brightness of the image output from the data glasses. Due to various factors, including an angle-, wavelength- and / or temperature-dependent diffraction efficiency of holograms, which can be used as free-space combiners in the data glasses, the homogeneity of the scanned images may not always be ensured.For calibration purposes, for example, white rectangles can be displayed next to each other within the user's field of view. One of the rectangles can then be selected and made brighter or darker using commands or user input until it is as bright as a neighboring rectangle. The brightness of a rectangle can be changed, for example, by adjusting the laser projector or the optical system of the data glasses. This is repeated with several rectangles until relative brightness information is available across the entire image. This can then be used to evenly illuminate the entire field of view during operation of the virtual retinal display, particularly without having to rely on the factory calibration, which may have become obsolete in the meantime.In particular, the size of the white areas, especially the rectangles, can be selected so that the probability of local brightness jumps or deviations is as low as possible. It is conceivable that the user can change the size or position of the white areas if noticeable brightness differences occur within one of the white areas.

[0018] Furthermore, it is proposed that the calibration image and / or the calibration pattern displayed to the wearer of the data glasses comprise a plurality of, preferably geometrically identical, colored areas, in particular rectangles, circles, or other shapes, whose color tones are adjusted to one another by the wearer of the data glasses, in particular individually, for the calibration of a surface color homogeneity of an image output from the data glasses, preferably by means of a surface-by-surface change of an RGB color mix of the image output from the data glasses. This advantageously enables a particularly simple calibration, in particular a field calibration, of the color homogeneity of the image output from the data glasses. In particular, the calibration of the color homogeneity using the colored areas can proceed analogously to the previously described calibration of the surface brightness, wherein in particular only the white areas are replaced by the colored areas.In particular, it is conceivable that for the calibration of color homogeneity each RGB color is individually adjusted / calibrated to one another and / or that the RGB laser beams are superimposed in the color homogeneity calibration and thus the user aims for the highest possible "whiteness" of the colored surfaces during calibration. In particular, the RGB color mix is ​​optimally coordinated when the superimposed RGB laser beams appear white to the wearer of the data glasses. During the calibration of color homogeneity the user can mark individual colored surfaces and, using commands or user inputs, adjust the respective colors red, green and blue to one another, e.g. by making them brighter, darker or by changing the wavelength, until the color impression of the colored surface is pure white for the user.A change in brightness or a change in wavelength of a color component can be achieved, for example, by adjusting the laser projector of the data glasses or the optical system of the data glasses.

[0019] If the calibration pattern displayed to the wearer of the data glasses comprises a plurality of areas with, preferably high-contrast, lines or shapes, for example line pair bars, whose image sharpnesses are adjusted to one another, in particular individually, by the wearer of the data glasses for the calibration of a planar image sharpness impression of an image output of the data glasses, a simple and / or rapid sharpness calibration can advantageously be enabled. A change in the sharpness impression of the calibration pattern can be achieved in particular by an adjustment on the laser projector and preferably by an adjustment of the optical system, e.g. via an electrically actuated lens with variable focus / a varioptic lens. Alternatively, the change in the sharpness impression of the calibration pattern can be achieved by shifting optical components in the beam path of the optical system of the data glasses, e.g.by means of adjusting screws or automated actuators. In addition, to achieve the change in sharpness impression, the image representing the calibration pattern could also be softened by software, e.g. if an image area is 'too sharp' and thus stands out compared to other image areas. For example, the user is shown high-contrast patterns, e.g. white and black separate lines, in one part of the field of view. The user then changes the sharpness of the lines using the adjustment made or instructed until they have a (high) contrast that is particularly appealing to them. Calibration is preferably carried out at several points in the user's field of view until the system can determine a good compromise sharpness setting for the entire image, e.g. by interpolation or averaging.

[0020] It is additionally proposed that the calibration pattern displayed to the wearer of the data glasses comprises a grating structure and / or a scale, wherein the wearer of the data glasses is requested to provide user feedback regarding which parts of the grating structure and / or the scale are currently perceptible, wherein coordinates of a center of a current field of view are determined based on this user feedback and wherein a center of a future image output of the data glasses is calibrated based on the determined coordinates, in particular via a control of the laser projector and / or the optical system. This advantageously enables a particularly simple calibration, in particular a field calibration, of the field of view. To determine the current field of view, the beamlet is preferably held stationary and displays, for example, the grating structure.To perform the calibration, the user tells the system / data glasses, for example, which intersection point of the grid structure is centrally located in front of them. The data glasses, in particular the virtual retinal display, can then, for example, adjust the image content of the output scanned image or a beam path of the virtual retinal display so that the new future center of the field of view is at this location and / or the image information to be displayed is arranged around the specific center of the field of view. It is also conceivable that the user can use this calibration or a process similar to this calibration to determine where the field of view should be, for example, not centered on their line of sight, but slightly offset to the right, left, top, or bottom. To do this, the user could read out the desired off-center coordinate of the calibration pattern designed as a grid structure and communicate it to the data glasses.The grid structure could be designed as a Cartesian grid. Alternatively, the grid structure could be formed by markings in the corners of the field of view.

[0021] If, in addition, the user feedback of the wearer of the data glasses is recorded by means of at least one microphone for recording voice commands, by means of a pressure switch on the data glasses, by means of a camera for gesture recognition, by means of an eye-tracking system of the data glasses for recording gaze commands or by means of an input into an external mobile device, such as a smartphone or a tablet, a particularly high level of user-friendliness and / or field calibration suitability can be advantageously achieved.

[0022] It is also proposed that a wearer-specific calibration setting be saved in an internal or external (e.g., cloud, mobile device, etc.) storage unit, in particular of the data glasses, for later retrieval, e.g., to reset the wearer-specific calibration setting. This can advantageously achieve a high level of user-friendliness. This can advantageously facilitate the use of the data glasses by multiple people. In particular, in all of the aforementioned cases, the data glasses save the calibration obtained in each case and use it / offer it for use in order to present the user(s) of the data glasses with an optimized image with optimized availability. In particular, the data glasses are designed to store multiple calibrations assigned to different users.

[0023] Furthermore, the computing unit for implementing the above-described method and / or the data glasses with the virtual retinal display and the computing unit are proposed. This advantageously makes it possible to provide data glasses with a high degree of user-friendliness, in particular with a field calibration option.

[0024] The method according to the invention, the computing unit according to the invention, and the data glasses according to the invention are not intended to be limited to the application and embodiment described above. In particular, the method according to the invention, the computing unit according to the invention, and the data glasses according to the invention may have a number of individual elements, components, units, and method steps that differs from the number stated herein to fulfill a functionality described herein. Furthermore, in the value ranges specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0025] drawing

[0026] Further advantages will become apparent from the following description of the drawings. The drawings illustrate an embodiment of the invention. The drawings, the description, and the claims contain numerous features in combination. Those skilled in the art will conveniently consider the features individually and combine them into useful further combinations. They show:

[0027] Fig. 1 a is a schematic representation of a top view of data glasses with a virtual retinal display,

[0028] Fig. 1 b a schematic representation of the data glasses from the front,

[0029] Fig. 2 is a schematic flow diagram of a method for calibrating the data glasses,

[0030] Fig. 3 shows an exemplary calibration image for a surface brightness calibration,

[0031] Fig. 4 shows an exemplary calibration image for a color homogeneity calibration,

[0032] Fig. 5 shows an exemplary calibration image for an image sharpness calibration,

[0033] Fig. 6 shows a first exemplary calibration image for a field-of-view calibration and

[0034] Fig. 7 shows a second exemplary calibration image for the field-of-view calibration.

[0035] Description of the embodiment

[0036] Figures 1a and 1b schematically show different views of data glasses 12. The data glasses 12 comprise a spectacle frame 62. The data glasses 12 comprise spectacle temples 32, 32'. The data glasses 12 comprise spectacle hinges 34, 34'. Each of the spectacle temples 32, 32' is hinged to the spectacle frame 62 via one of the spectacle hinges 34, 34'. The data glasses 12 comprise spectacle lenses 30, 30'. The spectacle lenses 30, 30' are enclosed in the spectacle frame 62. The data glasses 12 comprise a nose pad 38. The nose pad 38 is replaceable. Different nose pads 38 can be provided / suitable for different users of the data glasses 12. The data glasses 12 comprise an adjustment function for adjusting a distance 36 between the spectacle lenses 30, 30'. The data glasses 12 comprise a computing unit 26. The computing unit 26 is integrated into the data glasses 12.Alternatively, the computing unit 26 could also be integrated into an external mobile device 58, which is at least communicatively connected to the data glasses 12. The data glasses 12, in particular the computing unit 26, have a memory unit 60. The memory unit 60 is provided for storing wearer-specific calibration settings. Using the memory data of the memory unit 60, wearer-specific calibration settings can be reset or alternately set between different wearers of the same data glasses 12. It is also conceivable for the data glasses to automatically detect which of the stored calibration settings are currently appropriate, e.g., based on user recognition.

[0037] The data glasses 12 have a virtual retinal display 10. The virtual retinal display 10 is designed to project an artificially generated image content / scanned images 16 (see Figs. 3 to 7) directly onto the retina of an eye of a user of the data glasses 12. The data glasses 12 have a laser projector 14. The laser projector 14 is integrated, for example, into one of the temples 32, 32' of the data glasses 12. The laser projector 14 is designed to generate visible laser light. The visible laser light generates the scanned images 16. The laser projector 14 includes a MEMS mirror system (not shown) for scanning the visible laser light of the laser projector 14 to generate the two-dimensional scanned image 16. The data glasses 12 include a tilt mirror 40.The tilt mirror 40 is provided for controlling a position of the laser light emitted by the laser projector 14, in particular a beamlet of the virtual retinal display 10. The data glasses 12 have an optical system 18. The optical system 18 includes a deflection element 64 (see Figures 1a and 1b together). The deflection element 64 is designed as a holographic optical element (HOE). The deflection element 64 is integrated into one of the lenses 30, 30'. The deflection element 64 is provided for deflecting the laser light emitted by the laser projector 14 toward a pupil plane 66 of the data glasses 12. The pupil plane 66 of the data glasses 12 is defined by a plane in which an eye of a wearer of the data glasses 12 is positioned when the data glasses 12 are properly handled.The deflection element 64 is provided for focusing the laser light emitted by the laser projector 14 onto the pupil plane 66 of the data glasses 12. The deflection element 64 is provided for generating the beamlet. The optical system 18 can also comprise further optical elements (not shown here), such as a varifocal lens. The scanned images 16 generated by the laser projector 14 are projected as a beamlet into an eyebox of the data glasses 12 via the optical system 18 of the data glasses 12.

[0038] The data glasses 12 have an automated mechanical actuator 24. The automated mechanical actuator 24 is controlled by the computing unit 26 of the data glasses 12 to adjust the position of the beamlet during calibration of the data glasses 12. The data glasses 12 have a manual mechanical actuator 28. The manual mechanical actuator 28 is actuated by the wearer of the data glasses 12 to adjust the position of the beamlet during calibration of the data glasses 12. By actuating the manual mechanical actuator 28 or the automated mechanical actuator 24, a relative position and / or a relative inclination of the spectacle lens 30, 30' to the spectacle temples 32, 32' is set. By actuating the manual mechanical actuator 28 or the automated mechanical actuator 24, the spectacle hinge 34, 34' of the data glasses 12 is alternatively or additionally adjusted.By actuating the manual mechanical actuator 28 or the automated mechanical actuator 24, the distance 36 between the lenses 30, 30' of the data glasses 12 is alternatively or additionally adjusted. By actuating the manual mechanical actuator 28 or the automated mechanical actuator 24, the relative positioning and / or inclination of optical elements of the optical system 18 of the data glasses 12 is alternatively or additionally performed.

[0039] The data glasses 12 have a microphone 50 (see Fig. 1 b). The microphone 50 is provided for recording user feedback and / or voice commands from the wearer of the data glasses 12, in particular in connection with a calibration of the data glasses 12. The data glasses 12 have a loudspeaker 68 (see Fig. 1 b). The loudspeaker 68 is provided for outputting setting commands to the wearer of the data glasses 12 during a calibration, in particular an actuation of the manual mechanical actuator 28. The data glasses 12 have a push button switch 52 (see Fig. 1 b). The push button switch 52 is provided for manually setting / adjusting a state of the data glasses 12. The pressure switch 52 is provided for receiving user feedback from the wearer of the data glasses 12, in particular in connection with a calibration of the data glasses 12. The data glasses 12 have a camera 54 (see Fig. 1 b).The camera 54 is provided for gesture recognition. Depending on a recognized gesture, a state of the data glasses 12 is changed / adjusted during calibration. The camera 54 is provided for receiving user feedback from the wearer of the data glasses 12 via gesture recognition. The computing unit 26 is configured to process and recognize the gestures recorded by the camera 54. The data glasses 12 include an eye-tracking system 56. The eye-tracking system 56 is embodied, for example, as a known LFI eye-tracking system 56 based on exploiting the "bright pupil effect." However, alternative embodiments of eye-tracking systems 56 are also conceivable. The eye-tracking system 56 is provided for receiving user feedback from the wearer of the data glasses 12. The computing unit 26 is configured to process and evaluate the eye movements recorded by the eye-tracking system 56.For example, the detection of a specific gaze direction or a blink code (so-called gaze commands) can be recognized as user feedback by the eye-tracking system 56. Furthermore, the external mobile device 58 is designed to receive user feedback, e.g., via a touchscreen control of the mobile device 58.

[0040] Figure 2 shows a schematic flow diagram of a method for calibrating the data glasses 12. In at least one method step 70, the calibration, in particular the field calibration, is initiated by the user. The user can select from a variety of calibration options (surface brightness calibration, color homogeneity calibration, field-of-view calibration, etc.). Furthermore, it is conceivable that, after the start of the calibration, all necessary calibration steps are automatically performed one after the other. In at least one method step 72, scanned images 16 are generated by the laser projector 14 and projected as beamlets into an eyebox of the data glasses 12 via the optical system 18 of the data glasses 12. The beamlets generated in method step 72 contain calibration images 20 and / or calibration patterns 22.By imaging the image into the eyebox of the data glasses 12, the calibration images 20 and / or calibration patterns 22 are output to the current wearer of the data glasses 12 via the beamlets. In at least one method step 74, a surface brightness of the image output of the data glasses 12 is calibrated. In at least one method step 76, a color homogeneity of the image output of the data glasses 12 is calibrated. In at least one method step 78, an image sharpness impression of the image output of the data glasses 12 is calibrated. In at least one method step 80, a field of view of the data glasses 12 is calibrated. The calibration steps 74, 76, 78, 80 can be performed independently of one another in any desired order.In at least one further method step 82, which is in particular the same for all calibration steps 74, 76, 78, 80, user feedback provided by the wearer of the data glasses 12 based on their perception is recorded and used to calibrate the data glasses 12. The user feedback from the wearer of the data glasses 12 is recorded by at least the microphone 50 in the form of voice commands, by the push button 52 on the data glasses 12, by the camera 54 in the form of recognized gestures, by the eye-tracking system 56 in the form of gaze commands, or by an input into the external mobile device 58.

[0041] Based on the user feedback, in at least one method step 84, the automated mechanical actuator 24 is controlled by the computing unit 26 of the data glasses 12 to adjust the position of the beamlet for calibrating the data glasses 12. In method step 84, by controlling the automated mechanical actuator 24, for example, a position of the tilt mirror 40 of the optical system 18 of the data glasses 12, in particular modifying a position of the beamlet, can be adjusted. Alternatively or additionally, in a method step 86, actuation of the manual mechanical actuator 28 can be initiated to manually adjust the position of the beamlet.In method steps 84, 86, by actuating the manual mechanical actuator 28 or the automated mechanical actuator 24, a position relative to one of the lenses 30, 30' of the data glasses 12 and / or an inclination of the temples 32, 32' can be adjusted, the hinge 34 can be adjusted, the distance 36 between the lenses 30, 30' of the data glasses 12 can be adjusted, or a relative positioning and / or inclination of optical elements of the optical system 18 of the data glasses 12 can be adjusted. Alternatively or in addition to the use of actuators 24, 28, a manual conversion of the data glasses 12 for manually adjusting the position of the beamlet during calibration can be initiated in a method step 88. In method step 88, for example, the nose pad 38 of the data glasses 12 can be converted or replaced during the manual conversion of the data glasses 12.In method step 86, in which the manual adjustment is carried out, and / or in method step 88, in which the manual conversion is carried out, the data glasses 12 issues instructions to the wearer of the data glasses 12 via a visual display generated by the virtual retinal display 10 or via an acoustic sound output, e.g. of the data glasses 12 or the mobile device 58, as to how the manual mechanical actuator 28 must be actuated for the corresponding calibration of the data glasses 12 or which manual conversion of the data glasses 12 must be carried out for the corresponding calibration of the data glasses 12.

[0042] Alternatively or in addition to a mechanical adjustment by means of the actuators 24, 28 or to a conversion, in a method step 90 the laser projector 14 and / or the optical system 18 can be controlled by the computing unit 26 of the data glasses 12 during the calibration to adjust a brightness, a sharpness or a color tone of the reproduced calibration images 20 and / or calibration patterns 22.

[0043] In at least one further method step 92, a wearer-specific calibration setting is stored in the internal or external storage unit 60 of the data glasses 12 for later retrieval, e.g., for resetting the wearer-specific calibration setting.

[0044] During the calibration of the surface brightness in method step 74, a calibration image 20 comprising a plurality of geometrically identical white surfaces 42, 42' (cf. Fig. 3) is displayed to the wearer of the data glasses 12 in a sub-process step 94. The white surfaces 42, 42' in Fig. 3 are shown as rectangles by way of example. However, they could also be circles, polygons, or other shapes. The brightnesses of the white surfaces 42, 42' are individually adjusted to one another manually and / or automatically based on user feedback to calibrate the surface brightness of the image output of the data glasses 12. For this purpose, in a further sub-process step 96, one of the white surfaces 42 is selected by the wearer of the data glasses 12 or one of the white surfaces 42 is displayed to the wearer of the data glasses 12 as active.In Figure 3, this is done, for example, by displaying two small triangles above and below the active white area 42. By executing method step 90, in which the computing unit 26 controls the laser projector 14 and / or the optical system 18 to adjust the local area brightness, the brightness is iteratively adjusted (e.g., by commands or manual settings) until it is aligned with an adjacent white area 42' in the wearer's field of view. This is then repeated for the entire field of view.

[0045] During the calibration of color homogeneity in method step 76, a calibration image 20 comprising a plurality of geometrically identical colored areas 44, 44' (cf. Fig. 4) is displayed to the wearer of the data glasses 12 in a sub-process step 98. The colored (e.g., red, green, or blue) areas 44, 44' in Fig. 4 are shown as rectangles by way of example. However, they could also be circles, polygons, or other shapes. The color tones of the colored areas 44, 44' are individually adjusted to one another manually and / or automatically based on user feedback to calibrate the area brightness of the image output of the data glasses 12. For this purpose, in a further sub-process step 100, one of the colored areas 44 is selected by the wearer of the data glasses 12, or one of the colored areas 44 is displayed to the wearer of the data glasses 12 as active.In Figure 4, this is done, for example, by displaying two small triangles above and below the active colored area 44. By executing method step 90, in which the computing unit 26 controls the laser projector 14 and / or the optical system 18 to adjust the local color tone, the color tone is iteratively adjusted (e.g. by commands or manual settings) until it is aligned with a neighboring colored area 44' in the field of view of the wearer. This is then repeated for the entire field of view. Alternatively, the RGB color mix could also be calibrated directly during the calibration of the color homogeneity in method step 76, ie all light colors could be superimposed and the goal of the calibration could be the purest possible "whiteness of the colored area 44, 44'.

[0046] During the calibration of the image sharpness impression in method step 78, a calibration pattern 22 comprising a plurality of regions 46 with high-contrast lines or shapes, for example, line pair bars (see Fig. 5) is displayed to the wearer of the data glasses 12 in a sub-method step 102. The calibration pattern 22 comprises a plurality of regions 46 with high-contrast lines or shapes, for example, line pair bars (see Fig. 5). The sharpness impression of the regions 46 is adjusted to one another manually and / or automatically, individually or in groups, for the calibration of the image sharpness impression of the image output of the data glasses 12 based on the user feedback. By executing method step 90, in which the computing unit 26 controls the laser projector 14 and / or the optical system 18 to adjust the local image sharpness, the image sharpness of the calibration pattern 22 in a region 46 is iteratively adjusted (e.g., by commands or manual settings) until it is adjusted to the image sharpness of an adjacent region 46 in the wearer's field of view.Alternatively, an optical element, e.g., a varifocal lens, could be used to adjust the local image sharpness by performing one of the method steps 84, 86, in which an actuator 24, 28 is actuated automatically or manually. This is then repeated for the entire field of view.

[0047] During the calibration of the field of view in method step 80, a calibration pattern 22 comprising a grid structure 104 (see Fig. 6) or a scale 106 (see Fig. 7) is displayed to the wearer of the data glasses 12 in a sub-method step 108. In at least one further sub-method step 110, the wearer of the data glasses 12 is prompted by means of the virtual retinal display 10 to provide user feedback regarding which parts of the grid structure 104 or the scale 106 are perceptible to him / her and / or at which coordinates of the grid structure 104 or the scale 106 a center of the field of view currently perceptible to the wearer of the data glasses 12 lies. In a further sub-method step 48, a center of a future image output of the data glasses 12 is determined based on the coordinates thus determined.The new center of the future image output is determined by executing one or more of the process steps with the abbreviations 84, 86, 88 and 90, in particular via a software-based control, via a computer- or manually controlled actuator control and / or via a manual conversion of the data glasses 12.

Claims

Claims 1. A method for calibrating data glasses (12) having a virtual retinal display (10) (retinal scan display), wherein scanned images (16) are generated by a laser projector (14) of the virtual retinal display (10) and projected into an eyebox of the data glasses (12) as beamlets via an optical system (18) of the data glasses (12), wherein calibration images (20) and / or calibration patterns (22) are output to a wearer of the data glasses (12) by means of the beamlet, and wherein user feedback, in particular conscious, which is output by the wearer of the data glasses (12) based on their perception, in particular consciously, is detected and used to calibrate the data glasses (12), characterized in that for the calibration of the data glasses (12), an automated mechanical actuator (24) for adjusting a position of the beamlet is controlled by a computing unit (26) of the data glasses (12),or an actuation of a manual mechanical actuator (28) is initiated to manually adjust the position of the beamlet, or a manual conversion of the data glasses (12) is initiated to manually adjust the position of the beamlet, or the laser projector (14) and / or the optical system (18) are controlled by the computing unit (26) of the data glasses (12) to adjust a brightness, a sharpness or a color tone of the reproduced calibration images (20) and / or calibration patterns (22).

2. Method according to claim 1, characterized in that from the data glasses (12), in particular via a visual display generated by the virtual retinal display (10) or via an acoustic sound output, Instructions are given to the wearer of the data glasses (12) as to how the manual mechanical actuator (28) must be operated to calibrate the data glasses (12) or which manual conversion of the data glasses (12) must be carried out to calibrate the data glasses (12).

3. Method according to claim 1 or 2, characterized in that by actuating the manual mechanical actuator (28) or the automated mechanical actuator (24), a position relative to a spectacle lens (30, 30') of the data glasses (12) and / or an inclination of a spectacle temple (32, 32') of the data glasses (12), an adjustment of a spectacle hinge (34, 34') of the data glasses (12), an adjustment of a distance (36) between the spectacle lenses (30, 30') of the data glasses (12) or a relative positioning and / or inclination of optical elements of the optical system (18) of the data glasses (12) to one another is set.

4. Method according to one of the preceding claims, characterized in that during the manual conversion of the data glasses (12) a nose pad (38) of the data glasses (12) is converted or replaced.

5. Method according to one of the preceding claims, characterized in that by controlling the automated mechanical actuator (24) a position of at least one tilt mirror (40), in particular modifying a position of the beamlet, of the optical system (18) of the data glasses (12) is adjusted.

6. Method according to one of the preceding claims, characterized in that the calibration image (20) and / or the calibration pattern (22) displayed to the wearer of the data glasses (12) comprises a plurality of, preferably geometrically identical, white areas (42, 42'), in particular rectangles, circles or other shapes, the brightnesses of which are adjusted to one another, in particular individually, by the wearer of the data glasses (12) for the purpose of calibrating an area brightness of an image output from the data glasses (12).

7. Method according to one of the preceding claims, characterized in that the calibration image (20) and / or the calibration pattern (22) displayed to the wearer of the data glasses (12) comprises a plurality of, preferably geometrically identical, colored areas (44, 44'), in particular rectangles, circles or other shapes, the color tones of which are adjusted to one another, in particular individually, by the wearer of the data glasses (12), preferably by means of an area-by-area change in an RGB color mix of the image output of the data glasses (12), in order to calibrate a planar color homogeneity of an image output of the data glasses (12).

8. Method according to one of the preceding claims, characterized in that the calibration pattern (22) displayed to the wearer of the data glasses (12) comprises a plurality of areas (46) with, preferably high-contrast, lines or shapes, for example line pair bars, the image sharpnesses of which are adjusted to one another for the calibration of a planar image sharpness impression of an image output of the data glasses (12) by the wearer of the data glasses (12), in particular individually or in groups.

9. Method according to one of the preceding claims, characterized in that the calibration pattern (22) displayed to the wearer of the data glasses (12) comprises a grid structure (104) and / or a scale (106), wherein the wearer of the data glasses (12) is requested to provide feedback as user feedback regarding which parts of the grid structure (104) and / or the scale (106) are currently perceptible, wherein coordinates of a center of a current field of view are determined based on this user feedback, and wherein a center of a future image output of the data glasses (12) is calibrated based on the determined coordinates, in particular via a control of the laser projector (14) and / or the optical system (18).

10. Method according to one of the preceding claims, characterized in that the user feedback of the wearer of the data glasses (12) is recorded by means of at least one microphone (50) for recording voice commands, by means of a pressure switch (52) on the data glasses (12), by means of a camera (54) for gesture recognition, by means of an eye-tracking system (56) of the data glasses (12) for detecting gaze commands or by means of an input into an external mobile device (58), such as a smartphone or a tablet.

11. Method according to one of the preceding claims, characterized in that a carrier-specific calibration setting is stored in an internal or external storage unit (60) for later retrieval, e.g. for resetting the carrier-specific calibration setting.

12. Computing unit (26) for carrying out a method according to one of the preceding claims.

13. Data glasses (12) with a virtual retinal display (10) and with the computing unit (26) according to claim 12.

Images