Method and apparatus for non-contact user guidance in space for contactless optical recordings of fingerprint images

Abstract

The invention relates to a method and an apparatus for non-contact user guidance in space for contactless optical recordings of fingerprint images. The invention comprises generating a position specification (PE) for fingers or hand to be presented in a defined positioning plane (PE), which is visualized as a position specification for this purpose by means of a virtual display (11), recording images by means of a camera device (2), wherein at least parts of a finger or hand inner side in the positioning plane (PE) are recorded at a distance of at least 100 mm from the camera device (2) in order to define a contact-free measurement volume (21) having a depth of at most 30 mm, detecting the presented fingers with the aid of an illumination which allows at least one finger or hand recording for determining the position of the presented fingers or hand around the defined positioning plane (PE), and evaluating the finger or hand recordings at least with respect to the presence and position of a finger or hand inner side in order to control the virtual display (11) for displaying results or correction instructions for user guidance as feedback.

Description

title

[0001] Method and device for contactless user guidance in space for contactless optical recording of fingerprint images Technical field of the invention

[0002] The invention relates to a method and a device for contactless user guidance in space for contactless optical recordings of fingerprint images, in particular for the automated recording and processing of qualified fingerprints for databases for identification and identity verification of persons in conjunction with databases.

[0003] The outbreak of the coronavirus (SARS-CoV-2) epidemic triggered a global crisis with an immense impact on all public facilities where personal identities must be recorded or verified during significant flows of people, such as security and border control points, particularly at airports. Existing contact-based fingerprint scanners required disinfection after each use to prevent potential transmission of the disease to subsequent users. This increased the pressure to develop technologies and systems for contactless fingerprint capture, preferably of multiple fingers of one hand simultaneously. This required an intuitive user interface that allowed users to precisely position their fingers on a plane for optical fingerprint capture without any contact surfaces.

[0004] In a new generation of contactless optical fingerprint scanners based on structured illumination, the biometric object is no longer placed on the scanner but positioned within the object area of ​​the device's optical system. The term "fingerprint scanner" has become a generic term for devices that optically capture biometric objects, specifically the skin structures of fingers. It is irrelevant whether the device is designed for all of the aforementioned biometric objects or for only one or a subset of them. It is also immaterial how the captured image of the biometric object is generated, for example, whether the object is scanned or imaged in its entirety. The term "fingerprint" has historical roots.Here, it refers to any type of image of the biometric object, regardless of whether the object is pressed onto a physical surface during capture or positioned contactlessly within a measurement volume in space. The ultimate goal is to generate an image that corresponds to a classic fingerprint, whereby any captured 3D data can be transformed into 2D information. State of the art

[0005] Numerous solutions for non-contact optical fingerprint scanning are already known in the prior art, which utilize different aids for finger alignment.

[0006] US patent 8600 123 B2 describes a user interface for a contactless multi-finger scanning system. This interface provides hand positioning assistance on a screen located elsewhere than the scanning area, presented as a virtual representation. In addition to this on-screen support, a user guidance system using a type of light barrier is described. The user, looking through a window in the housing, uses the visible beams of this system to align or move their hand. Such assistance is difficult for untrained individuals to use.

[0007] US Patent 10460 145 B2 describes a device without user guidance and without a virtual display. While this device can resolve the papillary structures of multiple fingers, this does not result in a "true" 3D image, but rather in a rough 3D information generated by structured lighting, which is used only to rectify a 2D image.

[0008] JP 2017-067933 A discloses a virtual angle reflector array in which a projection element arrangement consists of a plurality of two-dimensionally arranged beam deflection elements designed such that a display located behind the projection element arrangement is projected in front of the front and becomes visible there as a free-floating bright display.

[0009] US patent 8224064 B1 discloses a system and method for 3D imaging using structured lighting, in which several (all) fingers of a hand are captured and a user interface is provided via a display. However, the hand is scanned within the device without being visible to the user, which—because the hand is not visible—increases the risk of accidental contact and is also uncomfortable for the user. Since the hand is inserted into the device in an uncontrolled manner, a virtual display for guiding the user's hand within the scan volume is not applicable.

[0010] Furthermore, WO 2016 / 038347 describes a method and a device for adapting haptic feedback, in which an acoustic field is modulated to provide a tactile sensation for the hand. The haptic feedback is generated by ultrasound, specifically by generating a multitude of ultrasound waves with a common focal point using a phase-coupled array of ultrasound transducers. This common focal point forms a haptic feedback point used for tactile sensation for the hand, and the response of the fingertips can be detected by a step function controller.

[0011] Furthermore, virtual displays have been known for some time, primarily used in connection with augmented reality (AR). One such display system is called a virtual retinal display (VNA, also known as a light field display), which projects a raster image directly onto the retina of the eye. The user thus has the impression of a screen floating in front of them. VNAs have so far mostly been used in combination with various AR systems for displaying virtual reality, preferably in mobile devices such as mobile phones, PDAs (Personal Digital Assistants), and various playback devices, where the image is projected onto the user's retina when placed in front of them.

[0012] Another virtual display technology utilizes the so-called "Air Floating Image" method, disclosed in US patent 2021 / 003858 A1. In this method, a display device emits light to show information, and an aerial image-generating element reflects the light from the display device multiple times to display a virtual image. A light direction-regulating element is positioned in the optical path from the display device to improve the visibility of the floating virtual image and reduce glare.

[0013] A further development, known as Floating Pictogram (FPT) technology, is disclosed in US patents 2023 / 0033280 A1 and 2023 / 0384615. These patents describe an input device that can be operated without physical contact. A display unit emits light to show information, an aerial image-generating element reflects the light emitted by the display unit multiple times to create a virtual image of a display plane with a switch in space, and a sensor detects the position of a target object approaching the switch in space, such as a finger, to determine in a detection unit whether the switch has been activated or not.

[0014] From DE 102020 131 513 B3, a device for non-contact optical imaging of a selected surface area of ​​a hand is known, in which the surface area to be imaged is guided non-contactably into a target position relative to an image acquisition unit. For this purpose, the device has a frame in which the hand is roughly inserted into a target position. Beyond displaying the image area, the device then provides the user with instructions for correcting the position or information about the scan status. This requires the user to look away from the object being imaged and focus on an output unit located elsewhere, which significantly complicates hand positioning. Description of the invention

[0015] The invention is based on the objective of finding a new method for contactless user guidance in space for contactless optical fingerprint scanning, which enables the user to better perceive how to achieve the correct finger or hand position and allows for improved contactless, high-resolution optical fingerprint scanning. A further objective is to eliminate the need for additional distance or range measuring devices to determine the current finger or hand position during fingerprint scanning.

[0016] According to the invention, the problem of a method for contactless user guidance in space for contactless optical recordings of fingerprint images is solved by the following steps: - Generating a position specification for freely floating fingers or an entire hand in a defined positioning plane, in which the finger or palm sides are to be presented stretched, wherein the positioning plane is visualized by means of a virtual display as a position specification for the fingers or hand to be presented, in order to enable an optically sharp image recording of the finger or palm sides of the fingers or hand presented in a recording area, - Taking pictures of the fingers or hand presented in the recording area, wherein at least parts of a finger or hand palm are recorded in the defined positioning plane with a distance between the camera device and the positioning plane of at least 100 mm in order to define a non-contact measurement volume for fingerprints of the presented fingers or hand with a measurement volume depth of at most 30 mm, - Capturing the presented fingers or hand in the recording area by means of lighting that allows at least one finger or hand recording around the defined positioning plane to determine the position of the fingertips or palms of the presented fingers or hand, and - Evaluation of currently recorded finger or hand images, at least regarding the presence and position of a finger or palm side, which is used to provide feedback on the finger or hand images in order to control the virtual display for additional display of results or correction instructions for user guidance in finger or hand positioning.

[0017] Advantageously, after evaluating the currently recorded finger or hand images, the virtual display is controlled with correction instructions and the illumination of the finger or hand palm is repeated until the finger or hand palm in the defined positioning plane allows for a sharp image recording of the finger or hand palm palm.

[0018] Preferably, corrective instructions for user guidance in the event of a deviation from the positioning plane are visualized by the virtual or a real display as representations in the form of pulsating or flashing finger structures, outlines, or areas, and upon reaching the positioning plane, the representations switch to a continuous light. The corrective instructions for user guidance in the event of deviations from the positioning plane can be displayed by the virtual or a real display as representations in the form of finger structures, outlines, or areas in a first color, in a second color when approaching the positioning plane, and in a third color upon reaching the positioning plane.

[0019] It has proven advantageous to generate haptic pressure elements within the positioning plane defined by the virtual display, using at least one method involving the superposition of ultrasonic waves or the creation of a gas flow curtain. Acoustic or haptic elements can thus create a tactile sensation, resulting in an apparent contact surface for finger or hand positioning. This haptic feedback provides tactile perception and generates a sense of touch. Haptics encompasses the sensory feedback of pressure, pressure changes, or flow, enabling the user to perceive the positioning plane as a contact surface. Alternatively or additionally, acoustic signals can be used, the volume or frequency of which increases or decreases with increasing deviation from the positioning plane.

[0020] Advantageously, lighting can be achieved using structured lighting by projecting a known regular pattern onto the defined positioning plane.

[0021] The recording of fingerprint or handprint images using the known pattern projected into the defined positioning plane can be used to generate an illumination structure for obtaining high-resolution images for 3D fingerprint images.

[0022] Preferably, the recording of fingerprint or handprint images using the known pattern projected into the defined positioning plane can also be used to generate a measurement pattern that is used to determine the distance between the finger or palm surfaces and the camera device, wherein, by means of spectrally limited emission of the measurement pattern and spectral selective detection in the camera device, a distance determination between the finger or palm surfaces and the camera device can be separated for each image recording.

[0023] It is advantageous for finger or hand images to determine the presence and position of a finger or palm for user guidance to be taken with a lower optical resolution than for generating high-resolution fingerprint images.

[0024] It proves advantageous if the structured lighting for high-resolution fingerprint images in the lighting device is only activated when a positioning of the finger or palm in the defined positioning plane has been detected by a contactless distance measuring unit.

[0025] To generate high-resolution fingerprint images, several images or an image sequence are advantageously recorded within a time period of < 1s using the camera device and checked for a predetermined image quality using an evaluation unit, and when the predetermined image quality is reached, a grayscale image is generated.

[0026] Preferably, the generator system for the virtual display can also generate structured lighting for capturing high-resolution fingerprint images.

[0027] Furthermore, the object is solved according to the invention with a device for contactless user guidance in space for contactless optical recordings of fingerprint images, comprising - a generator system for generating a position specification for freely suspended fingers or an entire hand, wherein the generator system creates a defined positioning plane in which the finger or hand surfaces of the fingers or hand to be presented are to be positioned in an extended position, by means of a virtual display as a visualization of the position specification, in order to enable optically sharp image recordings of fingers or a hand presented in a recording area, - a camera device for capturing images of fingers or a hand presented in the recording area, wherein the camera device is directed at at least parts of the inside of the fingers or hand in the defined positioning plane and has a distance to the positioning plane of at least 100 mm in order to generate a non-contact measurement volume for fingerprints of the presented fingers or hand with a measurement volume depth of at most 30 mm, - a lighting device directed into the recording area and allowing at least one finger or hand recording around the defined positioning plane to determine the position of the fingertips or palms of the presented fingers or hand, and - an evaluation unit for evaluating currently recorded hand or finger images at least with regard to the presence and position of a finger or palm, wherein the evaluation unit is set up to provide feedback for user guidance and to control at least the generator system for the virtual display in addition to displaying correction instructions or results.

[0028] The defined measurement volume is understood to be precisely the recording area within which the camera device is capable of capturing optically sharp images of the fingertips or palms. The measurement volume is defined here with respect to the camera device by its length and width as lateral dimensions, as well as a depth determined by the depth of field of the optics used in the camera device.

[0029] Advantageously, the generator system for the virtual display features a projection display and a projection element arrangement, with which the virtual display can be generated from image information and patterns of the projection display as a positioning plane floating in space for hand or finger positioning.

[0030] The projection display of the generator system is preferably a conventional self-illuminating display from the group LCD, LED, OLED or CRT monitor, in order to project the image information and patterns into the recording area on a side of the projection element arrangement facing away from the projection display, with distances and angles specified by the design of the projection element arrangement.

[0031] The projection display of the generator system can, in addition to the floating positioning plane generated by image information and patterns, expediently contain further image information with correction instructions from the group of symbols, pictograms, directional arrows, moving, pulsating or blinking finger structures, outlines or surfaces, as user guidance for correcting the position of the hand or fingers.

[0032] Advantageously, in addition to emitting structured illumination of the fingertips or palms, the projection display of the generator system can be equipped with at least one illumination structure or measurement pattern for capturing high-resolution fingerprint images.

[0033] The generator system for the virtual display can advantageously be equipped to control a real display for user guidance in addition to the virtual display.

[0034] The measuring volume of the device advantageously has a lateral dimension of 80 x 75 mm. 2 and 200 x 150 mm 2 and a depth of 15 to 30 mm. A lateral dimension of 100 x 80 mm is advantageous. 2 and 170 x 120 mm 2 and have a depth of 18 mm to 25 mm. Preferably, the measuring volume has lateral dimensions of 150 x 100 mm. 2 and a measuring volume depth of 20 mm.

[0035] It proves advantageous to have an ultrasonic sensor device and to be set up in such a way as to generate haptic pressure elements in the positioning plane by superimposing ultrasonic waves.

[0036] It is advantageous to have a gas nozzle arrangement and to configure it in such a way as to create a gas flow curtain as a haptic pressure element in the positioning plane.

[0037] An audio system can be usefully configured to generate tones of increasing volume or frequency depending on the extent of the deviation of the fingers or palms from the positioning plane. An additional audio playback device can be set up to provide acoustic feedback by changing the frequency of the played tone as the hand approaches the positioning plane (optimal recording distance to the camera). For example, as the hand position approaches the positioning plane, a tone is played with a frequency that increases the more the hand deviates from the positioning plane.

[0038] Preferably, a contactless distance measuring unit is provided for determining the position of the presented hand or fingers, comprising at least one sensor from the group consisting of a light barrier, auxiliary camera, confocal chromatic distance sensor, PMD sensor (photonic mixing device), and ultrasonic distance sensor. This reduces the latency in controlling the user interface. Latency, in this context, refers to the signal propagation time within the device, which represents the delay in the position measurement data for subsequent interaction or display of the finger or hand positioning result.

[0039] It proves particularly advantageous if the projection display, via the virtual display in combination with the camera system, is configured as a modified distance measurement unit to project at least one selected measurement pattern onto the projection plane. This allows the camera system to selectively capture the measurement pattern spectrally and then, using the evaluation unit, to separate it from the image and calculate the distance between the fingertips or palms and the camera system. The selected measurement pattern can be, for example, a geometric figure such as an ellipse, circle, rectangle, etc., preferably in multiple arrangements, or a grid or line pattern. The measurement pattern should preferably be emitted in a different spectrally limited color spectrum than an illumination structure used to calculate the 3D fingerprint, so that each pattern associated with the illumination structure, e.g., the color of the fingerprint, is not detected.B. Line pattern, captured fingerprint image contains the same measurement pattern, and the measurement pattern can be separated or extracted from the fingerprint image via channel separation using a color sensor. The captured image with the illumination structure for calculating the 3D fingerprint can then be corrected in the evaluation unit depending on the resulting image of the measurement pattern. This makes it possible to correct errors caused by hand movement within the depth of the measurement volume (z-direction of the camera setup) for each image. Thus, within a sequence of images captured with differently scaled illumination structures due to unwanted hand movements, corrections can be made as if the finger or hand had always been at the same distance from the camera setup.

[0040] The present invention provides a new method for contactless user guidance in space for contactless optical fingerprint scanning, enabling users to better perceive the correct finger or hand position and allowing for improved, high-resolution contactless optical fingerprint scanning. Furthermore, the invention eliminates the need for additional distance or range measuring devices to determine the current finger or hand position during fingerprint scanning.

[0041] It is understood that the features mentioned above and those to be explained below can be used not only in the combinations given, but also in other combinations or on their own, without leaving the scope of the present invention. Brief description of the drawings

[0042] The invention is explained in more detail below with reference to exemplary embodiments and the accompanying drawings, which also disclose essential features of the invention. However, the description of an exemplary embodiment with a plurality of elements or components is not to be interpreted as meaning that all of these elements or components are necessary for implementation. Rather, other exemplary embodiments may also contain alternative elements and components, fewer elements or components, or additional elements or components. Elements or components from different exemplary embodiments may be combined with one another unless otherwise specified. Modifications and variations described for one of the exemplary embodiments may also be applicable to other exemplary embodiments.To avoid repetition, identical or corresponding elements in different figures are designated with the same reference symbols and are not explained multiple times. The drawings show:

[0043] Fig. 1 shows a schematic representation of the device according to the invention for contactless fingerprint recording in side view and top view,

[0044] Fig. 2 shows a first embodiment of the device according to the invention with lighting device, camera device and generator for generating a virtual display in a schematic side view and top view.

[0045] Fig. 3 shows a schematic side view of a second embodiment of the device according to the invention with haptically supported user guidance and structured lighting integrated into the generator system for the virtual display.

[0046] Fig. 4 shows a schematic side view of a third embodiment of the device according to the invention with a further haptically supported user guidance and a contactless distance measuring unit,

[0047] Fig. 5 shows a schematic representation of a possible design for user guidance via the virtual display for different fingerprint scans.

[0048] Fig. 6A shows a possible design of a display for user guidance via the virtual display for a four-fingerprint scan of the right hand,

[0049] Fig. 6B shows a possible design of the display for user guidance from Fig. 6A for the animation of a correction instruction for a four-fingerprint scan.

[0050] Fig. 6C shows a possible design of the display for user guidance according to Figs. 6A and 6B to indicate the successful completion of the four-fingerprint scan.

[0051] Fig. 7A shows a schematic external view of the device according to the invention with user guidance exclusively via the virtual display,

[0052] Fig. 7B shows a first possible flowchart of a user guidance process for fingerprint capture with the device according to Fig. 7A,

[0053] Fig. 8A shows a schematic external view of the device according to the invention with user guidance via the virtual display and an additional real display.

[0054] Fig. 8B shows a second possible flowchart for user guidance during fingerprint capture using the device according to Fig. 8A.

[0055] Fig. 9 shows a flowchart of a fingerprint scan using a device according to the invention for the production of high-resolution fingerprint images, e.g. for entry in or comparison with entries from a database,

[0056] Fig. 10 shows a representation of a preferred illumination structure projected onto a presented finger for 3D recording of fingerprint images, which can also be used as a spectrally distinguishable measurement pattern for distance measurement, for determining structure sizes in 2D images, or for correcting errors due to hand movements during the recording of image sequences of the fingerprint images.

[0057] Fig. 11 shows a representation of measurement patterns preferably projected onto presented fingers of a hand for distance determination, which can be used as a spectrally separable measurement pattern for determining structural sizes or for correcting errors due to hand movements when taking fingerprint images.

[0058] Fig. 12 shows a diagram with spectral curves of sensor pixel sensitivities and LED emission behavior, which can be used in combination to separate measurement patterns projected with a narrowband LED in an associated color channel of the camera device from the fingerprint images. Detailed description of the drawings

[0059] The basic procedure for contactless user guidance in the room for the contactless optical recording of fingerprint images will be explained in detail below using a suitable device according to Fig. 1.

[0060] The device for contactless user guidance in space for contactless optical recording of fingerprint images, which is shown schematically in its basic structure in Fig. 1 as a cutaway side view and as a top view, includes a generator system 1 for generating a virtual display 11 (in short: generator system 1 for the virtual display), a camera device 2 for recording the inside of the hand or fingers of a hand presented in space (not shown), which is guided into a recording area AB by means of the generated virtual display 11, i.e. positioned orTo be directed, a lighting device 3, which is directed into the recording area AB and around a defined positioning plane PE by the camera device 2 allows a finger or hand recording at least to determine the position of fingers or palms of the presented hand, as well as an evaluation unit 4 for evaluating recorded hand or finger recordings at least with regard to the presence and / or position of a finger or palm, wherein the evaluation unit 4 is provided for outputting feedback for user guidance in order to control at least the generator system 1 for the virtual display 11 for displaying user instructions or results.

[0061] The top view shown in Fig. 1 depicts the virtual display 11 projected above the housing 5. This display presents symbols and patterns to the user of the device, representing a visible plane in space. In other words, image elements are projected in such a way that the user perceives the virtual display 11 as a surface floating freely in space, onto which a hand or fingers could be placed. The virtual display 11 is spatially aligned above the housing 5 such that, when viewing it through the PEA 13 in the background, the user is looking towards a dark interior or a dark surface of the housing 5. This ensures that the projected information and patterns exhibit high contrast against the dark background. This configuration for generating the high-contrast virtual display 11 is shown schematically in Fig. 3.

[0062] The process includes the following steps: - Generating a position specification for freely floating fingers or an entire hand in a defined positioning plane PE, in which the finger or palm sides are to be presented stretched, wherein the positioning plane PE is visualized by means of a virtual display 11 as a position specification for fingerprints or handprints in order to enable an optically sharp image recording of fingers or a hand presented in a recording area AB, - Capturing images of fingers or a hand presented in the recording area AB, wherein at least parts of a finger or palm surface are captured in the defined positioning plane PE with a distance of at least 100 mm between the positioning plane PE and the camera device 2, in order to define a non-contact measurement volume 21 for fingerprints of the presented fingers or hand with a depth of at most 30 mm, - Capturing the presented fingers or hand in the recording area AB by means of an illumination that allows at least one finger or hand image around the defined positioning plane PE to determine the position of the finger or palm surfaces of the presented fingers or hand, and - Evaluation of currently recorded finger or hand images, at least with regard to the presence and position of a finger or palm side, which is used to output feedback on the finger or hand images in order to control the virtual display 11 for additional display of results or correction instructions for user guidance in finger or hand positioning.

[0063] The virtual display 11 is generated using the image of a (real) projection display 12 (see Fig. 2), which generates a stereoscopic image in a plane defined in space for the virtual display 11 by means of a plurality of planar prismatic projection elements (not shown) arranged in a projection element arrangement 13.

[0064] Such a spatial projection is based on a device developed for virtual controls, as disclosed in US 2023 / 0384615 A1. This device describes a projection film layer with a plurality of prismatic projection elements, wherein a reflective film is provided on one of the two side faces of each projection element pointing in a first direction and on one of the two side faces of each prismatic projection element pointing in a second direction. Each projection element is surrounded by such a reflective film in such a way that the majority of the incident light is reflected by the reflective surface at the section on the side of the incident light surface, thus suppressing the primarily reflected light emitted by the light-emitting surface.

[0065] In this way, in the generator system 1, an image emitted by a projection display 12 is reflected multiple times at each of the projection elements of a projection element arrangement 13 specified above and emitted to a side of the planar projection element arrangement 13 opposite the projection display 12, thereby projecting the virtual display 11 into a positioning plane PE (which represents a reference plane for a possible fingerprint scan) in the space above an opening of the housing 5.

[0066] To generate or display the virtual display 11 as a positioning plane PE, planar shapes are preferably projected, such as at least one rectangular frame, preferably filled with hatching or a colored area. Honeycomb or circular patterns, dot or line patterns (which do not necessarily have to be regularly distributed), or symbols that can represent instructions for the user can also be used to fill the rectangular frame of the virtual display 11. Symbols for user instructions, e.g., for presenting a hand or individual fingers or for correcting their position, preferably include triangles, arrows, vignettes, pictograms, silhouettes, outlines, or similar symbols.

[0067] The recording area AB of the camera device 2, intended for the finger or palm of the hand, is illuminated by the lighting device 3 and is defined around the positioning plane PE by the virtual display 11 as the reference plane for the finger-hand presentation.

[0068] For image capture of fingers or an entire hand presented in the recording area AB, the camera device 2 is directed at least towards parts of the finger or hand interior in the defined positioning plane PE and has a distance to the positioning plane PE of at least 100 mm in order to generate a guaranteed non-contact measurement volume 21 for fingerprints of the presented fingers or hand, wherein the measurement volume 21 should have a depth of no more than 30 mm.

[0069] The lighting device 3 is directed into the recording area AB in such a way that at least a partial finger or hand recording is possible in the vicinity of the defined positioning plane PE to determine a position of the finger or hand surfaces of the presented fingers or hand.

[0070] The evaluation unit 4 analyzes the currently recorded hand or finger images, at least with regard to the presence and position of a finger or palm. The evaluation unit 4 is configured to provide user feedback and control the generator system 1 for the virtual display 11, not only to visualize the defined positioning plane PE but also to display correction instructions or results. Such correction instructions for adjusting the position or posture of the hand or finger can be displayed on the virtual display 11 as triangles, arrows, vignettes, pictograms, silhouettes, outlines, or similar elements. Alternatively, these correction instructions can also be displayed on a separate physical display 8 (not shown in Fig. 1).Some typical examples of concrete representations of correction instructions to be displayed to the user are shown and explained below in Figures 5 and 6A to 6C. As a supplementary measure, the correction instruction for the height of the hand position can also be supported by haptic elements, such as an airflow or ultrasonic pressure points, or acoustic elements, such as tones with a controlled frequency or volume, as described below for Figures 3 and 4.

[0071] The end of the correction instructions is typically communicated to the user as a success message by a sound and / or by changing the color of the frame, area or embedded symbols of the virtual display 11, by switching from flashing frames, area or symbols of the virtual display 11 to a continuous light or by displaying end symbols, e.g. with checkmarks or OK.

[0072] Only after the fingers or hand to be presented have been correctly positioned is the end of the correction instructions concluded with the success message for correctly positioned fingers and can be used directly to trigger a desired image capture (or capture sequence) of high-resolution fingerprint images.

[0073] Figure 2 shows a first embodiment of the device according to the invention – as in Figure 1, also as a vertical sectional view and as a top view. The sectional drawing shows the generator system 1 for the virtual display 11 in more detail. It consists of a projection display 12 for generating the image information to be displayed in space and an arrangement of projection elements 13 inclined to it, which comprises a plurality of projection elements (e.g., prisms) deflect the image information from the projection display 12 into multiple reflections and create an aerial image in the room above an exit window (not labeled) of the housing 5.

[0074] In the top view of Fig. 2, the virtual display 11 projected above the housing 5 is shown with a hand image 111 (e.g., a pictogram) that encourages the user to present an outstretched hand (in this example, the right hand), i.e., to seemingly "place" the hand on the virtual display 11. The illusion of an existing positioning plane PE can be further enhanced by projected lines or frames (here drawn as a dash-dot line). The projected hand image 111 also indicates to the user the lateral position within the positioning plane PE defined by the virtual display 11. This position lies within a recording area AB that can be captured with a sharp optical image by the camera device 2 and, in this example, is also bounded by the dash-dot line. A measuring volume 21 (only shown in Fig. 2) is located within the recording area AB.3) is defined for an optically high-resolution image acquisition around the positioning plane PE, wherein the lateral extent of the measurement volume 21 is limited by the field of view (FOV) of the camera device 2 and the depth of the measurement volume 21 is limited by the depth of field of the lens of the camera device 2.

[0075] For the recording of high-resolution fingerprint images, the presented finger or palm surfaces must be treated as 3D objects due to their curved surface, which can be captured in high resolution in different variations using the camera device 2: a) 3D image recordings using an illumination structure 31, which is characterized by precisely known, regular patterns, preferably line patterns, and is projected by the illumination device 3 into the recording area AB, b) 2D image recordings using distance information to the fingers or palms, which is obtained by distance sensors of an additional distance measuring unit 9 or by projection of a spectrally separable measurement pattern 91 (described only for Fig. 3), which, by means of spectrally limited emission of the measurement pattern 91 and spectral selective detection in the camera device 2, allows a distance determination to be separated for each image recording. c) 2D image acquisitions using 3D information obtained from 3D structures generated in the recording area AB, which may also be generated by the virtual display 11.

[0076] The lighting structures 31 used for the 3D scans preferably include periodic (regular) stripe or line patterns, grid structures or honeycomb patterns (formed from 3-, 4-, 5-, 6-, 8- or further n-gons), checkerboard patterns, touching or intersecting circular structures, or concentric circles, each as light-dark edge transitions or light-dark gradients (e.g., sinusoidal). In each case, the working method of known fringe projection techniques is applied [see, e.g., BG Frankowski et al. “Real-time 3D Shape Measurement with Digital Stripe Projection”, in: Proc, of SPIE, Vol. 3958 (2000), pp. 90-106; T. Peng et al. Algorithms for Constructing 3-D Point Clouds Using Multiple Digital Fringe Projection Patterns, CAD Conf., Bangkok, Thailand, June 2005].

[0077] The illumination structures 31 should be generated spectrally such that they can be detected in only one spectral channel of the camera device 2. Preferably, the green or the blue spectral channel can be used.

[0078] A particularly preferred illumination structure 31, which is projected by the illumination device 3 into the recording area AB, is shown in Fig. 10 and shows how the illumination structure 31, which is generated as a sine wave in phase shift on a presented finger, looks in the recorded fingerprint image.

[0079] The following pattern sequences can be used: - G8S1616 2-phase shift with Gray code - G8S4 4-phase shift with Gray code - S16 16 sine pattern (22.5° phase shift) - S4 4 sine pattern (90° phase shift) - FM 1 sine pattern - PFM 2 sine pattern (180° phase shift) - MFM 1 sine pattern + 1 bright image - NFM 1 sine pattern + 1 bright image - AM 2 sine pattern (90° phase shift) - APSn n Aperiodic sine patterns.

[0080] Furthermore, aperiodic patterns can also be used as illumination structures 31, which may be formed by non-regular spiderweb, spiral, or similar patterns. In any case, the exact geometry of the illumination structures 31 used must be known, and the structure sizes must be adapted and calibrated to a predetermined resolution of the fingerprint images.

[0081] The lighting device 3 preferably emits the lighting structure 31 in pulsed mode, in order in particular to project sequences of lighting structures 31 of different sizes of the same structure for varying the resolution of the image recordings.

[0082] Fig. 3 shows a second embodiment of the device according to the invention, in which, unlike the embodiment of Fig. 2, the lighting device 3 is integrated into the generator system 1 for the virtual display 11. However, the arrangement of a separate projector as a lighting device 3 for structured illumination of the palms or fingertips is merely symbolic, because the projection display 12 for generating the image information for the virtual display 11 can preferably also be used for the fingerprint scans to provide lighting structures 31, provided that the hand or fingers are correctly positioned within the measurement volume 21 of the recording area AB.

[0083] The illustration in Fig. 3 further clarifies that – as can be seen from the stylized line of sight of a user – certain requirements must be set for the dimensions of the virtual display 11, projection display 12, and PEA 13 in the generator system 1 so that the user, when viewing the virtual display 11, looks at a dark background inside the housing 5, ensuring that the projected information and patterns have a high contrast against the background. Alternatively, the virtual display 11 could also be projected outside the housing 5 if a homogeneous, dark floor is consistently available.

[0084] The illumination structure 31, as shown as a preferred example in Fig. 10, can also be used in the same form as a measurement pattern 91 for distance measurement of the 2D or 3D image recordings, wherein for this application the measurement pattern 91 can be projected from the illumination device 3 or from the generator system 1 directly from the projection display 12 onto the presented fingers or palms and can be separated in a defined spectral color (e.g. red, green or blue) which can be detected by a color sensor used (e.g. RGB sensor) in a separate color channel of the camera device 2, different from that of the illumination structure 31.For this purpose, the generator system 1 for the virtual display 11 can also include a control system for generating the measurement pattern 91. By means of the spectrally limited emission of the measurement pattern 91 and spectrally selective detection in the camera device 2, a distance determination between the fingertips or palms and the camera device 2 can be separated for each image capture if the measurement pattern 91 is emitted in pulsed mode. Further usable measurement patterns 91, which differ in the shape of the illumination structure 31 and can be expediently adapted to the shape of the fingerprints to be captured, are shown in Fig. 11, for example, for capturing a left hand. However, the measurement patterns 91 are not limited to the use of concentric patterns.

[0085] For the spectral separation of the measurement patterns 91 from the illumination structures 31 in the recorded fingerprint images, a partial match between the spectral sensitivity of spectral channels of the camera device 2 and the emission characteristics of LEDs used in the illumination device 3 or in the projection display is required. A diagram with LED spectral curves that can be used for the measurement patterns 91 and illumination structures 31 is shown in Fig. 12 in conjunction with the spectral channels of commonly used RGB color sensors of the camera device 2, from which suitable combinations of the spectral colors of the LED and sensor color channels can be selected to separate the illumination structures 31 and measurement patterns 91 from the fingerprint image recordings. The same applies to color selection for all other color systems. Preferably, according to the diagram in Fig.The spectral behavior of the LED and color sensor pixels in camera device 2, as shown in Figure 12, is that the illumination structures 31 in the blue and the measurement patterns 91 in the green spectral range are emitted and received, or vice versa. In both cases, a clear separation from the fingerprint images would be possible.

[0086] By emitting a selected known measurement pattern 91, distance information can be simultaneously acquired during a sequence of image acquisitions, which is available separately and in addition to each of the fingerprint images. A distance measuring unit 9, as shown as a separate element in Fig. 4, for capturing the position of the finger or palm in the depth of the measurement volume 21, is therefore not strictly necessary.

[0087] Fig. 3 additionally shows an ultrasonic transmitter 6 for haptic support of hand or finger positioning, in order to generate haptic pressure elements in the positioning plane PE by superimposing ultrasonic waves, which give the user the impression of a perceptible support surface for the fingers or hand. This is particularly advantageous for the presentation of individual fingers or both thumbs.

[0088] A third embodiment of the invention is shown in Fig. 4. In this embodiment, the lighting device 3 for generating the lighting structures 31 of the fingers / hand is again designed separately, although functional integration into the generator system 1 is also advantageously applicable here. The projection of the virtual display 11 is carried out in the same manner as already described for the previous figures. Likewise, the lighting structure 31 projected by the lighting device 3 is designed in accordance with the principles explained for Fig. 1.

[0089] In this embodiment of the device according to the invention, further advantageous components are provided to simplify or accelerate user guidance.

[0090] This concerns, firstly, the haptic support for positioning the palms of the hands or fingers in the positioning plane PE. In this example, a gas nozzle arrangement 7 is provided which generates a gas flow curtain 71 along the positioning plane PE and thus forms a haptic pressure element as a "contact surface" for the palms of the hands or fingers to be presented.

[0091] As an alternative or supplement to the distance measurement described in Fig. 3, the design shown in Fig. 4 achieves faster positioning of the hand "in the depth" of the measuring volume 21 by replacing the complex image evaluation of the high-resolution camera device 2 with a contactless distance measuring unit 9 for determining the position of the presented hand or fingers. This unit comprises at least one sensor from the group consisting of a light barrier, auxiliary camera, confocal chromatic distance sensor, PMD sensor (photonic mixing device), and ultrasonic distance sensor. Furthermore, the positioning of the fingers / hand can be acoustically supported by a loudspeaker using tones that are controlled, for example, in volume or frequency.

[0092] When the distance measuring unit 9 is implemented as an auxiliary camera, a measurement pattern 91, projected from the illumination device 3 or from the generator system 1 from the projection display 12 onto the presented fingers or palms, can again be used for measuring the distance of the 2D or 3D image recordings. Measurement patterns 91 as shown in Fig. 11 can be advantageously used, wherein the measurement pattern 91 is emitted in a defined spectral color (e.g., red, green, or blue) that can be detected by the auxiliary camera of the distance measuring unit 9 and is different from the spectral color of the illumination structure 31.

[0093] Furthermore, the measurement pattern 91 for acquisition by the auxiliary camera of the distance measuring unit 9 can also be used as a measurement pattern 91 that matches the illumination structure 31 in shape but differs spectrally. The generator system 1 for the virtual display 11 can additionally include a control for generating the measurement pattern 91. Due to its differing spectral emission compared to the illumination structure 31, the measurement pattern 91 is not acquired by the camera device 2, and the distance measuring unit 9 allows for direct distance determination between the fingers or palms and the camera device 2. This provides distance information that, separately from the high-resolution fingerprint images captured by the camera device 2 (preferably with the lower resolution of the auxiliary camera), is available more quickly for correcting the positioning of the user's fingers / hand.

[0094] Figure 5 shows several examples of how patterns can be displayed on the virtual display 11 or an additional physical display 8 (shown only in Figure 8A) to guide the user. These patterns can be displayed as animations or as corrective instructions. Figure 5 shows different finger or hand positions to be presented in five columns. In this example, the corrective instructions for the position are displayed by alternately switching between the respective image representation in line 1) and line 2), where line 1) is a symbolic representation of the currently detected error situation and line 2) represents the correct end position of the hand or finger. The alternating display of the actual and target state of the finger positioning continues until an image of the correctly positioned fingers has been captured.The display then changes to the success message or "Done" message shown in line 3).

[0095] Another embodiment for displaying the correction instructions is shown in Figures 6A to 6C. Here, the positioning of the hand to be presented for a four-finger photograph is preferably shown in three steps on the virtual display 11 (or optionally on a physical display 8, shown only in Figure 8A). In the upper part of Figure 6A of the virtual display 11 or physical display 8, a pictogram 111 of the hand to be presented (right hand) is displayed (projected). In the lower part of Figure 6A, the desired "placement position" of the so-called four-finger (as a finger silhouette 112) is shown in a background area, which is represented by a first color 115—or, as shown, as a free white or dark area—and is projected on the virtual display 11 or displayed on the physical display 8.

[0096] In Fig. 6B, the lateral misplacement of the hand is indicated by a directional arrow 113 pointing to a target field 114. In the lower part of the virtual display 11 or real display 8 used for the correction instruction, the finger placement position is left blank as a finger silhouette 112, while the background area is changed to a second color 116, preferably orange or red (shown as a hatched area), or to a hatched area to indicate the positioning as "incorrect".

[0097] Figure 6C shows the pictogram 111 of the correctly positioned hand with an additional checkmark in the upper part of the virtual display 11 or the real display 8, and in the lower part the background area is changed to a third color 117 (preferably green, shown in Figure 6C as a more widely spaced hatching) or to any other, preferably more widely spaced, dashed, or dotted hatching, which preferably also signals the simultaneous image capture of the correctly positioned fingers. This description is merely one embodiment of the correction instruction for a specific mispositioning of the fingers or hand. Further corrections are conceivable, e.g., regarding the distance to the camera device 2 = displacement of the presented fingers / hand in the depth of the measurement volume 21 (usually upwards or downwards), lateral displacement in the measurement volume 21 (e.g.,forwards and backwards), rotation within the measuring volume 21), since the hand can be moved freely in 3D space.

[0098] Figures 7A and 8A show the device described in Figures 1 to 4 in two embodiments, which differ in the way the user is guided. In Figure 7A, the user can rely solely on the animations projected by the virtual display 11 for presenting the hand or fingers, along with the correction instructions for their position or posture shown therein, when positioning their fingers or hand to be presented. In the embodiment according to Figure 8A, the animations for presenting the hand or fingers are essentially assigned to the virtual display 11, and the correction instructions and the results are predominantly reserved for a real display 8.

[0099] The design according to Fig. 7A has the advantage that the user can focus their undivided attention on the virtual display 11 in order to correctly position their fingers or hand in free space, i.e., without contact with a surface, as required for capturing high-resolution fingerprint images. The user guidance process, as shown in the flowchart in Fig. 7B, proceeds in the following steps.

[0100] The virtual display 11 prompts the user to present, i.e., to position an outstretched hand or individual fingers. Thumb in the positioning plane PE, indicated, with this prompt integrated into the representation of a support surface projected by the virtual display 11. This prompt can be represented, for example, by a pulsating image of the desired hand or fingers. However, a representation as shown above in Fig. 6A can also be chosen.

[0101] The camera unit 2, with the support of the lighting unit 3, continuously takes pictures, which are analyzed in the evaluation unit 4 to determine the presence and position of the hand or fingers. If the desired fingers are not present, the virtual display 11 repeatedly prompts the user to present the right or left hand, or specific fingers or thumbs, until the evaluation unit 4 detects the desired fingers in the images from the camera unit 2 and stops the pulsing of the hand or finger display.

[0102] The evaluation unit 4 then compares the position of the detected fingers or hand with a target position, determines the deviation, and, if a deviation is found, controls the virtual display 11 to show correction instructions and prompt the user to correct the position. This can be done as shown above in Fig. 6B and is repeated (adjusted) until the pictogram 111 of the hand is in the target field 114 and no longer shows a directional arrow 113, thus indicating that the target position has been reached. If further deviations are present (e.g., fingers spread too far apart), further correction instructions are displayed until no deviations remain. The virtual display 11 then switches to showing a success message, which can be displayed, for example, as shown in Fig. 6C.Alternatively, before the success message, while the high-resolution image capture is still in progress, the movement of a strip-shaped scan bar can be displayed over the hand silhouette 112 or over the hand pictogram 111 if the user is to be informed about a fingerprint capture currently being carried out by the camera device 2.

[0103] In the embodiment of the device according to Fig. 8A, in which a real display 8 in the form of a nearly vertical monitor is attached to the upper part of the housing 5, the process sequence is carried out according to the illustration of Fig. 8B.

[0104] The real display 8 shows a prompt to present or position an outstretched hand or individual fingers or thumbs in the positioning plane PE, which is generated by the virtual display 11 in the recording area AB. This prompt can be displayed, for example, by a pulsating image of the desired hand or fingers. Alternatively, a hand silhouette 112, as shown in Fig. 6A, can also be displayed by the virtual display 11 in the recording area AB.

[0105] The camera unit 2, with the support of the lighting unit 3, continuously takes pictures, which are analyzed in the evaluation unit 4 to determine the presence and position of the hand or fingers. If the desired fingers are not present, the request to present the right or left hand, or specific fingers or thumbs, is repeated on the real display 8, possibly supported by the virtual display 11, until the evaluation unit 4 recognizes the desired fingers in the images from the camera unit 2 and the pulsing of the hand or finger display on the real display 8 and / or the virtual display 11 stops.

[0106] The evaluation unit 4 then compares the position of the detected finger or hand with a target position, determines the deviation, and uses this information to control the physical display 8 and, if applicable, the virtual display 11 to display correction instructions and prompt the user to correct the position. This can be done as shown above in Fig. 6B and displayed in the same or a different way on the virtual display 11. The prompt is repeated, adapted, until the pictogram 111 of the hand pointing towards the target field 114, used in this example, no longer shows a directional arrow 113, and thus the target position has been reached. Then the physical display 8 and, if applicable, the virtual display 11 show a success message, which can be shown, for example, as in Fig. 6C, or a strip-shaped scan bar is moved across the hand silhouette 112 if information about the fingerprint scan subsequently performed by the camera device 2 is to be displayed.

[0107] Figure 9 schematically illustrates the process flow for one of the devices according to the invention described above, with subsequent high-resolution 3D fingerprint scanning. A prompt to present or position fingers or hand, for example as a pulsating fingerprint image or finger silhouette 112, is displayed via the virtual display 11 and any available real display 8. The virtual display 11 provides at least the projected representation of a contact surface as the recording area AB, or additionally a finger silhouette 112 as shown in Figure 6A.

[0108] The camera unit 2, with the support of the lighting unit 3 or the projection display 12, continuously generates images that are analyzed in the evaluation unit 4 for the presence and position of a hand or fingers. If the desired fingers are not present, the request to present the right or left hand, or specific fingers or thumbs, is repeated on the virtual display 11, or, if supported by the physical display 8, until the evaluation unit 4 recognizes the desired fingers in the images from the camera unit 2 and, for example, stops the pulsing of the hand or finger display on the virtual display 11 and / or physical display 8.

[0109] The evaluation unit 4 compares the position of the detected fingers or hand with a target position, determines the deviation, and uses this information to control the virtual display 11 to display correction instructions, prompting for position or alignment correction (e.g., for fingers that are too curved or protrude from the detection area AB). This can be done as shown above in Fig. 5 or according to Fig. 6B and / or additionally displayed on the physical display 8. The prompt is repeated, adjusted as needed, until switching between the actual and target positions according to Fig. 5 no longer reveals a difference, or until a pictogram 111 of the hand pointing towards the target field 114, as used in Fig. 6B, no longer displays a direction arrow 113, thus indicating that the target position has been reached. Then, the virtual display 11 and, if applicable, the physical display 8 show a success message, which may be a representation according to line 3 of Fig. 5 or the representation according to Fig. 6B.6C. Alternatively or immediately afterwards, a strip-shaped scan bar moving over a hand silhouette 112 can be displayed, which provides information about the progress of the fingerprint capture carried out by the camera device 2.

[0110] In a first embodiment of the invention, the camera device 2 switches to a recording mode during the fingerprint scan that allows for high-resolution 3D fingerprint imaging, in which the presented palms or fingertips are treated as 3D objects due to their curved surfaces. For this purpose, structured illumination, characterized by a precisely known illumination pattern 31, is directed by the illumination device 3 into the recording area AB. Periodic (regular) illumination structures can be used, such as stripe or line patterns, grid or honeycomb structures, checkerboard patterns, touching or overlapping circular structures, or concentric circles, etc., each as light-dark edge transitions or as light-dark gradients (e.g., sinusoidal).However, aperiodic lighting structures can also be used, which can be formed by non-regular spiderweb, spiral, or similar patterns. Orthogonal grids with sharp edge transitions can be used as a suitable example. Periodic stripe patterns as shown in Fig. 10, generated as sinusoidal patterns using a phase-shift technique, are particularly preferred.

[0111] In a preferred embodiment shown in Fig. 10, parallel line patterns with a sinusoidal light-dark gradient are projected onto the fingers or palms in the positioning plane PE as the illumination structure 31. This allows image processing software in the evaluation unit 4 to calculate a high-resolution 3D image of the captured fingerprints using the known periodic gradient of the line patterns projected as the illumination structure 31.

[0112] The successful completion of the fingerprint scan, usually a sequence of scans per presented finger or hand, is signaled by the virtual display 11, possibly additionally by an existing real display 8 and / or acoustically.

[0113] In a further embodiment of the invention, which can be implemented with a device according to Fig. 3, the generator system 1 can, by means of the projection display 12 via the virtual display 11 in combination with the camera device 2, also assume the function of a distance measuring unit 9 (shown only in Fig. 4) by projecting at least one selected measurement pattern 91 (e.g. a geometric figure, such as ellipse, circle, rectangle, etc., preferably in multiple arrangements according to the examples of Fig. 11 or also as a line or stripe pattern, as described for the lighting structure 31 according to Fig. 10).The measurement pattern 91 is emitted in a different color spectrum than an illumination structure 31 used for 3D image acquisition, which is emitted to calculate the 3D fingerprint. Thus, each 3D image acquired with the illumination structure 31 simultaneously contains a measurement pattern 91, which can be extracted as a separate image of the measurement pattern 91 via different color sensor channels of the camera device 2 by means of channel separation of the image acquisition. The captured image, provided with the illumination structure 31, for determining a 3D fingerprint image, which is separated (i.e., freed) from a measurement pattern image, can then be corrected in the evaluation unit 4 depending on the separated result image of the measurement pattern 91.

[0114] The measurement pattern 91 can be easily captured together with the fingerprint image via the camera device 2, and the distance to the presented fingers or hand can be calculated using the evaluation unit 4. A sufficient number of sub-patterns are displayed in the virtual display 11 to allow the dimensions (lateral extent, line thickness, etc.) of the measurement pattern 91 (and their sub-sizes) to be recorded. In addition to the line patterns already mentioned above, patterns from the schematically depicted shapes according to Fig. 11 can also be selected as the measurement pattern 91. The application of the measurement pattern 91 is as follows.

[0115] Since the lighting device 3 or the generator system 1 for the virtual display 11 does not use telecentric projection optics, but rather entocentric optics, and thus the projection beam path can be mathematically described using the intercept theorem, in which the recording plane of the camera device 2 is variable due to the variable position of the presented fingers / hand, the dimensions of the optically detected measurement pattern 91 deviate depending on the distance to the origin of the light source. The dimensions of the measurement pattern 91 become larger when the fingers or palms are farther away and smaller when they are closer. If the overall optical system (not specified) in the device according to the invention is calibrated to 500 ppi, the distance between the fingers or palms and the camera device 2 can be directly determined and corrected to a calibration value.Alternatively, the deviations within a sequence of images can be determined and corrected to a normal (e.g., median value of all determined distance values ​​of the sequence).

[0116] Determining the distance in a 2D scan results in a correction (scaling) of the lateral dimensions (usually xy extent) of the fingerprint image so that the 2D scan is correctly positioned within the calibrated 500 ppi space. For a 3D image captured with a projected illumination structure 31 (e.g., a line pattern for fringe projection), each scan must be individually corrected (scaled) to a normal before the calculation of the 3D fingerprint image (3D reproduction) begins. This allows for the correction of any mispositioning of the fingers / hand within the depth of the measurement volume 21 (usually referred to as z-error) for each image scan.

[0117] If illumination structure 31 and measurement pattern 91 are to be used together for capturing fingerprint images, a color sensor must be used in the camera device 2, and the illumination structure 31 must be projected at a different wavelength than the measurement pattern 91 in order to separate an image of the measurement pattern 91 for each capture. This makes it possible to detect and even correct errors caused by movement of the presented fingers / hand within the depth of the measurement volume 21 (z-direction of the camera device 2) for each image capture. Thus, within a sequence of fingerprint images captured with differently scaled illumination structure 31 due to unwanted hand movements, images can be corrected as if the fingers / hand had always been at the same distance from the camera device 2.

[0118] In an embodiment of the invention with simple 2D recordings of fingerprint images (i.e., image acquisitions without a projected illumination structure 31), the same principle of spectrally narrowband illumination with a measurement pattern 91 can be applied in the same way. This leads, with the same procedure of spectrally narrowband emission of the measurement pattern 91 by the illumination unit 3 or the projection display 12 and projection onto the positioning plane PE, as well as image acquisition with spectral separation of the measurement pattern image in the camera device 2, to the same possibility of correcting positional deviations of the fingers / hand in the depth of the measurement volume 21.

[0119] The above descriptions merely represent some examples of the inventive method for contactless user guidance in space for the contactless optical capture of fingerprint images using selected advantageously equipped devices. These do not limit the scope of the invention to further steps and elements aimed at improving the process control for user guidance and the acquisition of high-resolution fingerprint images. Reference list: 1 Generator system (for the virtual display) 11 Virtual display 111 Hand illustration / pictogram 112 Finger Silhouette 113 Directional arrow 114 Target area 115 first color display 116 second color display 117 third color display 12 Projection Display 13 Projection Element Arrangement (PEA) 14 User's Viewpoint 2 camera setup 21 measuring volumes 3 Lighting equipment 31 Lighting structure (line pattern) 4 evaluation units 5 cases 6 Ultrasonic transducer device 7 Gas nozzle arrangement 71 Gas flow curtain 8 real display 9 Distance measuring unit 91 measurement patterns (spectral narrowband) AB recording range PE Positioning plane (reference plane)

Claims

-33- Patent claims 1. Method for contactless user guidance in the room for contactless optical recording of fingerprint images, comprising the following steps: - Generating a position specification for freely floating fingers or an entire hand in a defined positioning plane (PE), in which the finger or palm sides are to be presented stretched, wherein the positioning plane (PE) is visualized by means of a virtual display (11) as a position specification for the finger or hand to be presented, in order to enable an optically sharp image recording of the finger or palm sides of fingers or a hand presented in a recording area (AB), - Taking pictures of fingers or a hand presented in the recording area (AB), wherein at least parts of a finger or hand palm are recorded in the defined positioning plane (PE) by means of a camera device (2) with a distance between camera device (2) and positioning plane (PE) of at least 100 mm in order to define a non-contact measurement volume (21) for fingerprints of the presented fingers or hand with a depth of at most 30 mm, - Capturing the presented finger or hand in the recording area (AB) by means of lighting that allows at least one finger or hand recording around the defined positioning plane (PE) to determine the position of the fingertips or palms of the presented finger or hand, and - evaluating currently recorded finger or hand recordings at least with regard to the presence and position of a fingertip or palm, which leads to the output of feedback on the finger or hand recordings in order to control the virtual display (11) for additional display of results or correction instructions for user guidance during finger or hand positioning.- 34 - 2. Method according to claim 1, wherein, after evaluating the currently recorded finger or hand images, the virtual display (11) is controlled with correction instructions and the illumination of the finger or hand surface is repeated until the finger or hand surface in the defined positioning plane (PE) enables a sharp image recording of the finger or hand surface.

3. Method according to claim 1 or 2, wherein correction instructions for user guidance in case of deviation from the positioning plane (PE) are visualized by the virtual display (11) or a real display (8) as representations in the form of pulsating or flashing finger structures, outlines or surfaces and, upon reaching the positioning plane (PE), the representations are switched to a continuous light.

4. Method according to claim 3, wherein correction instructions for user guidance in case of deviations from the positioning plane (PE) are displayed by the virtual display (11) or a real display (8) as a representation in the form of finger structures, outlines or surfaces in a first color, when approaching the positioning plane (PE) in a second color and when reaching the positioning plane (PE) in a third color.

5. Method according to any one of claims 1 to 4, wherein haptic pressure elements are generated in the positioning plane (PE) defined by the virtual display (11) by at least one measure consisting of superimposing ultrasonic waves or generating a gas flow curtain (71).

6. Method according to any one of claims 1 to 5, wherein the illumination is carried out by means of structured illumination by projecting an irregular or regular known pattern into the defined positioning plane (PE).

7. The method of claim 6, wherein the acquisition of the fingerprint or handprint images using the known pattern projected into the defined positioning plane (PE) is used to generate an illumination structure (31) for obtaining high-resolution images for 3D fingerprint images.

8. The method of claim 6, wherein the acquisition of the fingerprint or handprint images using the known pattern projected into the defined positioning plane (PE) is used to generate a measurement pattern (91) that is used to determine the distance between the finger or palm surfaces and the camera device (2), wherein, by means of spectrally selective emission of the measurement pattern (91) and spectral separation in the camera device (2), a distance determination between the finger or palm surfaces and the camera device (2) can be detected for each image acquisition.

9. Method according to any one of claims 6 to 8, wherein finger or hand images for determining the presence and position of a finger or palm surface for user guidance are taken with a lower optical resolution than for generating high-resolution fingerprint images.

10. Method according to one of claims 7 to 9, wherein in the lighting device (3) the structured lighting with a lighting structure (31) for high-resolution fingerprint images is only activated when a positioning of the finger or palm sides in the defined positioning plane (PE) has been at least transiently detected by a contactless distance measuring unit (9; 91).

11. Method according to one of claims 6 to 10, wherein, for the production of high-resolution fingerprint images, several images or an image sequence are recorded within a time period of < 1s by means of the camera device (2), checked for a predetermined image quality by means of an evaluation unit (4) and a grayscale image is produced when the predetermined image quality is achieved.

12. Method according to one of claims 1 to 11, wherein in the generator system (1) for the virtual display (11) a structured illumination is additionally generated for recording high-resolution fingerprint images.

13. Device for contactless user guidance in space for contactless optical recording of fingerprint images, comprising a generator system (1) for generating a position specification for freely suspended fingers or an entire hand, wherein the generator system (1) generates a defined positioning plane (PE), in which the fingertips or palms of the fingers or hand to be presented are to be positioned in an extended position, by means of a virtual display (11) as a visualization of the position specification, in order to enable optically sharp image recordings of fingers or a presented hand presented in a recording area (AB), - a camera device (2) for capturing images of fingers or a hand presented in the recording area (AB), wherein the camera device (2) is directed at at least parts of the inside of the fingers or hand in the defined positioning plane (PE) and has a distance to the positioning plane (PE) of at least 100 mm in order to generate a non-contact measuring volume (21) for fingerprints of the presented fingers or hand with a depth of at most 30 mm, - a lighting device (3) directed into the recording area (AB) and allowing at least one finger or hand recording around the defined positioning plane (PE) for determining the position of the finger or palm surfaces of the presented fingers or hand, and - an evaluation unit (4) for evaluating currently recorded hand or finger images at least with regard to the presence and position of a finger or palm, wherein the evaluation unit (4) is set up to provide feedback for user guidance and to control at least the generator system (1) for the virtual display (11) in addition to displaying correction instructions or results.

14. Device according to claim 13, wherein the generator system (1) for the virtual display (11) comprises a projection display (12) and a projection element arrangement (13) with which the virtual display (11) can be generated from image information and patterns of the projection display (12) as a positioning plane (PE) floating in space for hand or finger positioning.

15. Device according to claim 14, wherein the projection display (12) of the generator system (1) is a conventional self-illuminating display from the group consisting of LCD, LED, OLED, or CRT monitor to display the image information and patterns with through- 37 - to project the projection element arrangement (13) at design-specific distances and angles into the recording area (AB) on a side of the projection element arrangement (13) facing away from the projection display (12).

16. Device according to claim 14 or 15, wherein the projection display (12) of the generator system (1) has, in addition to the floating positioning plane (PE) generated by image information and patterns, further image information with correction instructions from the group consisting of symbols, pictograms, directional arrows, moving, pulsating or flashing finger structures, outlines or surfaces, as user guidance for correcting the position of the hand or fingers.

17. Device according to any one of the preceding claims 13 to 16, wherein the projection display (12) of the generator system (1) is additionally designed with at least one illumination structure (31) or a measurement pattern (91) for receiving high-resolution fingerprint images in addition to emitting structured illumination of the finger or palm surfaces.

18. Device according to any one of the preceding claims 13 to 17 wherein the generator system (1) for the virtual display (11) is equipped to control a real display (8) for supplementary user guidance in addition to the virtual display (11).

19. Device according to any one of the preceding claims 13 to 18, wherein the measuring volume (21) has a lateral dimension of between 80 x 75 mm 2 and 200 x 150 mm 2 and has a depth of 15 to 30 mm.

20. Device according to any one of the preceding claims 13 to 18, wherein the measuring volume (21) has a lateral dimension of between 100 x 80 mm2 and 170 x 120 mm 2 and has a depth of 18 mm to 25 mm.

21. Device according to any one of the preceding claims 13 to 18, wherein the measuring volume (21) has a lateral dimension of 150 x 100 mm 2 and has a depth of 20 mm.-38- 22. Device according to any one of the preceding claims 13 to 21, wherein an ultrasonic transmitter device (6) is configured to generate haptic pressure elements in the positioning plane (PE) by superimposing ultrasonic waves.

23. Device according to any one of the preceding claims 13 to 22, wherein a gas nozzle arrangement (7) is configured to generate a gas flow curtain (71) as a haptic pressure element in the positioning plane (PE).

24. Device according to any one of the preceding claims 13 to 23, wherein a non-contact distance measuring unit (9) is provided for determining the position of the presented hand or fingers, comprising at least one sensor from the group consisting of light barrier, auxiliary camera, confocal chromatic distance sensor, PMD sensor (photonic mixing device) and ultrasonic distance sensor.

25. Device according to one of the preceding claims 13 to 23, wherein the projection display (12) is configured via the virtual display (11) in combination with the camera device (2) as a modified distance measuring unit (9) to project at least one selected measurement pattern (91) into the projection plane (PE) and to record the measurement pattern (91) spectrally selectively by means of the camera device (2) and to calculate, by means of the evaluation unit (4), a distance between finger or palm surfaces and camera device (2).

Images