Apparatus and method for producing a lens-free or glasses-free facial image of a person wearing glasses

Description

[0001] The invention relates to a device for generating a spectacle-free, preferably biometric, facial image of a bespectacled person. Furthermore, the invention relates to a method for generating a spectacle-free facial image of the bespectacled person.

[0002] Images, especially passport photos, should enable the person depicted to be identified as clearly as possible. Wearing glasses can make such identification more difficult, especially if it is automated. Glasses, especially glasses with distinctive frames, can alter the impression of a face, and biometric characteristics, such as the distance between the eyes, can also appear distorted or altered due to the optical effects of the lenses. Thus, glasses impair the performance of biometric facial recognition systems. One way to improve identification is to ask the user to remove their glasses while the facial image or passport photo is being taken. However, this is not only unpleasant, especially for people with severe visual impairment, but can lead to them squinting tightly in order to read instructions on a terminal or control panel.This counteracts the desired neutral look of the person during the identification process or the image creation process.

[0003] Maninchedda et al., Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4608 - 4617, July 21-26, 2017, Honolulu, Hawaii, USA, disclose a method that can be carried out using a mobile phone for the 3D facial reconstruction of people wearing glasses. A height profile is first created from a series of images taken using the mobile phone, and the presence of glasses is detected. If the sum of all gradients of the height profile is greater than a predetermined threshold, this indicates the presence of glasses. In the next step, the glasses are segmented from the rest of the image, assuming that the glasses differ from the rest of the face due to color or height profile, that the glasses are symmetrical, that the eye is covered by the glasses, and that the glasses form a uniform surface.In the next step, the 3D face is reconstructed by optimizing the alignment, model fitting and by optimizing the height profile by adjusting height differences.

[0004] Proesmans et al., Proceedings of the International Conference on Image Processing, pages 647-650, September 16-19, 1996, Lausanne, Switzerland, demonstrates a device consisting of a camera, a projection device, and a detector that enables a 3D reconstruction of a face using structured illumination. A 3D model is thus generated using structured illumination.

[0005] US Pat. No. 9,202,105 A discloses a method for facial recognition and presentation attack detection, for example, based on predefined landmarks in an image. The device is configured as a computer / mobile phone / tablet with one or more cameras.

[0006] US 2002 / 0136435 A1 discloses a system with a VIS and an IR camera for face recognition, wherein the image information of the VIS image and the IR image is used for face recognition in order to improve the quality of face recognition.

[0007] Wong and Zhao, Information Fusion 14, pages 163 - 176, 2013, shows an apparatus and a method for removing spectacle lenses by combining the image information from a VIS image with the image information of a thermal IR image.

[0008] JP 2008 232 776 A discloses a device and method for measuring a three-dimensional image of an object using two cameras and a projection device that generates a point cloud. The evaluation unit is configured to compensate for the optical effects caused by spectacle lenses based on the face illuminated by the projection device and generates a distance image of the object.

[0009] WO 2016 / 111327 discloses a facial recognition apparatus and method for a car. The device comprises an infrared camera and two projection means that emit light in the near-infrared range and project a striped pattern onto a person's face, comprising areas of higher intensity and lower intensity.

[0010] CN 10 225 359 0B discloses a contour detection element for improving the image quality of cameras with a first infrared camera and a second camera detecting in the visible range. Furthermore, two projection means are provided that illuminate a face or an object with a vertical or horizontal line pattern to remove optical effects, particularly at the contours of the image.

[0011] The invention is therefore based on the object of providing a device and a method for generating a facial image of a person wearing glasses, by means of which a glasses-free, biometric facial image can be created.

[0012] The problem concerning the device is solved by a device having the features of claim 5.

[0013] This device for generating a spectacle-free, biometric facial image of a bespectacled person comprises a camera for capturing the person's face and an evaluation unit connected to the camera. Furthermore, a projection means for illuminating the person's face, in particular for projecting a pattern onto the face, is provided. The evaluation unit is designed to compensate for optical effects caused by spectacle lenses using a method for capturing three-dimensional objects and to remove the spectacles from an image using image processing.

[0014] The camera is used to create an image of the person, with the evaluation unit being designed in other words to remove the glasses from the image by means of image processing. Using the face illuminated by the projection means, it is possible to compensate for the optical effects caused by the glasses. These optical effects are, for example, refraction at the edge of the glass or distortion due to the curvature of the glass or the like. This will be discussed again elsewhere. The evaluation unit can be designed in the form of a computer and comprise various input and / or output units. The evaluation unit preferably comprises an independent processor (e.g. a microprocessor) with an executable real-time operating system (RTOS) stored in its memory and, if applicable,one or more applications for generating a spectacle-free or frame-free facial image of a bespectacled person.

[0015] The device according to the invention not only removes the glasses themselves through image processing, but also compensates for the optical effects of the lenses by illuminating the face with the projection device. This creates a facial image of the person as if the person were not wearing glasses at all. The device can be used both to create an identification image, i.e., a facial image for use in a passport, and to generate an identification image for comparison with a passport photo, for example, in an ABC (automated boarder control) system.

[0016] According to the invention, the projection means is designed to illuminate the person's face using structured lighting. This method for capturing three-dimensional objects is also known as the fringe projection method.

[0017] Preferably, the projection means emits light in the visible wavelength range from at least approximately 400 nanometers (nm) to at least approximately 780 nanometers. Alternatively or additionally, the projection means is designed to emit light in the non-visible wavelength range, i.e. below at least approximately 400 nanometers or above at least approximately 780 nanometers. Such a design of the projection means improves the detection options of the, preferably structured, illumination and enables better separability of the images taken by the camera and the detection of the, preferably structured, illumination. In addition, comfort for the person is increased because in such a case, a stripe or grid pattern used on the person's face remains invisible to them.In this context, it is particularly preferred if the projection means emits light in the near infrared ("NIR") wavelength range, ie between at least approximately 780 nanometers to at least approximately 3 micrometers (µm).

[0018] Within the scope of the present invention, a near-infrared camera ("NIR camera") is preferably provided for detecting the preferably structured illumination generated by the projection means. The NIR camera is preferably designed to image the same facial area as the camera. This allows the image of the NIR camera and the image of the camera to be easily correlated.

[0019] Preferably, the camera and the NIR camera are directed toward a beam splitter that transmits as much of the incoming light in the NIR wavelength range as possible and reflects as much of the incoming light in the visible wavelength range as possible. The use of a beam splitter allows the device to be kept compact.

[0020] In an alternative embodiment, the projection means itself comprises an NIR camera. In such a case, the projection means comprises an NIR emitter for, preferably structured, illumination of the person's face and an NIR detector for detecting the NIR structure reflected in the person's face.

[0021] To capture a biometric facial image, it is advantageous if the camera can detect wavelengths in the visible range. The visible wavelength range extends from approximately 400 nanometers to approximately 780 nanometers.

[0022] An alternative embodiment, however, provides for the camera to be configured to capture light in both the near infrared ("NIR") and visible ("VIS") ranges. This camera, configured as a NIR-VIS camera, is then assigned two interchangeable blocking filters, which can be moved into or out of a beam path between the person's face and the NIR-VIS camera. The first blocking filter has a passband for wavelengths of light in the NIR, and the second blocking filter has a passband for wavelengths of visible light. This allows both the (preferably structured) illumination and the facial image to be captured using a single camera. This results in a compact design with only a few components.The blocking filters that can be adjusted in the beam path allow the preferably structured illumination in the NIR wavelength range to be detected separately from the facial image in the visible wavelength range.

[0023] To improve the resolution—and thus increase the image quality of a facial image—it is also advantageous to have at least one additional camera for capturing at least one additional facial image. Using the additional camera makes it possible to supplement any missing image information that may have arisen when removing the glasses frame and / or compensating for the optical effects of the lenses. In other words, the additional camera collects additional image information of the face.

[0024] In this context, it is particularly advantageous that the additional camera of the device is positioned at a different angle of inclination than the camera. This allows for two different viewing angles of the person's face.

[0025] An alternative embodiment provides that the projection means is designed to illuminate the person's face at an inclination angle relative to the horizontal, preferably in a structured manner. In particular, the projection means is arranged such that the illumination occurs obliquely from above or obliquely from below relative to the horizontal or relative to the face. In an advantageous embodiment, the projection means is designed as a projector.

[0026] It has proven advantageous if the projection means is designed to project a line structure onto the person's face. For better detection and compensation of the optical effects caused by the spectacle lenses, it is also advantageous if a second projection means is provided for generating a preferably structured illumination, preferably in the form of a second line structure, so that, together with the first projection means, a grid pattern can be generated on the person's face. By using a grid pattern, the optical effects can be detected more closely. In a further alternative embodiment, it is also possible for a grid pattern to be projected directly onto the face using a single projection means.In a preferred embodiment, the illumination may form a line pattern radiated at the angle of inclination or a grid pattern radiated at the angle of inclination on the person's face.

[0027] The problem concerning the method is solved by a method according to the features of claim 1.

[0028] The method for generating a glasses-free, biometric facial image of a bespectacled person comprises the following steps: A) Capturing a first image of a person's face using a camera; B) Determining the pixels in the first image assigned to the spectacle lenses using an evaluation unit; C) Illuminating, preferably structured illuminating, the person's face and thereby generating a pattern on the face and on the spectacle lenses using a projection means, preferably using a projection means for structured illumination; D) Capturing a second image of the person's face, preferably illuminated in a structured manner, preferably using the camera or a second camera; and E) Determining and compensating for optical effects caused by the spectacle lenses within the first image based on a pattern distortion detected in the second image using the evaluation unit, in order to generate the spectacle-free facial image of the person.

[0029] This process allows a person to wear the glasses during the recording process. Nevertheless, a nearly unambiguous facial image is created, allowing for improved – possibly automatic – identification of the person. The person's glasses are subsequently removed, largely eliminating the optical effects they cause.

[0030] When creating portraits for machine-readable travel documents (MRTD), it is recommended that the image contain approximately 120 information-bearing pixels at the interocular distance so that structures one millimeter (mm) in size remain recognizable in the image. In contrast, both the device and the method preferably use a higher resolution. The interocular distance, for example, encompasses more than 360 pixels, which carry the image information. This results in portrait images with a total image width of more than 1,800 pixels. These values are guidelines, so deviations, even downwards, are permitted as long as they are not excessive. A downward deviation of a maximum of 10 percent appears possible. Furthermore, the images are preferably extracted from a video stream.All this ensures that sufficiently good data is available for implementing the method according to the invention, especially for compensating for pattern distortion. The larger amount of data allows for faster compensation of the effects caused by the spectacle lenses, and offsets and distortion corrections can be eliminated more easily.

[0031] The pixels assigned to the lenses or the pixels assigned to the glasses are determined, for example, by edge detection, contrast detection or color detection.

[0032] If the camera is configured as a NIR-VIS camera, both the first image and the second image can be captured with the NIR-VIS camera. If the camera is configured as a VIS camera operating only in the visible wavelength range, the second image can be captured with a second camera, in particular an NIR camera.

[0033] The optical effects caused by the lenses include aberrations, the refraction of light at the edge of the lenses, as well as the imaging effects caused by convex and / or concave lens shapes, which are generally referred to as "distortion".

[0034] According to the invention, the pattern distortion is detected by determining points of discontinuity in the pattern in the second image using the evaluation unit. These points of discontinuity in the pattern arise from the refraction of light at the edge of the spectacle lens. By determining the points of discontinuity within the second image, an initial correction of the optical effects or distortion in the first image can be made. Alternatively or additionally, the pattern distortion can be detected by determining points of kinks in the pattern. At these points, the pattern curve therefore has a kink, which indicates a different refraction of light at the edge of the spectacle lens. At these points, no clear tangent can be drawn to the line curve, which is why this can be referred to as a discontinuity in the first derivative of a function corresponding to the line curve.

[0035] In order to also remove the spectacle frame from the facial image of the first image, it has proven advantageous if, preferably, at least step A) is followed by the following step: Determining frame pixels in the first image that are assigned to a pair of glasses frames by means of the evaluation unit, and substituting the frame pixels to generate a glasses-free facial image.

[0036] The frame pixels can be substituted either by extrapolating the pixels adjacent to the frame pixels. This represents a simple and sufficient substitution option for comparatively thin frames. However, it is preferable if the frame pixels are substituted via an additional image, possibly with the aid of another camera. The additional image can then supplement missing image information that arose when removing the frame and / or when compensating for the optical effects of the lenses.

[0037] For the same reason, an alternative embodiment also makes sense to supplement the image information missing after compensating for pattern distortion, which occurs particularly when using convex lenses at the edge of the lenses. In particular, the method is intended to additionally include the following steps: Checking the facial image for missing image information; determining the false pixels corresponding to the missing image information; and substituting the false pixels to generate a lensless facial image.

[0038] False pixels are therefore defined as those pixels that result from missing image information after compensating for the optical effects caused by the lenses or the curvature of the lenses. In a simpler embodiment, the replacement of false pixels can again be achieved by extrapolating the pixels adjacent to the false pixels.

[0039] However, for the substitution of false pixels in thick lenses, especially convex lenses and / or thick spectacle frames, it has also proven useful if the procedure includes the following steps: Capturing another image of the person's face at an angle different from the first image; determining replacement pixels corresponding to the frame pixels and / or the false pixels within the another image, and substituting the frame pixels and / or the false pixels.

[0040] This allows the areas of the face obscured by the spectacle frame to be supplemented and / or the missing image information resulting from the pattern distortion compensation to be replaced. In one embodiment, the additional image can be captured by the camera. In this context, it has proven advantageous to instruct the subject to tilt their head in different directions while the image is being captured.

[0041] In an alternative embodiment, the further image is generated by a further camera, at an angle of inclination different from that of the camera of the first image.

[0042] In order to simplify the compensation of the optical effects caused by the spectacle lenses, the invention provides that the at least one projection means projects a striped pattern onto the face of the person.

[0043] In another embodiment of the invention, the evaluation unit is designed to perform quality control by checking the resolution or image information of the facial image. For this purpose, one or more requirements for an identification image are stored in a database of the evaluation unit, which is stored in a memory. If the facial image does not meet one or more requirements, it is discarded, and the process for generating a lens-free or spectacle-free facial image of the person wearing glasses begins again.

[0044] As part of the quality control process, it can also be checked whether any further false pixels or frame pixels in the form of defective pixels are present, and whether these need to be replaced. If additional image information is then required, one or more additional images of the person's face can be captured at a different angle than the first and subsequent images. These steps are repeated until a facial image is generated that meets the specified requirements for an identification image in terms of position, resolution, etc.

[0045] The compensation of optical effects can be further simplified by storing frequently used lenses or frames as models with the corresponding compensation data in the evaluation unit's database. In this context, it is advantageous if the following additional steps are provided, preferably at least after step A): Determination of the shape of the spectacle lens and comparison with compensation data stored in a database of the evaluation unit, which corresponds to predefined spectacle models or predefined lens models. Compensation of the optical effects caused by the spectacles using the corresponding compensation data.

[0046] In the following, the invention is explained in more detail using exemplary embodiments shown in the drawing; in which: Fig. 1 shows a device for generating a spectacle-free facial image of a person wearing glasses, based on a schematic side view; Fig. 2 shows a second device for generating a spectacle-free facial image of a person wearing glasses using a further camera, based on a schematic side view; Fig. 3 shows a third device for generating a spectacle-free facial image of a person wearing glasses using two projection means, based on a schematic side view; Fig. 4 shows a fourth device for generating a spectacle-free facial image of a person wearing glasses using an NIR camera, based on a schematic side view; and Fig. 5 shows a schematic representation of the projection of the structured illumination onto the person's face and onto the lenses.

[0047] Figure 1shows a device 100 for generating a spectacle-free, biometric facial image of a person wearing glasses, comprising a camera 101 for capturing a face 104 of the person and an evaluation unit 102 connected to the camera 101 via a communication link 113. The evaluation unit 102 is configured to compensate for the optical effects caused by spectacle lenses 114 using a method for capturing three-dimensional objects. Furthermore, a projection means 103 is provided for illuminating the face 104 of the person, in particular for projecting a pattern onto the face 104. The device 100 shown utilizes a stripe projection method, so that the projection means 103 is configured to illuminate the face 104 of the person using structured illumination 108 (also called "structured light" or "coded light").

[0048] Openings are formed in the housing front of the device 100 facing the person, through which the beam path 106 between the camera 101 and the face 104, as well as the structured illumination 108, can pass. In the example shown, the camera 101 is directed toward a mirror 105, so that the beam path 106 of the camera 101 is folded to create a greater distance between the camera 101 and the face 104. The projection means 103 generates a structured illumination 108 in the form of a striped pattern 115, wherein the face 104 of the person is illuminated in a structured manner by the projection means 103 at an inclination angle relative to the horizontal.

[0049] In the first embodiment, the structured illumination 108 of the face 104 is achieved by mounting the projection means 103 at an angle in the housing 107 and directing it toward the person's face 104. The projection means 103 generates structured illumination 108 in the non-visible wavelength range, namely the NIR wavelength range. The camera 101 is therefore designed as a NIR-VIS camera 112, so that the camera 112 can detect light in both the NIR wavelength range and the visible wavelength range.

[0050] In order to be able to record an image in the visible wavelength range of the face 104 and an image in the NIR wavelength range of the structured illumination 108, the camera 112 is additionally assigned two interchangeable blocking filters 109, which can be adjusted in and out of the beam path 106 running between the camera 112 and the face 104. The adjustability is indicated by the double arrow. The first blocking filter 110 has a transmission range which largely transmits light in the NIR wavelength range, i.e., it blocks wavelengths outside the NIR wavelength range, so that at least visible light is blocked. The second blocking filter 111 has a transmission range which largely transmits light in the visible wavelength range, so that light in the NIR wavelength range is at least almost completely blocked or absorbed.

[0051] The method for generating a lens-free or spectacle-free, preferably biometric facial image comprises the following steps: First, a first image of a person's face 104 is captured using camera 101, in this case, NIR-VIS camera 112. This first image is captured while the second blocking filter 111 is located in the beam path 106 of NIR-VIS camera 112, so that only wavelengths in the visible wavelength range are transmitted and detected by NIR-VIS camera 112.

[0052] In a next step, the evaluation unit 102, which communicates with the NIR-VIS camera 112 via a communication link 113, determines the pixels assigned to the lenses 114 in the first image. This can be done, for example, via edge detection, contrast detection, color detection, or even via the increase in the transmission of UV light in the UVA wavelength range through the lenses 114 (increase in transmission in the wavelength range of 380 nanometers compared to the transmission at 320 nanometers).

[0053] In addition, the person's face 104 is illuminated in a structured manner by the projection means 103, whereby a pattern in the form of a striped pattern 115 is projected onto the face 104 and onto the spectacle lenses 114. A second image of the person's face 104 illuminated in a structured manner is then captured by the NIR-VIS camera 112, with the first blocking filter 110 now arranged in the beam path 106, so that light in the NIR wavelength range is largely transmitted.

[0054] Finally, the evaluation unit 102 determines the optical effects caused by the spectacle lenses 114 within the first image and compensates for them using a pattern distortion detected in the second image. The pattern distortion is caused partly by the refraction of light at the spectacle lenses 114 but also by the curvature of the spectacle lenses 114. The goal of compensating for the pattern distortion is to create an image in the first image that would correspond to a facial image without glasses.

[0055] With concave lenses 114, the eye region appears smaller than in reality, while with convex lenses it appears larger. The compensation of the optical effects caused by the lenses 114 and / or the glasses takes place in several steps.

[0056] As in Figure 5As shown schematically, the structured illumination 108 shifts due to the refraction of the light at the edge 116 of the lenses 114. This creates discontinuities and / or kinks in the line progression of the pattern at the edge 116 of the lenses 114. This can be understood by following the progression of the lines 115a, 115b, 115c, 115d. Furthermore, the pattern of the structured illumination 108 will spread out in a pincushion-like or barrel-like manner due to the curvature of the lens, so that this change in the progression of the pattern is first captured in the second image and then compensated for using this data in the first image.

[0057] Furthermore, it is determined at the beginning whether the glasses are rimless glasses or whether the glasses have a frame. This can be done, for example, using a contrast gradient or color recognition. If a frame is present, the frame pixels in the first image that are assigned to the frame of the glasses are determined by means of the evaluation unit 102. These are then substituted, i.e. removed and replaced by other pixels in order to generate not only a facial image without lenses, but also one without frames. In the case of very thin and inconspicuous frames, the frame or frame pixels can be substituted by extrapolating the pixels adjacent to the frame pixels.

[0058] For a more precise and higher-quality substitution, however, additional image information is necessary. This is generated, for example, by the NIR-VIS camera 112 capturing at least one additional image in the visible wavelength range using the second blocking filter 111, with the person additionally receiving an instruction to move or tilt their head into a position different from the first image. The evaluation unit 102 determines the replacement pixels corresponding to the frame pixels within the additional image and substitutes the frame pixels with the replacement pixels to supplement the areas of the face 104 obscured by the glasses frame.

[0059] Furthermore, the evaluation unit 102 can perform a quality control, i.e., it determines whether all frame pixels have been replaced. If any missing image information is still present, this is compensated for—if possible—by the additional image, or additional images of the person's face 104 are captured at different angles of inclination of the face 104 until the quality control determines that the facial image meets the quality requirements stored in the evaluation unit 102, i.e., quality parameters with regard to resolution and image information, etc.

[0060] Alternatively, if it has been determined that the glasses do not have a frame, or additionally if a frame has been detected, the facial image is checked for missing image information. These false pixels corresponding to the missing image information are identified and then substituted. In the case of only thin and / or concave lenses, substitution by extrapolating the pixels adjacent to the false pixels may be sufficient. Alternatively or additionally, the substitution can also be carried out using the image information from the further or additional images. The replacement pixels in the further or additional images corresponding to the false pixels are identified, and the false pixels are replaced by replacement pixels. Here, too, a quality control is preferably carried out to determine whether the missing image information has been replaced and whether the stored quality requirements with regard to resolution and image information are met.This can be done analogously to the quality control described with regard to the spectacle frame.

[0061] The device 100 and the method can also be used to capture a biometric facial image of a person who is not wearing glasses. In this case, the evaluation unit 102 will determine that no glasses are present, so that no structured illumination 108 of the face is applied. The first image is then subjected to a quality control by the evaluation unit 102, and if necessary, further or additional images are captured by the camera 101 or another camera 120.

[0062] In Figure 2A second alternative embodiment of the device 100 according to the invention is shown. It differs from the first embodiment in that an additional camera 120 is provided for recording at least one additional facial image. The additional camera 120 is aligned at a different inclination or recording angle with respect to the person's face 104 than that of camera 101. The additional camera 120 serves to generate additional image information to compensate for frame and / or false pixels. It is therefore also connected to the evaluation unit 102 via a communication link 121. The method differs in this embodiment in that the additional image is recorded not by the camera 101 but by the additional camera 120.

[0063] In Figure 3A third alternative embodiment of the device 100 according to the invention is shown. In contrast to the second embodiment, it has a second projection means 130, wherein the two projection means 103, 130 are arranged such that a grid pattern 131 is generated on the face 104 of the person by means of structured illumination 108. The projection means 103, 130 are arranged such that they illuminate the face 104 of the person from an angle above and from an angle below. By generating a grid pattern 131, the optical distortion can be detected more closely and thus more accurately and thus better compensated. A grid pattern 131 generated in this way is shown schematically, for example, in Figure 5can be seen, wherein each of the projection means 103, 130 projects its own stripe pattern 115. In an alternative embodiment, it is also possible for only a single one of the projection means 103, 130 to generate the complete grating pattern 131. In a further embodiment, not shown, this device 100 of the third embodiment can also comprise an additional camera 120 sensitive in the visible wavelength range - as described for the second embodiment - for recording at least one additional image and thus for collecting additional image information.

[0064] In Figure 4An alternative fourth embodiment of the device 100 according to the invention is shown. This differs from the third embodiment in that the camera 101 is designed as a VIS camera 117 for detecting the face 104 in the visible wavelength range, and in that an NIR camera 140 is additionally provided for detecting the structured illumination 108 in the NIR wavelength range. The VIS camera 117 and the NIR camera 140 are both directed towards a beam splitter 141, with the beam splitter 141 transmitting light in the NIR wavelength range and reflecting light in the visible wavelength range. The NIR camera 140 is also connected to the evaluation unit 102 via a communication link 142. The VIS camera 117 can also record the additional images necessary to substitute the false pixels and / or the frame pixels.Alternatively, the device 100 may also have an additional camera 120 (not shown) which records the further and additional images in the visible wavelength range at different inclination angles.

[0065] The device according to Figure 4 The method performed differs from that of the third embodiment in that the first image is taken by the VIS camera 117 and the second image is taken by the NIR camera 140. LIST OF REFERENCE SYMBOLS

[0066] 100Device 101Camera 102Evaluation unit 103Projection device 104Face 105Mirror 106Beam path 107Housing 108Structured illumination 109Blocking filter 110First blocking filter 111Second blocking filter 112NIR-VIS camera 113Communication connection (camera) 114Lens 115Fringe pattern 115aLine 115bLine 115cLine 115dLine 116VIS camera 120Additional camera 121Communication connection (additional camera) 130Second projection device 131Grid pattern 140NIR camera 141Beam splitter 142Communication connection (NIR camera)

Claims

1. A method of generating a spectacle-free facial image of a spectacled person, comprising the following steps: A) Recording a first image of a face (104) of the person by means of a camera (101); B) Determination of the pixels assigned to the spectacle lenses (114) in the first image by means of an evaluation unit (102); C) Illuminating the face (104) of the person and thereby creating a pattern on the face (104) and on the spectacle lenses (114) by means of a projection means (130), wherein a pattern in the form of a stripe pattern (115) is projected onto the face (104) of the person and onto the spectacle lenses (114) by the at least one projection means (130); D) Capturing a second image of the illuminated face (104) of the person by means of the camera (101) or by means of a second camera; and E) Determination and compensation of optical effects within the first image caused by the spectacle lenses (114) on the basis of a pattern distortion detected in the second image by means of the evaluation unit, in order to generate the spectacle-free facial image of the person, the pattern distortion being detected by determining discontinuity points and / or kink points of the pattern in the second image by means of the evaluation unit (102).

2. The method according to claim 1, characterized by the following step - Determining frame pixels in the first image which are assigned to a spectacle frame of the spectacles by means of the evaluation unit (102), and substituting the frame pixels to generate a spectacle-free facial image.

3. The method according to claim 1 or 2, characterized by the following steps: - Checking the facial image for missing image information; - Determining the false pixels corresponding to the missing image information; and - Substituting the false pixels to produce a lensless facial image.

4. The method according to claim 2 or 3, characterized by the following steps: - Taking a further image of the person's face at an angle different from the first image; - Determining the substitute pixels corresponding to the frame pixels and / or the false pixels within the further image, and - Substituting the replacement pixels for the frame pixels and / or the false pixels.

5. A device (100) for generating a spectacle-free facial image of a spectacled person according to the method according to one of the preceding claims, having a camera (101) for detecting a face (104) of the person, having a projection means (103) for structured illumination of the face (104) of the person, the projection means (103) being designed to project a pattern in the form of a stripe pattern (115) onto the face (104) of the spectacled person and onto the spectacle lenses (114), and with an evaluation unit (102) which is connected to the camera (101) and which is designed, in step E), to compensate for the optical effects caused by spectacle lenses (114) by means of a method for detecting three-dimensional objects on the basis of the pattern drawing detected in step E) and to remove the spectacles from an image by means of image processing, using the face (104) illuminated in a structured manner by the projection means (103).

6. The device (100) according to claim 5, characterized in that the projection means (103) emits light in the non-visible wavelength range.

7. The device (100) according to claim 5 or 6, characterized in that a NIR camera is provided for detecting the illumination generated by the projection means (103).

8. The device (100) according to any one of claims 5 to 7, characterized in that the camera (101) is formed as an NIR-VIS camera (112), in that the NIR-VIS camera (112) is assigned to two exchangeable blocking filters (109), which are adjustable in or out of a beam path (106) between the face (104) and the NIR-VIS camera (112), and in that the first blocking filter (110) has a transmission range for wavelengths of light in the NIR and the second blocking filter (111) has a transmission range for wavelengths of visible light.

9. The device (100) according to any one of claims 5 to 8, characterized in that at least one further camera (120) is provided for recording at least one further facial image.

10. The device (100) according to any one of claims 5 to 9, characterized in that the person's face (104) can be illuminated by the projection means (103) at an angle of inclination with respect to the horizontal.

11. The device (100) according to claim 10, characterized in that a second projection means (130) is provided for generating a grid pattern (131) on the face (104) of the person.