Biological recognition system, biological recognition method and electronic device

By using SWIR cameras and SWIR bandpass filters in the biometric system, the distinction between real and fake skin based on the brightness of the SWIR spectrum is solved, and the problem of difficulty in identifying in low-light environments in the prior art is improved. Accuracy and security of identification.

CN119049097APending Publication Date: 2024-11-29SONY SEMICON SOLUTIONS CORP

Patent Information

Application Number
CN202410268437.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-03-08
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing biometric systems are difficult to effectively distinguish real skin from fake skin made of fake skin in low-light environments or in the presence of occlusions, resulting in increased identification difficulty.

Method used

Short-wave infrared (SWIR) camera is used to collect images of biological parts and distinguish them based on the brightness of the SWIR spectrum. The identification accuracy is improved using technologies such as SWIR bandpass filter and automatic exposure mode.

Benefits of technology

It achieves efficiently distinguishing between real and fake skin in the presence of low-light environments and occlusions, improving the accuracy and security of biometrics, especially in preventing attacks from human skin masks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049097A_ABST
    Figure CN119049097A_ABST
Patent Text Reader

Abstract

The present disclosure is directed to improving biological identification capabilities based on a SWIR camera. A biometric recognition system includes or is connected to a SWIR camera, and includes a resolution. The SWIR camera acquires SWIR images based on the SWIR spectrum. The discrimination unit receives the SWIR image, discriminates whether the biological site corresponding to the SWIR image is real skin or false skin based on at least the brightness of the SWIR image, and outputs the discrimination result. A biological recognition method comprises: a step of acquiring an SWIR image based on an SWIR spectrum; and a step of receiving the SWIR image, distinguishing whether the biological site corresponding to the SWIR image is real skin or false skin based on at least the brightness of the SWIR image, and outputting the distinguishing result. An electronic device includes: a memory storing computer executable code; and a processor that executes the code to perform the following: receive a SWIR image acquired based on the SWIR spectrum; and distinguishing whether the biological part corresponding to the SWIR image is real skin or false skin at least based on the brightness of the SWIR image, and outputting the distinguishing result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present technology relates to a biometric recognition system, a biometric recognition method, and an electronic device, and more particularly, to a biometric recognition system, a biometric recognition method, and an electronic device capable of improving biometric authentication capabilities based on a SWIR (short-wave infrared) camera. Background Art

[0002] Face recognition has become very popular and can be applied in many fields. However, attempts to attack face recognition have never stopped. For example, criminals sometimes wear human skin masks when committing crimes. Some human skin masks can already simulate real faces to a high degree, and can even simulate blood vessels, pores, and skin textures such as wrinkles on the mask. These human skin masks can successfully crack many conventional face recognition systems (such as mobile phone lock screen systems, supermarket face payment systems, company or community access control systems, bank ATM or safe deposit box systems, etc.). These conventional face recognition systems use RGB (red, green and blue) cameras, so it is difficult to distinguish between real faces and human skin masks.

[0003] Similarly, in conventional biometric systems, biometric methods and electronic devices that identify fingerprints, palm prints, foot prints or irises, there is also the problem of difficulty in detecting (identifying) fake skin made of counterfeit materials such as fingerprint films, palm print films, foot print films or iris films. Summary of the invention

[0004] [Technical issues to be solved]

[0005] In existing biometric identification systems and methods, images of biological parts are collected, the images are preprocessed, features are extracted from the preprocessed images, the extracted features are matched and searched with feature templates stored in a database, and then the identity information of the biological in the image is judged based on the similarity. However, the image information collected by existing biometric identification systems and methods using the RGB spectrum is very limited, so it is often difficult to distinguish between real skin and fake skin made of counterfeit materials (such as human skin masks, fingerprint films, palm print films, foot print films or iris films, etc.).

[0006] Especially in low-light environments or in the presence of obstructions such as fog, smoke, haze, dust, plastic bags or glass, the images collected using the RGB spectrum are very unclear and the image details are insufficient, making it more difficult to identify and recognize organisms.

[0007] Accordingly, the present disclosure intends to provide a biometric recognition system, a biometric recognition method, and an electronic device that can simply and efficiently distinguish between real skin and fake skin based on a SWIR camera, thereby improving the biometric identification capability.

[0008] [Technical solution to the problem]

[0009] In order to solve the above problems, one aspect of the present disclosure provides a biometric recognition system, which includes a SWIR camera or is connected to a SWIR camera, and includes a resolution unit. The SWIR camera collects SWIR images for each of at least one part of a biological body based on a SWIR spectrum. The resolution unit receives the SWIR image collected by the SWIR camera, and determines whether the part corresponding to the SWIR image is real skin or fake skin based on at least the brightness of the SWIR image, and outputs the resolution result.

[0010] Another aspect of the present disclosure provides an electronic device, comprising a memory and a processor. The memory stores a code that can be executed by a computer. The processor is configured to execute the code to implement the following operations: receiving a SWIR image, the SWIR image being collected for each part of at least one part of a biological body based on a SWIR spectrum; and distinguishing whether the part corresponding to the SWIR image is real skin or fake skin based at least on the brightness of the SWIR image, and outputting the distinguishing result.

[0011] Another aspect of the present disclosure provides a biometric identification method, which includes a SWIR image acquisition step and a discrimination step. In the SWIR image acquisition step, SWIR images are respectively acquired for each of at least one part of the biological body based on the SWIR spectrum. In the discrimination step, the acquired SWIR image is received, and the part corresponding to the SWIR image is discriminated based on at least the brightness of the SWIR image to determine whether it is real skin or fake skin, and the discrimination result is output.

[0012] [Beneficial Effects]

[0013] The biometric recognition system, biometric recognition method, and electronic device (which are sometimes collectively referred to as SWIR recognition solutions) disclosed in the present invention for improving biometric authentication capabilities based on a SWIR camera have the following effects.

[0014] (1) The SWIR recognition scheme can simply and efficiently identify human skin masks and overcome the shortcomings of current ordinary face recognition systems.

[0015] (2) From the simulation results, the SWIR recognition scheme can identify human skin masks more easily than the recognition schemes using RGB cameras, near-infrared (NIR) cameras or depth cameras.

[0016] (3) Using a bandpass filter of 1400nm to 1500nm, the SWIR recognition scheme has the best effect in identifying human skin masks.

[0017] (4) Even for wet human skin masks, the SWIR recognition solution can identify them.

[0018] (5) Even when a person has makeup on his or her face, the SWIR recognition scheme can distinguish whether it is a real face or a human skin mask.

[0019] (6) Regarding the problem that the brightness of SWIR images may decrease as the distance to the face becomes farther, the present disclosure also proposes the following three solutions: (i) using AE (Auto Exposure) on mode; (ii) using a light source array to obtain uniform light; and / or (iii) using depth information for joint judgment.

[0020] (7) The SWIR recognition scheme helps to improve the security level of the face recognition system to prevent attacks by human skin masks.

[0021] (8) The SWIR recognition solution can be applied to, for example, facial recognition payment systems, ATM (Automated Teller Machine) systems, and access monitoring systems for important places / ports (such as customs). BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a block diagram showing a configuration example of a biometrics authentication system according to a first embodiment of the present disclosure.

[0023] Figure 2 : is a block diagram showing a configuration example of a SWIR camera in the biometrics authentication system according to the first embodiment of the present disclosure.

[0024] Figure 3 A comparison of images of real human faces and human skin masks using four situations: RGB camera, NIR camera, depth camera, and SWIR camera is shown.

[0025] Figure 4 Comparison of images of a real human face and a human skin mask in the case of an RGB camera and in the case of a SWIR camera using different SWIR bandpass filters is shown.

[0026] Figure 5is a graph showing quantitative evaluation in the case of an RGB camera and in the case of a SWIR camera using different SWIR bandpass filters.

[0027] Figure 6 A comparison of images of a real human face, a dry human skin mask, and a wet human skin mask is shown using a 1450 nm bandpass filter on a SWIR camera.

[0028] Figure 7 (a) shows an image of a face with makeup captured with an RGB camera.

[0029] Figure 7 (b), (c), and (d) show the comparison of images of a human face with makeup, a human face without makeup, and a human skin mask when the SWIR camera uses a 1450nm bandpass filter.

[0030] Figure 8 : is a block diagram showing a configuration example of a discrimination section in the biometrics authentication system according to the first embodiment of the present disclosure.

[0031] Fig. 9 is a block diagram showing a configuration example of a biometrics authentication system according to a second embodiment of the present disclosure.

[0032] Fig.10 is a block diagram showing a configuration example of a biometrics authentication system according to a third embodiment of the present disclosure.

[0033] Fig.11 : is a block diagram showing a configuration example of an identification section in a biometrics identification system according to a third embodiment of the present disclosure.

[0034] Fig.12 An example of three network structures constituting the MTCNN algorithm used when performing face alignment in the recognition unit is shown.

[0035] Fig.13 An operation flow chart of a face recognition system as one example of a biometric recognition system according to a third embodiment of the present disclosure is shown.

[0036] Fig.14 is a block diagram showing a schematic configuration of a biometrics authentication system according to a fourth embodiment of the present disclosure.

[0037] Fig.15 An operation flowchart of a face recognition system as one example of a biometric recognition system according to a third modification of the fourth embodiment of the present disclosure is shown.

[0038] Fig.16 A light source array consisting of a plurality of halogen lamps is shown. DETAILED DESCRIPTION

[0039] A mode for carrying out the present technology (hereinafter, referred to as an embodiment) will now be described. The description will be made in the following order.

[0040] 1. Biometric Authentication System of First Embodiment

[0041] [Overview of a construction example of a biometric authentication system]

[0042] [light source]

[0043] [Configuration example of SWIR camera]

[0044] [Effect of SWIR camera]

[0045] [Three solutions to the problem that SWIR image brightness decreases as the distance from the face increases]

[0046] [Structure example of the resolution unit]

[0047] 2. Biometric Authentication System of Second Embodiment

[0048] 3. Biometric Authentication System of the Third Embodiment

[0049] [Configuration Example of Identification Section]

[0050] 4. Biometric Authentication System of Fourth Embodiment

[0051] 5. Electronic devices

[0052] 6. Application examples

[0053] The first embodiment according to the present disclosure will be described below.

[0054] [1. Biometric Authentication System of First Embodiment]

[0055] [Overview of a construction example of a biometric authentication system]

[0056] Figure 1 1 is a block diagram showing a configuration example of a biometric authentication system according to a first embodiment of the present disclosure. Figure 1 As shown, the biometric authentication system 100A mainly includes a SWIR camera 10 and a resolution unit 20. Figure 1 In the example shown, the SWIR camera 10 is included in the biometrics recognition system 100A, but the SWIR camera 10 may be provided outside the biometrics recognition system 100A and connected to the biometrics recognition system 100A. In the present disclosure, the case where the SWIR camera 10 is included in the biometrics recognition system 100A is described as an example.

[0057] The light source 2 disposed near the subject 1 emits irradiation light to the subject 1, and a part of the irradiation light is reflected by the subject 1 and enters the SWIR camera 10 as incident light. Figure 1 The subject 1 shown in the figure is the upper body of a human body, but it can be at least one part of a living being, for example, any combination of the face, fingers, palms, soles, irises, etc.

[0058] The SWIR camera 10 itself can obtain the spectral information of the visible light band and the SWIR band in the incident light. The present disclosure intends to highlight the special effect of the light in the SWIR band in the incident light. Therefore, the SWIR camera 10 can be additionally configured with a SWIR bandpass filter to allow only the light in the SWIR band (i.e., the SWIR spectrum) to pass through. In this way, the SWIR camera 10 collects the SWIR image of the subject 1 based on the SWIR spectrum in the incident light. This will be referred to later. Figure 2 Further description is given of the details of the configuration of the SWIR camera 10 .

[0059] The location of the SWIR bandpass filter is not particularly limited, as long as it is located upstream of the light receiving surface (imaging surface) of the sensor portion of the SWIR camera 10 in the light incident direction (i.e., only the SWIR spectrum of the incident light is allowed to pass through so as to form an image on the light receiving surface). For example, the SWIR bandpass filter may be built into the SWIR camera 10, or alternatively, may be disposed outside the SWIR camera 10 and between the lens of the SWIR camera 10 and the subject 1. By disposing the SWIR bandpass filter outside the SWIR camera 10, it can be easily maintained and replaced.

[0060] In the present disclosure, the spectral range of the SWIR bandpass filter may be, for example, 1350 nm to 1600 nm, preferably 1400 nm to 1600 nm, more preferably 1400 nm to 1550 nm, and most preferably 1400 nm to 1500 nm. The spectral range of the SWIR bandpass filter may optimally be approximately 1450 nm.

[0061] Under normal circumstances, the subject 1 whose image is captured by the SWIR camera 10 should be real skin (such as a real human face, etc.), but criminals sometimes use fake skin made of counterfeit materials (such as masks, etc.). In order to detect fake skin to prevent crime, in the biometric system 100A, a classification model corresponding to the different parts of the organism to be identified (such as face, fingers, palms, soles, irises, etc.) will be pre-trained as an image classifier that can distinguish between fake skin and real skin. In the present disclosure, the classification model can be an AI (artificial intelligence) classification model, a threshold classification model, or a traditional classification model such as SVM (Support Vector Machine). Figure 1 In the embodiment, the classification unit 20 is deployed with a trained classification model. After the SWIR image collected by the SWIR camera 10 is input to the classification unit 20, the classification unit 20 identifies the SWIR image based on the classification model and classifies it as real skin or fake skin. The structure of the classification unit 20 and the training of the classification model will be further described later.

[0062] When the input SWIR image is classified as fake skin, the classification unit 20 outputs a classification result indicating “fake skin.” When the input SWIR image is classified as real skin, the classification unit 20 outputs a classification result indicating “real skin.”

[0063] [light source]

[0064] In the present disclosure, the SWIR camera may use, for example, a halogen lamp as a light source.

[0065] Halogen lamps emit light of a full wavelength range (similar to sunlight), and have the advantages of high light quantity and wide wavelength, and therefore are particularly suitable for analysis and detection in a wide wavelength range. In the present disclosure, the light source may be a single light source or multiple light sources.

[0066] Under the condition of a single artificial light source, the setting position of the light source can be determined based on the threshold function obtained as follows: first, let the subject 1 be 1 meter away from the light source 2 and the SWIR camera 10, and then determine the optimal brightness threshold at 1 meter = (real face image brightness at 1 meter + mask image brightness at 1 meter) / 2 according to the actual situation, and set the threshold function = optimal brightness threshold at 1 meter / distance squared. The distance here can be determined, for example, by the depth information provided by the depth sensor.

[0067] Under the condition of multiple light sources, the locations of these light sources can be determined according to the usage scenario and location characteristics of the application example. For example, the area where these light sources are set can usually be divided into several blocks, and by using a illuminance meter to measure in each block, if the illuminance of each block is similar, it is considered that the location of these light sources is better.

[0068] [Configuration example of SWIR camera]

[0069] Generally speaking, the wavelength of visible light is 400nm to 780nm, and the wavelength of near infrared (NIR) light is 780nm to 2526nm. Among near infrared light, the wavelength of short-wave infrared (SWIR) light can cover the range of 900nm to 1700nm. Therefore, SWIR cameras can capture images that conventional visible light sensors cannot capture.

[0070] Figure 2 1 is a block diagram showing a configuration example of a SWIR camera in a biometric authentication system according to a first embodiment of the present disclosure. Figure 2 As shown, the SWIR camera 10 includes, for example, an optical element 101, an imaging element (sensor unit) 102, a signal processing unit 103, a display unit 105, and a storage unit 106. The optical element 101 includes a lens composed of one or more lenses, and guides imaging light (incident light) from the subject 1 to the imaging element 102 so that it is imaged on the light receiving surface of the imaging element 102.

[0071] Although not shown, the imaging element 102 includes a semiconductor substrate in which a plurality of pixels are formed, each pixel having a photoelectric conversion region. The photoelectric conversion region photoelectrically converts received incident light to generate a signal having a charge corresponding to the amount of received incident light as an (analog) pixel signal.

[0072] In the above description, an example is given in which a SWIR bandpass filter is formed separately and then installed between the optical element 101 and the imaging element 102 in the SWIR camera 10, or detachably installed outside the SWIR camera 10 and located between the optical element 101 and the object 1. As another optional example, instead of forming a SWIR bandpass filter separately, a SWIR filter layer is formed in a stacked manner on the light incident side of the semiconductor substrate of the imaging element 102 of the SWIR camera 10. The spectral range of the SWIR filter layer can be the same as that of the above-mentioned SWIR bandpass filter.

[0073] The signal processing section 103 performs various signal processing on the pixel signal output from the imaging element 102. The image (image data) obtained as a result of the signal processing performed by the signal processing section 103 can be provided to the display section 105 to be displayed thereon, and can also be provided to the storage section 106 to be stored (recorded) therein. For example, the signal processing section 103 may include an AD (analog-to-digital) conversion section 131 and a DSP (Digital Signal Processor) circuit 132, the AD conversion section 131 converts the analog pixel signal from the imaging element 102 into a digital pixel signal, and the DSP circuit 132 performs processing such as amplification, correction, calculation, etc. on the digital pixel signal from the AD conversion section 131, and then outputs the processed signal.

[0074] [Effect of SWIR camera]

[0075] In order to confirm the effect of the SWIR camera, the inventors of the present disclosure conducted tests through various experimental examples. In these experimental examples, for the sake of convenience, face recognition is used as an example of biometric recognition. In this case, the fake skin made of counterfeit materials is called a "human skin mask" or "mask".

[0076] <1. First Experimental Example: Comparison of SWIR Camera with RGB Camera, NIR Camera, and Depth Camera>

[0077] The following is a comparison with an RGB camera, a NIR camera, and a depth camera to illustrate the effectiveness of the SWIR camera in distinguishing images of real human faces and human skin masks.

[0078] Figure 3 The comparison of images of real human faces and human skin masks using RGB camera, NIR camera, depth camera and SWIR camera is shown. Figure 3 In the figure, the upper figure corresponds to the image of a real human face, and the lower figure corresponds to the image of a human skin mask. Figure 3 (a) in the figure corresponds to the RGB camera, and the camera model used is iPhone14pro max. Figure 3 (b) corresponds to a NIR camera, where the spectral range of the NIR bandpass filter used is approximately 940 nm. Figure 3 (c) in FIG. 5 corresponds to the depth camera. Figure 3 (d) in the figure corresponds to a SWIR camera, where the camera model used is Omron STC-LBS132POE-SWIR, the sensor is IMX990, the lens is Kowa LM8HC-VIS-SW, and the spectral range of the SWIR bandpass filter is approximately 1450 nm. Figure 3The images (b) and (c) are both from the indirect time-of-flight (iToF) camera technology.

[0079] from Figure 3 As can be seen, in the RGB camera, the image of the real face and the image of the human skin mask have similar brightness. The same is true for the NIR camera and the depth camera. Compared with the RGB camera, the NIR camera, and the depth camera, the SWIR camera can have a more obvious difference in the image brightness between the real face and the human skin mask. The reason is that the real human face is rich in liquids such as water under the skin, and water absorbs light in the SWIR band (for example, 1450nm in this case), so the resulting SWIR image is almost black. On the contrary, the counterfeit material used to make the mask has no water (or very little water), so the resulting SWIR image is relatively bright. Using this principle, the SWIR camera can simply and efficiently distinguish between real skin and fake skin.

[0080] In the following Table 1, the present disclosure provides a quantitative evaluation of the image brightness of a real human face and a human skin mask and the difference between the two in four cases using an RGB camera, a NIR camera, a depth camera, and a SWIR camera.

[0081] [Table 1] Quantitative evaluation of RGB camera, NIR camera, depth camera and SWIR camera

[0082]

[0083] In the above Table 1, the Y value represents the image brightness value of the object, and m (meter) is the unit of the depth information (distance) of the object.

[0084] It is also obvious from Table 1 that when using a SWIR camera to take pictures, the brightness difference between the case with a human skin mask and the case without a human skin mask (real human face) is very obvious, so it is easy to detect whether a human skin mask is worn.

[0085] <2. Second Experimental Example: Using Different SWIR Bandpass Filters>

[0086] During the image acquisition process of the SWIR camera, a SWIR bandpass filter may be used to perform imaging based on the SWIR spectrum. The inventors of the present disclosure tested the effects of different SWIR bandpass filters on the SWIR camera. Figure 4 A comparison of images of a real human face and a human skin mask is shown for the case of an RGB camera and for the case of a SWIR camera using different SWIR bandpass filters. Figure 4In the figure, the upper figure corresponds to the image of a real human face, and the lower figure corresponds to the image of a human skin mask. Specifically, Figure 4 (a1) in the figure shows the comparison of the images of real human face and human skin mask captured when the camera model of the RGB camera is iPhone14pro max. Figure 4 (a2) to (a7) in FIG. 5 show the comparison of the images of the real human face and the human skin mask collected when the SWIR camera uses different SWIR bandpass filters. More specifically, Figure 4 (a2) to (a6) in FIG. 4 are cases where a SWIR bandpass filter is used (spectral ranges are 1300nm, 1350nm, 1450nm, 1550nm, and 1600nm, respectively). Figure 4 (a7) is the case where no SWIR bandpass filter is used (spectral range is 300nm to 1700nm).

[0087] from Figure 4 It can be seen that in the case of an RGB camera, the image of a real face and the image of a human skin mask have almost the same brightness, so it is difficult to distinguish whether it is a real face or a human skin mask by the image brightness. In the case of a SWIR camera, the image of a real face and the image of a human skin mask have a more obvious difference in brightness, so it is easier to distinguish between a real face and a human skin mask. According to the test results, from a subjective judgment, when the spectral range of the bandpass filter is 1400nm to 1650nm, the SWIR camera has better resolution performance. Furthermore, when the spectral range of the bandpass filter is approximately 1450nm, 1550nm or 1600nm, the SWIR camera has better resolution performance.

[0088] and Figure 4 Correspondingly, the present disclosure provides in Table 2 below a quantitative evaluation of the image brightness of a real human face and a human skin mask and the difference between the two in an RGB camera and in a SWIR camera using different SWIR bandpass filters.

[0089] [Table 2] Quantitative evaluation of RGB cameras and SWIR cameras using different SWIR bandpass filters

[0090]

[0091] Figure 5 It is a curve chart showing the quantitative evaluation in the case of RGB camera and in the case of SWIR camera using different SWIR bandpass filters, wherein the vertical axis (percentage = (Y value of human skin mask - Y value of real face) / Y value of human skin mask) corresponds to the difference (Gap) in Table 2.

[0092] According to Table 2 and Figure 5 From the quantitative evaluation curve in the figure, it can be seen that when a 1450nm bandpass filter is used, the image brightness (Y value) of the human skin mask collected by the SWIR camera is the largest difference from the image brightness (Y value) of the real face (e.g. Figure 5 In addition, the above difference is also great when a 1550nm or 1600nm bandpass filter is used.

[0093] <3. Third Experimental Example: Wet Human Skin Mask>

[0094] The above has described the experimental example of the case where the human skin mask has no moisture (or contains very little moisture). Due to the concern that criminals may use a wet human skin mask to attack the biometric recognition system, the inventor of the present disclosure also tested the detection effect of the SWIR camera on the wet human skin mask.

[0095] Figure 6 The comparison of the images of real human face, dry human skin mask and wet human skin mask is shown in Table 3 below, with the SWIR camera using a 1450nm bandpass filter. Figure 6 (a), (b), and (c) in the figure respectively give the quantitative evaluation of the image brightness (Y value).

[0096] [Table 3] Quantitative evaluation of SWIR cameras using a 1450nm bandpass filter

[0097] Facial images Real face Dried human skin mask Wet human skin mask Y Value 14.7 124.9 111.4

[0098] The wet human skin mask made by the inventor of the present disclosure contains a lot of water on its surface. As can be seen from Table 3, although the image brightness of the wet human skin mask is reduced by about 10% compared with the dry human skin mask, the image brightness is still much higher than the image brightness of a real human face. This means that the SWIR camera of the present disclosure can easily identify the wet human skin mask.

[0099] <4. Fourth Experimental Example: Makeup on the Face>

[0100] Often, the person to be identified has makeup on his or her face. Due to concerns that the makeup may cover the real face and cause false detection by the SWIR camera, the inventors of the present disclosure also tested the detection effect of the SWIR camera for situations where makeup is on the face of the person.

[0101] Figure 7 (a) shows an image of a face with makeup captured by an RGB camera. Concealer, foundation, blush, highlighter, shadow liquid, eye shadow, lipstick, etc. are applied on different parts of the face. Figure 7(b), (c), and (d) show the comparison of images of a face with makeup, a face without makeup, and a human skin mask when the SWIR camera uses a 1450nm bandpass filter. Figure 7 (b), (c), and (d) in the figure give the quantitative evaluation of the image brightness (Y value) respectively.

[0102] [Table 4] Quantitative evaluation of SWIR cameras using a 1450nm bandpass filter

[0103]

[0104] As can be seen from Table 4, although adding some cosmetics to a human face slightly increases the brightness of the SWIR image compared to a human face without cosmetics, the brightness is still much lower than the image brightness of a human skin mask. This means that the SWIR camera disclosed in the present invention can easily distinguish between a real human face with cosmetics and a human skin mask.

[0105] [Three solutions to the problem that SWIR image brightness decreases as the distance from the face increases]

[0106] In the application example of the present disclosure, regarding the brightness of the SWIR image, if only a single artificial light source is used, the brightness of the SWIR image will decrease as the distance of the subject (e.g., face) becomes farther, which will cause trouble in setting the brightness threshold for distinguishing between a human skin mask and a real face. In order to solve this problem, the present disclosure proposes the following three methods.

[0107] (1) When the SWIR camera is capturing images, the AE on (auto exposure on) mode is used and a fixed AE target value is set, whereby the brightness of the SWIR image does not change much as the distance to the face becomes farther.

[0108] (2) When the SWIR camera acquires images, a light source array is used as the light source to produce uniform light in the application example.

[0109] In the application example of the present disclosure, it is preferable to adopt a light source array in which a plurality (two or more) of light sources of the same type are used. Fig.16 The circle in FIG. 1 shows a light source 2 consisting of a single halogen lamp. Fig.16A light source array composed of multiple halogen lamps is shown in the figure. The arrangement of the light source array is not particularly limited in the present disclosure, but can be determined according to the usage scenario and location characteristics of the application example. For example, the light sources used to form the light source array can be placed around the working area of ​​the application example, and the light source array can be arranged to produce uniform light for the working area. Using a illuminance meter to check whether the illumination in each place in the working area is similar can help confirm whether the light source array produces uniform light. In this environment, because of the presence of uniform light, the brightness of the facial area will not change no matter where the facial area is or how far it is from the SWIR camera.

[0110] This method can not only solve the problem in the application example of the present disclosure, but also solve the problem in any other SWIR application examples (such as food sorting, etc.) using a single artificial light source. The uniform light from the light source array can keep the SWIR image brightness constant in any application example.

[0111] (3) When distinguishing between real human faces and human skin masks based on SWIR image brightness, depth (distance) information (which can be obtained by, for example, a ToF (Time of Flight) camera or a dual camera) is used to estimate the SWIR image brightness at different distances.

[0112] The SWIR image brightness can be estimated by the formula Y1×(1 / L^2), where Y1 is the SWIR image brightness of a face 1m away using a single artificial light source, and L is the face distance. In addition, the brightness threshold for distinguishing between human skin masks and real faces can be set as a function of the face distance.

[0113] First, the present disclosure needs to confirm a suitable brightness threshold T1 in the application example at a distance of 1m. Since the facial distance L (depth) can be obtained by a ToF camera or a dual-camera RGB camera, the present disclosure sets a threshold function T = T1 × (1 / L^2). Such a threshold function T can be applied to the application examples of the present disclosure at any distance.

[0114] This method can not only solve the problem in the application example of the present disclosure, but also solve the problem in any other SWIR application examples (such as food sorting, etc.) using a single artificial light source. By combining with the depth information and the threshold function T of the present disclosure, no side effects will occur when the brightness of the SWIR image changes with different distances.

[0115] [Structure example of the resolution unit]

[0116] Figure 8A block diagram showing an example of the configuration of a discrimination unit in a biometric authentication system according to a first embodiment of the present disclosure. Figure 8 As shown, the identification unit 20 includes, for example, a ROI frame setting unit (face detection unit) 201, a ROI frame cropping unit 202, and a classification unit 203. Here, face recognition is still used as an example for description.

[0117] For the SWIR image from the SWIR camera 10, the ROI frame setting unit (face detection unit) 201 determines the size and position of the face in the SWIR image by detection (e.g., manual detection), that is, determines "where the face is". If there is a face in the SWIR image, the ROI frame setting unit (face detection unit) 201 sets a boundary box containing only the face in the SWIR image (i.e., encircles the face with the ROI frame (regions-of-interest box)) and outputs the coordinates of the ROI frame, thereby outputting the SWIR image including the ROI frame to the ROI frame cropping unit 202.

[0118] The ROI frame cropping unit 202 crops the ROI frame image containing only the face from the SWIR image to facilitate classification, conversion, feature extraction, analysis or recognition in subsequent stages.

[0119] The ROI frame image cropped by the ROI frame cropping unit 202 is input to the classification unit 203, which classifies the ROI frame image as real skin or fake skin based on a classification model (such as an AI classification model, a threshold classification model or a traditional classification model) and outputs the classification result.

[0120] It should be understood that the ROI frame setting unit (face detection unit) 201 and the ROI frame cropping unit 202 do not have to be arranged in the resolution unit 20, but can also be arranged in series between the SWIR camera 10 and the resolution unit 20, or can be merged in a preprocessing unit (not shown) located between the SWIR camera 10 and the resolution unit 20.

[0121] As a modified example, a conversion unit (not shown) may be provided between the ROI frame setting unit (face detection unit) 201 and the ROI frame cropping unit 202, or between the ROI frame cropping unit 202 and the classification unit 203. The conversion unit converts the SWIR image including the ROI frame from the ROI frame setting unit (face detection unit) 201 or the ROI frame image from the ROI frame cropping unit 202 into a recognizable image, so as to perform feature extraction from the recognizable image, and the extracted features are used together with the image brightness for classification or recognition.

[0122] The following two examples are described by way of example regarding the classification model used in the classification unit 203 .

[0123] <1. Example of AI classification model>

[0124] exist Figure 8 In the classification unit 203 shown, an AI (artificial intelligence) classification model may be deployed. In addition to classification based on the brightness of the SWIR image (the water content of the subject), the AI ​​classification model may also be combined with object features (such as facial contours, facial features, shape and position, skin texture, skin color, etc.) that can be extracted from the SWIR image by the AI ​​classification model as needed for classification. The following is an example of the training process of the AI ​​classification model, which includes the following stages.

[0125] (1) Dataset preparation

[0126] (1-1) Data collection step: Collect sufficient data related to the AI ​​classification model to be trained from reliable data sources. For example, in the present disclosure, it is necessary to train an image classifier that can distinguish between masks and real faces, so a large amount of image data of masks and real faces must be collected.

[0127] (1-2) Data cleaning step: After collecting data, irrelevant data is removed and the accuracy and consistency of the data are ensured, for example, through deduplication processing, missing value processing, outlier filtering processing, data standardization processing, error correction processing, etc.

[0128] (1-3) Data division step: After cleaning the data, the data is divided into a training data set, a validation data set, and a test data set. The training data set is used to train the AI ​​classification model, the validation data set is used to verify the accuracy of the AI ​​classification model or optimize the AI ​​classification model, and the test data set is used to test the performance of the AI ​​classification model.

[0129] (2) Model design or selection stage: Design or select a machine learning algorithm suitable for the AI ​​classification model to be trained.

[0130] (2-1) Regarding image classification algorithms: In the present disclosure, it is necessary to distinguish between images of masks and real faces. Therefore, a CNN (Convolutional Neural Networks) architecture, which is one of the representative algorithms of deep learning, may be selected, such as a MobileNet architecture or an EfficientNet architecture.

[0131] (2-2) Regarding detection and feature extractor algorithms: For example, you can choose VGGNet (a deep convolutional neural network model proposed by the Computer Vision Geometry Group of the University of Oxford), ResNet (a deep neural network model based on residual learning proposed by Microsoft Research), or Transformer (a deep learning model based on the self-attention mechanism proposed by Google), etc.

[0132] (3) Model training phase: Use the training dataset and deep learning architecture to train the model.

[0133] (3-1) Find a suitable loss function for classification to distinguish masks from real faces.

[0134] For classification training, for example, a cross entropy loss function may be used.

[0135] Regarding the training of the detector and feature extractor, for example, an L1 loss function may be used.

[0136] (3-2) Find a suitable optimization function, such as the SGD (Stochastic Gradient Descent) algorithm or the Adam (Adaptive Moment Estimation) algorithm, to reduce the loss.

[0137] (3-3) Other parameters, such as learning rate, batch size, etc., can be set to improve the training process.

[0138] (4) Model optimization and evaluation phase

[0139] (4-1) The validation dataset can be used to verify the accuracy of the model or to optimize the model.

[0140] (4-2) Techniques such as cross-validation and grid search can be used to adjust model parameters to improve model performance.

[0141] (4-3) The test dataset can be used to evaluate the performance of the model such as accuracy, precision, recall, etc.

[0142] (5) Model deployment phase: After completing the training, optimization and evaluation of the model, the model is deployed to the application or device of the biometric system (in this case, the classifier 203).

[0143] When the ROI frame image cropped by the ROI frame cropping unit 202 is input into the classifier 203 , the classifier 203 can distinguish whether the face in the ROI frame image is a mask or a real face based on the deployed AI classification model.

[0144] <2. Example of threshold classification model>

[0145] exist Figure 8 In the classification unit 203 shown, a threshold classification model or a traditional classification model such as SVM (Support Vector Machine) can be deployed. For example, in the example where the classification unit 203 is deployed with a threshold classification model, a threshold for distinguishing whether the part corresponding to the input SWIR image is a mask or a real face can be pre-set based on the threshold classification model. The threshold can be an image brightness threshold. If the input SWIR image has a brightness value lower than the threshold, it is classified as a real face by the threshold classification model; if the input SWIR image has a brightness value above the threshold, it is classified as a mask by the threshold classification model. Of course, the threshold classification model trained for different parts of the organism (such as face, fingers, palms, soles, irises, etc.) can set different corresponding thresholds for distinguishing whether the part belongs to real / fake skin.

[0146] As a variation, more than two brightness thresholds may be set. If the brightness of the input SWIR image is above the highest threshold, it is determined to be fake skin and an alarm is output. If the brightness of the input SWIR image is below the lowest threshold, it is determined to be real skin, and face recognition is performed directly and the recognition result is output. If the brightness of the input SWIR image is between the highest threshold and the lowest threshold or within other intermediate threshold intervals, other features other than brightness (such as facial contours, facial features, textures, colors, etc. extracted from the SWIR image) may be further combined for judgment.

[0147] When the ROI frame image cropped by the ROI frame cropping unit 202 is input into the classifier 203, the classifier 203 can distinguish whether the part corresponding to the image is a mask or a real face by comparing the brightness of the image with the threshold based on the deployed threshold classification model.

[0148] [2. Biometric Authentication System of Second Embodiment]

[0149] Fig. 91 is a block diagram showing an example of the construction of a biometric system according to a second embodiment of the present disclosure. The biometric system 100B according to the second embodiment differs from the biometric system 100A according to the first embodiment in that a prompting unit 50a is further provided. When the resolution result from the resolution unit 20 indicates that the part corresponding to the input SWIR image is fake skin, the prompting unit 50a prompts by generating a specific sound, picture and / or text. For example, the prompting unit 50a may emit an alarm tone and / or may display a picture or text message indicating "fake face" or "not allowed to pass".

[0150] [3. Biometric Authentication System of the Third Embodiment]

[0151] Fig.10 is a block diagram showing a configuration example of a biometrics authentication system according to a third embodiment of the present disclosure.

[0152] like Fig.10 As shown, the biometrics authentication system 100C of the third embodiment differs from the biometrics authentication system 100A of the first embodiment in that a biometric database storage unit 40, a monitor 50b, and a recognition unit 60 are further provided.

[0153] In the third embodiment, when the input SWIR image is identified as fake skin, the identification unit 20 outputs the identification result indicating "fake skin" to the monitor 50b. All or part of the functions of the prompt unit 50a described in the second embodiment can be incorporated into the monitor 50b in the third embodiment. When the input SWIR image is identified as real skin, the identification unit 20 outputs the identification result indicating "real skin" and the SWIR image of the real skin (here, the ROI frame image) to the recognition unit 60.

[0154] When the recognition unit 60 receives the SWIR image output from the resolution unit 20, it recognizes the identity of the creature in the SWIR image based on the pre-stored biological image data template obtained from the biological database storage unit 40, and outputs the recognition result to the monitor 50b for display. At this time, in addition or alternatively, the monitor 50b can emit a prompt sound of "allowed to pass", can display a picture or text information indicating "allowed to pass", "this is the person", etc., or can provide instructions for opening the door, etc.

[0155] In the biological database storage unit 40 , an image of at least one part of at least one biological user is stored in advance in a one-to-one correspondence with the identity (for example, name, etc.) of the user.

[0156] [Configuration Example of Identification Section]

[0157] Fig.11 : is a block diagram showing a configuration example of an identification unit in a biometric identification system according to a third embodiment of the present disclosure. Fig.11 As shown, the recognition unit 60 includes, for example, a preprocessing unit 601, an alignment unit 602, a representation unit 603, and a matching unit 604. The following description continues with the example of face recognition.

[0158] The SWIR image of the area identified as real skin from the identification unit 20 (as mentioned above, at this time it is a ROI frame image containing only the face) is input to the preprocessing unit 601, which performs preprocessing such as noise reduction on the ROI frame image and outputs the ROI frame image obtained after the preprocessing to the alignment unit 602.

[0159] If the information details in the ROI frame image are not obvious enough and thus difficult to identify, then preferably, an image enhancement function may also be performed in the preprocessing unit 601. For example, the image enhancement function may enhance the image details by methods such as sharpening, high-pass filtering, increasing image brightness, or increasing image contrast. As an example, the image enhancement function may be incorporated into the preprocessing unit 601. As another example, the image enhancement function may also be implemented by an image enhancement unit (not shown) disposed between the resolution unit 20 and the preprocessing unit 601 of the recognition unit 60 or between the preprocessing unit 601 and the alignment unit 602.

[0160] The face of the same biological subject may present different postures and expressions in different image sequences, which is not conducive to face recognition. Therefore, the alignment unit 602 is provided to play the role of face alignment, so that the ROI frame images received from the preprocessing unit 601 are transformed into a unified standardized angle and posture. Regarding the algorithm for face alignment, the examples provided here can be, for example, the MTCNN (Multi-task convolutional neural network) algorithm or the LAB (Look at Boundary) algorithm. Fig.12 An example of three network structures for constituting the MTCNN algorithm used when performing face alignment in the recognition unit is shown. The alignment unit 602 can, for example, use the MTCNN algorithm to perform face alignment. Among the three network structures for constituting the MTCNN algorithm, P-Net is a proposal network, R-Net is a refinement network, and O-Net is an output network.

[0161] In the alignment unit 602, the face alignment process, the image conversion process, the feature point selection and positioning process, and the cropping process can be performed in sequence, or the image conversion process, the feature point selection and positioning process, the face alignment process based on the feature points, and the cropping process can be performed in sequence, but the present disclosure is not limited to these two sequences. For example, if the image conversion process and the feature point selection and positioning process have been completed in the resolution unit 20, there is no need to repeat these processes in the alignment unit 602. Here, only the former of these two sequences is taken as an example for specific description. During the face alignment process, for the ROI frame image from the preprocessing unit 601, the alignment unit 602 transforms the face position in the ROI frame image through algorithms such as affine transformation or perspective transformation to align the face; during the image conversion process, the alignment unit 602 converts the ROI frame image after face alignment into a recognizable image format; during the feature point selection and positioning process, the alignment unit 602 selects facial feature points (such as eyebrows, eyes, nose, mouth and facial contours, etc.) from the recognizable ROI frame image by using grayscale image processing technology and marks the position coordinates of these key positioning points; in addition, during the cropping process, the alignment unit 602 further crops the ROI frame image into a face image that only retains the key positioning points.

[0162] Continue to refer to Fig.11 The standardized face image from the alignment unit 602 is input to the representation unit 603, which converts the pixel values ​​of the face image into a compact and discriminable feature vector through feature modeling (e.g., encoding). Ideally, the face of the same biological subject in all cases should be mapped to a similar feature vector.

[0163] After batch preprocessing, a large amount of image data from the biological database storage unit 40 is batch-processed and then subjected to feature extraction to obtain an image data template. In Table 5 described below, as an example of an image data template, a situation is given in which the extracted face feature vectors are stored in a one-to-one correspondence with the biological identity (e.g., name) in the face database. However, the image data template used in the present disclosure is not limited to this example.

[0164] [Table 5] An example of image data template

[0165] Face Database feature Glory [-0.2…] Paul [1.0…] Euan [2.1…] other ……

[0166] Next, the matching unit 604 compares and matches the discernible feature vector from the representation unit 603 with the image data template obtained from the biological database storage unit 40, determines whether the two belong to the same biological subject based on the difference or similarity score, and outputs the recognition result to the monitor 50b for display.

[0167] As some variations, the pre-processing unit 601 and the alignment unit 602 in the third embodiment do not have to be disposed in the recognition unit 60, but may be disposed upstream of the recognition unit 60, or may be omitted depending on the needs of actual application examples.

[0168] Fig.13 The operation flow chart of the face recognition system as an example of the biometric recognition system of the third embodiment of the present disclosure is shown. For example, the operation flow chart includes the following steps (b1)-(b11). Fig.13 In steps (b1)-(b3), the upper side image corresponds to the case of a real human face, and the lower side image corresponds to the case of a human skin mask. The following is a detailed description of each step (b1)-(b11).

[0169] (b1) Image input step: In this step, the user (the organism to be identified, such as a person) places the upper body close to the lens of the SWIR camera, places the face in the viewfinder of the lens and keeps looking directly at the lens. When the blue progress bar is completed and it shows that everything is ready, the user's SWIR image is obtained. At this time, the light used for imaging uses the SWIR spectrum of the incident light.

[0170] (b2) The step of performing face detection to set the ROI frame: performing detection (e.g., manual detection) on the SWIR image obtained in step (b1), and if a face is found, manually setting the ROI frame to include the face, and outputting the user's SWIR image including the ROI frame.

[0171] (b3) Perform a cropping step: crop the SWIR image including the ROI frame obtained in step (b2) into an ROI frame image containing only the face.

[0172] (b4) The step of classifying using a classification model: that is, identifying or judging the ROI frame image obtained in step (b3) to distinguish whether it is a real face or a mask.

[0173] If it is identified as a real face in step (b4), the subsequent steps (b5) to (b11) are continued for identification. On the contrary, if it is identified as a mask in step (b4), the face recognition system emits an alarm prompt sound and outputs a picture or text information used to represent a "fake face", and the subsequent steps are no longer executed. That is, in the face recognition system of the present disclosure, if the input image is identified as a mask through the classification model in step (b4), then the image will not be passed.

[0174] In the classification step of step (b4), AI classification models such as MobileNet-v2, EfficientNet or EfficientNet-Lite can be used for classification, or a threshold classification model can be used for classification, or a traditional classification model such as SVM (Support Vector Machine) can be used for classification.

[0175] (b5) Image enhancement step: that is, for the ROI frame image identified as a real face in step (b4), the details of the ROI frame image are enhanced by methods such as sharpening, high-pass filtering, improving image brightness or increasing image contrast.

[0176] (b6) Perform a transformation step: convert the enhanced ROI frame image in step (b5) into a recognizable image format, select facial feature points in the recognizable image (such as key positioning points of eyebrows, eyes, nose, mouth and facial contours) and mark the position coordinates of these key positioning points.

[0177] (b7) further performing a cropping step: further cropping the ROI frame image obtained in step (b6) into a face image that only retains the key facial positioning points.

[0178] (b8) Performing the step of aligning facial features: that is, for the facial image obtained in step (b7), correcting the posture of the facial image, such as tilt, through geometric transformation (such as affine, rotation, scaling, etc.), so as to obtain a facial image after aligning facial images at different angles according to the same standard (for example, moving eyebrows, eyes, nose, mouth, facial contours, etc. to the same position).

[0179] (b9) performing a characterization step: that is, extracting facial features from the facial image after face alignment in step (b8), thereby converting (e.g., encoding) the pixel values ​​of the facial image into features that can characterize facial features, so as to obtain a string of feature vectors with a fixed length to be identified. Here, in the facial feature extraction process, known architectures such as VGGNet, ResNet, or Transformer can be used.

[0180] (b10) Performing face matching to search for the person: comparing the feature vector to be identified with the face feature vectors that are pre-collected and pre-stored in the face recognition system and correspond to the biological identity, so as to search for the person in the face database. Here, when comparing the feature vectors, the difference between the feature vector to be identified and the pre-stored feature vectors can be calculated. If the difference is lower than a preset threshold, the feature vector is considered to be matched; if the difference is higher than the preset threshold, the feature vector is considered to be mismatched. The lower the preset threshold is set, the lower the possibility of misidentification. It is also possible to use similarity instead of the above-mentioned difference to make a judgment.

[0181] (b11) Step of drawing a conclusion: Based on the above face matching, if the corresponding personal identity information is found in the face database, a conclusion is drawn, for example, "this is that person".

[0182] It should be understood that the order of the conversion process, cropping process, and alignment process in the above steps (b6)-(b8) is not particularly limited, and can be changed or deleted according to the actual recognition method used. For example, if the image conversion process has been completed before the classification step, this process is no longer required in the recognition stage. For example, the positioning of facial key points in the above step (b6) does not have to be completed in step (b6), but can also be completed in step (b3) or step (b5). For example, the image enhancement step is not necessary.

[0183] In addition, there may usually be a liveness detection step between the above steps (b1) to (b4). For example, the user performs a single action or a combination of actions such as blinking, opening the mouth, shaking the head, and nodding according to the prompt tone. In this way, technologies such as facial key point positioning and face tracking are used to verify whether the user performing the action is a real living person.

[0184] Compared with the conventional technology in which face recognition is performed by using an RGB camera, the face recognition system to which this technology is applied can provide more reliable face recognition results.

[0185] [4. Biometric Authentication System of Fourth Embodiment]

[0186] Fig.14 is a block diagram showing a schematic configuration of a biometrics authentication system according to a fourth embodiment of the present disclosure.

[0187] Unlike the biometrics recognition system 100C in the third embodiment of the present disclosure described above, which uses only the SWIR image based on the SWIR spectrum, the biometrics recognition system 100D in the fourth embodiment of the present disclosure is further provided with an RGB camera 80 and a synthesis unit 90. The RGB camera 80 performs imaging based on visible light among incident light to obtain an RGB image. The synthesis unit 90 outputs a synthesized image by using the SWIR image based on the SWIR spectrum and the RGB image based on the visible light spectrum.

[0188] Specifically, in Fig.14 In the biometric recognition system 100D shown, the SWIR camera 10 is used in combination with the RGB camera 80. On the one hand, when the SWIR image acquired by the SWIR camera 10 is identified as real skin by the identification unit 20, the SWIR image is input to the synthesis unit 90. On the other hand, the RGB image acquired by the RGB camera 10 is also input to the synthesis unit 90. The synthesis unit 90 synthesizes the SWIR image and the RGB image by performing a channel superposition operation on the features of interest, and outputs the synthesized image to the recognition unit 60 for recognition.

[0189] As a first modification example of the fourth embodiment of the present disclosure, a depth camera (not shown) or an NIR camera (not shown) may be used instead of the RGB camera 80 .

[0190] Although the fourth embodiment of the present disclosure and its first variation have described an example in which the SWIR camera 10 can be used in combination with any one of the RGB camera, the depth camera, and the NIR camera, as a second variation of the fourth embodiment of the present disclosure, the SWIR camera 10 can be used in combination with any two of the RGB camera, the depth camera, and the NIR camera. In addition, as a third variation of the fourth embodiment of the present disclosure, the SWIR camera 10 can be used in combination with the RGB camera, the depth camera, and the NIR camera.

[0191] It should be noted that the image enhancement function described as a preferred example in the third embodiment can also be applied to the fourth embodiment of the present disclosure and its various variations. Fig.14 For illustration, an image enhancement unit (not shown) may be provided between the synthesis unit 90 and the recognition unit 60, or the image enhancement function may be incorporated into the recognition unit 60 or the synthesis unit 90. The present disclosure is not particularly limited to this as long as the image enhancement process can be performed before the recognition step.

[0192] Fig.15 An operation flowchart of a face recognition system as one example of a biometric recognition system according to a third modification of the fourth embodiment of the present disclosure is shown. Fig.15 Operation flow chart and Fig.13 The difference of the operation flow chart of the third embodiment shown is that face detection and cropping steps for images captured by other special cameras such as RGB camera, depth camera and NIR camera are added, and image synthesis step is added. Fig.13 Similar steps are described in Fig.15 In the face detection and cropping steps in , face detection and ROI frame setting are performed for the SWIR image captured by the SWIR camera, the RGB image captured by the RGB camera, the depth image captured by the depth camera, and the NIR image captured by the NIR camera, respectively, and the ROI frame image is cropped (refer to Fig.13 Steps (b1) to (b3) in Fig.15 In the classification step, a classification model (which can be an AI classification model, a threshold classification model or a traditional classification model) is trained to distinguish whether the creature in the SWIR image is real skin or a mask based on at least the brightness of the SWIR image (refer to Fig.13 If it is identified as a mask in the classification step, an alarm tone is sounded and a picture or text information representing a "fake face" is output; if it is not identified as a mask in the classification step, Fig.15 In the image synthesis step shown, the ROI frame images of the SWIR image, RGB image, depth image and NIR image are synthesized, and the synthesized image is output for recognition. The subsequent recognition process can refer to Fig.13 Steps (b5) to (b11) described in .

[0193] exist Fig.15 In the image synthesis step shown, it is not necessary to synthesize all of the SWIR image, RGB image, depth image and NIR image. The image can be output based on one of the SWIR image, RGB image, depth image, NIR image as needed, or a synthesized image can be output based on any combination of two or more of them.

[0194] The biometric identification method disclosed herein can refer to Fig.13 or Fig.15The method can be implemented by the operation flow chart described in, for example, at least some steps in the operation flow chart can be included. The biometric identification method includes at least the following distinguishing step: wherein, a SWIR image is received, and whether the part corresponding to the SWIR image is real skin or fake skin is distinguished based on at least the brightness of the SWIR image, and the distinguishing result is output. The biometric identification method can also include the following SWIR image acquisition step: wherein, the SWIR image is respectively acquired for each part of at least one part of the organism based on the SWIR spectrum. The biometric identification method can also optionally include the operation steps or features already described in the previous description of the biometric identification system.

[0195] [5. Electronic devices]

[0196] The disclosed technology of distinguishing whether a biological part in a SWIR image is real skin or fake skin based on at least the brightness of the SWIR image can be applied to not only biometric systems and biometric methods, but also electronic devices. For example, the functions of the distinguishing unit, the identifying unit, the synthesizing unit, etc. described above in the biometric system can be implemented in a code manner.

[0197] As an example of an electronic device, it may be a smart phone with a camera function, a face-swiping payment device, an access control device, an ATM cash machine, etc. For example, the electronic device includes: a memory storing a code executable by a computer; and a processor configured to execute the code to implement the following operations: acquiring or receiving a SWIR image, wherein the SWIR image is acquired for each part of at least one part of a biological body based on a SWIR spectrum; and distinguishing whether the part corresponding to the SWIR image is real skin or fake skin based at least on the brightness of the SWIR image, and outputting the distinguishing result.

[0198] The electronic device may include a SWIR camera for collecting the SWIR image, or may be connected to a SWIR camera to receive the SWIR image collected by the SWIR camera. The electronic device may also include or may be connected to other special cameras such as an RGB camera, a depth camera, and a NIR camera.

[0199] The electronic device may be configured to further include at least one of a display, a speaker, a microphone, a communication module, a sensor, a touch panel, and a controller, and these components as well as the above-mentioned memory, processor, etc. may be connected to each other via a bus.

[0200] When the processor executes the code, the controller can function as an application processing unit and an operating system processing unit.

[0201] The display can display an operation screen to execute the processing performed by the application processing unit and the operating system processing unit, and the touch panel can obtain the touch operation performed by the user on the operation screen displayed on the display. In addition, the display can also display images, data, and various processing results processed by the electronic device.

[0202] The speaker and microphone can perform voice output and voice collection.

[0203] The communication module can perform network communication via a communication network (such as the Internet, a public telephone line network, etc.), a long-distance communication network (such as the so-called 4G lines and 5G lines for wireless mobile bodies, WAN (wide area network), or LAN (local area network), etc.), or short-distance wireless communication (such as Bluetooth (TM), or NFC (near field communication)).

[0204] [6. Application examples]

[0205] The technology disclosed in the present invention relates to the principle of distinguishing between real skin and counterfeit materials used to make fake skin. Therefore, in addition to being applicable to face recognition technology, the technology can also be applied to technologies for distinguishing fake fingerprints, fake palm prints, fake foot prints or fake irises, etc. In addition, the technology can also be applied to any other fields that require identification based on SWIR images collected by a SWIR camera.

[0206] The biometric recognition system disclosed in the present invention has the following advantages in application examples compared to the recognition system using RGB cameras because it uses a SWIR camera:

[0207] ·Enables enhanced material detection: SWIR cameras can detect chemicals (such as moisture, chemical components, etc.) that are not visible in the RGB spectrum. Different substances have different characteristics in absorbing SWIR light. When photographed with a SWIR camera, their images will show different colors. Therefore, using a SWIR camera can achieve the distinction between powders of the same color but different types (salt, sugar, MSG, etc.), liquids (water, oil, etc.), and plastics, for example, it can be used in impurity sorting applications.

[0208] Improved visibility in low light: SWIR cameras perform better in low light conditions than RGB cameras, thus providing clear images in all lighting conditions.

[0209] Ability to see through obstructions: SWIR cameras can see through obstructions such as fog, smoke, haze, dust, plastic bags or glass by using the SWIR spectrum, which is difficult for RGB cameras to achieve.

[0210] Ability to inspect food defects based on moisture content: Moisture in food (e.g., damp bread, bruised fruit, crops with pests) absorbs SWIR light, causing the image to darken. Different moisture contents cause different degrees of darkening. Therefore, by using a SWIR camera to measure the moisture content in food, it is possible to detect whether the food is a defective product or to classify agricultural products.

[0211] · Capable of non-invasive imaging: SWIR cameras can perform imaging through the surface of the object without invading it, without causing damage. Therefore, SWIR cameras are very suitable for detecting internal defects of wafers and the position of alignment keys in industry. In addition, SWIR cameras are also very suitable for medical and biological applications, such as medical and aesthetic material testing, as well as non-invasive diagnosis and imaging in dermatology and vascular research.

[0212] It should be noted that the effects described herein are merely illustrative and not restrictive. The present technology may additionally provide effects other than those described herein.

[0213] It should be understood by those skilled in the art that the embodiments of the present technology are not limited to those described above. Various changes and combinations may be made without departing from the spirit and scope of the present technology.

[0214] It should be noted that the present technology can also adopt the following technical solutions.

[0215] (1) A biometric recognition system comprising a SWIR camera or connected to a SWIR camera and comprising a resolution unit, wherein:

[0216] The SWIR camera collects SWIR images for each of at least one part of the biological body based on the SWIR spectrum, and

[0217] The distinguishing unit receives the SWIR image acquired by the SWIR camera, distinguishes whether the part corresponding to the SWIR image is real skin or fake skin based at least on the brightness of the SWIR image, and outputs the distinguishing result.

[0218] (2) A biometric authentication system according to (1) above, wherein:

[0219] The distinguishing unit distinguishes whether the portion corresponding to the SWIR image is real skin or fake skin by means of a classification model.

[0220] (3) The biometric authentication system according to (2) above, wherein:

[0221] The classification model is an AI classification model deployed in the discrimination unit after training, and is capable of discerning whether the part corresponding to the SWIR image is real skin or fake skin based on the brightness of the SWIR image and features extracted from the SWIR image.

[0222] (4) The biometric authentication system according to (2) above, wherein:

[0223] The classification model is a threshold classification model deployed in the discrimination unit after training, and discriminates whether the part corresponding to the SWIR image is real skin or fake skin by comparing the brightness of the SWIR image with a preset threshold.

[0224] (5) The biometric authentication system according to (4) above, wherein:

[0225] When the brightness of the SWIR image is equal to or higher than the threshold, the discrimination unit outputs a discrimination result indicating that the part is fake skin, and

[0226] When the brightness of the SWIR image is lower than the threshold, the discrimination unit outputs a discrimination result indicating that the portion is real skin.

[0227] (6) The biometric recognition system according to any one of (1) to (5) above, further comprising:

[0228] The prompting unit is configured to provide a prompt by generating a specific sound, picture and / or text when the identification result indicates that the part is fake skin.

[0229] (7) The biometric recognition system according to any one of (1) to (6) above, further comprising:

[0230] The identification unit outputs the SWIR image of the part to the identification unit when the identification result indicates that the part is real skin.

[0231] The recognition unit recognizes the identity of the living being in the SWIR image based on image data including at least the SWIR image.

[0232] (8) The biometric authentication system according to (7) above, wherein:

[0233] The recognition unit recognizes the identity of the living being in the SWIR image by matching image data including at least the SWIR image with a pre-stored image data template corresponding to the identity of the living being.

[0234] (9) The biometric recognition system according to (7) or (8) above, further comprising:

[0235] An image enhancement unit performs image enhancement processing on the SWIR image before the recognition unit recognizes the identity of the living being in the SWIR image.

[0236] (10) A biometric authentication system according to any one of (1) to (9) above, wherein:

[0237] The SWIR camera acquires the SWIR image by using a light source array for generating uniform light as a light source and / or by using an automatic exposure on mode, and / or

[0238] The classification unit performs classification based on brightness and depth information of the SWIR image.

[0239] (11) The biometric authentication system according to any one of (1) to (10) above, further comprising:

[0240] The light source is a halogen lamp.

[0241] (12) The biometric authentication system according to (11) above, wherein:

[0242] A plurality of the halogen lamps are arranged to form a light source array for generating uniform light.

[0243] (13) The biometric recognition system according to any one of (1) to (12) above, further comprising:

[0244] The SWIR bandpass filter is disposed inside the SWIR camera, or is detachably disposed outside the SWIR camera and located between the lens of the SWIR camera and the at least one part of the organism.

[0245] (14) A biometric authentication system according to (13) above, wherein:

[0246] The spectral range of the SWIR bandpass filter is 1400nm to 1600nm.

[0247] (15) The biometric authentication system according to (13) or (14), wherein:

[0248] The spectral range of the SWIR bandpass filter is 1400nm to 1550nm.

[0249] (16) A biometric authentication system according to any one of (13) to (15) above, wherein:

[0250] The spectral range of the SWIR bandpass filter is 1400nm to 1500nm.

[0251] (17) A biometric authentication system according to any one of (13) to (16) above, wherein:

[0252] The spectral range of the SWIR bandpass filter is approximately 1450 nm.

[0253] (18) A biometric authentication system according to any one of (1) to (17) above, wherein:

[0254] The at least one part is at least one of a face, a fingerprint, a foot print, a palm print, and an iris, and

[0255] The fake skin is at least one of a corresponding mask, a fingerprint film, a foot print film, a palm print film, and an iris film.

[0256] (19) A biometric authentication system according to (18) above, wherein:

[0257] When the at least one part is a face, the distinguishing unit performs the following operations:

[0258] performing face detection on the SWIR image from the SWIR camera;

[0259] Setting a bounding box containing a face in the SWIR image based on the face detection;

[0260] cropping the SWIR image to retain the image within the bounding box; and

[0261] The image within the bounding box is resolved.

[0262] (20) The biometric recognition system according to (19) above, further comprising:

[0263] The recognition unit outputs the image within the boundary box to the recognition unit when the recognition result indicates that the part is real skin.

[0264] The recognition unit recognizes the identity of the creature in the image within the bounding box by sequentially performing image enhancement processing, face alignment processing, face characterization processing and face matching processing on the image within the bounding box.

[0265] (21) A biometric authentication system according to any one of (1) to (20) above, wherein:

[0266] The biometric recognition system also includes or is further connected to:

[0267] at least one of an RGB camera, a depth camera, and a NIR camera, which respectively collects an RGB image, a depth image, and a NIR image for each part of the organism based on a corresponding one of a visible light spectrum, distance information, and a NIR spectrum, and

[0268] The biometric recognition system further comprises:

[0269] A synthesis unit, when the discrimination result indicates that the part is real skin, synthesizes the SWIR image with at least one of the RGB image, the depth image and the NIR image, and outputs the synthesized image for identification.

[0270] (22) Electronic devices, including:

[0271] a memory storing code executable by a computer; and

[0272] A processor is configured to execute the code to implement the following operations:

[0273] receiving SWIR images, wherein the SWIR images are respectively acquired for each of at least one part of the biological body based on the SWIR spectrum; and

[0274] Whether the part corresponding to the SWIR image is real skin or fake skin is distinguished based at least on the brightness of the SWIR image, and the distinguished result is output.

[0275] (23) The electronic device according to (22) above, wherein:

[0276] The wavelength range of the SWIR spectrum is 1400nm to 1600nm.

[0277] (24) The electronic device according to (22) or (23), wherein:

[0278] The SWIR image is acquired by a SWIR camera, and

[0279] The electronic device is connected to the SWIR camera.

[0280] (25) Biometric identification methods, including:

[0281] SWIR image acquisition step: that is, acquiring a SWIR image for each of at least one part of the biological body based on the SWIR spectrum; and

[0282] Discrimination step: that is, receiving the acquired SWIR image, distinguishing whether the part corresponding to the SWIR image is real skin or fake skin based at least on the brightness of the SWIR image, and outputting the distinction result.

[0283] (26) The biometric identification method according to (25) above, further comprising:

[0284] A prompting step, when the identification result shows that the part is fake skin, a prompt is given with a specific sound, picture and / or text in the prompting step.

[0285] (27) The biometric identification method according to (25) or (26) above, further comprising:

[0286] Identification step: That is, when the discrimination result indicates that the part is real skin, the identity of the living being in the SWIR image is identified based on image data including at least the SWIR image in the identification step.

[0287] (28) A biometric authentication method according to any one of (25) to (27) above, wherein:

[0288] In the SWIR image acquisition step, a light source array for generating uniform light is used as the light source and / or an automatic exposure on mode is used, and / or

[0289] In the distinguishing step, the distinguishing is performed based on the brightness and depth information of the SWIR image.

[0290] (29) The biometric identification method according to (27) or (28) above, further comprising:

[0291] Image enhancement step: that is, before performing the recognition in the recognition step, image enhancement processing is performed on the SWIR image.

[0292] (30) A biometric authentication method according to any one of (25) to (29) above, wherein:

[0293] The wavelength range of the SWIR spectrum is 1400nm to 1600nm.

[0294] (31) The biometric identification method according to any one of (25) to (30) above, further comprising:

[0295] A step of collecting RGB images, depth images and NIR images for each part of the organism based on at least one of a visible light spectrum, distance information and a NIR spectrum; and

[0296] Synthesis step: that is, when the discrimination result indicates that the part is real skin, the SWIR image is synthesized with at least one of the RGB image, the depth image and the NIR image in the synthesis step, and the synthesized image is output for identification.

[0297] [Explanation of Reference Numerals]

[0298] 1: Subject

[0299] 2: Light source

[0300] 100A, 100B, 100C, 100D: Biometric systems

[0301] 10: SWIR Camera

[0302] 101: Optical components

[0303] 102: Imaging element (sensor part)

[0304] 103: Signal Processing Unit

[0305] 105: Display unit

[0306] 106: Storage

[0307] 20: Discrimination

[0308] 201: ROI frame setting unit (face detection unit)

[0309] 202: ROI frame cropping

[0310] 203: Classification Department

[0311] 40: Biological database storage department

[0312] 50a: Prompt

[0313] 50b: Monitor

[0314] 60: Identification Department

[0315] 601: Pre-processing department

[0316] 602: Alignment

[0317] 603: Representation Department

[0318] 604: Matching section

[0319] 80: RGB Camera

[0320] 90: Synthesis Department

Claims

1. A biometric recognition system comprising a SWIR camera or connected to a SWIR camera and comprising a resolution unit, wherein: The SWIR camera collects SWIR images for each of at least one part of the biological body based on the SWIR spectrum, and The distinguishing unit receives the SWIR image acquired by the SWIR camera, distinguishes whether the part corresponding to the SWIR image is real skin or fake skin based at least on the brightness of the SWIR image, and outputs the distinguishing result.

2. The biometric identification system according to claim 1, wherein: The distinguishing unit distinguishes whether the portion corresponding to the SWIR image is real skin or fake skin by means of a classification model.

3. The biometric identification system according to claim 1, further comprising: The prompting unit is configured to provide a prompt by generating a specific sound, picture and / or text when the identification result indicates that the part is fake skin.

4. The biometric identification system according to claim 1, further comprising: The identification unit outputs the SWIR image of the part to the identification unit when the identification result indicates that the part is real skin. The recognition unit recognizes the identity of the living being in the SWIR image based on image data including at least the SWIR image.

5. The biometric identification system according to claim 1, wherein: The SWIR camera acquires the SWIR image by using a light source array for generating uniform light as a light source and / or by using an automatic exposure on mode, and / or The classification unit performs classification based on brightness and depth information of the SWIR image.

6. The biometric identification system according to claim 1, further comprising: The SWIR bandpass filter is disposed inside the SWIR camera, or is detachably disposed outside the SWIR camera and located between the lens of the SWIR camera and the at least one part of the organism.

7. The biometric identification system according to claim 6, wherein: The spectral range of the SWIR bandpass filter is 1400nm to 1600nm.

8. The biometric recognition system according to any one of claims 1 to 7, wherein: The biometric recognition system also includes or is further connected to: at least one of an RGB camera, a depth camera, and a NIR camera, which respectively collects an RGB image, a depth image, and a NIR image for each part of the organism based on a corresponding one of a visible light spectrum, distance information, and a NIR spectrum, and The biometric recognition system further comprises: A synthesis unit, when the discrimination result indicates that the part is real skin, synthesizes the SWIR image with at least one of the RGB image, the depth image and the NIR image, and outputs the synthesized image for identification.

9. Electronic devices, including: A memory storing a code executable by a computer; and A processor is configured to execute the code to implement the following operations: receiving SWIR images, wherein the SWIR images are respectively acquired for each of at least one part of the biological body based on the SWIR spectrum; and Based at least on the brightness of the SWIR image, it is determined whether the portion corresponding to the SWIR image is real skin or fake skin, and the determination result is output.

10. Biometric identification methods, including: SWIR image acquisition step: that is, acquiring SWIR images for each part of at least one part of the organism based on the SWIR spectrum; and Discrimination step: that is, receiving the acquired SWIR image, distinguishing whether the part corresponding to the SWIR image is real skin or fake skin based at least on the brightness of the SWIR image, and outputting the distinction result.

Citation Information

Patent Citations

  • False image detection

    CN115376212A

  • False image detection

    CN115393920A

  • Spoof detection using proximity sensors

    US20170257770A1

Cited By

  • Biometric identification system, biometric identification method and electronic device

    WO2025185384A1