Electronic devices with facial recognition capabilities

Abstract

An auxiliary filtering device for face recognition is provided. This auxiliary filtering device is used to exclude ineligible objects to be identified based on the relative relationship between object distance and image size, changes in the image over time, and / or feature differences between images acquired by different cameras, so as to avoid the possibility of cracking face recognition using photos or videos.

Description

[0001] This application is a divisional application of Chinese invention patent application No. 201810778568.4, filed on July 16, 2018, entitled "Auxiliary Filtering Device for Face Recognition and Activation Method of Electronic Device". Technical Field

[0002] This invention relates to a face recognition system, and more particularly to an auxiliary filtering device and electronic device activation method for excluding ineligible individuals before or after identity verification using face recognition. Background Technology

[0003] With the advancement of digital electronic devices, various types of data are increasingly stored digitally. Therefore, the protection of digital data has become a crucial issue. Electronic security locks using biometrics are widely applied in various electronic devices, including those employing fingerprint, iris, facial, and voiceprint recognition. Biometrics typically utilizes machine learning for identity verification to achieve personal security locks, effectively enhancing the protection of personal information.

[0004] However, using facial recognition in biometrics for electronic security locks still carries the risk of being hacked. For example, using a photo or video of the identified person could still unlock the lock, exposing personal information to risk. Of course, in some cases, multiple biometric methods can be combined to increase protection, such as fingerprint and facial recognition, but this would increase the product cost.

[0005] In light of this, the industry needs a method that can avoid using photos or videos to crack identity verification methods that rely on facial recognition. Summary of the Invention

[0006] The present invention provides an auxiliary filtering device for facial recognition and a method for activating an electronic device, which is used to perform an additional filtering procedure in addition to using facial recognition for identity identification, thereby excluding unqualified individuals to be identified, thus preventing the use of photos or videos to crack facial recognition-based electronic security locks.

[0007] This invention provides an electronic device comprising a first camera, a second camera, and a processor. The first camera is used to acquire a first image. The second camera is used to acquire a second image. The processor is used to calculate the face position and face size in the first image and the second image respectively, and compares the calculated first face size at the first face position in the first image or the second face size at the second face position in the second image with a relative preset face size range to exclude ineligible objects, wherein the number of pixels in the second camera is less than the number of pixels in the first camera.

[0008] The present invention also provides an electronic device comprising a first camera, a second camera, and a processor. The first camera is used to acquire a first image at a first time. The second camera is used to acquire a second image at the first time. The processor is used to determine that the face is a qualified object when it is determined that the first image and the second image contain different facial features and facial characteristics of the same face, wherein the number of pixels in the second camera is less than the number of pixels in the first camera.

[0009] The auxiliary filtering device of the present invention can operate before or after identity verification using facial recognition to increase the protection capability of electronic security locks.

[0010] The auxiliary filtering device of this invention does not require calculating the depth of every point on the face to create a three-dimensional image during the filtering process. In the subsequent face recognition process, a three-dimensional image can be created according to the algorithm used for identity verification.

[0011] To make the above and other objects, features and advantages of the present invention more apparent, a detailed description will be provided below with reference to the accompanying drawings. Furthermore, in the description of the present invention, the same components are represented by the same reference numerals, which will be stated first. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of an electronic device using the auxiliary filtering device for face recognition according to the present invention.

[0013] Figure 2 This is a block diagram of an auxiliary filtering device for face recognition according to one embodiment of the present invention.

[0014] Figures 3A-3C This is a schematic diagram illustrating the operation of the auxiliary filtering device for face recognition according to an embodiment of the present invention.

[0015] Figure 4 This is a schematic diagram of the operation of an auxiliary filtering device for face recognition according to another embodiment of the present invention.

[0016] Figure 5 This is a schematic diagram of the operation of an auxiliary filtering device for face recognition according to another embodiment of the present invention.

[0017] Figure 6-7 This is a flowchart illustrating a startup method for an electronic device according to certain embodiments of the present invention.

[0018] Explanation of reference numerals in the attached figures

[0019] 10 Electronic devices

[0020] 11 First Camera

[0021] 12 Second Camera

[0022] 13 screens

[0023] 15 processors

[0024] 17 Memory Detailed Implementation

[0025] This invention can be applied to electronic security locks in electronic devices that utilize facial recognition as an identity verification algorithm for unlocking. Before or after identity verification, this invention employs an additional auxiliary filtering device to filter and exclude ineligible individuals, thereby preventing the electronic security lock from being cracked by unauthorized individuals using photos or videos of correct faces, thus enhancing its protection capabilities.

[0026] The exclusion of ineligible individuals described in this invention means that even if the identity recognition result is correct, the electronic device is still not turned on because the auxiliary filtering device determines that the face may be a photograph or video. Not turning on, for example, means not turning on the screen, but only turning on the camera and the processor executing the filtering algorithm.

[0027] Please refer to Figure 1 This is a schematic diagram of an electronic device 10 using the facial recognition auxiliary filtering device according to an embodiment of the present invention. In this embodiment, the electronic device 10 is described using a mobile phone as an example, but the present invention is not limited thereto. The electronic device 10 can be any device using an electronic security lock, such as a notebook computer, tablet computer, personal digital assistant (PDA), monitoring system, workstation, vehicle control system, etc.

[0028] Taking a mobile phone as an example, the electronic device 10 includes a first camera 11 and a second camera 12. Figure 1 The positions of the first camera 11 and the second camera 12 shown are merely illustrative and are not intended to limit the invention. The first camera 11 and the second camera 12 are preferably positioned horizontally relative to the user. In other embodiments (e.g., using position and size filtering), the first camera 11 and the second camera 12 may also be positioned vertically relative to the user or on a diagonal line.

[0029] In one embodiment, the first camera 11 is the front camera of the mobile phone, such as a camera located on the side of the screen 13. Furthermore, the second camera 12 is preferably configured to be always on, and the first camera 11 is only activated when the second camera 12 detects a face. Therefore, to save power, the number of pixels in the second camera 12 (e.g., less than 640×480) is much smaller than the number of pixels in the first camera 11 (e.g., more than 1920×1080). However, in other embodiments, if system power consumption is not a significant consideration, the second camera 12 may have the same number of pixels as the first camera 11. The number of pixels refers to the number of pixels in the pixel arrays comprised by the first camera 11 and the second camera 12.

[0030] Please refer to Figure 2 The diagram shown is a block diagram of an auxiliary filtering device for face recognition according to an embodiment of the present invention, including a first camera 11, a second camera 12, a processor 15, and memory 17. It must be noted that, although... Figure 2 The auxiliary filtration device and the electronic device 10 are represented by different functional blocks, which are for illustrative purposes only and not intended to limit the invention. The auxiliary filtration device may be included in the electronic device 10 or be a separate device located outside the electronic device 10, without any particular limitation.

[0031] The first camera 11 includes, for example, a solid-state image sensor for receiving its field of view (...). Figure 1 The first image IF1 is obtained by receiving light from the area in front of the screen 13 of the electronic device 10 (displayed as the area in front of the screen 13). The second camera 12 includes, for example, a solid-state image sensor for receiving light from its field of view (the area in front of the screen 13 of the electronic device 10). Figure 1 The second image IF2 is obtained by capturing light (also shown as the area in front of the screen 13 of the electronic device 10). The second camera 12 is a separate sensor, different from the proximity sensor. As mentioned above, the first camera 11 and the second camera 12 may have the same or different number of pixels, so the first image IF1 and the second image IF2 may have the same or different sizes.

[0032] The processor 15 is electrically coupled to the first camera 11 and the second camera 12 via a bus or signal line to receive the first image IF1 and the second image IF2 respectively, and performs post-processing such as face recognition and object filtering (detailed examples are described below). The processor 15 also controls the first camera 11 and the second camera 12 to acquire images. The processor 15 may include, for example, a digital signal processor (DSP), a microprocessor (MCU), a central processing unit (CPU), or an application-specific integrated circuit (ASIC), and the post-processing can be implemented by software, hardware, firmware, or a combination thereof, depending on the application.

[0033] The memory 17 includes, for example, volatile or non-volatile memory, for pre-storing algorithms for face detection, identity recognition based on face recognition, object filtering, parameters used in the calculation process, and the correspondence between object positions and object sizes in the image (detailed examples are described later). These parameters are, for example, face detection, identity recognition, and object filtering parameters obtained through machine learning. After receiving the first image IF1 and the second image IF2, the processor 15 performs face detection, identity recognition, and object filtering operations using the built-in algorithms and by accessing the parameters in memory 17.

[0034] In this invention, the processor 15 can use known algorithms to perform face detection and identity recognition without specific limitations. The invention adds an object filtering function before or after the identity recognition process to exclude ineligible objects. In this invention, ineligible objects refer to faces in photographs or videos, which may represent correct or incorrect identities.

[0035] Please refer to Figures 3A-3C The diagram shown is an operational schematic of the auxiliary filtering device for face recognition according to an embodiment of the present invention. It is assumed that when the face is at a first distance D1 from the electronic device 10 (e.g., ... Figure 3A As shown), the first image IF acquired by the first camera 11 11 Includes the first face O 11 The second image IF acquired by the second camera 12 21 Includes second face O 21 (like Figure 3B As shown). Assume that when the face is at a second distance D2 from the electronic device 10 (e.g., ... Figure 3A As shown), the first image IF acquired by the first camera 11 12 Includes the first face O 12 The second image IF acquired by the second camera 12 22 Includes second face O 22 (like Figure 3C (As shown).

[0036] The processor 15 then calculates the first image IF1 (e.g., IF...). 11 IF 12 ) and the second image IF2 (e.g., IF 21 IF 22 The system calculates the face position and size in memory () and compares the results with the corresponding positions and sizes stored in memory (17) to exclude ineligible objects. For example... Figure 3B In the process, when processor 15 calculates the first image IF 11 The position of the face in the image is at a distance d. 11 (corresponding distance D1) and face size A 11 At that time, the processor 15 compares the memory 17 with the memory at a distance of d. 11 The corresponding preset face size (e.g., size range) when face size A 11 When the first face O is within the preset size range, 11 (i.e., the current face) is the eligible object. Similarly, when processor 15 calculates the second image IF 21 The position of the face in the image is at a distance d. 21 (corresponding distance D2) and face size A 21 At that time, processor 15 compares the face size A 21 Is it within the distance d specified in memory 17? 21 Within the corresponding preset size range, the second face O is determined. 21 Whether (the current face) is a suitable candidate. Regarding... Figure 3C The judgment method is similar Figure 3B Therefore, it will not be elaborated further here. Processor 15 preferably compares the first image IF1 (e.g., IF...) simultaneously. 11 IF 12 ) and the second image IF2 (e.g., IF 21 IF 22 Does the relationship between the face position and face size conform to the preset correspondence?

[0037] The memory 17 preferably pre-stores a range of object sizes relative to different object locations for comparison by the processor 15. It must be noted that, although... Figure 3B-3C The text only displays the distance along the left and right directions in the graph (e.g., d). 11 d 12 d 21 d 22The image shown is a one-dimensional location of a face, used only for simplification. The face location in this invention can also include a two-dimensional location (e.g., distance from the top or bottom of the image), each two-dimensional location having a corresponding size or size range and stored in memory 17. The relative relationships stored in memory 17 can be set before shipment by actually measuring multiple faces. Therefore, when attempting to unlock using a photo or video, the electronic security lock cannot be successfully unlocked because the relative relationships cannot be matched.

[0038] In this invention, after the auxiliary filtering device completes the identification of ineligible objects and determines that the current face is not from a photograph or video (i.e., does not belong to the ineligible object category), the processor 15 further performs identity recognition based on the first image IF1 through face recognition. In some embodiments, the processor 15 may also first perform identity recognition based on the first image IF1 through face recognition, and only use the auxiliary filtering device to identify ineligible objects after the identity is correctly identified; in this embodiment, if the identity is incorrect, the auxiliary filtering device will no longer operate. In other words, the processor 15 itself has the function of performing identity recognition based on the first image IF1. In this invention, the processor 15 further utilizes the first image IF1 and the second image IF2 to filter non-living objects to improve security.

[0039] Furthermore, to further enhance security, the processor 15 can further filter for non-living objects based on multiple first images IF1 and / or second images IF2. For example, see [reference needed]. Figure 4 As shown, it is an operational schematic diagram of an auxiliary filtering device for face recognition according to another embodiment of the present invention. Figure 4 The processor 15 displays two images acquired by the first camera 11 and / or the second camera 12 at two different times t1 and t2, as a first image IF1 and / or a second image IF2. The processor 15 also excludes unsuitable subjects based on multiple first images IF1 acquired by the first camera 11 at different times. When there is no difference in expression between the multiple first images IF1 (e.g., images at different times t1 and t2), the processor 15 determines that the current face in the first image IF1 is an unsuitable subject (because the photo does not produce expression changes); conversely, when there is a difference in expression between the multiple first images IF1 (e.g., images at different times t1 and t2), the processor 15 determines that the current face in the first image IF1 is a suitable subject. Figure 4 The differences in facial expressions include open / closed eyes, smiling / not smiling, etc., but this invention is not limited to these. The processor 15 can use known algorithms to determine facial expressions. It must be noted that... Figure 4 The two images displayed can be two of multiple consecutive images, but they do not necessarily have to be two images acquired at consecutive sampling times.

[0040] Similarly, the processor 15 also performs liveness detection based on multiple second images IF2 acquired by the second camera 12 at different times. To avoid using consecutive images for unlocking, this embodiment is preferably used in conjunction with the relative relationship between position and size in the previous embodiment.

[0041] Please refer to Figure 5 The diagram shown is an operational schematic of an auxiliary filtering device for face recognition according to another embodiment of the present invention. In this embodiment, the processor 15 further excludes ineligible objects based on the image difference between a first image IF1 acquired by the first camera 11 and a second image IF2 acquired by the second camera 12.

[0042] For example, the first camera 11 acquires a first image at a first time t and the second camera 12 acquires a second image at the first time t, according to Figure 1 The configuration is such that the first camera 11 is used to acquire the image of the left half of the face, therefore Figure 5 The first image in IF L This indicates that, because the second camera 12 is used to acquire images of the right half of the face, therefore Figure 5 The second image in IF R It is understood that the portion of the face captured by the first camera 11 and the second camera 12 is determined by their configuration positions and is not limited to the examples of this invention. Furthermore, although... Figure 5 This indicates that the first camera 11 and the second camera 12 acquire images at substantially the same time t, but the present invention is not limited thereto. In other embodiments, the first camera 11 and the second camera 12 may acquire images IF at different times. L IF R There are no specific restrictions.

[0043] The processor 15 calculates the first image IF. L The first facial feature (e.g., left ear) in the image is used to calculate the IF of the second image. R The second facial feature (e.g., the right ear) in the second image is used when the first facial feature is not included in the second image IF. R Furthermore, the second facial feature is not included in the first image IF. L When, determine the first image IF L and the second image IF R The faces in the image are eligible objects.

[0044] In other words, if photos and videos are used for unlocking, the first camera 11 and the second camera 12 will capture the same facial features and characteristics. Therefore, the first image IF can be used... L and the second image IF RThe images contain different facial features and characteristics of the same person to identify a living individual. It should be noted that the differences in the images are not limited to the ears described in this description, but may also include other facial features such as facial angles, birthmarks, and contours.

[0045] The three methods described above for filtering ineligible objects can be used in combination to enhance security.

[0046] Please refer to Figure 6 The diagram shows a flowchart of a startup method for an electronic device according to an embodiment of the present invention. This startup method is applicable to... Figure 1-2 The electronic device 10 includes a first camera 11, a second camera 12, a memory 17, and a processor 15. The startup method includes the following steps: always keeping the second camera on to acquire a second image and keeping the first camera off (step S61); using the processor to perform face detection based on the second image (step S62); only turning on the first camera to acquire a first image when a face is detected (step S63); using the processor to determine an ineligible object based on the first image and the second image (step S64); when the face in the second image does not belong to the ineligible object, determining whether to turn on the electronic device through face recognition (steps S65-S66).

[0047] Step S61:

[0048] In this invention, the second camera 12 is preferably always on to acquire the second image IF2 at a sampling frequency. During this time, apart from the continuous operation of the second camera 12 and the processor 15 performing the face detection algorithm, other components of the electronic device 10 are preferably in sleep mode to reduce power consumption. That is, the first camera 11 is off until the second camera 12 detects a face. "Always on" here means that the second camera 12 and the processor 15 are on and operating as long as they have access to power.

[0049] Step S62:

[0050] The processor 15 then uses its built-in face detection algorithm to determine whether the acquired second image IF2 contains a face. It must be noted that in this step, the processor 15 does not identify the identity of the face, but only whether the object contained in the second image IF2 is a face. The face detection algorithm can use any known algorithm, as long as it can detect a face.

[0051] Step S63:

[0052] The processor 15 activates the first camera 11 to acquire the first image IF1 only when it determines that the second image IF2 contains a face. As mentioned earlier, this face detection step does not involve identity recognition. Preferably, the first camera 11 acquires images synchronously with the second camera 12, but this is not a limitation. In other embodiments, the first camera 11 and the second camera 12 can also acquire images in a time-sharing manner.

[0053] Step S64:

[0054] Next, the processor 15 can, according to the above... Figures 3A-3C Methods 4 and 5 are used to determine whether the current face is a qualified object.

[0055] For example Figures 3A-3C In the process, processor 15 calculates the face positions (e.g., d) in the first image IF1 and the second image IF2, respectively. 11 d 12 d 21 d 22 ) and face size (e.g., A) 11 A 12 A 21 A 22 The system calculates the results and compares them with the corresponding information in memory 17 to determine whether the current face is an ineligible object; wherein, memory 17 pre-stores the corresponding relationship between the position of an object and the size of the object in the image.

[0056] As mentioned earlier, in order to increase filtration capacity, Figures 3A-3C It can also be paired with Figure 4 The filtering method involves the processor 15 determining the ineligible object based on the facial expression differences between multiple first images IF1 acquired by the first camera 11 at different times (e.g., t1, t2) or multiple second images IF2 acquired by the second camera 12 at different times (e.g., t1, t2). As mentioned earlier, the processor 15 can use known facial expression recognition algorithms for facial expression recognition.

[0057] In another embodiment, the processor 15 calculates a first face feature in the first image IF1 and a second face feature in the second image IF2. When the first face feature is not included in the second image IF2 and the second face feature is not included in the first image IF1, the processor 15 determines that the face in the second image IF2 does not belong to the ineligible object. Figure 5 As shown.

[0058] Other implementation methods. Figure 5 It can also be paired with Figure 4 The filtering method has already been explained, so it will not be repeated here.

[0059] Step S65:

[0060] When the processor 15 determines that the current face in the first image IF1 and / or the second image IF2 is a live person (i.e., not an ineligible object), the processor 15 performs identity recognition through face recognition. When the number of pixels in the first image IF1 is much greater than the number of pixels in the second image IF2, the processor 15 preferably uses the first image IF1 for face recognition. However, when the number of pixels in the second image IF2 is sufficient for face recognition, the processor 15 may also use the second image IF2 for face recognition.

[0061] Step S66:

[0062] Finally, when the processor 15 recognizes the current face as a registered face (i.e., the correct identity), it activates the electronic device 10, such as turning on the screen 13 and other components, allowing the user to control the operation. In this invention, due to the additional step of filtering ineligible objects (e.g., step S64), the electronic security lock can be effectively prevented from being cracked by photos or videos. As long as an ineligible object is confirmed, the electronic device 10 will not be activated.

[0063] Please refer to Figure 7 This is a flowchart of an electronic device startup method according to another embodiment of the present invention. Figure 7 and Figure 6 The difference lies in the reversed order of filtering ineligible objects and performing face recognition. That is, the face recognition described in this invention can be performed before or after judging the ineligible object. Figure 7 The detailed implementation methods for each step are the same as those for the following steps. Figure 6 Therefore, I will not elaborate further here.

[0064] In some embodiments, the auxiliary filtering device may further include a light source to provide the light required for the first camera 11 and the second camera 12 to acquire images.

[0065] In summary, known face recognition methods cannot distinguish between facial images in photos or videos and real faces, thus posing a risk of reduced data security. Therefore, this invention also provides an auxiliary filtering device for face recognition. Figures 1 to 2 ) and the method of starting up electronic devices ( Figures 6 to 7 This technology, by performing liveness detection in addition to identity verification, helps improve data security by confirming that the face being viewed is not the one in the photo or video.

[0066] While the present invention has been disclosed through the foregoing examples, it is not intended to limit the invention. Any person skilled in the art to which this invention pertains can make various modifications and alterations without departing from the spirit and scope of the invention. Therefore, the scope of protection of this invention shall be determined by the appended claims.

Claims

1. An electronic device comprising: A first camera, used to acquire a first image; A second camera, used to acquire a second image; as well as The processor is configured to calculate the face position and face size in the first image and the second image respectively, and compare the calculated first face size of the first face position in the first image or the second face size of the second face position in the second image with a corresponding preset face size range to exclude ineligible objects, wherein different face positions have corresponding face size ranges. The second camera has fewer pixels than the first camera.

2. The electronic device of claim 1, wherein the electronic device is a mobile phone, and the first camera is the front camera of the mobile phone.

3. The electronic device according to claim 1, wherein the processor further performs identity recognition based on the first image via facial recognition.

4. The electronic device according to claim 1, wherein the processor further... The ineligible objects are excluded based on multiple first images acquired by the first camera at different times, and When there is no difference in expression among the plurality of first images, the current face in the first image is determined to be the ineligible object.

5. The electronic device according to claim 1, wherein the processor further... The ineligible objects are excluded based on multiple second images acquired by the second camera at different times, and When there is no difference in expression among the multiple second images, the current face in the second image is determined to be the ineligible object.

6. The electronic device of claim 1, wherein the processor further excludes the ineligible object based on an image difference between a first image acquired by the first camera and a second image acquired by the second camera.

7. An electronic device, the electron comprising: A first camera, which is used to acquire a first image at a first moment; A second camera is used to acquire a second image at the first time. as well as The processor is configured to determine whether a face is a living, qualified object when it is determined that the first image and the second image contain different facial features and characteristics of the same face. The second camera has fewer pixels than the first camera.

8. The electronic device of claim 7, wherein the electronic device is a mobile phone, and the first camera is the front camera of the mobile phone.

9. The electronic device of claim 7, wherein the processor further identifies an individual based on the first image using facial recognition.

10. The electronic device according to claim 7, wherein the processor further... Unqualified objects are excluded based on multiple first images acquired by the first camera at different times or multiple second images acquired by the second camera at different times, and When there is no difference in expression between the plurality of first images or the plurality of second images, the current face in the first image or the plurality of second images is determined to be the ineligible object.

Images