Equipment integrating face recognition and cat eye functions and control method thereof

Abstract

The invention relates to the technical field of intelligent door locks, in particular to equipment integrating face recognition and cat eye functions and a control method thereof. The equipment comprises an optical module, a visual intercom module and a control module, wherein the control module is connected to the optical module and the visual intercom module; wherein the optical module is used for collecting a to-be-recognized image and sending the to-be-recognized image to the control module; the visual intercom module is used for displaying the to-be-recognized image and / or sending the to-be-recognized image to a mobile terminal; and the control module is used for performing face recognition on the to-be-recognized image according to a preset recognition algorithm to obtain a recognition result, and sending an unlocking instruction to target equipment when the recognition result is a target user. According to the equipment, the internal space occupation of the door lock can be reduced, and the function completeness and the door opening reliability are both considered.

Description

Technical Field

[0001] This invention relates to the field of smart door lock technology, and in particular to a device and its control method that integrates facial recognition and peephole functions. Background Technology

[0002] With the development of technology, smart door locks, as the first line of defense for home security, have gradually replaced traditional mechanical door locks and become one of the core categories in the home furnishing market.

[0003] Currently, the mainstream smart door locks on the market mainly rely on card swiping and fingerprint recognition for unlocking. Card swiping requires users to carry a physical card, which carries the risk of card loss or theft and duplication. Fingerprint recognition can solve the problems of card recognition, but due to the limitations of the recognition technology, the success rate drops significantly when the user's fingers are wet, damaged, or covered in dirt, making it impossible to achieve stable and convenient door unlocking in all scenarios.

[0004] Furthermore, most current smart locks adopt a split configuration of the lock body plus an external USB camera to meet users' needs for visual viewing of the outside scene. To achieve core functions such as door opening recognition, security monitoring, and communication linkage, the smart lock motherboard needs to integrate multiple independent functional modules, such as a fingerprint recognition module and a USB camera module. Different modules often use different communication protocols (such as UART protocol, USB interface, etc.). This multi-module, multi-protocol distributed architecture design not only leads to complex motherboard circuit layout and increased wiring difficulty, but also significantly occupies the limited internal space of the lock board.

[0005] Therefore, how to provide a device that can reduce the space occupied inside the door lock while ensuring both functionality and reliability of door opening is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] To address the aforementioned technical problem of significantly increasing door lock space due to the integration of various functional modules, this invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides a device integrating face recognition and peephole functions, comprising: an optical module, a video intercom module, and a control module, wherein the control module is connected to the optical module and the video intercom module; wherein the optical module is used to acquire an image to be recognized and send the image to be recognized to the control module; the video intercom module is used to display the image to be recognized and / or send the image to be recognized to a mobile terminal; the control module is used to perform face recognition on the image to be recognized according to a preset recognition algorithm, obtain a recognition result, and send an unlock command to the target device when the recognition result is a target user.

[0008] Beneficial effects: By using the images captured by the optical module for both facial recognition unlocking and monitoring display, the device does not need to integrate many modules, thereby reducing the size of the device, reducing the space occupied inside the door lock plate, and lowering costs. In addition, by adopting facial recognition for door opening, the problem of fingerprint recognition not being able to achieve stable and convenient door opening in all scenarios can be solved, thereby improving the reliability of door opening.

[0009] Furthermore, the optical system includes a camera submodule and an infrared floodlight.

[0010] Beneficial effects: By setting up infrared floodlights, the necessary illumination for clear imaging can be provided in low-light scenarios, thereby improving the success rate of subsequent face recognition.

[0011] Furthermore, the video intercom module includes a speaker, a microphone, a video encoder, and an audio codec; wherein the audio codec is used to receive data sent by the speaker and the microphone, and the video encoder is used to receive data sent by the camera submodule.

[0012] Beneficial effects: By setting up speakers and microphones, users can communicate without opening the door, improving the user experience.

[0013] Furthermore, the control module communicates with the target device via a serial communication interface. The control module includes a main control chip, which stores the recognition algorithm.

[0014] In a second aspect, the present invention provides a control method for a device integrating face recognition and cat-eye functions as described in the first aspect, comprising: in response to receiving a start command, sending a control command to an optical module to acquire an image to be recognized; sending the image to be recognized to a video intercom module so that the video intercom module displays the image to be recognized and / or sending the image to be recognized to a mobile terminal; performing face recognition on the image to be recognized according to a preset recognition algorithm to obtain a recognition result; and if the recognition result is a target user, sending an unlock command to the target device.

[0015] Furthermore, face recognition is performed on the image to be recognized according to a preset recognition algorithm, including: extracting face features from the image to be recognized; determining whether the face features match the face features in a pre-stored face feature database; if so, performing liveness detection on the image to be recognized.

[0016] Beneficial effects: By performing liveness detection after a successful match, unauthorized intrusion can be prevented, thereby improving the reliability of facial recognition for door opening.

[0017] Further, determining whether the facial features match facial features in a pre-stored facial feature database includes: calculating the matching degree between the facial features and facial features in the pre-stored facial feature database based on Euclidean distance or cosine similarity; if the matching degree is greater than a threshold, it is determined to be a match.

[0018] Further, the image to be identified includes a first image and a second image, and the identification algorithm is a deep learning algorithm; the liveness detection of the image to be identified includes: extracting features from the first image and the second image respectively to obtain a first feature and a second feature; fusing and concatenating the first feature and the second feature using an attention mechanism to obtain a liveness value; determining whether the liveness value is greater than a liveness threshold, and if so, determining that it is a target user.

[0019] Furthermore, the expression for calculating the living body value is as follows:

[0020] In the formula, The liveness value of the image to be identified. For trainable weights, As the first feature, As the second feature, For the Sigmoid function, This is a feature concatenation operator.

[0021] Furthermore, the method also includes: determining whether the audio receiving or playback function is enabled; if so, sending a control command to the microphone in the video intercom module to collect audio, or sending a control command to the speaker in the video intercom module to play audio.

[0022] The beneficial effects of this invention are as follows: by using facial recognition to unlock the door, the problems of easy loss of cards and high failure rate of fingerprint recognition are avoided, thereby improving the reliability of door opening; by sharing an optical module for facial recognition and monitoring display, the door lock plate only needs to be connected to one module to realize the functions of facial recognition door opening and external monitoring, reducing the internal space occupied by the door lock plate and improving development efficiency. Attached Figure Description Figure 1 This is a schematic structural block diagram illustrating a device integrating face recognition and cat-eye functions according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating, schematically, a control method for a device integrating face recognition and cat-eye functionality according to an embodiment of the present invention; Figure 3 This is a schematic block diagram illustrating the structure of a control system for a device integrating face recognition and cat-eye functions according to an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] Figure 1 This is a schematic structural block diagram illustrating a device integrating face recognition and cat-eye functionality according to an embodiment of the present invention.

[0026] In a first aspect, the present invention provides a device integrating facial recognition and peephole functionality. Specifically, as... Figure 1 As shown, the device of the present invention includes an optical module, a video intercom module, and a control module. The optical module is used to acquire an image to be identified and to send the image to the control module; the video intercom module is used to display the image to be identified and / or send the image to a mobile terminal; the control module, connected to the optical module and the video intercom module, is used to perform facial recognition on the image to be identified according to a preset recognition algorithm, obtain a recognition result, and send an unlock command to the target device when the recognition result matches the target user. The mobile terminal can be a user's mobile phone, tablet computer, etc.; the target device is a door lock control device that unlocks the door upon receiving the unlock command; the optical module is connected to the control module via a MIPI signal line.

[0027] Specifically, the optical module includes a camera submodule and an infrared floodlight. The camera submodule is used to acquire the image to be identified outside the door, which includes IR (infrared) images and RGB (red, green, blue) images; the infrared floodlight is used to provide an infrared light source to ensure that a clear image to be identified can be obtained in low-light scenarios such as at night, thereby improving the success rate of face recognition.

[0028] In one embodiment, the camera submodule includes an infrared camera and a visible light camera. The infrared camera is used to acquire IR images, and the visible light camera is used to acquire RGB images.

[0029] In other optional embodiments, the camera submodule may also be a camera consisting of a single image sensor that simultaneously receives visible light and infrared light signals, and then separates the RGB color channel and IR infrared channel data through a back-end ISP (Image Signal Processor) to generate IR images and RGB images respectively.

[0030] By using a camera composed of a single image sensor to acquire the image to be recognized, the size of the device can be further reduced, thereby reducing the space occupied inside the door lock panel. Compared to face recognition using only IR or RGB images, using both IR and RGB images can improve the reliability of face recognition.

[0031] In one embodiment, the video intercom module includes a speaker, a microphone, a video encoder, and an audio codec. The speaker outputs audio data; the microphone receives audio data; the audio codec receives audio data from the microphone and speaker via the UAC (USB Audio Class) protocol and performs PCM (Pulse Code Modulation) encoding on the audio data; the video encoder receives image data from the camera submodule via the UVC (USB Video Class) protocol and performs MJPEG (Motion JPEG) encoding on the image data.

[0032] This video intercom module can monitor the situation outside the door in real time, displaying images captured by the optical module. Furthermore, it also features an intercom function, receiving audio data from the target user / authorized user via a microphone and playing the audio data back through a speaker. It's understandable that this video intercom module corresponds to a peephole (doorbell).

[0033] In one embodiment, communication with external devices can be achieved via a USB interface, meaning that external devices can initiate the monitoring and intercom functions of the video intercom module at any time via USB.

[0034] In one embodiment, the control module includes a main control chip that stores a recognition algorithm and a facial feature database. Furthermore, the control module can connect to the target device via a serial communication interface.

[0035] By enabling the video intercom module to share the same optical module for monitoring and the control module for facial recognition, multiple modules are eliminated. The door lock only needs to interface with one module to achieve both peephole and facial recognition unlocking functions, ensuring functional completeness and reducing the internal space occupied by the door lock plate, thereby accelerating production efficiency and reducing production costs. Furthermore, compared to card or fingerprint unlocking, this invention avoids the problems associated with contact-based recognition methods such as card and fingerprint recognition, improving the stability and convenience of door opening. Moreover, the device of this invention can achieve data interaction with external devices through only two communication methods, simplifying the motherboard circuitry and reducing costs.

[0036] Figure 2This is a flowchart illustrating, schematically, a control method for a device integrating face recognition and cat-eye functionality according to an embodiment of the present invention.

[0037] In a second aspect, the present invention provides a control method for a device integrating facial recognition and cat-eye functionality. Specifically, as shown in the figure... Figure 2 As shown, the method of the present invention includes the following steps.

[0038] S101. In response to receiving the start command, send a control command to the optical module to acquire the image to be recognized.

[0039] In one embodiment, a start signal / command can be triggered by a PIR (Passive Infrared Sensor) to control the optical module to acquire the image to be identified. Specifically, when a person is detected entering a preset sensing range and remains there for a certain period of time (up to a time threshold), the PIR sensor sends a start command to the control module via a serial communication interface. Upon receiving the start command, the control module controls the optical module to acquire the image outside the door, thereby obtaining the image to be identified. The sensing range can be set to 0.5~1.5m, and the time threshold can be set to 0.5~2 seconds.

[0040] In an optional embodiment, a temperature sensor can also trigger the start signal. Specifically, the temperature sensor acquires the temperature change outside the door. When the detected temperature reaches the range of human body temperature, the temperature sensor sends a start command to the control module. The control module then controls the optical module to acquire an image of the outside of the door, thereby obtaining the image to be identified. The human body temperature range can be set to 36~38℃.

[0041] S102. Send the image to be identified to the video intercom module so that the video intercom module displays the image to be identified and / or sends the image to be identified to the mobile terminal.

[0042] After receiving the image to be identified, the controller module immediately sends the image to the video intercom module via the UVC protocol, so that the video intercom module can display the image in a timely manner, or send the image to the user's mobile phone so that the user can check the situation outside the door at any time.

[0043] In one embodiment, the method of the present invention further includes: determining whether the audio receiving or playback function is enabled; if so, sending a control command for collecting audio to the microphone in the video intercom module, or sending a control command for playing audio to the speaker in the video intercom module, thereby enabling the intercom function so that the user can communicate with people outside the door without opening the door.

[0044] S103. Perform face recognition on the image to be recognized according to the preset recognition algorithm to obtain the recognition result. If the recognition result is the target user, send an unlock command to the target device.

[0045] In this embodiment, the identification results include target users and non-target users; the identification algorithm is a deep learning algorithm, such as a convolutional neural network; the images to be identified include a first image and a second image, wherein the first image is an IR image and the second image is an RGB image.

[0046] Prior to this, the method of the present invention further included: if a command to register a face was triggered, controlling the optical module to acquire a face image, extracting face features from the face image, generating a unique ID (Identification), storing it in a face feature database in the format of ID + face features, and outputting the registration result. It can be understood that one ID corresponds to a set of face features and one target user. The command to register a face can be triggered by a user with registration authority.

[0047] In one embodiment, the method of the present invention further includes: preprocessing the image to be recognized, such as filtering and denoising, locating the face region, etc.

[0048] Specifically, upon receiving the start command, facial features are extracted from the image to be identified captured by the optical module. These facial features are then compared one by one with facial features in the facial feature database. It is then determined whether the facial features match those in the pre-stored database. If they match, it indicates that the user is the target user, and liveness detection is performed on the image to be identified; otherwise, the user is determined to be a non-target user. The facial features referred to may include nose width, eye corner angle, and eye distance, among other things.

[0049] More specifically, the matching degree between the facial feature and facial features in a pre-stored facial feature database can be calculated using Euclidean distance or cosine similarity. If the matching degree is greater than a threshold, it is considered a match. In this embodiment, the threshold can be set to 0.9.

[0050] In one embodiment, before performing liveness detection, the method of the present invention further includes: training a backbone network such as ResNet or MobileNet using a large number of live face images and fake face images, so that the backbone network such as ResNet or MobileNet can clearly identify the key distinguishing features between live and fake faces, so that when performing liveness detection in the future, the backbone network such as ResNet or MobileNet can extract the key distinguishing features from the image to be identified.

[0051] By training backbone networks such as ResNet or MobileNet using both live and spoofed face images, these networks can focus on distinguishing key features, thereby improving the accuracy of subsequent liveness detection while reducing the amount of data that needs to be processed. These distinguishing key features can include subcutaneous blood distribution characteristics in key facial regions, infrared reflectance of different facial areas, and skin texture.

[0052] In one embodiment, performing liveness detection on the image to be identified specifically includes: extracting features from a first image and a second image respectively to obtain a first feature and a second feature; wherein the first feature corresponds to the feature of the IR image and the second feature corresponds to the feature of the RGB image.

[0053] An attention mechanism is used to fuse and concatenate the first and second features to obtain a liveness value. It is then determined whether this liveness value is greater than a liveness threshold. If it is, the user is identified as a target user; otherwise, they are identified as a non-target user. The liveness value ranges from 0 to 1, and the liveness threshold can be set to 0.9, which is an empirical value determined based on a validation set curve. In other optional embodiments, those skilled in the art can set the liveness threshold according to actual needs, for example, to 0.85.

[0054] Compared to using features from a single image for liveness detection, this method improves the accuracy of liveness detection by combining features from both IR and RGB images.

[0055] In one embodiment, the expression for calculating the liveness value is:

[0056] In the formula, The liveness value of the image to be identified. For trainable weights, As the first feature, As the second feature, For the Sigmoid function, This is a feature concatenation operator.

[0057] By performing liveness detection after a user's facial features are successfully matched with facial features in the facial feature database, non-live forgery carriers such as 3D models and videos can be blocked, preventing non-target users from successfully unlocking the door through facial recognition, thereby improving the reliability of facial recognition unlocking.

[0058] In an optional embodiment, if the liveness score is between 0.7 and 0.9, dynamic verification is triggered. For example, the user is randomly asked to perform actions such as "blinking" or "turning their head." If the user passes the verification, they are identified as the target user. By triggering dynamic verification when the liveness score is low, the probability of false positives can be reduced.

[0059] The method of this invention not only enables the use of images acquired by the optical module for facial recognition unlocking, but also for display in a video intercom module, realizing a peephole function, allowing real-time viewing of the outside environment. By performing liveness detection on the image to be recognized after successful matching, the possibility of unauthorized unlocking using spoofs can be avoided, thereby improving the reliability of identity authentication.

[0060] Figure 3 This is a schematic block diagram illustrating the structure of the control system of the device integrating face recognition and cat-eye functions according to this embodiment.

[0061] In a third aspect, the present invention also provides a control system for a device integrating facial recognition and cat-eye functionality. For example... Figure 3 As shown, the control system includes a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement the control method for a device integrating face recognition and cat-eye functions as described in the second aspect of the present invention.

[0062] The control system also includes other components well known to those skilled in the art, such as communication interfaces. Their settings and functions are known in the art and will not be described in detail here.

[0063] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions that can be stored or otherwise maintained by such a computer-readable medium.

[0064] In the description of this specification, "multiple" means at least two, such as two, three or more, unless otherwise explicitly specified. Furthermore, the steps described above are for clarity only; in implementation, they can be combined into one step or some steps can be broken down into multiple steps, as long as they include the same logical relationships.

[0065] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A device integrating facial recognition and peephole functions, characterized in that, include: The system comprises an optical module, a video intercom module, and a control module, wherein the control module is connected to the optical module and the video intercom module; wherein... The optical module is used to acquire the image to be identified and send the image to be identified to the control module; The video intercom module is used to display the image to be identified and / or send the image to be identified to the mobile terminal; The control module is used to perform face recognition on the image to be recognized according to a preset recognition algorithm, obtain the recognition result, and send an unlock command to the target device when the recognition result is the target user.

2. The device integrating face recognition and cat-eye functions according to claim 1, characterized in that, The optical system includes a camera submodule and an infrared floodlight.

3. The device integrating face recognition and cat-eye functions according to claim 2, characterized in that, The video intercom module includes a speaker, a microphone, a video encoder, and an audio codec; wherein the audio codec is used to receive data sent by the speaker and the microphone, and the video encoder is used to receive data sent by the camera submodule.

4. The device integrating face recognition and cat-eye functions according to claim 3, characterized in that, The control module communicates with the target device through a serial communication interface. The control module includes a main control chip, which stores the recognition algorithm.

5. A control method for a device integrating face recognition and cat-eye functions as described in any one of claims 1 to 4, characterized in that, include: In response to receiving the start command, a control command is sent to the optical module to acquire the image to be recognized; Send the image to be identified to the video intercom module so that the video intercom module displays the image to be identified and / or sends the image to be identified to the mobile terminal; The image to be identified is subjected to facial recognition according to a preset recognition algorithm to obtain a recognition result. If the recognition result is that of the target user, an unlock command is sent to the target device.

6. The control method for the device integrating face recognition and cat-eye functions according to claim 5, characterized in that, Facial recognition is performed on the image to be recognized according to a preset recognition algorithm, including: Extract facial features from the image to be identified; Determine whether the facial features match those in the pre-stored facial feature database; if so, perform liveness detection on the image to be identified.

7. The control method for the device integrating face recognition and cat-eye functions according to claim 6, characterized in that, Determining whether the facial features match facial features in a pre-stored facial feature database includes: The matching degree between the facial features and the facial features in the pre-stored facial feature database is calculated based on Euclidean distance or cosine similarity. If the matching degree is greater than the threshold, it is determined to be a match.

8. The control method for the device integrating face recognition and cat-eye functions according to claim 6, characterized in that, The image to be identified includes a first image and a second image, and the identification algorithm is a deep learning algorithm. Performing liveness detection on the image to be identified includes: Feature extraction is performed on the first image and the second image respectively to obtain the first feature and the second feature; An attention mechanism is used to fuse and concatenate the first feature and the second feature to obtain a liveness value; Determine whether the liveness value is greater than the liveness threshold; if so, determine that the user is the target user.

9. The control method for the device integrating face recognition and cat-eye functions according to claim 8, characterized in that, The expression for calculating the living body value is: In the formula, The liveness value of the image to be identified. For trainable weights, As the first feature, As the second feature, For the Sigmoid function, This is a feature concatenation operator.

10. The control method for the device integrating face recognition and peephole according to claim 5, characterized in that, Also includes: Determine whether the audio receiving or playback function is enabled. If so, send a control command to the microphone in the video intercom module to collect audio, or send a control command to the speaker in the video intercom module to play audio.

Images