Unlocking method and device of intelligent door lock and nonvolatile storage medium

Abstract

The invention discloses an unlocking method and device of an intelligent door lock and a nonvolatile storage medium. The method comprises the following steps: acquiring image data; identifying the image data, and comparing the to-be-identified face information with the face information of the resident personnel stored in the local edge storage device in an edge storage mode under the condition that the to-be-identified face information exists in the image data; under the condition that the comparison between the to-be-recognized face information and the face information of the resident personnel fails, determining that the personnel corresponding to the to-be-recognized face information is the external visitor; and receiving request information of the temporary password sent by the visitor, and unlocking the intelligent door lock based on the temporary password of the visitor when the request information passes. The technical problems that the face recognition process needs a long time and the unlocking efficiency of the intelligent door lock is low due to the fact that the face data needs to be continuously transmitted to the terminal in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of smart terminals, and more specifically, to a method and device for unlocking a smart door lock and a non-volatile storage medium. Background Art

[0002] As people's security awareness gradually increases, various access control systems have been developed. Common access control systems include: password recognition access control system, card recognition access control system and biometric access control system. Password recognition access control system has poor security, passwords are easy to leak, can only be controlled in one direction, there is no record of opening the door, and the buttons are prone to damage or failure. Card recognition access control system can be divided into contact card access control system (magnetic stripe card, barcode card) and contactless card (also called induction card, radio frequency card) access control system according to the type of card. Contact cards are easy to wear and easy to be disturbed by external magnetic fields, which makes contact cards easy to fail, and the information in the card is easy to be copied, so the security is poor. Contactless cards are more durable, faster to read, and more secure than contact cards, but the cards are easy to lose and damage. Biometric access control system recognizes people's biological characteristics, such as fingerprint, palm, iris, and finger vein. At present, the stability and accuracy of biometric recognition are still being further improved.

[0003] Facial recognition access control systems are increasingly being used in our daily lives. The facial recognition methods mentioned above in the related technologies only recognize faces in the database. When there are visitors, manual registration is still required. Visitors will have to wait for a long time, and the recognition efficiency is low and the safety of entry and exit cannot be guaranteed. When performing facial recognition, facial data needs to be continuously transmitted to the terminal, which results in a long recognition process and low efficiency.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device and non-volatile storage medium for unlocking a smart door lock, so as to at least solve the technical problem in the related art that facial data needs to be continuously transmitted to a terminal, resulting in a long facial recognition process and low efficiency in unlocking the smart door lock.

[0006] According to one aspect of an embodiment of the present application, a method for unlocking a smart door lock is provided, including: acquiring image data; identifying the image data, and when it is identified that there is facial information to be identified in the image data, comparing the facial information to be identified with facial information of a resident stored in a local edge storage device using edge storage; when the comparison between the facial information to be identified and the facial information of the resident fails, determining that the person corresponding to the facial information to be identified is a visitor; receiving request information for a temporary password sent by the visitor, and when the request information is approved, opening the smart door lock based on the temporary password of the visitor.

[0007] In some embodiments of the present application, after acquiring the image data, the method also includes: randomly generating a preset number of facial movements, and detecting the completion degree of the facial movements of the target object corresponding to the image data, and when the completion degree is greater than a first preset threshold, determining that the image data is a non-static picture; when the completion degree is less than or equal to the first preset threshold, determining that the image data is a static picture, and triggering an early warning.

[0008] In some embodiments of the present application, the method also includes: the facial information of the resident personnel is stored in the local edge storage device using the following edge storage method: the data packet corresponding to the facial information of the resident personnel is encrypted, and the encrypted data packet is divided into multiple shards according to a preset size; a hash value is calculated for each shard to obtain a first hash value corresponding to each shard, and a redundant shard corresponding to each shard is determined, wherein the redundant shard is used for data recovery in the event of data loss; each shard and the corresponding redundant shard are distributed and stored on the edge nodes in the local edge storage device.

[0009] In some embodiments of the present application, the method also includes: performing a hash value calculation on the first shard to obtain a second hash value corresponding to the first shard, wherein the first shard is any shard on an edge node stored in a local edge storage device; obtaining a third hash value corresponding to the first shard, and comparing the second hash value and the third hash value corresponding to the first shard, wherein the third hash value is obtained by performing a hash value calculation on the first shard before the first shard is stored in the edge node; when the second hash value and the third hash value are consistent, determining that the first shard has not changed; when the second hash value and the third hash value are inconsistent, determining that the first shard has changed, and performing data recovery on the first shard based on the redundant shards of the first shard, the first hash value of the first shard, and the second hash value of the first shard.

[0010] In some embodiments of the present application, the facial information to be identified is compared with the facial information of the resident personnel stored in a local edge storage device using edge storage, including: vectorizing the facial information to be identified and the facial information of the resident personnel respectively to obtain a first feature vector and a second feature vector; calculating the similarity between the first feature vector and the second feature vector, and comparing the facial information to be identified with the facial information of the resident personnel based on the similarity.

[0011] In some embodiments of the present application, facial information to be identified is compared with facial information of permanent personnel based on similarity, including: determining a first similarity between a first feature vector and a second feature vector using a first method, and determining a second similarity between the first feature vector and the second feature vector using a second method, wherein the first similarity is used to represent the distance between the first feature vector and the second feature vector, and the second similarity is used to represent the cosine similarity between the first feature vector and the second feature vector; when the first similarity is less than a second preset threshold, determining that the facial information to be identified has been compared with the facial information of the permanent personnel; or, when the second similarity is greater than a third preset threshold, determining that the facial information to be identified has been compared with the facial information of the permanent personnel; or, determining a target similarity based on the first similarity and the second similarity, and when the target similarity is greater than a fourth preset threshold, determining that the facial information to be identified has been compared with the facial information of the permanent personnel.

[0012] In some embodiments of the present application, opening a smart door lock based on a temporary password of a visitor includes: when the visitor is identified for the first time, receiving request information for a temporary password sent by the visitor, wherein the request information includes: a preset voice command issued by the visitor, the dynamic facial information of the visitor, and the current location of the visitor is located in a preset location area; after the request information meets preset conditions, determining the temporary password corresponding to the visitor; performing verification based on the temporary password and the request information respectively to obtain a first verification result and a second verification result; if both the first verification result and the second verification result are passed, opening the smart door lock; when the visitor is not identified for the first time, verifying the visitor only through the request information.

[0013] According to another aspect of an embodiment of the present application, a smart door lock unlocking device is also provided, including: an acquisition module for acquiring image data; an identification module for identifying the image data, and when it is identified that there is facial information to be identified in the image data, comparing the facial information to be identified with the facial information of a resident stored in a local edge storage device using edge storage; a determination module for determining that a person corresponding to the facial information to be identified is a visitor when the comparison between the facial information to be identified and the facial information of the resident fails; a receiving module for receiving request information for a temporary password sent by an outsider, and when the request information is approved, opening the smart door lock based on the temporary password of the outsider.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned unlocking method of the smart door lock.

[0015] According to another aspect of an embodiment of the present application, an electronic device is also provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above-mentioned unlocking method of the smart door lock is executed when the program is running.

[0016] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, and the computer instructions are executed by a processor to implement the above-mentioned method of unlocking the smart door lock.

[0017] In an embodiment of the present application, the method of acquiring image data and identifying the image data is adopted; when it is identified that there is facial information to be identified in the image data, the facial information to be identified is compared with the facial information of the resident personnel stored in the local edge storage device by edge storage; when the comparison between the facial information to be identified and the facial information of the resident personnel fails, it is determined that the person corresponding to the facial information to be identified is an outsider; receiving a request information for a temporary password sent by the outsider, and when the request information is passed, the smart door lock is opened based on the temporary password of the outsider. By adopting the edge storage method to store the facial information of the resident personnel in the local edge storage device, the time for acquiring the facial information of the resident personnel during identification is greatly reduced, and when the comparison between the facial information to be identified and the facial information of the resident personnel fails, it is determined that the person corresponding to the facial information to be identified is an outsider, and when the request information sent by the outsider is passed, the smart door lock is opened based on the temporary password of the outsider, thereby achieving the technical effect of shortening the identification time and improving the identification efficiency, thereby solving the technical problem in the related technology that the facial data needs to be continuously transmitted to the terminal, resulting in a long facial recognition process and low efficiency in unlocking the smart door lock. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing an unlocking method of a smart door lock provided in an embodiment of the present application;

[0020] Figure 2 It is a flowchart of a method for unlocking a smart door lock provided in an embodiment of the present application;

[0021] Figure 3 It is a flowchart of another method for unlocking a smart door lock provided in an embodiment of the present application;

[0022] Figure 4 This is a schematic diagram of a process for an external visitor to obtain a temporary password according to an embodiment of the present application;

[0023] Figure 5 It is a structural schematic diagram of an unlocking device of a smart door lock provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0028] Infrared camera: The wavelength from 0.76 to 400 microns in the spectrum is called infrared. Infrared is invisible light. All substances above absolute zero (-273.15℃) can produce infrared. Modern physics calls it heat rays.

[0029] A visible light camera, or visible light camera sensor, is an imager that collects visible light (400-700nm) and converts it into electrical signals, then organizes that information to render images and video streams. Visible light cameras utilize light with wavelengths of 400-700nm, which is the same spectrum perceived by the human eye. Visible light cameras are designed to create images that replicate human vision, capturing red, green, and blue wavelengths (RGB) of light for accurate color representation. Modern security and surveillance cameras do this in HD or higher resolutions and come with a variety of lens options for wide-angle or telephoto views to identify targets and objects in a scene.

[0030] Edge Storage: Mainly used in the field of edge computing. Edge storage refers to the technology of storing data at the source of data generation or close to the source. Compared with traditional centralized storage, edge storage can reduce the need to transmit data to the central server, reduce network bandwidth consumption, reduce data transmission latency, improve data access speed and application performance, and enhance data privacy protection and security. In scenarios such as the Internet of Things, 5G networks, and smart cities, edge storage is becoming increasingly important because it can effectively process large amounts of real-time data and support low-latency and high-bandwidth application requirements.

[0031] In the related art, the facial recognition of multiple access control recognition systems only recognizes faces in the database. When there are visitors, manual registration is still adopted. Visitors will have to wait for a long time, the recognition efficiency is low, and the safety of people entering and leaving cannot be guaranteed. When performing facial recognition, the facial data needs to be continuously transmitted to the terminal, resulting in a long time for the recognition process and low efficiency. Therefore, there is a problem in the related art that the facial data needs to be continuously transmitted to the terminal, resulting in a long time for the facial recognition process and low efficiency for smart door locks to unlock. In order to solve this problem, a relevant solution is provided in the embodiment of the present application, which is described in detail below.

[0032] According to an embodiment of the present application, an embodiment of a method for unlocking a smart door lock is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal for implementing the unlocking method of a smart door lock is shown. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0034] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the unlocking method of the smart door lock in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the unlocking method of the smart door lock described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0038] In the above operating environment, the embodiment of the present application provides a flowchart of a method for unlocking a smart door lock, such as Figure 2 As shown, the method comprises the following steps:

[0039] Step S202, acquiring image data, for example, acquiring image data collected in real time by a camera or other sensor device, and the image data may be, for example, dynamic facial image data.

[0040] The following is a specific embodiment: detect the light intensity, and when the light intensity is greater than a preset value, collect the face image based on the visible light camera, and when the light intensity is less than the preset value, collect the face image based on the infrared light camera. There are at least two different types of cameras, the first camera is a visible light camera, and the second camera is an infrared camera. It should be noted that in general, visible light cameras are used to collect face images, but in dim light conditions, the performance of visible light cameras is poor, and infrared cameras are required to add face images. The smart door lock maintains a communication connection with both the visible light camera and the infrared camera.

[0041] After acquiring the image data, a preset number of facial movements are randomly generated, and the completion degree of the facial movements of the target object corresponding to the image data is detected. When the completion degree is greater than a first preset threshold (for example, 90%), the image data is determined to be a non-static picture; when the completion degree is less than or equal to the first preset threshold, the image data is determined to be a static picture, and an early warning is triggered. When an outbound visitor uses a non-dynamic photo for identification, the device automatically issues a warning and notifies the authorized administrator that the photo has been leaked. The early warning includes: the identification time, the leaked photo, and the facial information of the current outbound visitor; the following is a specific embodiment:

[0042] A predetermined number of facial movements are randomly generated, such as shaking the head to the left, shaking the head to the right, nodding up and down, closing the left / right eye, etc., and the above facial movements are displayed on the display interface or presented to the target object in the form of voice, and the completion degree of the above movements by the target object is detected. If the target object cannot complete the above movements, the image data is determined to be a static picture, and the subsequent unlocking process is terminated. If the target object can complete the above movements, it is determined that the target object is alive and the subsequent unlocking process continues.

[0043] After determining that the target object is alive, you can also use intention information (such as human eye sight, voice, and action, etc.) to determine whether the target object has a real intention to unlock. You can use human eye sight, voice, and action, etc. to determine whether the target object has the intention to unlock. This can be achieved through the following steps:

[0044] Detect whether the target object's line of sight is consistent with a preset direction. If the detection result indicates that the line of sight is consistent with the preset direction, it is determined that the target object has the intention to unlock the door; or detect the target object's voice information through a microphone. If the target object's voice information indicates unlocking the door, it is determined that the target object has the intention to unlock the door; or capture the target object's hand movement through a camera. If the hand movement is a predetermined movement, it is determined that the target object has the intention to unlock the door.

[0045] Step S204, identifying the image data, and when it is identified that there is facial information to be identified in the image data, comparing the facial information to be identified with the facial information of the resident personnel stored in the local edge storage device in an edge storage manner.

[0046] In the technical solution provided in step S204, the local edge storage device can be a local server, a gateway device or other smart terminal device, etc. The image data is stored in the edge node of the local edge storage device. The edge node can refer to a computing and storage resource located near the geographical or logical location where the data is generated and used. In the smart door lock system, these edge nodes can be the smart door lock itself or a local server, gateway device or other smart terminal device directly connected to the smart door lock. They can directly process and store the facial dynamic information data captured by the smart door lock camera without transmitting the data to a remote central server or cloud for processing. Edge nodes can perform preliminary processing on the raw data collected from sensors such as cameras, such as image denoising, resizing, color correction, etc., to improve the quality of data and the efficiency of subsequent processing; because edge nodes are close to data sources, they can quickly respond to data collection and processing needs and reduce latency, which is especially important for facial recognition scenarios that require real-time processing; edge nodes can store pre-processed facial dynamic information data and use distributed storage technology to ensure data availability and fault tolerance; edge nodes can apply encryption technology and blockchain technology to ensure the security and non-tamperability of data during storage and transmission; when distributing and storing data in the system, edge nodes can ensure data integrity and recoverability by applying technologies such as erasure codes even if some nodes fail; edge nodes control data access rights through smart contracts to ensure that only authorized users and devices can access stored facial information.

[0047] In the technical solution provided in step S204, the facial information of the resident personnel can be stored in the local edge storage device in the following edge storage manner: encrypt the data packet corresponding to the facial information of the resident personnel, and divide the encrypted data packet into multiple shards according to a preset size; calculate the hash value for each shard to obtain the first hash value corresponding to each shard, and determine the redundant shard corresponding to each shard, wherein the redundant shard is used for data recovery in the event of data loss; and store each shard and the corresponding redundant shard on the edge nodes in the local edge storage device.

[0048] The following is a specific embodiment:

[0049] During the first entry, the facial images of the resident personnel (i.e. the facial information mentioned above) are collected in real time through cameras or other sensor devices; before the data is transmitted to the edge storage, the collected facial images need to be preprocessed, such as image denoising, resizing, color correction, etc. This helps to improve the accuracy of facial recognition. During the resizing, deep learning algorithms are applied to automatically detect image quality problems (such as blur and occlusion) and optimize the processing. In particular, lightweight machine learning models are deployed on edge nodes to achieve fast preprocessing and preliminary analysis of facial images. The image preprocessing work is completed quickly at the edge nodes, reducing the need for data backhaul to the central server, and recording the access and modification history of facial recognition data. The preprocessed facial images are transmitted to the edge node of the local edge storage device through the network. Since edge storage devices are usually deployed close to the data source, transmission delays and network bandwidth consumption can be reduced.

[0050] Before data is stored, the data packets corresponding to the facial information of the resident personnel are encrypted using advanced encryption technology (such as quantum encryption). Then, the hash value of the encrypted data packet is anchored to the blockchain. Anchoring the hash value to the blockchain can ensure the integrity and non-tampering of the data packet during storage and transmission.

[0051] The specific storage process is as follows: the data packet corresponding to the encrypted facial information is sharded and stored on multiple edge nodes using distributed storage technology to improve data availability and fault tolerance: the size of each shard is determined according to the capacity and performance of the local edge storage device, and the shard size should balance storage efficiency and recovery time; the data packet of the facial information is divided into multiple shards according to the determined size. If the data packet size is not an integer multiple of the shard size, it can be filled in the last shard; a hash value (i.e., the first hash value mentioned above, the shard data of the encrypted data packet is input into a hash algorithm (e.g., SHA-256 (Secure Hash Algorithm) etc.) for processing, and the hash algorithm traverses the entire data packet and generates a fixed-length hash value through a series of mathematical operations) or a checksum is calculated for each shard to facilitate error detection during storage and transmission; an erasure code technology, such as Reed-Solomon coding, is used to generate additional redundant shards for the original shards. These redundant shards allow the system to recover the original data in the event of partial data loss; the generated shards and redundant shards are distributed and stored on different edge nodes to improve data availability and fault tolerance. Each shard and the corresponding redundant shards are distributed and stored on the edge nodes in the local edge storage device.

[0052] When it is necessary to verify whether the data packet stored on the edge node has been modified, the hash value of the data packet can be recalculated and compared with the hash value on the blockchain. If the two match, it means that the data packet has not been tampered with; if they do not match, it indicates that the data packet may have been modified or damaged at a certain link, so that data security issues can be discovered and handled in a timely manner, for example, by the following method: a hash value calculation is performed on the first shard to obtain a second hash value corresponding to the first shard, wherein the first shard is any shard on the edge node stored in the local edge storage device; a third hash value corresponding to the first shard (i.e., the first hash value of the slice written into the blockchain before the first shard is stored in the edge node) is obtained, and the second hash value corresponding to the first shard is compared with the third hash value, wherein the third hash value is obtained by performing a hash value calculation before the first shard is stored in the edge node; when the second hash value is consistent with the third hash value, it is determined that the first shard has not changed; when the second hash value is inconsistent with the third hash value, it is determined that the first shard has changed, and data recovery is performed on the first shard based on the redundant shards of the first shard, the first hash value of the first shard, and the second hash value of the first shard.

[0053] In some special scenarios, such as self-service business places, since there may be a lot of data stored, in order to avoid the impact of system failures, redundant shards can be stored on edge nodes that may produce different types of failures. It is necessary to ensure that redundant shards are not stored on edge nodes that may be affected by the same failure at the same time. The specific implementation process is as follows: Identify the fault domain, such as fault nodes generated or associated with the same power supply, network switch or physical location, or for example, build a directed graph to represent the dependencies between the components in the system. Use a graph traversal algorithm (for example, depth-first traversal or breadth-first traversal) to identify strongly connected components, which usually represent fault domains, specifically:

[0054] Construct a directed graph to represent the dependencies between components in the system. Use a graph traversal algorithm (e.g., depth-first traversal or breadth-first traversal) to identify strongly connected components, thereby determining the fault domain; or, classify shards (classified by access time or frequency) and edge nodes (classified by performance indicators), for example, classify shards corresponding to redundant shards according to access time or frequency, and store shards with higher access frequencies and redundant shards in edge nodes with higher performance indicators.

[0055] Deploy smart contracts on the blockchain to manage data access permissions and review data access requests to ensure that only authorized objects can access the data. Through blockchain technology, data can be synchronized between different edge nodes, and the immutability of blockchain can be used for data backup. It can automatically learn and update the facial recognition model stored in the blockchain based on new facial dynamic information (i.e. the facial information mentioned above) without human intervention; combined with the blockchain's smart contracts, it can automatically detect and respond to abnormal access patterns and improve the security of the system. When necessary, facial information in edge storage can also be integrated and analyzed with other data sources to obtain richer insights and information. It is also necessary to delete or archive the stored facial information regularly, which helps protect user privacy and free up storage space.

[0056] Through the above process, edge storage can effectively store and manage facial information while ensuring the security and real-time nature of the data.

[0057] In the scheme provided in step S204, there are multiple ways to compare the facial information to be identified with the facial information of the resident stored in the local edge storage device using edge storage, for example: respectively vectorizing the facial information to be identified and the facial information of the resident to obtain a first feature vector and a second feature vector; calculating the similarity between the first feature vector and the second feature vector, and comparing the facial information to be identified with the facial information of the resident based on the similarity.

[0058] There are many ways to compare the facial information to be identified with the facial information of the resident based on similarity, for example: using a first method (for example, indicating similarity by calculating distance similarity, distance includes but is not limited to: Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, etc.) to determine the first similarity of the first feature vector and the second feature vector, and using a second method (for example, indicating similarity by calculating cosine similarity) to determine the second similarity of the first feature vector and the second feature vector, wherein the first similarity is used to represent the distance between the first feature vector and the second feature vector, and the second similarity is used to represent the cosine similarity between the first feature vector and the second feature vector; when the first similarity is less than the second preset threshold, it is determined that the facial information to be identified and the facial information of the resident have been compared; or, when the second similarity is greater than the third preset threshold, it is determined that the facial information to be identified and the facial information of the resident have been compared; or, based on the first similarity and the second similarity, a target similarity is determined, and when the target similarity is greater than a fourth preset threshold, it is determined that the facial information to be identified and the facial information of the resident have been compared.

[0059] The following is a specific embodiment:

[0060] The collected facial information to be identified is converted into a first feature vector, and the facial information of the stored resident personnel is converted into a second feature vector. The similarity between the first feature vector and the second feature vector is determined by comparing the first feature vector and the second feature vector (distance similarity or cosine similarity or target similarity), for example:

[0061] The distance similarity (i.e., the first similarity mentioned above) is less than a second preset threshold (e.g., 0.2), and it is determined that the facial information to be identified passes the comparison with the facial information of the resident, and the smart door lock is opened.

[0062] Or when the cosine similarity is greater than a third preset threshold (for example, 0.9), it is determined that the facial information to be identified has been compared with the facial information of the resident, and the smart door lock is opened. The cosine value between the angle between two vectors in a vector space is used as a measure of the difference between two individuals. The cosine value is close to 1 and the angle tends to 0, indicating that the two vectors are more similar. The cosine value is close to 0 and the angle tends to 90 degrees, indicating that the two vectors are less similar. In geometry, the cosine of the angle can be used to measure the difference in the directions of two vectors. The range of the cosine of the angle is [-1,1]. The larger the cosine of the angle, the smaller the angle between the two vectors, and the smaller the cosine of the angle, the larger the angle between the two vectors. When the directions of the two vectors coincide, the cosine of the angle takes the maximum value of 1, and when the directions of the two vectors are completely opposite, the cosine of the angle takes the minimum value of -1.

[0063] Or the target similarity is obtained based on the distance similarity and the cosine similarity (after normalizing the respective determined similarities, a weighted operation is performed to obtain the target similarity). When the target similarity is greater than a preset threshold (for example, 90%), it is determined that the collected facial information to be identified and the stored facial information of the resident meet the similarity requirement, and the smart door lock is opened.

[0064] Step S206, when the facial information to be identified fails to be compared with the facial information of the resident personnel, it is determined that the person corresponding to the facial information to be identified is a visiting person.

[0065] Step S208, receiving a request message for a temporary password sent by a visitor, and if the request message is approved, opening the smart door lock based on the temporary password of the visitor.

[0066] In the technical solution provided in step S208, there are multiple ways to open the smart door lock based on the temporary password of the visitor, for example: when the visitor is identified for the first time, a request information for the temporary password sent by the visitor is received, wherein the request information includes: a preset voice command issued by the visitor, the dynamic facial information of the visitor, and the current location of the visitor is located in a preset location area; after the request information meets the preset conditions, the temporary password corresponding to the visitor is determined; verification is performed based on the temporary password and the request information respectively to obtain a first verification result and a second verification result; if both the first verification result and the second verification result are passed, the smart door lock is opened; when the visitor is not identified for the first time, the visitor is verified only through the request information.

[0067] The following is a specific embodiment:

[0068] When the visitor is identified for the first time, the request information of the visitor is verified: the visitor needs to say a specific voice command first, which is uploaded to the temporary password generation system of the smart door lock for verification; the dynamic facial information of the visitor (such as fingerprints, facial features, irises, etc.) is extracted to verify the identity of the visitor; the location of the visitor is checked to verify his / her identity. The visitor can only be successfully verified if he / she is within a specific area or distance to prevent malicious use of biometric information. That is, after the dynamic facial information and voice command are verified, the location of the visitor is also detected. When the location is in the area corresponding to the visited person (such as a community), it is determined that the identity authentication is passed and a temporary password is issued (the temporary password can be sent to the visitor through mobile phone text messages, push notifications from the smart door lock APP, or the permission administrator contacts the visitor to inform him / her).

[0069] For the authority administrator, when the outside visitor is identified, the device automatically issues an early warning, and the terminal device prompts the authority administrator to apply for visiting permissions for the outside visitor. The early warning information includes: the identification time and the facial information of the current outside visitor. When the authority administrator has confirmed the visiting permissions of the outside visitor, after recording the facial information or photo of the outside visitor, the application information and temporary password of the outside visitor (that is, the first verification result and the second verification result mentioned above) are required to be verified simultaneously. After the verification is passed, the outside visitor can enter.

[0070] When it is not the first time to identify a visitor, the visitor is verified only by requesting information, and the smart door lock is opened after the verification is passed.

[0071] The visitor confirms to delete the facial information or photo after the visit, and the permission administrator deletes the facial information or photo of the visitor when the visitor enters or leaves.

[0072] For example, a self-service apartment or hotel can transfer photos of guests to the local edge storage in advance. When the guest checks in for the first time, facial information and a temporary password are required for simultaneous verification. At this time, the dynamic facial information is uploaded to the local edge storage. When the guest is identified for the second time, no temporary password is required, only the dynamic facial information needs to be identified. After the stay, the guest can selectively delete their own dynamic facial information and photos. Alternatively, the dynamic facial information of permanent family members can be uploaded to the local edge storage, and the smart door lock can be unlocked through facial recognition.

[0073] If there are multiple smart door locks with access control in the venue, you can choose to set the corresponding authority level for each smart door lock (or you can choose not to set it, then all access control will be set to low-level authority by default), such as high-level authority, intermediate authority, and low-level authority. The temporary password generated at this time is only used for outsiders to open access control with low-level authority, that is, outsiders are only granted the authority to unlock the access control with low-level authority once. Outsiders need to apply to the authority manager for other permissions, and can only obtain other permissions after the application is passed (outsiders cannot apply for the unlocking authority corresponding to the access control with high-level authority); long-term residents have the unlocking authority of access control with low-level and intermediate authorities. If they want to obtain the unlocking authority corresponding to the access control with high-level authority, they need to apply to the authority manager for the unlocking authority, and can only obtain it after the application is passed.

[0074] It should be noted that the image data corresponding to the facial information of resident personnel and visitors in all the above steps can be stored in the edge node of the local edge storage device. The facial information of visitors is stored in the same way as the facial information of resident personnel. The authority administrator can manage it locally, or upload the facial information to the cloud, store it in the terminal, and perform cloud-side security protection.

[0075] Through the above steps S202-S208, when there are external visitors, the device automatically warns and the recognition system notifies the authorized manager to record the photo leakage. At this time, the authorized manager can set the visual smart door lock for strong protection, and facial recognition and dynamic temporary passwords are required at the same time before entry, which reduces the risk of facial data leakage and improves the safety of personnel entry and exit. The edge storage method makes deployment easier, from the centralization of storage to the marginalization of storage, and the data processing is divided into two parts: end processing and central processing. With the increasing performance of end devices, each end device is no longer a simple data acquisition device. Processing classification and stratification becomes easier. Therefore, the data reading speed can be improved, thereby speeding up the recognition speed and improving the recognition efficiency; effectively utilizing the network bandwidth, when using central storage, all data is transferred to the central storage, and its bandwidth demand is extremely large. When the storage is moved to the edge storage, the network bandwidth is saved and the utilization rate of the network bandwidth is improved. And when there is a lot of stored data, the central storage cannot save all the data, and usually the data can only be saved for a predetermined number of days; the fault tolerance is stronger. When the central storage is used, any network problems or data problems will cause service interruption. When the end device has a certain processing capability, the requirements for the network are reduced after the data is stored on the end. If a network interruption occurs, it will only affect a small part of the functions, and the problem can be handled locally, so it is more fault-tolerant. Improve data security, store data on the end, establish a point-to-point link between the end device and the data center, and allow data to be transmitted privately, thereby improving data security.

[0076] The present application also provides a flowchart of another method for unlocking a smart door lock. Figure 3 As shown, the visual smart door lock authority manager can store the dynamic facial information of permanent personnel (i.e. the facial information of the permanent personnel mentioned above) in the smart door lock system platform (i.e. the local edge storage device mentioned above). When the visual smart door lock (i.e. the smart door lock mentioned above) recognizes facial information outside the door (i.e. the facial information to be identified above), it is first compared with the stored facial information or photos (i.e. the facial information of the permanent personnel mentioned above). After the comparison is successful, the door lock (i.e. the smart door lock mentioned above) can be opened. If the comparison fails and the facial information to be identified does not match the stored facial information or photos, the facial information or photos of the visitor will be transmitted to the authority manager's terminal device and an application for visiting authority will be made (i.e. the request information for receiving the temporary password sent by the visitor mentioned above). In order to protect the facial information or photos of the visitor, the visitor can choose to automatically delete the facial information or photos after entering when applying for entry. The implementation methods of the above steps have been described in. Figure 2 The embodiments are described in detail in the embodiments and will not be repeated here.

[0077] The present application embodiment also provides a schematic diagram of a process for an external visitor to obtain a temporary password, such as Figure 4 As shown, first perform voice command authentication (i.e., the preset voice command issued by the above-mentioned visitor). If the voice command authentication is successful, perform biometric authentication (i.e., the dynamic facial information of the above-mentioned visitor). If any of the above-mentioned voice command authentication or biometric authentication fails, resume voice command authentication. If the biometric authentication is successful, perform location authentication (i.e., the current location of the above-mentioned visitor is in the preset location area), and obtain a temporary password if the location authentication is successful.

[0078] The present application embodiment provides a schematic diagram of the structure of an unlocking device of a smart door lock, such as Figure 5 As shown, the device comprises:

[0079] The acquisition module 502 is used to acquire image data.

[0080] The acquisition module 502 is also used to encrypt the data packet corresponding to the facial information of the resident personnel, and divide the encrypted data packet into multiple slices according to the preset size; perform hash value calculation on each slice to obtain the first hash value corresponding to each slice, and determine the redundant slice corresponding to each slice, wherein the redundant slice is used to recover data in the event of data loss; distribute and store each slice and the corresponding redundant slice on the edge node in the local edge storage device. Perform hash value calculation on the first slice to obtain the second hash value corresponding to the first slice, wherein the first slice is any slice on the edge node stored in the local edge storage device; obtain the third hash value corresponding to the first slice, compare the second hash value corresponding to the first slice with the third hash value, wherein the third hash value is the hash value calculated before the first slice is stored in the edge node; when the second hash value is consistent with the third hash value, determine that the first slice has not changed; when the second hash value is inconsistent with the third hash value, determine that the first slice has changed, and recover the data of the first slice based on the redundant slice of the first slice, the first hash value of the first slice, and the second hash value of the first slice.

[0081] The recognition module 504 is used to recognize image data. When it is recognized that there is facial information to be recognized in the image data, the facial information to be recognized is compared with the facial information of the resident personnel stored in the local edge storage device using edge storage.

[0082] The recognition module 504 is also used to randomly generate a preset number of facial movements, and detect the completion degree of the facial movements of the target object corresponding to the image data. When the completion degree is greater than a first preset threshold, the image data is determined to be a non-static picture; when the completion degree is less than or equal to the first preset threshold, the image data is determined to be a static picture, and an early warning is triggered.

[0083] The recognition module 504 is also used to vectorize the facial information to be recognized and the facial information of the resident personnel respectively to obtain a first feature vector and a second feature vector; calculate the similarity between the first feature vector and the second feature vector, and compare the facial information to be recognized and the facial information of the resident personnel based on the similarity.

[0084] The recognition module 504 is also used to determine the first similarity between the first feature vector and the second feature vector using a first method, and determine the second similarity between the first feature vector and the second feature vector using a second method, wherein the first similarity is used to represent the distance between the first feature vector and the second feature vector, and the second similarity is used to represent the cosine similarity between the first feature vector and the second feature vector; when the first similarity is less than a second preset threshold, it is determined that the facial information to be identified has been compared with the facial information of the resident personnel; or, when the second similarity is greater than a third preset threshold, it is determined that the facial information to be identified has been compared with the facial information of the resident personnel; or, based on the first similarity and the second similarity, a target similarity is determined, and when the target similarity is greater than a fourth preset threshold, it is determined that the facial information to be identified has been compared with the facial information of the resident personnel.

[0085] The determination module 506 is used to determine that the person corresponding to the facial information to be identified is a visitor when the facial information to be identified fails to be compared with the facial information of the resident person.

[0086] The receiving module 508 is used to receive a request message for a temporary password sent by a visitor, and if the request message is approved, the smart door lock is opened based on the temporary password of the visitor.

[0087] The receiving module 508 is also used to receive request information for a temporary password sent by a visitor when the visitor is identified for the first time, wherein the request information includes: a preset voice command issued by the visitor, the dynamic facial information of the visitor, and the current location of the visitor is in a preset location area; after the request information meets the preset conditions, determine the temporary password corresponding to the visitor; perform verification based on the temporary password and the request information respectively to obtain a first verification result and a second verification result; if both the first verification result and the second verification result are passed, open the smart door lock; when the visitor is not identified for the first time, verify the visitor only through the request information.

[0088] It should be noted that the various modules in the unlocking device of the above-mentioned smart door lock can be program modules (for example, a set of program instructions to implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0089] The embodiment of the present application also provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned unlocking method of the smart door lock. For example, image data is acquired; image data is recognized, and when facial information to be recognized exists in the image data, the facial information to be recognized is compared with the facial information of the resident personnel stored in the local edge storage device in an edge storage manner; when the facial information to be recognized fails to be compared with the facial information of the resident personnel, it is determined that the person corresponding to the facial information to be recognized is an outsider; a request information for a temporary password sent by the outsider is received, and when the request information is passed, the smart door lock is opened based on the temporary password of the outsider.

[0090] The embodiment of the present application also provides an electronic device, which includes a processor, and the processor is used to run a program, wherein the above method of unlocking the smart door lock is executed when the program is running. For example, image data is obtained; image data is identified, and when it is identified that there is facial information to be identified in the image data, the facial information to be identified is compared with the facial information of the resident personnel stored in the local edge storage device using edge storage; when the facial information to be identified fails to be compared with the facial information of the resident personnel, it is determined that the person corresponding to the facial information to be identified is an outsider; a request for a temporary password sent by the outsider is received, and when the request is passed, the smart door lock is opened based on the temporary password of the outsider.

[0091] According to another aspect of the embodiment of the present application, a computer program product is also provided, including a computer program, which implements the above unlocking method of the smart door lock when executed by a processor. For example, image data is acquired; image data is identified, and when facial information to be identified is identified in the image data, the facial information to be identified is compared with the facial information of the resident personnel stored in the local edge storage device in an edge storage manner; when the facial information to be identified fails to be compared with the facial information of the resident personnel, it is determined that the person corresponding to the facial information to be identified is an out-of-town visitor; a request for a temporary password sent by an out-of-town visitor is received, and when the request is approved, the smart door lock is opened based on the temporary password of the out-of-town visitor.

[0092] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0097] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for unlocking a smart door lock, characterized in that: include: Get image data; Identify the image data, and when it is identified that there is facial information to be identified in the image data, compare the facial information to be identified with facial information of a resident stored in a local edge storage device in an edge storage manner; In the case where the facial information to be identified fails to be compared with the facial information of the resident personnel, determining that the person corresponding to the facial information to be identified is an outside visitor; Receive the request information of the temporary password sent by the outsider, and if the request information is approved, open the smart door lock based on the temporary password of the outsider.

2. The method according to claim 1, characterized in that After acquiring the image data, the method further includes: randomly generating a preset number of facial actions, and detecting the degree of completion of the facial actions by the target object corresponding to the image data, and determining that the image data is a non-static picture when the degree of completion is greater than a first preset threshold; When the degree of completion is less than or equal to a first preset threshold, the image data is determined to be a static picture, and an early warning is triggered.

3. The method according to claim 1, characterized in that The method further includes: storing the facial information of the resident personnel in a local edge storage device in the following edge storage manner, including: Encrypting a data packet corresponding to the facial information of the resident personnel, and dividing the encrypted data packet into a plurality of fragments according to a preset size; Calculate a hash value for each shard to obtain a first hash value corresponding to each shard, and determine a redundant shard corresponding to each shard, wherein the redundant shard is used to recover data in case of data loss; Each shard and the corresponding redundant shard are distributed and stored on edge nodes in the local edge storage device.

4. The method according to claim 3, characterized in that The method further comprises: Performing hash value calculation on a first shard to obtain a second hash value corresponding to the first shard, wherein the first shard is any shard on an edge node stored in the local edge storage device; Obtain a third hash value corresponding to the first shard, and compare the second hash value and the third hash value corresponding to the first shard, wherein the third hash value is a hash value calculated before the first shard is stored in the edge node; When the second hash value is consistent with the third hash value, determining that the first shard has not changed; When the second Hash value is inconsistent with the third Hash value, it is determined that the first shard has changed, and data recovery is performed on the first shard based on redundant shards of the first shard, the first Hash value of the first shard, and the second Hash value of the first shard.

5. The method according to claim 1, characterized in that The facial information to be identified is compared with the facial information of the resident personnel stored in the local edge storage device using edge storage, including: Vectorizing the facial information to be identified and the facial information of the resident respectively to obtain a first feature vector and a second feature vector; The similarity between the first feature vector and the second feature vector is calculated, and based on the similarity, the facial information to be identified is compared with the facial information of the resident.

6. The method according to claim 5, characterized in that The facial information to be identified is compared with the facial information of the resident based on the similarity, including: Determine a first similarity between the first feature vector and the second feature vector in a first manner, and determine a second similarity between the first feature vector and the second feature vector in a second manner, wherein the first similarity is used to represent the distance between the first feature vector and the second feature vector, and the second similarity is used to represent the cosine similarity between the first feature vector and the second feature vector; When the first similarity is less than a second preset threshold, it is determined that the facial information to be identified passes the comparison with the facial information of the resident; or: When the second similarity is greater than a third preset threshold, it is determined that the facial information to be identified passes the comparison with the facial information of the resident; or: A target similarity is determined based on the first similarity and the second similarity, and when the target similarity is greater than a fourth preset threshold, it is determined that the facial information to be identified passes the comparison with the facial information of the resident.

7. The method according to claim 1, characterized in that Opening the smart door lock based on the temporary password of the visitor includes: When the visitor is identified for the first time, a request message for a temporary password sent by the visitor is received, wherein the request message includes: a preset voice command issued by the visitor, dynamic facial information of the visitor, and the current location of the visitor is in a preset location area; After the request information meets the preset conditions, determine the temporary password corresponding to the external visitor; Perform verification based on the temporary password and the request information respectively to obtain a first verification result and a second verification result; If both the first verification result and the second verification result are passed, opening the smart door lock; When the outside visitor is not identified for the first time, the outside visitor is verified only through the request information.

8. An unlocking device for a smart door lock, characterized in that: include: An acquisition module, used for acquiring image data; A recognition module, configured to recognize the image data, and when it is recognized that there is facial information to be recognized in the image data, compare the facial information to be recognized with facial information of a resident stored in a local edge storage device in an edge storage manner; A determination module, configured to determine that the person corresponding to the facial information to be identified is an outside visitor if the facial information to be identified fails to be compared with the facial information of the resident person; The receiving module is used to receive the request information of the temporary password sent by the outsider, and when the request information is passed, the smart door lock is opened based on the temporary password of the outsider.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the unlocking method of the smart door lock described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program, when running, executes the unlocking method of the smart door lock described in any one of claims 1 to 7.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by the processor, the unlocking method of the smart door lock described in any one of claims 1 to 7 is implemented.

Images