Intelligent door lock system based on multi-modal verification and timestamp encryption and unlocking scheme

Abstract

The invention provides an intelligent door lock system based on multi-modal verification and timestamp encryption and an unlocking scheme, belongs to the technical field of smart homes, and aims to solve the problems of personal health and privacy and easiness in environmental interference of a door lock combined with multiple authentication modes. The intelligent door lock system comprises a main control chip, a dynamic password generation module, a Bluetooth module, an NFC module, a camera module, an optical communication module, a lock body module and a storage module. The unlocking method comprises the steps that S1, the unlocking mode of a user is judged through the main control chip; s2, calling a corresponding module for processing according to an unlocking mode selected by a user, executing a corresponding instruction according to a processing result, and transmitting the instruction to a lock body module; and S3, uploading the unlocking record of this time to an unlocking log, and uploading a switching log to a storage module for storage so as to be convenient for a user to call.

Description

TECHNICAL FIELD

[0001] The present application relates to a smart door lock system and unlocking scheme based on multi-modal verification and timestamp encryption, belonging to the technical field of smart home. BACKGROUND

[0002] The smart door lock industry has developed rapidly in recent years and has become an important part of the smart home and Internet of Things fields. With the increasing awareness and acceptance of smart home products by consumers, the market demand for smart door locks continues to grow. Currently, the global smart door lock market size is about $7403 million in 2024, and is expected to reach $33130 million by 2031, with a compound annual growth rate (CAGR) of 24.5%.

[0003] The technical level of smart door locks is constantly improving, with increasingly rich functions. Fingerprint recognition, facial recognition, password input, and other biometric technologies are widely used, significantly improving the security, convenience, and user experience of smart door locks. In addition, the trend of smart door locks integrating with smart home systems is evident, such as linking with smart cameras, smart lighting, and other devices to achieve comprehensive protection of home security.

[0004] Authentication methods: Currently, the main unlocking methods for smart door locks include fingerprint recognition, facial recognition, password input, NFC, Bluetooth, Wi-Fi, and other mainstream products support 2-3 unlocking methods (such as fingerprint + password + NFC), but multi-modal data is not deeply integrated. Here is a specific analysis: 1. Fingerprint recognition: Most smart door locks currently support fingerprint recognition, with continuously improving recognition speed and accuracy. For example, Xiaomi Smart Door Lock 2 uses a new generation of semiconductor fingerprint recognition technology with built-in AI algorithms, with a fingerprint recognition speed of ≤0.5 seconds and an accuracy of ≥99.51%.

[0005] 2. Facial recognition: Facial recognition technology is also increasingly widely used in smart door locks. Some high-end products such as Desmann smart door locks use 3D cameras to establish user facial models and perform verification through live detection and facial recognition algorithms.

[0006] 3. NFC and Bluetooth: NFC and Bluetooth unlocking methods are popular with users due to their convenience. For example, Huawei smart door locks support unlocking through mobile phone NFC or Bluetooth.

[0007] 4. Optical communication technology: Optical communication unlocking is an innovative technology that is less common on the market, but similar visible light communication technology has been researched and applied.

[0008] Defects and deficiencies of the prior art: At present, fingerprint recognition and face recognition technology has become the market mainstream due to its convenience and security, but traditional door locks mostly use single authentication methods such as mechanical keys, digital passwords or fingerprint recognition, which are highly dependent on single environmental conditions and cannot adapt to diverse scenarios. Secondly, if a certain authentication method fails, the door lock cannot be opened. In addition, the security is relatively low and easy to be cracked or stolen. With the development of technology, some intelligent door locks begin to use a combination of multiple authentication methods to improve security and user experience, such as: Fingerprint recognition + digital password: Its unlocking method is limited to contact unlocking, which has problems for personal health and privacy. For example, during the epidemic period, users may reject to frequently expose to the object of touching the door lock. The fingerprint recognition module leaves fingerprint traces during the identification process, which can be extracted and copied by criminals for illegal unlocking, and at the same time prevents illegal access to user passwords through physical channels. Environmental impact: The fingerprint recognition module is sensitive to environmental humidity, temperature and finger surface state (such as moisture, dirt and wear), which may cause identification failure Face recognition + Bluetooth recognition: highly dependent on wireless communication such as wifi, and easily interfered by metal door body and electromagnetic shielding environment (such as bank vault); NFC and Bluetooth unlocking methods require user devices (such as mobile phones) to support specific functions, which may limit the use of some users. For example, some old mobile phones may not support NFC function.

[0009] At present, symmetric encryption technology is commonly used in intelligent door locks, which uses the same key for encryption and decryption, but the key management has risks. Most intelligent door locks use static passwords or dynamic password generation mechanisms based on time synchronization. If the static password is known, it will be leaked. If the time is known, the dynamic password generation mechanism based on time synchronization may be reversed to deduce the encryption logic, which has low security. High-quality face recognition modules have high costs, which increases the overall cost of intelligent door locks. SUMMARY

[0010] To solve the problems of personal health and privacy and the vulnerability to environmental interference of door locks combined with multiple authentication methods, the present application proposes an intelligent door lock system and unlocking scheme based on multi-modal verification and timestamp encryption.

[0011] The technical solution adopted by the present application to solve the above problems is: the intelligent door lock system based on multi-modal verification and timestamp encryption proposed by the present application comprises: a main control chip, a dynamic password generation module, a Bluetooth module, an NFC module, a camera module, an optical communication module, a lock body module and a storage module; The master control chip is responsible for the control and coordination of the modules in the system, receives data from the NFC module, camera module, and optical communication module, and compares the corresponding data with the data in the storage module. If they match, the unlocking instruction is sent to the lock body module. If they do not match, the unlocking fails. The dynamic password generation module is used to store the 5 groups of different time sequences stored in the storage module for random selection and random difference value, and to perform AES-258 symmetric encryption on the password after random selection and random difference value, to generate a dynamic password and transmit it to the lock body module. The Bluetooth module is used for close-range communication with the user's mobile phone, and for initialization setting, user management, and unlocking record query of the door lock. The NFC module uses an encrypted communication method to read the NFC card number and NFC-related information of the NFC device and transmit them to the master control chip. The lock body module uses an electronic control mechanical lock body with automatic locking and unlocking functions, and is used to receive instructions from the master control chip for unlocking. The camera module is used for face recognition and two-dimensional code scanning. The optical communication module is used to obtain unlocking information from the user's app call to the mobile phone flash light emitting high-frequency visible light, and to convert the optical signal into an electrical signal for transmission to the master control chip. The storage module is used to store user information, unlocking records, unlocking pre-stored information, 5 groups of mutually different time sequences, and authorized card numbers.

[0012] Further, the master control chip receives the read NFC card number and NFC-HCE. For the NFC card number, it is compared with the authorized card number in the storage module. If the comparison is successful, the unlocking instruction is sent to the lock body module. When the NFC-HCE is obtained, the phone's own NFC function is called to make the user's phone act as an NFC simulation card and communicate with the door lock. The password is sent to the door lock through near field communication and transmitted to the door lock module. The door lock module checks the password and dynamic password to determine whether to unlock.

[0013] The master control chip receives the face information scanned by the camera module and matches it with the user information in the storage module. If the match is successful, the unlocking instruction is sent to the lock body module. The master control chip receives the electrical signal transmitted by the optical communication module and decodes it to extract the unlocking information. It is matched with the unlocking pre-stored information in the storage module. If the match is successful, the unlocking instruction is sent to the lock body module.

[0014] The unlocking scheme of the intelligent door lock based on multi-modal verification and dynamic encryption includes: Step 1: Determine the user's unlocking method through the master control chip. Step 2: According to the user's selection of the unlocking mode, the corresponding module is called for processing, and the main control chip accepts the processing data from each module, compares the data with the data in the storage module, and if it matches, sends an unlocking instruction to the lock body module, if it does not match, the unlocking fails; Step 3: Upload this unlocking record to the unlocking log, and upload the switch log to the storage module for saving, to facilitate user call.

[0015] Further, in step 2, if the user selects face recognition, the camera module automatically collects the user's face image, and the image is enhanced through active infrared fill light method; The enhanced image is preprocessed, wherein the preprocessing operation includes but is not limited to grayscale, noise reduction and normalization; A deep learning algorithm is used, and a convolutional neural network trained with batch face data is used to extract features from the preprocessed face image to generate a face feature vector, and the face feature vector is transmitted to the main control chip; The main control chip compares it with the face feature template in the user information in the storage module, calculates the similarity of the face feature vector and the face feature template, and when the similarity exceeds the preset threshold, it is considered to be matched successfully, and the main control chip sends an unlocking instruction to the lock body module, and when the similarity does not exceed the preset threshold, the unlocking fails.

[0016] Further, in step 2, if the user selects NFC unlocking, the NFC module listens to the surrounding NFC signal in real time, when a device with NFC function is detected close, the NFC module reads the NFC card number or NFC-HCE from the NFC device using encrypted communication method, when the NFC card number is obtained, the read NFC card number is transmitted to the main control chip, the main control chip receives the read NFC card number, and compares it with the authorized card number in the storage module, if it matches, sends an unlocking instruction to the lock body module, if it fails, the unlocking fails; When NFC-HCE is obtained, the phone's own NFC function is called, and the user's mobile phone is used as an NFC simulation card and a door lock to communicate, the password is sent to the door lock through near field communication and transmitted to the door lock module, and the door lock module verifies the transmitted password and the generated dynamic password to determine whether to unlock.

[0017] Further, the generation of the dynamic password specifically includes: The master control chip calls 5 groups of different time sequences stored in the storage module and transmits the time sequences to the dynamic password generation module; each group of sequences is dynamically increased in time according to different time rules, and one group is randomly selected from the 5 groups of different time sequences, and a password is formed by random interpolation of 3 positions in the selected sequence through random number generation, wherein the inserted values include 64 types of characters, and the password is subjected to AES-258 symmetric encryption; a new dynamic password is generated immediately after the user completes the unlocking, and the dynamic password is sent out every 10 minutes.

[0018] Further, if the user selects light communication unlocking in step 2, the app calls the flashlight function of the mobile phone, transmits the unlocking information to the light communication module through high-frequency visible light, the light communication module converts the received light signal into an electrical signal through the visible light receiving unit and transmits it to the master control chip, the master control chip decodes the received electrical signal, extracts the unlocking information, and matches it with the pre-stored unlocking information in the storage module, if the matching is successful, the unlocking instruction is sent to the lock body module, if the matching fails, the unlocking fails.

[0019] Further, during the transmission of the unlocking information through high-frequency visible light, the high-frequency visible light is converted into a 01 sequence according to the on-off information of the high-frequency visible light, and during the information encoding process of the unlocking visible light receiving unit, the continuous 5 high-level signals are the start bits, the middle 72 bits are the data bits, the last 1 bit is the parity check bit, and the last continuous 5 low-level signals are the end bits.

[0020] Further, if the user selects two-dimensional code unlocking in step 2, the user's mobile phone APP generates a dynamic two-dimensional code, the dynamic two-dimensional code is scanned by the camera module, the information in the dynamic two-dimensional code is obtained, and the obtained information is transmitted to the master control chip. The master control chip decodes the scanned dynamic two-dimensional code information, extracts the user ID, timestamp and encryption key information contained therein, compares the extracted user ID with the authorized user ID in the storage module, verifies the unlocking authority of the user, and performs encryption and verification on the two-dimensional code information according to the timestamp and encryption key, if the verification and verification are passed, the master control chip sends an unlocking instruction to the lock body module.

[0021] The beneficial effects of the present application are: (1) For the single verification mode of the traditional door lock, the multi-modal fusion verification mechanism proposed in the application reduces the environmental dependence of the door lock verification and improves the applicability in different scenarios. Even if a certain authentication method fails, the user can still unlock through other methods, improving the reliability of the system. For example, in a weak light environment, face recognition and two-dimensional code recognition are difficult, and light communication unlocking and NFC unlocking can be used. In a severe electromagnetic interference scenario, light communication can be used. In addition, the light communication technology is not limited by device compatibility and can be used as long as there is a light source. In the case of low battery power, the energy consumption of the mobile phone driving the flash is less. In addition, in the night scene, the light communication method is also more convenient for the user to unlock (2) For the traditional door lock security encryption technology, the application adopts the method of storing multiple time sequences internally, randomly interpolating and then encrypting, using a dynamic password update mechanism and AES encryption technology, and the password is updated every 10 minutes. Only send out information, not receive information, effectively prevent password cracking and malicious attacks. Five groups of self-walking time sequences are stored internally, one of which is randomly selected for random interpolation and then encrypted, increasing the complexity and unpredictability of the password.

[0022] (3) The intelligent door lock of the application has low cost, and the self-made light sensor (selecting VEML6040 module) costs only 1 / 5 of the NFC chip (such as ST25R3911B) and the face module (such as U3 of Obitongguang). The face recognition of the application is deployed in the domestic Ruikai micro LuckFox Pico Pro / Max, and the cost is less than 80 yuan, which is much lower than the cost of general face recognition hardware devices. On the premise of realizing the same performance, the hardware required for the model deployment is also the lowest in the country. The cost of the door lock can be greatly reduced.

[0023] (4) One-way communication mechanism: only necessary dynamic passwords are stored in the door lock when designing, and the communication line for uploading to the cloud is refused, and the communication line is a local self-organizing private network, and external network signals are refused. Input, that is, do not receive external control instructions, effectively avoid external network attacks, and increase the security of the door lock (5) The deep learning model used in the application is deployed on the edge device, and the data related to user privacy is only operated and verified locally without involving the cloud, ensuring that privacy data is not leaked and providing better security protection. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 It is a structure diagram of an intelligent door lock system based on multi-modal verification and timestamp encryption; Figure 2 It is a circuit diagram of a visible light signal receiving unit in a self-made 20KHz light communication module; Figure 3PCB design diagram of visible light signal receiving unit in self-made 20KHz optical communication module Figure 4 Schematic diagram of visible light signal receiving unit in self-made 20KHz optical communication module Figure 5 PN532 NFC schematic diagram for NFC communication Figure 6 Camera module schematic diagram Figure 7 Edge deployment device schematic diagram of face recognition neural network Figure 8 Industrial design structure diagram of intelligent door lock system Figure 8 , 1 - door lock module, 2 - NFC module, 3 - ESP32S3 main control module, 4 - camera module, 5 - light sensor module Figure 9 Schematic diagram of door lock central processor ESP32S3 in door lock module Figure 10 PCB design diagram of self-made door lock core processing module Figure 11 Schematic diagram of self-made door lock core processing module Figure 12 Flowchart of intelligent door lock unlocking scheme based on multi-modal verification and dynamic encryption Figure 13 Flowchart of face recognition DETAILED DESCRIPTION

[0025] Specific implementation one: combined with Figures 1-9 Explain this embodiment, as Figure 1 shown, the structure of the intelligent door lock system based on multi-modal verification and timestamp encryption described in this embodiment includes: The main control chip selects a high-performance, low-power microcontroller, ESP32S3, as shown in Figure 9 , responsible for the control and coordination of modules within the system, receiving data from the NFC module, camera module, optical communication module, and comparing the corresponding data with the data in the storage module. If matched, send the opening instruction to the lock body module. If not matched, the opening fails.

[0026] The custom communication encoding protocol stored in the main control chip is shown in Table 1: Table 1

[0027] The dynamic password generation module is used to randomly select and randomly differ 5 different time series stored in the storage module, and then perform AES-258 symmetric encryption on the randomly selected and randomly differred passwords to generate dynamic passwords.

[0028] The Bluetooth module, specifically the Bluetooth Low Energy (BLE) module, is used for short-range communication with the user's mobile phone to perform door lock initialization settings, user management, and unlock record query.

[0029] The NFC module integrates an NFC reader / writer, using encrypted communication to read the NFC card number and NFC-HCE of NFC devices and transmit them to the main control chip. The NFC module should comply with relevant standards such as ISO 14443 to ensure compatibility and security. The PN532NFC is used for NFC communication. Figure 5 As shown.

[0030] Camera modules, such as Figure 6 As shown, a high-definition camera is used for face recognition and QR code scanning. The camera has good low-light performance and autofocus, adapting to usage needs in different environments. During face recognition, the camera captures a face image and transmits it to the main control chip for processing.

[0031] like Figure 7 As shown, the facial recognition system of this invention is deployed on the domestically produced Rockchip LuckFox Pico Pro / Max, at a cost of no more than 80 yuan, far lower than the cost of general facial recognition hardware. Under the premise of achieving the same performance, the hardware required for this model is also the lowest in the country. This can greatly reduce door lock costs.

[0032] The optical communication module calls the visible light receiving unit to obtain unlocking information from the high-frequency visible light emitted by the user's mobile phone flashlight via the app, and converts the optical signal into an electrical signal to transmit it to the main control chip; The circuit diagram of the visible light receiving unit in the self-made 20kHz optical communication module is as follows: Figure 2 As shown, the PCB design diagram of the visible light signal receiving unit in the self-made 20kHz optical communication module is as follows. Figure 3 As shown; the visible light signal receiving unit in the self-made 20kHz optical communication module is as follows. Figure 4 As shown, the cost of a self-made optical sensor (using the VEML6040 module) is only 1 / 5 of that of an NFC chip (such as ST25R3911B) and a face module (such as Orbbec U3).

[0033] The lock body module uses an electronically controlled mechanical lock body, which has automatic locking and unlocking functions and is used to receive instructions from the main control chip to unlock.

[0034] The PCB design diagram of the self-made door lock core processing module is as follows:Figure 10 As shown in the self-made door lock core processing module as Figure 11 As shown, The storage module uses a large-capacity flash memory to store user information, unlocking records, unlocking pre-stored information, five groups of different time sequences and authorized card numbers. The storage module has data encryption and backup functions to prevent data loss and illegal reading. The connection mode between each module includes: the self-made light sensor module connects the GPIO17 of the single-chip microcomputer ESP32S3 through digital output (OUTPUT) pin, and the power supply connects the 3.3v of the single-chip microcomputer. The IRQ of PN532 connects GPIO4, the RESET connects GPIO5, the SDA connects GPIO13, the SCL connects GPIO11, and the vcc connects the 3.3v of the ESP32S3. The ESP32S3 is cross-connected through the RX pin and the UART4_TX_M1 pin of the LuckFox Pico Pro / Max, and the GEN is common, and the LuckFox Pico Pro / Max is connected with the SC3336 COM V2. The door lock core is an electromagnetic lock, and the power supply is connected with the GPIO47 of the ESP32S3 through a high-power MOS tube, and the electromagnetic lock current is controlled by the signal output by the ESP32S3, thereby controlling the opening and closing of the door lock. The spatial positional relationship of each module includes that the camera for face recognition is located at the uppermost part of the door lock, and the visible light communication module, the ESP32S3 single-chip microcomputer, the LuckFox Pico Pro / Max, the PN532 and the electromagnetic lock are sequentially arranged below.

[0035] In order to improve the integration of the door lock and simplify the installation, the application designs a door lock core processing module as Figure 11 Each component is directly inserted on the PCB and can be used, and the required elements of the intelligent door lock system are directly connected by the circuit board, which ensures the stability and firmness of the connection, and a 3D shell of the door lock is independently designed to effectively protect all structures of the door lock. In the embodiment, the industrial design structure of the intelligent door lock system is as shown in Figure 8 .

[0036] In addition to the above settings, the embodiment additionally provides a user management system and a data backup and recovery module. User management system: develop a matching mobile phone APP and cloud server for user management. Users can register, log in, add family members, set unlocking permissions, view unlocking records and other operations through the APP. The APP and the cloud server use a secure communication protocol for data transmission to ensure the security of user information. The cloud server is responsible for storing user registration information, unlocking permissions and other data, and synchronizing data with the door lock. Data backup and recovery: The cloud server regularly backs up user information, unlocking records, and other data to prevent data loss. In the event of a door lock replacement or malfunction, users can restore data from the cloud server through the mobile app, ensuring the normal use of the door lock.

[0037] Specific implementation method two: combination Figure 12 and Figure 13 This embodiment is illustrated as follows: Figure 12 The unlocking scheme of the intelligent door lock system based on multi-modal verification and dynamic encryption according to the embodiment includes the following steps: S1: Determine the user's unlocking method through the main control chip; S2: According to the user's selected unlocking method, call the corresponding module for processing, and the main control chip accepts the processing data from each module, compares the data with the data in the storage module, if matched, sends the unlocking instruction to the lock body module, if not matched, the unlocking fails; S201: Face recognition unlocking; As shown in Figure 13 The steps of face recognition unlocking include: S20201: Face image acquisition: When the user approaches the door lock, the camera automatically starts and collects the user's face image. The camera can use active infrared fill light technology to improve the image quality in low light environment. S20202: Image preprocessing: Perform grayscale, noise reduction, normalization, and other preprocessing operations on the collected face image to improve the accuracy of subsequent feature extraction. S20203: Feature extraction: Use deep learning algorithms, convolutional neural networks (CNN), to extract features from the preprocessed face image and generate a face feature vector. In the training stage, use a large amount of face image data to train the CNN model to improve the recognition accuracy of the model. S20204: Feature comparison: Compare the extracted face feature vector with the registered face feature template stored in the Flash memory. Use Euclidean distance or cosine similarity algorithm to calculate the similarity between the feature vectors. When the similarity exceeds the set threshold, it is determined that the matching is successful, and the main control chip sends the unlocking instruction to the lock body. S202: NFC unlocking; S20201: NFC device detection: The NFC module continuously listens to the surrounding NFC signals, and when it detects a device with NFC function approaching, it starts the NFC communication process. S20202: Data reading: The NFC module reads the NFC card number or NFC-HCE from the NFC device (such as a mobile phone or access card). When reading data, use encrypted communication method to prevent data from being stolen or tampered with. S20203: Permission verification: compare the read NFC card number with the authorized NFC card number stored in the Flash memory to verify the user's unlocking permission. If the comparison is successful, the master chip sends an unlocking instruction to the lock body; When the NFC-HCE is acquired, the phone's own NFC function is called, and the user's phone is used as an NFC simulation card to communicate with the door lock. The password is sent to the door lock module through near field communication, and the door lock module checks the transmitted password and the generated dynamic password to determine whether to unlock.

[0038] Dynamic password generation: the master chip calls the 5 groups of different time sequences stored in the storage module and transmits them to the dynamic password generation module. Each group of sequences is dynamically incremented in time according to different time rules, and a group is randomly selected from the 5 groups of different time sequences, and a random number is generated to randomly insert values from 3 positions in the selected sequence to form a password. The inserted values include various 64 characters, and the password is AES-258 symmetrically encrypted. A new dynamic password is generated immediately after the user completes unlocking, and the dynamic password is sent out every 10 minutes.

[0039] S203: Dynamic two-dimensional code unlocking; S20301: Two-dimensional code scanning: the user opens the phone APP and generates a dynamic two-dimensional code. The camera of the door lock scans the two-dimensional code on the phone screen to obtain the information in the two-dimensional code. The camera should have fast recognition and error correction capabilities to ensure the accuracy of two-dimensional code scanning.

[0040] S20302: Information decoding: the master chip decodes the scanned two-dimensional code information and extracts the user ID, timestamp, encryption key, etc. contained therein. S20303: Permission verification and encryption verification: compare the extracted user ID with the authorized user ID stored in the Flash memory to verify the user's unlocking permission. At the same time, according to the timestamp and encryption key, the two-dimensional code information is encrypted and verified to prevent the two-dimensional code from being forged or reused. If the verification and verification are passed, the master chip sends an unlocking instruction to the lock body. S204: Light communication unlocking; S20401: Light signal transmission and reception: the user calls the phone's flashlight function through the app to emit unlocking information (such as encrypted identification code) through high-frequency visible light. The light communication module of the door lock receives the light signal and converts it into an electrical signal to transmit to the master chip. The light communication module has good directivity and anti-interference ability to ensure accurate transmission of the light signal. S20402: Signal decoding and verification: the master chip decodes the received electrical signal and extracts the unlocking information. Then, the extracted unlocking information is compared with the pre-stored information stored in the Flash memory to verify the user's unlocking authority. If the comparison is successful, the master chip sends an unlocking instruction to the lock body. The application sets up a one-way communication mechanism, and only necessary dynamic passwords are retained for the communication line of uploading to the cloud when the door lock is designed, and the communication line is a local self-organizing private network, and the input of external network signals is refused, that is, external control instructions are not received, which effectively avoids external network attacks and increases the security of the door lock.

[0041] After replacing each module with the model in embodiment one, the workflow includes: ESP32S3 as the intelligent door lock center processor, real-time generation of dynamic passwords, and regular updating to the cloud. Visible light communication and NFC communication are directly deployed on ESP32S3, and in parallel, Freeotos receives external door opening signals in real time while generating dynamic passwords. If it is a visible light signal and an NFC signal, the internal logic is processed and judged whether to open the door and record the opening behavior. Face recognition and temporary two-dimensional code recognition are deployed in LuckFox Pico Pro / Max, and the machine learning model is locally deployed to determine whether to open the lock only in the local LuckFox Pico Pro / Max, and then the result is transmitted to ESP32S3 through UART communication. ESP32S3 also uses Freeotos to listen to data from LuckFox Pico Pro / Max in real time, and according to the data result, the on-off control of the electromagnetic lock is realized, and the opening log is recorded and uploaded.

[0042] S3: upload the opening log of this time to the opening log, and upload the switch log to the storage module for saving, to facilitate user call.

[0043] In summary, for the traditional door lock security encryption technology, we use the method of internal storage of multiple time series for random interpolation and then encryption, use the dynamic password update mechanism and AES encryption technology, the password is updated every 10 minutes, and only sends out and does not receive information, effectively preventing the password from being cracked and malicious attacks. 5 groups of self-walking time series are stored internally, and one of them is randomly selected for random interpolation and then encryption, which increases the complexity and unpredictability of the password.

[0044] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed with preferred embodiments as above, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical contents without departing from the technical solution of the present application, and can make equivalent embodiments with equivalent changes, as long as they do not depart from the technical solution of the present application and are within the spirit and principle of the present application. Any simple modification, equivalent replacement and improvement of the above embodiments, as long as they are within the protection scope of the present application, are still within the protection scope of the present application.

Claims

1. A smart door lock system based on multi-modal authentication and timestamp encryption, characterized in that, The application relates to a smart lock system, which comprises a main control chip, a dynamic password generation module, a Bluetooth module, an NFC module, a camera module, an optical communication module, a lock body module and a storage module. The main control chip is responsible for the control and coordination of the modules in the system, receives data from the NFC module, the camera module and the optical communication module, compares the corresponding data with the data in the storage module, and sends an unlocking instruction to the lock body module if the data match, or fails to unlock if the data do not match. The dynamic password generation module is used for storing 5 groups of different time sequences in the storage module for random selection and random difference value, and performing AES-258 symmetric encryption on the password after random selection and random difference value to generate a dynamic password which is transmitted to the lock body module. The Bluetooth module is used for close-range communication with a user's mobile phone to initialize the door lock, manage users and query the unlocking record. The NFC module reads the NFC card number and NFC-HCE of an NFC device in an encrypted communication mode and transmits the data to the main control chip. The lock body module adopts an electronic control mechanical lock body and has automatic locking and unlocking functions and is used for receiving an instruction from the main control chip to unlock. The camera module is used for face recognition and two-dimensional code scanning. The optical communication module calls a visible light receiving unit to obtain unlocking information from high-frequency visible light emitted by a mobile phone flash through an app and converts the optical signal into an electrical signal which is transmitted to the main control chip. The storage module is used for storing user information, unlocking records, pre-stored unlocking information, 5 groups of different time sequences and authorized card numbers. The main control chip receives the read NFC card number and NFC-HCE, compares the NFC card number with the authorized card numbers in the storage module, and sends an unlocking instruction to the lock body module if the NFC card number matches; when the NFC-HCE is obtained, the mobile phone is used as an NFC simulation card to communicate with the door lock, the password is sent to the door lock through near field communication and is transmitted to the door lock module, and the door lock module checks the password and the dynamic password to determine whether to unlock.

2. The smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 1, wherein, The main control chip receives face information scanned by the camera module and matches the face information with the user information in the storage module, and sends an unlocking instruction to the lock body module if the face information matches. The main control chip receives the electrical signal transmitted by the optical communication module, decodes the electrical signal, extracts the unlocking information and matches the unlocking information with the pre-stored unlocking information in the storage module, and sends an unlocking instruction to the lock body module if the unlocking information matches. The application further relates to a smart lock system unlocking method.

3. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption, applied to the smart door lock system based on multi-modal verification and timestamp encryption according to any one of claims 1-2, characterized in that, Step 1: determining the unlocking mode of a user through the main control chip; Step 2: calling corresponding modules to process according to the unlocking mode selected by the user, receiving processing data from the modules by the main control chip, comparing the data with the data in the storage module, and sending an unlocking instruction to the lock body module if the data match, or failing to unlock if the data do not match; Step 3: uploading the unlocking record of this time to the unlocking log and uploading the switching log to the storage module for storage to facilitate user calling. If the user selects face recognition in step 2, the camera module automatically collects the face image of the user and performs image enhancement through an active infrared fill light method.

4. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 3, wherein, ​ The enhanced image is preprocessed, wherein the preprocessing operation includes but is not limited to graying, noise reduction and normalization; A deep learning algorithm is adopted, and a convolutional neural network trained by batch face data is used to extract features of the preprocessed face image to generate a face feature vector, which is transmitted to the master control chip; The master control chip compares the face feature vector with the face feature template in the user information in the storage module, calculates the similarity of the face feature vector and the face feature template, and when the similarity exceeds a preset threshold, it is considered that the matching is successful, and the master control chip sends an unlocking instruction to the lock body module; when the similarity does not exceed the preset threshold, the unlocking fails.

5. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 3, wherein, In step 2, if the user selects NFC unlocking, the NFC module listens to the surrounding NFC signals in real time, when a device with NFC function is detected close, the NFC module reads the NFC card number or NFC-HCE from the NFC device in an encrypted communication mode, when the NFC card number is obtained, the read NFC card number is transmitted to the master control chip, the master control chip receives the read NFC card number and compares it with the authorized card number in the storage module, if the matching is successful, an unlocking instruction is sent to the lock body module, if the matching fails, the unlocking fails; when the NFC-HCE is obtained, the phone's own NFC function is called, the user's mobile phone is used as an NFC simulation card to communicate with the door lock, the password is sent to the door lock through near field communication and transmitted to the door lock module, and the door lock module verifies the transmitted password and the generated dynamic password to determine whether to unlock.

6. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 5, wherein, The generation of the dynamic password specifically includes: The master control chip calls 5 groups of different time sequences stored in the storage module and transmits them to the dynamic password generation module, each group of sequences is dynamically increased in time according to different time rules, and a group is randomly selected from the 5 groups of different time sequences, and a random number is generated, 3 positions in the selected sequence are randomly interpolated to form a password, wherein the interpolated values include 64 types of characters, and the password is AES-258 symmetrically encrypted, a new dynamic password is generated immediately after the user completes the unlocking, and the dynamic password is sent out every 10 minutes.

7. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 3, wherein, In step 2, if the user selects optical communication unlocking, the app calls the mobile phone flash function, sends the unlocking information to the optical communication module through high-frequency visible light, the optical communication module converts the received optical signal into an electrical signal through the visible light receiving unit and transmits it to the master control chip, the master control chip decodes the received electrical signal, extracts the unlocking information, and matches it with the pre-stored unlocking information in the storage module, if the matching is successful, an unlocking instruction is sent to the lock body module, if the matching fails, the unlocking fails.

8. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 7, wherein, During the transmission of the unlocking information through high-frequency visible light, the high-frequency visible light is converted into a 01 sequence according to the on-off information of the high-frequency visible light, and during the encoding process of the unlocking visible light receiving unit, the continuous 5 high-level signals are the start bits, the middle 72 bits are the data bits, the last bit is the parity check bit, and the last continuous 5 low-level signals are the end bits.

9. The unlocking scheme of the smart door lock system based on multi-modal verification and timestamp encryption as claimed in claim 3, wherein, If the user selects the two-dimensional code unlocking in step 2, a dynamic two-dimensional code is generated by the user's mobile phone APP, the dynamic two-dimensional code is scanned by the camera module, the information in the dynamic two-dimensional code is obtained, and the obtained information is transmitted to the main control chip; The main control chip decodes the scanned dynamic two-dimensional code information, extracts the user ID, timestamp and encryption key information contained therein, compares the extracted user ID with the authorized user ID in the storage module, verifies the unlocking permission of the user, and performs encryption verification on the two-dimensional code information according to the timestamp and the encryption key. If the verification and verification pass, the main control chip sends an unlocking instruction to the lock body module.

Images