Intelligent lock control system and method based on embedded edge AI

Abstract

The invention relates to an intelligent lock control system and method based on an embedded edge AI. The method comprises the following steps: firstly, building a system hardware platform by taking an edge intelligent computing K210 chip as a main component; secondly, face information acquisition is realized by adopting an OpenMV machine vision module; thirdly, training a face recognition model based on a YOLO deep learning network to realize face registration and recognition; then, an HC-08BLE4.0 Bluetooth communication module is adopted to realize data interaction between the WeChat applet and the intelligent lock, and transmission information is encrypted through an AES algorithm, so that the transmission efficiency is improved, and the communication security is ensured; finally, the system is externally connected with a matrix keyboard to achieve panel function operation of the intelligent lock, and meanwhile a motor drive circuit is externally expanded to achieve clutch control of the intelligent lock. The system has the functions of user management, face registration, face recognition unlocking, WeChat unlocking, record query and the like, and the intelligent lock WeChat applet improves the convenience of user operation. The intelligent lock control system designed by the invention has the characteristics of light weight, low cost and low power consumption, and better meets the design and market requirements of the intelligent lock.

Description

Technical Field

[0001] The present invention relates to the field of smart lock technology, and in particular to a smart lock control system and method based on embedded edge AI. Background Art

[0002] With the development of Internet of Things and wireless communication technology, the concept of smart life has gradually entered people's field of vision, and various smart home products have emerged in the market. At the same time, as the gatekeeper of home security, traditional mechanical locks and NFC recognition access control systems can no longer meet people's needs for home life security,

[0003] Smart door locks are gaining popularity due to their intelligence, security, and convenience. With the development of new biometric technologies, facial and fingerprint recognition are also being applied to smart locks. This improves security while optimizing the user experience, providing efficient and reliable identity authentication solutions for smart locks in various scenarios, including homes and businesses.

[0004] Currently, most smart locks on the market use fingerprint recognition technology to authenticate user identities. The uniqueness of fingerprints also ensures the security of smart locks. However, in actual use, fingerprint locks have also exposed some problems. The first is the limitation of contact operation: fingerprint recognition requires users to directly contact the sensor, which requires extremely high finger cleanliness and humidity. If the fingers are stained, sweaty, or cracked, the recognition success rate will drop significantly. In humid or low-temperature environments, the sensor surface is prone to condensation or failure, further affecting the user experience. Secondly, it has poor adaptability to specific groups: children's fingerprint features are shallow, the fingerprints of the elderly are easily blurred due to aging skin, and some special occupational groups (such as construction workers and chemical practitioners) have damaged fingerprint features due to long-term wear and tear on their hands or contact with corrosive substances, making it difficult to complete effective recognition. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a smart lock control system and method based on embedded edge AI. The lightweight, low-cost and low-power consumption characteristics of the smart lock control system are more in line with the design and market demand of smart locks.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions: an intelligent lock control system based on embedded edge AI, comprising: a main control module, a data acquisition module, a face recognition module, a communication module, an input module, a control module, and a clutch module;

[0007] The data acquisition module, face recognition module, input module and clutch module are connected to the main control module; the control module is connected to the main control module through the communication module; the input module collects face information and sends it to the main control module.

[0008] In a preferred embodiment, the communication module is specifically a Bluetooth communication module, the control module and the communication module exchange data in a two-way manner, and the communication module and the main control module exchange data in a two-way manner.

[0009] In a preferred embodiment, the data acquisition module and the main control cabinet module are integrated into one by using a K210 development board; the data acquisition module specifically collects facial information through a camera.

[0010] In a preferred embodiment, the input module is specifically a matrix keyboard module, and the input module key judgment adopts a row and column scanning method, with all the upper four bits output as low level and the lower four bits output as high level; when the lower four bits of the received data are not all high level, it means that a key is pressed, and then the received data value is used to determine which column the key is pressed; then conversely, the upper four bits are output as high level and the lower four bits are output as low level, and then the row of the key is pressed is determined based on the received upper four bits value, so that the row and column values ​​can be determined, thereby determining which key is pressed; when the four-digit number input on the keyboard is consistent with the set temporary password, the main control module will drive the clutch module to complete the unlocking operation.

[0011] In a preferred embodiment, the rotation angle and direction of the clutch module motor are controlled by PWM signals, and the main control module outputs pulse signals with different duty cycles to control the rotation angle and direction of the motor, thereby controlling the turbine to realize the opening and closing of the door lock.

[0012] The present invention also provides a smart lock control method based on embedded edge AI, which adopts the smart lock control system based on embedded edge AI;

[0013] Step 1: Build the system hardware platform;

[0014] Step 2: Use the OpenMV machine vision module to collect facial information;

[0015] Step 3: Train the face recognition model based on the YOLO deep learning network to achieve face registration and recognition;

[0016] Step 4: Use the Bluetooth communication module to realize data interaction, and the transmitted information is encrypted by the AES algorithm; the AES algorithm encryption adopts a 128-bit key length, and the communication data is padded with 0 if it is less than 128 bits; the round transformation operation is the core link of the AES algorithm, and the system adopts 10 rounds of encryption transformation;

[0017] Step 5: The hardware system connects an external matrix keyboard to realize the panel function operation of the smart lock, and at the same time expands the external motor drive circuit to realize the clutch control of the smart lock.

[0018] In a preferred embodiment, in step 2, model training is first performed:

[0019] Step A21: First, grayscale and normalize all images;

[0020]

[0021] H(x,y)=I(x,y) gamma

[0022] I(x,y) is the pixel value of the local grayscale image; x is the horizontal coordinate value of the pixel point; y is the vertical coordinate value of the pixel point; G represents grayscale processing; gamma is the weight; H(x,y) is the normalized grayscale pixel value;

[0023] Step A22: performing convolution operation on the processed image with respect to pixel size and gradient to obtain the HOG feature vector of the image;

[0024] Step A23: Use the HOG feature vector as the initial value and input it into the SVM machine learning classification algorithm;

[0025] Step A24: setting SVM parameters;

[0026] Step A25: Export the face detection model file after SVM classifier training.

[0027] In a preferred embodiment, in step 2, a two-layer cascaded convolutional neural network is used. The first layer of the neural network performs a rough positioning of five key points, including the left eye, right eye, nose tip, left corner of the mouth, and right corner of the mouth; the second layer of the neural network performs an accurate positioning of the five key points. The process is as follows:

[0028] Step B21: Image preprocessing: crop the image to 448×448 pixels and segment it into 7×7×30 grids using a convolutional network.

[0029] Step B22: Each grid predicts the category of an object and can directly give the probability of the object;

[0030] Step B23: Assign different confidence levels to each grid. The higher the confidence level, the more accurate the location of an object. The 30 grids together represent an image. Combined with the loss function, the object location is predicted. The loss function is as follows:

[0031] L(o,c,O,C,l,g)=λ1L conf (o,c)+λ2L cla (O,C)+λ3L loc (l,g)

[0032] In the above formula, λ1, λ2, λ3 are balance coefficients, L conf , Lcla and L loc They are confidence loss, classification loss, and positioning loss, respectively. o is the overlap between the predicted target bounding box and the true target bounding box, ranging from 0 to 1. c is the prediction confidence. O indicates whether there are other categories of targets in the predicted box. C is the target probability, l is the distance to the center, and g is the aspect ratio.

[0033] In a preferred embodiment, in step 2, HOG feature vector extraction includes:

[0034] Step C21: grayscale the face image, i.e., treat the image as a three-dimensional image;

[0035] Step C22: normalizing the color of the face image and adjusting the image contrast;

[0036] Step C23: Subdivide the entire processed image into small regions, and calculate the gradient direction and gradient magnitude of each small region as the feature identifier of the small region;

[0037] Step C24: Finally, the feature identifiers of each small area are combined to form the feature value of the entire image.

[0038] In a preferred embodiment, the registered user face data is stored in the features.txt folder in the SD card to build a local face database, and the Euclidean distance between the face data features collected by the camera and the local face database is compared.

[0039] Output recognition results;

[0040]

[0041] P is the Euclidean distance, x i and y i They represent the feature identifiers of two different face data respectively; i is the feature sequence number of the face data; N is the total dimension of the feature;

[0042] The facial data feature values ​​collected by the camera are cyclically compared with the feature values ​​in the features.txt folder. The threshold is defined as the similarity of the Euclidean distance between the two feature values. When the Euclidean distance between the two feature values ​​is higher than the set threshold, they are judged to be the same person.

[0043] Compared with existing technologies, the present invention offers the following advantages: The intelligent lock control system designed in this invention is lightweight, low-cost, and consumes low power. It integrates the functions of mainstream intelligent lock systems on the market with Bluetooth remote control, further aligning with intelligent lock design and market demand, and achieving a secure, reliable, and convenient intelligent lock system. Test results demonstrate that the designed intelligent lock control system operates stably, has high reliability, and exhibits a high facial recognition rate. Compared with traditional intelligent locks, it reduces manufacturing costs while improving security and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A block diagram of the system hardware of a preferred embodiment of the present invention;

[0045] Figure 2 This is a diagram of the K210 chip architecture of a preferred embodiment of the present invention;

[0046] Figure 3 The functional distribution of the K210 development board of the preferred embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of a 4*4 matrix keyboard according to a preferred embodiment of the present invention;

[0048] Figure 5 This is a wiring diagram of a communication module according to a preferred embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the relationship between the clutch module motor output angle and input pulse width in a preferred embodiment of the present invention;

[0050] Figure 7 This is a flow chart of a face detection algorithm according to a preferred embodiment of the present invention;

[0051] Figure 8 This is a flowchart of facial key point positioning according to a preferred embodiment of the present invention;

[0052] Figure 9 CNN algorithm flow chart of the preferred embodiment of the present invention;

[0053] Figure 10 This is a YOLO network structure diagram of a preferred embodiment of the present invention;

[0054] Figure 11 Flowchart of the YOLO algorithm according to the preferred embodiment of the present invention;

[0055] Figure 12 HOG algorithm flow chart of the preferred embodiment of the present invention;

[0056] Figure 13 This is a diagram of the serial communication data format of a preferred embodiment of the present invention;

[0057] Figure 14 This is a flow chart of AES communication encryption and decryption according to a preferred embodiment of the present invention;

[0058] Figure 15 A schematic diagram of the system function flow of a preferred embodiment of the present invention;

[0059] Figure 16 This is a schematic diagram of the WeChat applet interface and successful Bluetooth connection in a preferred embodiment of the present invention;

[0060] Figure 17 This is a schematic diagram of the facial key point positioning test results of a preferred embodiment of the present invention;

[0061] Figure 18 This is a schematic diagram of face recognition success rates at different thresholds according to a preferred embodiment of the present invention;

[0062] Figure 19 This is a diagram of the WeChat applet lock switch operation in a preferred embodiment of the present invention;

[0063] Figure 20 Schematic diagram of unlocking record query according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0065] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0066] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0067] refer to Figure 1-20The present invention provides an intelligent lock control system and method based on embedded edge AI, which includes hardware and software parts; the hardware system is mainly designed with the edge intelligent computing K210 chip. The face recognition module adopts the OpenMV machine vision module, and builds the face recognition algorithm through the model file to effectively improve the face recognition rate and accuracy. The keyboard module adopts a 4*4 matrix keyboard, which only needs to occupy 8 I / O ports to achieve 16-bit key input. The Bluetooth communication module adopts the HC-08BLE4.0 module. Bluetooth 4.0 technology introduces low-power mode for the first time, which improves the connection speed, data transmission stability and transmission distance. Its largest application scenario is low-power short-range small-scale data communication. The clutch module adopts motor control, and outputs different PWM signals through the I / O port control to realize the motor rotation of different angles and directions to realize the opening and closing of the door lock.

[0068] The overall hardware module design is as follows Figure 1 As shown in the figure, the WeChat mini-program, serving as the overall system's control module, enables two-way data exchange with the Bluetooth communication module. The Bluetooth communication module also uses UART serial communication to communicate with the K210 development board, achieving the design goal of enabling remote control of the smart lock via the WeChat mini-program. The K20 board communicates with the WeChat mini-program via Bluetooth, receiving commands for unlocking, facial registration, and temporary passwords, and returning unlocking records and facial registration results.

[0069] The face recognition module of this system uses the OpenMV machine vision module, which is low-cost, scalable, and supports Python development. By running the image processing algorithm through Python scripts, face recognition can be quickly achieved.

[0070] The K210 development board is an MCU based on the RISC-V reduced instruction set. Compared with other single-chip microcomputer devices, its most prominent feature is that the chip architecture integrates a neural network hardware accelerator KPU, which can perform convolutional neural network operations with high performance. It is capable of high-performance machine vision and auditory processing, and comes with a high-definition camera, which is also a data acquisition module. The camera sends the collected facial data to the K210 chip, which uses the facial recognition algorithm to quickly analyze the facial data. This feature greatly improves the facial recognition rate and accuracy of the smart lock designed in this system. Figure 2 The following is the K210 chip architecture diagram. It has a rich set of peripherals such as I2C, DVP, PWM, etc. At the same time, when using Python language to develop face recognition programs, there is no need to consider register allocation, which reduces repetitive work and allows developers to focus more on the realization of the product's own functions. Figure 3Figure 2 shows the functional layout of the development board. This system uses a geometric feature extraction method. Geometric feature-based face recognition is a relatively intuitive traditional face recognition method. It uses a camera to collect facial data in real time. By running the face recognition algorithm, each user's facial data is stored as feature values ​​on the onboard SD card. By establishing a local face database, this data can be preserved even when the power is off.

[0071] The input module of this system uses a 4*4 matrix keyboard, such as Figure 4 As shown in the figure, I / O port resources are often at a premium in microcontroller systems. When multiple keys are used, a matrix keyboard is chosen to conserve I / O ports. In a matrix keyboard, each horizontal and vertical line is connected at its intersection, not directly, but through a key. This allows 8 ports to create 4*4=16 keys, double the number of keys used directly for a keyboard. The more keys there are, the more distinct they become. In practice, only 11 keys are needed: 0-9 and a delete key.

[0072] Key press detection uses a row-column scanning method, where the upper four bits are all low and the lower four bits are high. If the lower four bits of the received data are not all high, it indicates a key has been pressed. The received data value is then used to determine which column the key has been pressed. Conversely, the upper four bits are high and the lower four bits are low. The received upper four bits are then used to determine which row the key has been pressed. This method determines the row and column values, and thus the key that has been pressed. When the four-digit keypad input matches the set temporary password, the development board activates the clutch to unlock the door.

[0073] The communication module utilizes Bluetooth. Bluetooth low energy technology, an innovative technology first introduced in Bluetooth 4.0, serves as the bridge for data exchange between the K210 development board and the WeChat mini-program. This module uses the HC-08BLE 4.0. The HC-08 Bluetooth serial port module offers low power consumption, dual-mode Bluetooth, simple operation, integrated master / slave functionality, and excellent communication performance. Its application scenario is focused on low-power, short-range, small-volume data communication. Bluetooth technology is a key component of wireless communications. Bluetooth 4.0 provides a low-energy, low-cost solution for short-range radio transmission.

[0074] The HC-08 Bluetooth serial port module not only supports Bluetooth protocols, but also serial communication protocols. The HC-08 Bluetooth serial port module communicates with the development board using the UART serial port, using four ports: VCC and GND for power, TXD for serial data transmission, and RXD for serial data reception. Because the K210 development board supports custom I / O ports, this system defines the development board's I / O9 and I / O10 as the board's TXD and RXD, which are connected to the HC-08's RXD and TXD, respectively. Figure 5 The following is a wiring diagram. This enables normal serial port protocol communication. The HC-08 uses Bluetooth communication to exchange data with the WeChat applet. During use, simple AT commands to set the same parameters enable connection between modules. This eliminates the need for users to deeply understand the Bluetooth protocol stack; they only need to customize their own Bluetooth services in the application layer.

[0075] The clutch module of this system uses motor control to realize the opening and closing of the door lock. The rotation angle and direction of the clutch motor are controlled by PWM signals. The K210 development board controls the rotation angle and direction of the motor by outputting pulse signals with different duty cycles, thereby controlling the turbine to realize the opening and closing of the door lock. The control signal of the motor is a PWM signal with a period of 20ms, in which the pulse width ranges from 0.5ms to 2.5ms, and the corresponding rotation position is 0-180 degrees, which changes linearly. Figure 6 shown.

[0076] The principle of PWM signal-controlled motors relies on a standard circuit within the motor that generates a reference signal with a period of 20ms and a width of 1.5ms. A signal comparator compares the applied PWM signal with the reference signal to determine its direction and magnitude, thereby generating a motor rotation signal. Motors require PWM signals for control. This system uses the I / O2 port to output PWM signals with varying duty cycles, thereby achieving motor rotation in a specified direction and angle. This system uses three pulse signals to control the forward and reverse rotation of the clutch, thereby achieving the desired on / off lock function.

[0077] The following introduces the software implementation part of this solution:

[0078] Face recognition is the process of comparing a face detected in a still image or video frame with a local database of facial data to identify a matching face for identity verification and authentication. The face recognition system of the present invention uses a method based on geometric features, offering advantages such as fast recognition speed, high accuracy, and ease of implementation.

[0079] In face recognition, face detection is the first step. Face detection involves detecting the position, size, or key facial features of a face from an image or video frame. The most important step in face detection is training the face detection model. Considering the actual usage of smart locks, the model trained in this system intentionally prioritizes the elderly and children. A training set of 55 images was used, including 18 of elderly people and 22 of children, with an additional 4 images prepared as a test set. Although the test set was relatively small, the final test results showed that the model training was effective. This model utilizes the Dlib training model, a powerful machine learning library that supports large amounts of data processing, boasts numerous face recognition algorithms, and offers high recognition accuracy. It is widely used in object detection, deep learning, and other fields. The training algorithm for the face recognition model used in this invention is based on the HOG feature extraction algorithm combined with the SVM machine learning classification algorithm. Among the algorithms used in conjunction with the HOG feature extraction algorithm, the SVM machine learning classification algorithm performs best, producing a superior trained model.

[0080] The complete training process is:

[0081] (1) First, grayscale and normalize all images;

[0082]

[0083] H(x,y)=I(x,y) gamma

[0084] I(x,y) is the pixel value of the local grayscale image; G represents grayscale processing; gamma is generally taken as 0.5.

[0085] (2) Perform convolution operation on the processed image with respect to pixel size and gradient to obtain the HOG feature vector of the image.

[0086] (3) The feature vector is used as the initial value and input into the SVM machine learning classification algorithm.

[0087] (4) Set the SVM parameters. Most parameters such as max_iter (maximum number of iterations) and kernel (kernel function) use the default values. The more important parameters are the penalty coefficient C, which is the tolerance for detection errors. The higher the C value, the less tolerant the error is. The smaller the C value, the greater the probability of error. Therefore, the size of the C value will affect the quality of the training model.

[0088] (5) After SVM classifier training, the face detection model file is exported. The algorithm flow chart is as follows Figure 7 shown.

[0089] The quality of the facial key point detection algorithm directly determines the speed and accuracy of face recognition. The facial key point positioning method used by the model of the present invention is to first pre-process the image, use the facial key point coordinates as image labels, use the convolutional neural network to deeply learn the mathematical relationship between the input facial image and the key point coordinates, and finally derive the key point detection model. The process is as follows Figure 8 shown.

[0090] The present invention uses a 5-point key point detection model for the face, and the 5 key points are the left eye, right eye, nose tip, left corner of the mouth and right corner of the mouth. The 5-point key point detection model used in this system is to realize the positioning of the 5 key points of the face through a two-layer cascade CNN (convolutional neural network). Convolutional neural networks perform very well in applications such as image processing and target detection, and are one of the most successful algorithms in the field of computer deep learning. Since the actual facial data collection environment of smart locks is relatively complex and is greatly affected by the user's expression and the light intensity of the shooting environment, there are large errors in the key points located by a single neural network model. Therefore, this model uses a two-layer cascaded convolutional neural network to improve the precision and accuracy of face positioning. The first layer of the neural network performs rough positioning of the 5 key points, and the second layer of the neural network performs accurate positioning of the 5 key points. As Figure 9 The figure shows the flow chart of the CNN algorithm.

[0091] This model uses the YOLO efficient open source deep learning framework. The network structure of YOLO is as follows Figure 10 The model is based on the cascade of multiple convolutional neural networks, which directly predict the category and location of the detected object through deep learning.

[0092] YOLO is an image recognition and positioning algorithm based on deep learning. It is widely used in the field of machine vision. Its extremely fast computing speed gives it a unique advantage in real-time video processing. The YOLO algorithm principle process is as follows Figure 11 The basic process is:

[0093] (1) Image preprocessing: the image size is cut to 448×448 and the convolutional network is used to segment the image into a 7×7×30 grid.

[0094] (2) Each grid can predict the category of an object and directly give the probability of the object.

[0095] (3) Each grid is assigned a different confidence level. The higher the confidence level, the more accurate the location of an object. The 30 grids together represent an image. Combined with the loss function, the object location is predicted. The loss function is as follows:

[0096] L(o,c,O,C,l,g)=λ1Lconf (o,c)+λ2L cla (O,C)+λ3L loc (l,g)

[0097] In the above formula, λ1, λ2, λ3 are balance coefficients, L conf , L cla , L loc They are confidence loss, classification loss and positioning loss respectively, o is the overlap between the predicted target bounding box and the true target bounding box, and the value is between 0 and 1, c is the prediction confidence, O indicates whether there are other categories of targets in the predicted box, C is the target probability, l is the distance to the center, and g is the aspect ratio.

[0098] The database is Olivetti Faces, a 400-image facial database from New York University. Next, we define the basic components of the CNN, including the LeNetConvPoolLayer (convolutional sampling layer), HiddenLayer (hidden layer), and LogisticRegression (classification layer). The next step is to load the dataset and generate a detection model.

[0099] The feature extraction model is an important step in digitizing facial data. A good facial feature extraction algorithm should have good distinguishability, meaning it is sensitive to facial features and highly reliable. Considering the actual use of smart locks, this paper ultimately selected the HOG feature extraction algorithm. Its advantages in extracting geometric features and effectively reducing shadows and lighting changes contribute to improved facial recognition accuracy in smart locks.

[0100] HOG feature is a feature descriptor that is a statistical feature of the gradient information of a local area of ​​an image. Its theoretical basis is that the local area in an image can be well described by the directional density of the gradient or edge. The idea of ​​the HOG feature extraction algorithm is to calculate and count the gradient direction and gradient amplitude of the image. The algorithm flow is as follows Figure 12 shown

[0101] The specific algorithm steps are:

[0102] (1) Grayscale the face image, that is, treat the image as a three-dimensional image.

[0103] (2) Normalize the color of the face image and adjust the image contrast. This approach can effectively reduce the impact of shadows and lighting changes.

[0104] (3) The entire processed image is subdivided into small areas, and the gradient direction and gradient amplitude of each small area are calculated as the feature identifier of this small area.

[0105] (4) Finally, the feature identifiers of each small area are combined to form the feature value of the entire image.

[0106] The feature value and the user name are matched and stored in the features.txt folder in the SD card to save the user's face data.

[0107] Face recognition implementation:

[0108] The three steps above yield three model files, which are then used to build the face recognition program. Face comparison involves comparing facial image features and analyzing key facial features to determine whether they represent the same person. This system uses a facial feature value comparison method, storing registered user facial data in the features.txt file on the SD card to build a local face database. The system then compares the Euclidean distance between the facial data captured by the camera and the local face database to output the recognition results.

[0109]

[0110] P is the Euclidean distance, x i and y i Respectively represent feature identifiers of two different face data.

[0111] The previous step obtained the user's facial feature values, realizing the digitization of facial data. In this step, the facial feature values ​​will be compared to determine whether the user is the same person. The facial data feature values ​​collected by the camera are cyclically compared with the feature values ​​in the features.txt folder. The threshold is defined as the similarity of the Euclidean distance between the two feature values. When the Euclidean distance of the two feature values ​​is higher than a certain threshold, it is determined that they are the same person. The setting of the threshold will affect the accuracy of face recognition. If the threshold is too high, it is easy to mistakenly recognize the same face as different faces, resulting in accidents where the user cannot unlock the door. If the threshold is too low, it is easy for unauthorized users to unlock the door normally, which will reduce the security of the smart lock. In the experiment, by comparing the accuracy of face recognition under different thresholds under the same experimental conditions, the optimal threshold of the algorithm was found to be 85. Specific experimental data will be explained in the specific embodiments and descriptions.

[0112] Information interaction between smart lock and mini program:

[0113] Since the K210 development board lacks a Bluetooth communication chip, a Bluetooth serial port module is required to enable data interaction with the WeChat mini-program. This module supports not only Bluetooth technology communication but also serial communication. Using a Bluetooth serial port module effectively solves the data interaction problem between the development board and the WeChat mini-program.

[0114] Universal Asynchronous Receiver / Transmitter (UART) is used for serial data communication between hardware devices. UART has the characteristics of low cost, high reliability, long transmission distance and easy implementation. Data is transmitted bit by bit, one character at a time, and the transmitted characters always start with a start bit and end with a stop bit. The protocol frame of serial communication generally includes a start bit, a stop bit, a parity bit, a baud rate, etc. Figure 13 The data format of serial communication is shown in the figure. The K210 development board of the present invention transmits and receives commands and transmits data with the WeChat applet through the HC-08 Bluetooth serial port module.

[0115] Before using the HC-08 Bluetooth serial port module, you need to set the module parameters. The parameter design of the HC-08 Bluetooth serial port module is very simple. Use a USB to TTL module to connect the computer and the Bluetooth module. After powering on, open the serial port assistant and use AT commands to set the module parameters. The parameters are set to 9600 baud rate, 0 parity bit, 8 data bits, 1 stop bit, 1000ms timeout, and 4096 buffer size. After the settings are completed, connect the Bluetooth serial port module to the data transmission interface of the development board. Since the I / O port of K210 supports custom functions, this greatly reduces the difficulty of using serial port communication. You only need to set I / O9 and I / O10 to TXD and RXD on the development board to achieve serial port communication with the Bluetooth module. The specific definition function is as follows:

[0116] fm.register(9,fm.fpioa.UART1_TX, force=True)

[0117] fm.register(10,fm.fpioa.UART1_RX, force=True)

[0118] At the same time, the development board parameter settings must be consistent with the Bluetooth module. The specific setting functions are as follows:

[0119] uart_A=UART(UART.UART1,9600,8,1,0,timeout=1000,read_buf_len=4096)

[0120] The sending and receiving functions are as follows:

[0121] uart_A.write()

[0122] uart_A.read()

[0123] In the actual experiment, the WeChat applet served as the host and connected to the Bluetooth module. The Bluetooth module forwarded the data received from the WeChat applet to the K210 development board via serial communication, thus enabling remote control of the smart lock via the WeChat applet. The transmitted data can be viewed through the serial terminal of the development software (MaixPy IDE).

[0124] BLE (Bluetooth Low Energy) technology offers low cost, high stability, and low power consumption. Bluetooth communication is a one-to-one or one-to-many, bidirectional communication. Bluetooth technology stipulates that Bluetooth communication is between two devices, one of which must be a master and the other a slave. Before communication can begin, the master device searches for and initiates pairing. Only after a connection is established can communication begin.

[0125] Bluetooth allows electronic devices to communicate over short distances and is used by billions of devices worldwide. This system uses a one-to-one communication between the Bluetooth module and the WeChat applet. The Bluetooth module is configured in slave mode using AT commands, while the WeChat applet is in host mode. Before the HC-08 connects to the WeChat applet, it must first broadcast a signal. This broadcast packet is sent every 1 second, known as the broadcast time. Bluetooth technology requires that each broadcast packet requires the Bluetooth module to activate its RF receiver for a certain period of time to receive pairing requests from the host. Although the RF module consumes a lot of power when enabled, it is not constantly on, but rather periodically, resulting in very low overall power consumption.

[0126] The BLE 4.0HC-08 device used in this system supports full-duplex data transmission with a transmission distance of up to 80 meters. During the communication process, users do not need to have a deep understanding of the Bluetooth protocol stack; they only need to customize their own Bluetooth services in the application layer.

[0127] To facilitate the Bluetooth module's recognition of WeChat Mini Program commands, this system customizes some of the communication protocols between the Bluetooth module and WeChat Mini Program, as shown in Table 1.

[0128]

[0129] Table 1 Custom Bluetooth communication protocol communication data format is shown in Table 2.

[0130]

[0131] Table 2 Communication data format

[0132] For example, when unlocking via Bluetooth, the data content sent by the WeChat applet is 0X11010000. After unlocking, the Bluetooth module will return the data: 0x11010101 (unlocking successful) or 0x11010102 (unlocking failed).

[0133] Because Bluetooth uses 2.4 GHz radio waves to transmit and receive data, it is vulnerable to third-party data theft and attacks. To improve the security of smart lock Bluetooth communications and protect user privacy, this invention incorporates the Advanced Encryption Standard (AES) into Bluetooth communications to enhance the confidentiality of data exchanges.

[0134] The AES encryption algorithm is a symmetric encryption algorithm. The so-called symmetry means using the same key to encrypt and decrypt communication data. The key production is obtained by mutual negotiation between the communicating parties. Therefore, the security of the symmetric encryption algorithm is mainly determined by the key. AES adopts the idea of ​​iterative encryption, which has high computational efficiency and good confidentiality. The key length of AES can be 128 bits and 16 bytes or higher 192 bits and 256 bits. The longer the key length, the higher the encryption strength. At the same time, the encryption and decryption operations are greater and take longer. In combination with the user's demand for fast opening and closing of smart locks, the system adopts a 128-bit key length. Since the communication data of the smart lock is less, the communication data needs to be padded with 0s if it is not 128 bits. The core link of the AES algorithm is the round transformation operation. The system adopts 10 rounds of encryption transformation. The algorithm has high computational efficiency and high security. The AES encryption and decryption process is as follows: Figure 14 shown.

[0135] Overall usage process and steps:

[0136] To sum up, the unlocking steps of the smart lock control system based on embedded edge AI are as follows: Figure 15 ,as follows:

[0137] Step (1), face information registration, in administrator mode, users can register and delete users through the face recognition module.

[0138] Step (2): unlocking by face recognition. Users who have registered their face can unlock by face recognition by approaching the camera. The face recognition accuracy is high.

[0139] Step (3): Bluetooth unlocking. The WeChat applet controls the smart lock to unlock via Bluetooth communication.

[0140] Step (4): Temporary user unlocking. The WeChat applet generates a temporary password function. The user can also unlock the door by entering the temporary password through the matrix keyboard.

[0141] Step (5), unlocking record. All unlocking methods will be sent to the WeChat applet via Bluetooth and recorded in the background.

[0142] Functional test of each hardware module:

[0143] WeChat applet interface and Bluetooth connection are successful as follows Figure 16 shown.

[0144] In the actual connection cable, the 4x4 matrix keypad occupies eight I / O ports connected to the K210 main development board. In actual use, in addition to the numeric keys 0-9, there's also a clear password button, allowing users to re-enter their temporary password if they enter it incorrectly. The K210 main board primarily runs facial recognition code, receives data from the matrix keypad and Bluetooth, and controls the motor drive. The system is powered by connecting the K210 development board to a computer via USB, which also serves as the data communication channel between the development software (MaixPy) and the board. The model files and code required for facial recognition are downloaded to the K210 development board via USB. The HC-08 Bluetooth serial port module and clutch are powered by the 5V and 3.3V power supplies on the K210 development board. The Bluetooth module occupies two I / O ports on the K210 development board for serial communication. The clutch requires one I / O port to receive the PWM signal output by the development board, which drives the motor to rotate the turbine in the specified direction and angle to lock the door. The development board can collect facial data normally, Bluetooth can also establish a connection with the WeChat applet and realize real-time data exchange, the matrix keyboard also successfully realizes the temporary password unlocking function, and the motor can also work normally.

[0145] Face recognition program testing and analysis:

[0146] The face recognition module is the focus of this system's development. Before running the face recognition program, you need to download the three trained face recognition model files (face detection model, facial key point detection model, and facial feature value extraction model) to the K210 development board's FLASH memory. These three model files are used to build the face recognition program. While the program is running, the camera will display the captured face in real time in the upper right corner of the development software.

[0147] When training the face detection model, the parameter C of the SVM algorithm was specifically mentioned because the C value will affect the quality of the face recognition model. During the face recognition program testing phase, the control variable method was used to set the models trained with different C values ​​for experimental comparison. The test items included the accuracy of face recognition and the time required for model training.

[0148] Table 3 shows the experimental data. This data analyzes the relationship between the C value, face detection accuracy, and model training time. It shows that when the C value is less than 10, model training takes less time, but the trade-off is lower accuracy. Increasing the C value improves accuracy, but at the expense of significantly longer training times. Considering both accuracy and training time, a C value of 10 was ultimately chosen for training. The final results demonstrate that the face detection model performs very well.

[0149]

[0150] Table 3 Relationship between C value, model prediction accuracy and training time

[0151] The present invention uses a two-layer cascade CNN to detect key points on the face. Here, the trained key point positioning model of the face is actually tested. The test results are as follows: Figure 17 .

[0152] Face recognition is achieved by cyclically calculating the Euclidean distance between the collected facial data feature values ​​and the local face database in the SD card. It is important to pay attention to the threshold parameter. The setting of the threshold will affect the accuracy of face recognition. Too high or too low will reduce the user experience of the smart lock. Setting different thresholds will affect the accuracy of face recognition, such as Figure 18 As shown in the figure, by analyzing the experimental line graph, when the threshold is too low, the error rate of face recognition is very high, and when the threshold is very high, the success rate of face recognition decreases. Based on the comprehensive experimental data, the optimal threshold is set to 85.

[0153] The facial recognition program has been tested above and the overall test results are good. Next, we'll store the facial data. After the development software is successfully connected to the K210 development board, click Run. The face image display area will show the face captured by the camera in real time, and the serial terminal will display some program operation data. First, register the face, run the program, and press the Boot button. The serial terminal will display some data.

[0154] As you can see, start 3 indicates that three facial feature values ​​have already been stored, so the fourth face data is being recorded. The data after bytearray represents the feature values ​​stored in the features.txt folder on the SD card, indicating that face registration was successful.

[0155] WeChat applet interaction test:

[0156] Power on the development board, which will then power the Bluetooth module. The blue LED on the HC-08 Bluetooth device will flash at a fixed frequency, indicating that the Bluetooth device is broadcasting an event. Use the WeChat mini-program to scan for surrounding devices. When the blue light on the Bluetooth device turns solid, the Bluetooth connection is successful. A successful Bluetooth connection prompt will also appear on the WeChat mini-program.

[0157] The Bluetooth connection is successful, and the next step is to test the Bluetooth data communication. You can send a specified command through the WeChat applet to see if the Bluetooth device can receive it. In the experiment, you can view the command received by the Bluetooth device through the development software serial terminal. The test command sent is to turn on and off the lock. The WeChat applet sends a command such as Figure 19, indicating successful data reception and return to the WeChat Mini Program. The next step is to modify the smart lock program, adding custom command settings and mapping smart lock operations to user commands. After each user operation, the smart lock will return the corresponding data to the WeChat Mini Program. This enables two-way interaction between the WeChat Mini Program and the smart lock, greatly improving the user experience.

[0158] System overall functional test:

[0159] After the control system design of the face recognition smart lock is completed, the entire control system is tested as a whole, specifically the following steps:

[0160] (1) The user uses the WeChat applet to enable Bluetooth lock. The test results are as follows Figure 19 .

[0161] (2) The user unlocks the door using a temporary password.

[0162] (3) Face registration in administrator mode.

[0163] (4) Unlocking by face recognition.

[0164] (5) Check the unlocking record of the smart lock. The test results are as follows: Figure 20 .

[0165] The present invention proposes a smart lock control system and method based on embedded edge AI. The system consists of a face recognition module, a key matrix module, a WeChat mini-program control module, a Bluetooth communication module, and a clutch module. The face recognition module runs a face recognition program, acquires user face data through a camera, implements user face data collection, compares the collected face data with a local face database, implements face comparison and recognition, and returns the results to the WeChat mini-program via the Bluetooth module. The Bluetooth communication module uses Bluetooth communication technology to implement data exchange with the WeChat mini-program, and the WeChat mini-program implements remote control of the smart lock through the Bluetooth module. The matrix key module is designed to cooperate with the WeChat mini-program to implement the temporary password unlocking function. The WeChat mini-program module is the main control module, implementing face registration, Bluetooth unlocking, and temporary password unlocking functions. All smart lock operations will return data to the WeChat mini-program. The WeChat mini-program will record the user and time of each unlocking in the background. The clutch module is used to simulate the opening and closing of an actual door lock. The smart lock control system designed by this invention features lightweight, low cost, and low power consumption. It integrates the functions of mainstream smart lock systems on the market with Bluetooth remote control, further aligning with smart lock design and market demand, and achieving a secure, reliable, and convenient smart lock system. Test results demonstrate that the designed smart lock control system operates stably, has high reliability, and exhibits a high facial recognition rate. Compared to traditional smart locks, it reduces manufacturing costs while improving security and user experience.

Claims

1. A smart lock control system based on embedded edge AI, characterized in that: include: Main control module, data acquisition module, face recognition module, communication module, input module, control module and clutch module; The data acquisition module, face recognition module, input module and clutch module are connected to the main control module; the control module is connected to the main control module through the communication module; the input module collects face information and sends it to the main control module.

2. The smart lock control system based on embedded edge AI according to claim 1 is characterized in that: The communication module is specifically a Bluetooth communication module. The control module and the communication module exchange data in a two-way manner. The communication module and the main control module exchange data in a two-way manner.

3. The smart lock control system based on embedded edge AI according to claim 1, characterized in that: The data acquisition module and the main control module are integrated into one by using the K210 development board; the data acquisition module specifically collects facial information through a camera.

4. The smart lock control system based on embedded edge AI according to claim 1, characterized in that: The input module is specifically a matrix keyboard module, and the input module key judgment adopts a row and column scanning method, through which the upper four bits are all output as low level and the lower four bits are output as high level; when the lower four bits of the received data are not all high level, it means that a key is pressed, and then the received data value is used to judge which column the key is pressed; then conversely, the upper four bits are output as high level and the lower four bits are output as low level, and then the row of the key is pressed is judged according to the received upper four bits value, so that the row and column values ​​can be determined, thereby judging which key is pressed; when the four-digit number input on the keyboard is consistent with the set temporary password, the main control module will drive the clutch module to complete the unlocking operation.

5. The smart lock control system based on embedded edge AI according to claim 1, characterized in that: The rotation angle and direction of the clutch module motor are controlled by PWM signals, and the main control module outputs pulse signals with different duty cycles to control the rotation angle and direction of the motor, thereby controlling the turbine to realize the opening and closing of the door lock.

6. A smart lock control method based on embedded edge AI, characterized in that: A smart lock control system based on embedded edge AI as described in any one of claims 1-5 is adopted; Step 1: Build the system hardware platform; Step 2: Use the OpenMV machine vision module to collect facial information; Step 3: Train the face recognition model based on the YOLO deep learning network to achieve face registration and recognition; Step 4: Use the Bluetooth communication module to realize data interaction, and the transmitted information is encrypted by the AES algorithm; the AES algorithm encryption adopts a 128-bit key length, and the communication data is padded with 0 if it is less than 128 bits; the round transformation operation is the core link of the AES algorithm, and the system adopts 10 rounds of encryption transformation; Step 5: The hardware system connects an external matrix keyboard to realize the panel function operation of the smart lock, and at the same time expands the external motor drive circuit to realize the clutch control of the smart lock.

7. The smart lock control method based on embedded edge AI according to claim 6, characterized in that: In step 2, the model is trained first: Step A21: First, grayscale and normalize all images; H(x,y)=I(x,y) gamma I(x,y) is the pixel value of the local grayscale image; x is the horizontal coordinate value of the pixel point; y is the vertical coordinate value of the pixel point; G represents grayscale processing; gamma is the weight; H(x,y) is the normalized grayscale pixel value; Step A22: performing convolution operation on the processed image with respect to pixel size and gradient to obtain the HOG feature vector of the image; Step A23: Use the HOG feature vector as the initial value and input it into the SVM machine learning classification algorithm; Step A24: setting SVM parameters; Step A25: Export the face detection model file after SVM classifier training.

8. The smart lock control method based on embedded edge AI according to claim 7 is characterized in that: In step 2, a two-layer cascaded convolutional neural network is used. The first layer of the neural network performs a rough positioning of five key points, including the left eye, right eye, nose tip, left corner of the mouth, and right corner of the mouth; the second layer of the neural network performs an accurate positioning of the five key points. The process is as follows: Step B21: Image preprocessing: crop the image to 448×448 pixels and segment it into 7×7×30 grids using a convolutional network. Step B22: Each grid predicts the category of an object and can directly give the probability of the object; Step B23: Assign different confidence levels to each grid. The higher the confidence level, the more accurate the location of an object. The 30 grids together represent an image. Combined with the loss function, the object location is predicted. The loss function is as follows: L(o,c,O,C,l,g)=λ1L conf (o,c)+λ2L cla (O,C)+λ3L loc (l,g) In the above formula, λ1, λ2, λ3 are balance coefficients, L conf , L cla and L loc They are confidence loss, classification loss, and positioning loss, respectively. o is the overlap between the predicted target bounding box and the true target bounding box, ranging from 0 to 1. c is the prediction confidence. O indicates whether there are other categories of targets in the predicted box. C is the target probability, l is the distance to the center, and g is the aspect ratio.

9. The smart lock control method based on embedded edge AI according to claim 8, characterized in that: In step 2, HOG feature vector extraction includes: Step C21: grayscale the face image, i.e., treat the image as a three-dimensional image; Step C22: normalizing the color of the face image and adjusting the image contrast; Step C23: Subdivide the entire processed image into small regions, and calculate the gradient direction and gradient magnitude of each small region as the feature identifier of the small region; Step C24: Finally, the feature identifiers of each small area are combined to form the feature value of the entire image.

10. The smart lock control method based on embedded edge AI according to claim 9, characterized in that: Store the registered user's face data in the features.txt folder on the SD card to build a local face database. Then, compare the Euclidean distance between the face data features collected by the camera and the local face database to output the recognition results. P is the Euclidean distance, x i and y i They represent the feature identifiers of two different face data respectively; i is the feature sequence number of the face data; N is the total dimension of the feature; The facial data feature values ​​collected by the camera are cyclically compared with the feature values ​​in the features.txt folder. The threshold is defined as the similarity of the Euclidean distance between the two feature values. When the Euclidean distance between the two feature values ​​is higher than the set threshold, they are judged to be the same person.

Images