Intelligent door lock suitable for aging and using method thereof

Abstract

The invention provides an intelligent door lock suitable for aging and a using method thereof, and relates to the technical field of intelligent door locks, the intelligent door lock comprises a door lock body, a control module, a faint detection system, an anomaly detection system, an emergency communication module and an interaction module.The method comprises the steps that the faint detection system monitors whether old people faint or not in real time; if the elderly faint, the control module controls the emergency communication module to dial a family number and a platform emergency call, and if the elderly feels uncomfortable at the doorway, the control module dials the family number and the platform emergency call based on the emergency communication module through an emergency button or a voice call module; and identifying the abnormal behavior of the old people according to the set abnormity early warning rule based on the abnormity detection system. The key defects of a traditional scheme in the aspects of accuracy, reliability, usability and the like are overcome, and a landing and sustainable home safety solution is provided for the aging society.

Description

Technical Field

[0001] The present invention relates to the technical field of smart door locks, and in particular to an aging-friendly smart door lock and a use method thereof. Background Art

[0002] Smart door locks are improved from traditional mechanical locks and are more intelligent and simple in terms of user security, identification, and management. Smart door locks are the executive components of door locks in access control systems. Smart door locks are different from traditional mechanical locks and are composite locks with security, convenience, and advanced technology. Existing smart door locks are often designed for young people, with novel styles, full functions, and a variety of unlocking methods, but they often ignore the use of the elderly.

[0003] In addition, according to statistics, about 30% of the elderly over 65 years old have falls every year, and more than 50% of them occur in the transition area between indoor and outdoor (such as doorways). Traditional solutions mainly rely on wearable devices (such as smart bracelets), but they are inconvenient to wear and the elderly often do not like to wear them. How to combine smart door locks with the daily life monitoring of the elderly and then launch an aging-friendly smart door lock has become a major problem.

[0004] Therefore, it is very necessary to design an aging-friendly smart door lock and a method of using the same. Summary of the invention

[0005] In order to overcome the deficiencies of the prior art, an object of the present invention is to provide an aging-friendly smart door lock and a method of using the same.

[0006] To achieve the above object, the present invention provides the following solutions: The present invention provides an aging-friendly smart door lock and a method for using the same, comprising: a door lock body, a control module, a fainting detection system, an abnormality detection system, an emergency communication module and an interactive module, wherein the control module, the fainting detection system, the abnormality detection system, the emergency communication module and the interactive module are arranged on the door lock body, and the fainting detection system, the abnormality detection system, the emergency communication module and the interactive module are connected to the control module.

[0007] Preferably, the fainting detection system includes an inside-door fainting detection system and an outside-door fainting detection system, the inside-door fainting detection system includes an infrared pyroelectric sensor and a sound recognition module, the infrared pyroelectric sensor and the sound recognition module are arranged on the inner door panel of the door lock body, the infrared pyroelectric sensor is used to monitor human body micro-motion signals, the sound recognition module is used to identify falling and impact sounds, cries for help and long groaning sounds, the outside-door fainting detection system includes an outside-door camera and a fainting recognition module, the outside-door camera is used to collect video information of the elderly, the fainting recognition module is used to determine whether the elderly have fainted based on the video information of the elderly, and the infrared pyroelectric sensor, the sound recognition module, the outside-door camera and the fainting recognition module are all connected to the control module.

[0008] Preferably, the anomaly detection system is used to identify abnormal behaviors of the elderly through set anomaly warning rules.

[0009] Preferably, the emergency communication module includes an eSIM card module and a WIFI module. When there is WIFI, it is connected to the server through the WIFI module. Otherwise, it is connected through the eSIM card module. Both the eSIM card module and the WIFI module are connected to the control module.

[0010] Preferably, the interactive module includes an emergency button, a display screen and a voice call module, the emergency button is set on the inner door panel and the outer door panel of the door lock body, the display screen and the voice call module are set on the inner door panel of the door lock body, and the emergency button, the display screen and the voice call module are all connected to the control module.

[0011] The present invention also provides a method for using an aging-friendly smart door lock, comprising: Step 1: The fainting detection system monitors whether the elderly person faints in real time; Step 2: If the elderly person faints, the control module controls the emergency communication module to call the family number and the platform emergency number; Step 3: If the elderly person feels unwell at the door, he or she can dial the family number and the platform emergency number through the emergency button or voice call module based on the emergency communication module; Step 4: Based on the anomaly detection system, the abnormal behavior of the elderly is identified according to the set anomaly warning rules.

[0012] Preferably, in step 1, the fainting detection system monitors in real time whether the elderly person faints, specifically: Step 101: Detecting whether an elderly person faints inside the door based on a fainting detection system inside the door; Step 102: Detect whether the elderly person faints outside the door based on the outside fainting detection system.

[0013] Preferably, in step 101, whether the elderly person faints inside the door is detected based on the door fainting detection system, specifically: The infrared pyroelectric sensor detects human body micro-movement signals. If relevant signals are detected, the sound recognition module is activated. Based on the sound recognition module, it is determined whether there is a fall, impact sound, cry for help, or long groaning sound. If so, it is determined that the elderly person has fainted inside the door.

[0014] Preferably, in step 102, whether the elderly person faints outside the door is detected based on the outside fainting detection system, specifically: Collecting video information outside the door based on the camera outside the door; The fainting recognition module builds a graph convolutional action recognition model and trains it based on a preset data set; The collected video information outside the door is input into the trained graph convolution action recognition model to identify the fainting situation of the elderly.

[0015] Preferably, in step 4, the abnormal behavior of the elderly is identified based on the abnormal detection system according to the set abnormal warning rules, specifically: Obtain the number of times the elderly open their doors every day, the duration of time they go out, and the high-frequency time periods; Collect the number of times the elderly open the door on that day. If it drops by 70% compared to the average value within 7 days, it is judged that the elderly have an abnormality; Collect whether the elderly have gone out during the high-frequency time period of the day. If they have not gone out for three consecutive days, it is considered that something is wrong with the elderly. Collect the time the elderly have been out. If they have not returned after more than 12 hours, it is considered that the elderly have an abnormality. If it is determined that an abnormality has occurred in the elderly, the control module will call the family number and the platform emergency number through the emergency communication module.

[0016] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: The present invention provides an aging-friendly smart door lock and a method of using the same. The smart door lock includes a door lock body, a control module, a fainting detection system, an abnormality detection system, an emergency communication module and an interactive module. The method includes a fainting detection system that monitors whether the elderly faint in real time. If the elderly faint, the control module controls the emergency communication module to dial a family number and a platform emergency number. If the elderly feel unwell at the door, the emergency button or voice call module dials the family number and the platform emergency number based on the emergency communication module, and the abnormal behavior of the elderly is identified based on the abnormality detection system according to the set abnormal warning rules. The present invention uses the smart door lock as a physical carrier to construct an elderly health protection system that integrates risk monitoring, data analysis, and emergency response. Through technical innovations such as multimodal sensor fusion, edge computing architecture, and privacy protection design, the key defects of traditional solutions in terms of accuracy, reliability, and ease of use are solved, providing a feasible and sustainable home safety solution for an aging society. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 This is a schematic diagram of the external structure of the smart door lock; Figure 2 This is a schematic diagram of the internal structure of a smart door lock; Figure 3 It is a schematic diagram of the process of the present invention; Figure 4 Schematic diagram of the DCNN model structure; Figure 5 This is the structure diagram of the adaptive graph convolution layer; Figure 6 It is a structural diagram of the weighted adaptive graph convolution layer; Figure 7 It is a schematic diagram of the STE module structure; Figure 8 This is a schematic diagram of the backbone network STE-GCN structure; Fig. 9 Schematic diagram of the STE-MSFF-GCN action recognition model structure. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The purpose of the present invention is to provide an aging-friendly smart door lock and its use method, using the smart door lock as a physical carrier to build a health protection system for the elderly that integrates risk monitoring, data analysis, and emergency response. Through technical innovations such as multimodal sensor fusion, edge computing architecture, and privacy protection design, the key defects of traditional solutions in terms of accuracy, reliability, and ease of use are solved, providing a feasible and sustainable home safety solution for an aging society.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 1 and Figure 2 As shown, the present invention provides an aging-friendly smart door lock, including: a door lock body, a control module, a fainting detection system, an abnormality detection system, an emergency communication module and an interactive module, the control module, the fainting detection system, the abnormality detection system, the emergency communication module and the interactive module are arranged on the door lock body, and the fainting detection system, the abnormality detection system, the emergency communication module and the interactive module are connected to the control module.

[0023] The fainting detection system includes an inside-door fainting detection system and an outside-door fainting detection system, the inside-door fainting detection system includes an infrared pyroelectric sensor and a sound recognition module, the infrared pyroelectric sensor and the sound recognition module are arranged on the inside-door panel of the door lock body, the infrared pyroelectric sensor is used to monitor human micro-motion signals, the sound recognition module is used to recognize falling and impact sounds, cries for help and long-term groans, the outside-door fainting detection system includes an outside-door camera and a fainting recognition module, the outside-door camera is used to collect video information of the elderly, the fainting recognition module is used to determine whether the elderly fainted based on the video information of the elderly, and the infrared pyroelectric sensor, the sound recognition module, the outside-door camera and the fainting recognition module are all connected to the control module; The camera outside the door captures the image of the elderly when the door is opened for the first time every day, and sends it to the WeChat account of the family through the emergency communication module for their relatives to view.

[0024] The anomaly detection system is used to identify abnormal behaviors of the elderly based on the set anomaly warning rules.

[0025] The emergency communication module includes an eSIM card module and a WIFI module. When there is WIFI, it is connected to the server through the WIFI module. Otherwise, it is connected through the eSIM card module. The eSIM card module and the WIFI module are both connected to the control module. Based on the eSIM card module and the WIFI module, information transmission is realized, and the door lock related information is transmitted to the server. The server displays the door lock related information to the family WeChat account for viewing. Among them, the door lock related information includes the following reminders: Silent reminder: If the door lock is not opened or closed for a certain period of time, the door lock will send a WeChat message to the family number to remind you; Door not locked reminder: After the door is unlocked, if it is not locked for a certain period of time, the door lock will send a WeChat message to the family number to remind you; Reminder for incorrect door opening: If the door is opened incorrectly multiple times (such as incorrect fingerprint, incorrect password, etc.), the door lock will send a WeChat message to the family account to remind you; Door lock low battery reminder: When the battery power is lower than a certain value, the door lock will send a WeChat message to the family number to remind you.

[0026] The interactive module includes an emergency button, a display screen and a voice call module. The emergency button is set on the inner door panel and the outer door panel of the door lock body. The display screen and the voice call module are set on the inner door panel of the door lock body. The emergency button, the display screen and the voice call module are all connected to the control module.

[0027] The door lock body of the present invention adopts a semi-automatic door lock setting, and the handle of the semi-automatic door lock adopts the setting of lifting up to lock (optional door lock power-off function) and pressing down to open quickly: the traditional handle locks the door and opens the door, which can be within the reach of the user. It uses the main lock tongue to lock the door, lift the handle to lock the main lock tongue, and press the handle down to open the door lock. It adopts the lever principle, and it is convenient and light to lock and open the door, which is within the power of the elderly (one-press quick open). It does not have a separate safety door plug, and cancels the indoor safety plug-in door lock function of the traditional handle lock. To avoid the inability to accurately and quickly find the safety plug knob and open it smoothly in the dark or in a panic, there is only the one-press quick open function of the door handle to ensure that it is within the ability of the elderly. When there is a special situation where it is impossible to open the door indoors, the door can be unlocked outdoors for emergency relief; According to specific needs, sensitive biometric unlocking methods (at least two) are set. In addition to passwords and mechanical keys, the door opening method also uses fingerprint recognition (recognizing worn or blurred fingerprints) with semiconductor + AI algorithms and face recognition for the elderly (or palm vein). If you forget the password and do not have a mechanical key, you can use your own biometrics to open the door, which is more convenient for the elderly to use; According to specific needs, set a digital key password to replace the touch screen. The keys are raised and have a clear touch, supporting blind operation. It is equipped with a mechanical key emergency function. When the battery is out of power and the door cannot be opened, the door can be opened by a mechanical key or by using the Type-C port for power supply; The voice call module is equipped with three buttons to bind the smart lock to the mobile phone at the same time. The first two buttons are bound to the family number first, and the third button is fixed to the 24-hour service phone number of the smart platform. The alarm function can be set according to specific needs. When the battery is low or the wrong operation is repeated multiple times (3 times), the alarm will sound and the alarm information can be sent to the family number mobile phone at the same time; Low battery alarm: voice prompt at 20%, APP push notification to the family number at 10%. The family number phone receives the alarm information and can remotely unlock or make emergency processing functions; Silent reminder: If the door is not opened within a certain period of time, a WeChat or phone call will be sent to the family number to remind you; Unlocked door reminder: If the door is not locked for a certain period of time, a WeChat or phone call will be sent to the family number to remind you; Indoor cat-eye video, the indoor screen will automatically turn on when the doorbell is pressed, so that the elderly can see the visitor's face clearly and talk to him / her, and then decide whether to open the door. They can also press the button at any time to check the outdoor situation. The fingerprint and palm vein data of the present invention are not uploaded to the cloud, but only stored in the lock body encryption chip.

[0028] like Figure 3 As shown, the present invention also provides a method for using an aging-friendly smart door lock, comprising: Step 1: The fainting detection system monitors whether the elderly person faints in real time; Step 2: If the elderly person faints, the control module controls the emergency communication module to call the family number and the platform emergency number; Step 3: If the elderly person feels unwell at the door, he or she can dial the family number and the platform emergency number through the emergency button or voice call module based on the emergency communication module; Step 4: Based on the anomaly detection system, the abnormal behavior of the elderly is identified according to the set anomaly warning rules.

[0029] In step 1, the fainting detection system monitors whether the elderly person faints in real time, specifically: Step 101: Detecting whether an elderly person faints inside the door based on a fainting detection system inside the door; Step 102: Detect whether the elderly person faints outside the door based on the outside fainting detection system.

[0030] In step 101, the fainting detection system inside the door detects whether the elderly person faints inside the door, specifically: The infrared pyroelectric sensor detects human micro-motion signals. If relevant signals are detected, the sound recognition module is activated to determine whether there is a fall, a collision, a cry for help, or a long groan. If so, it is determined that the elderly person has fainted inside the door. A detailed description of whether a falling sound, a cry for help, or a long groan occurs based on the sound recognition module: The present invention adopts a sound recognition algorithm based on a Bayesian optimization algorithm for sound recognition; First, the Bayesian optimization algorithm is introduced: The Bayesian optimization algorithm is widely used in many neural network models to adjust the hyperparameters of the model due to its superior performance. Therefore, in the present invention, the Bayesian optimization method is also selected to select the hyperparameters. The two core components of the Bayesian optimization algorithm are the prior function (PF) and the acquisition function (ACF). The prior function PF is usually unknown. In this case, the kernel function based on the Gaussian process (GP) can be used as the prior function, that is, the GP algorithm is used to model the objective function. The acquisition function is used to select the hyperparameter group to be observed during the iteration process; Assume that when using the Bayesian optimization algorithm to select hyperparameters, the objective function to be optimized is defined as:

[0031] Among them, the objective function Set to the accuracy of the model on the validation set, represents a combination of hyperparameters, which in the present invention is a combination of four hyperparameters: learning rate, regularization coefficient, descent rate, and depth of the first layer of the model; Then, the optimization problem to be solved can be expressed as:

[0032] in, is a combination of hyperparameters A set of hyperparameters in the optimization problem can be expressed as: Find the optimal set of parameters in , so that the accuracy of the model on the validation set Reach the maximum value; The essence of the Bayesian optimization algorithm can be regarded as a sequential decision problem. In other words, multiple iterations are required in the process of finding the optimal hyperparameter group. Assume that the total number of iterations is , in each iteration, we need to select the hyperparameter combination Select a group To calculate the value of the corresponding objective function However, due to the influence of other factors, the observed targets are usually noisy values, and the actual output result of this iteration is not , but:

[0033] in, It obeys Gaussian distribution, that is is the variance of the noise, and the newly observed set of values Add to the observation results, and then select the observation data for the next iteration through the acquisition function ACF ; The ACF function is used to solve the balance problem between model exploration and exploitation. Exploration is to obtain sampling points in areas that have not been sampled, and exploitation is to sample in areas where the global optimal solution is most likely to appear based on the posterior distribution. The result set at this time is , according to the prediction obtained by GP algorithm, its mean and variance can be expressed as: and ,in, Can be regarded as development, and The corresponding method is exploration. The expression of the acquisition function ACF can be obtained through commonly used methods such as Gaussian Process-Upper Confidence Bound (GP-UCB), Expected Improvement (EI), Entropy Search, and Thompson Sampling (TS); The present invention uses an improved DCNN model based on Bayesian optimization to realize sound recognition, wherein a basic deep convolutional neural network is usually formed by a combination of multiple convolutional layers and fully connected layers. The wide application of networks such as VGG-Nets has verified that as the network depth increases, the extracted high-dimensional features can better characterize the sound signal. However, the deepening of the network depth will also aggravate the overfitting problem of the model. In order to alleviate the overfitting problem, the model usually introduces Dropout to reduce certain training parameters, and uses data enhancement methods to expand the training data set. However, data enhancement not only increases the overall computational cost, but also may introduce junk data. It is feasible to use 1×1 convolution kernels to perform convolution on feature maps of different layers, and adjust the depth of the output features on the basis of maintaining the original planar structure of the feature map, so as to achieve data dimensionality reduction on the basis of ensuring the original structure. At present, in network models built based on deep CNN (such as ResNet50, etc.), 1×1 convolution kernels are usually used to solve the connection problem between different layers of the model and reduce the dimensionality of the input feature map; The present invention utilizes the characteristics of 1×1 convolution kernel that does not change the original input feature plane structure and the weight sharing of convolution layer, and realizes the dimension increase of input features by adjusting the depth of convolution kernel. The DCNN model used in the present invention is composed of 4 convolution layers and 3 fully connected layers. The specific parameters are as follows: Figure 4 As shown in the figure, the convolution kernel size of the first layer Conv1 of the model is set to 1×1. In order to avoid the local optimal problem of the model caused by the number of layers selected manually, the Bayesian optimization algorithm is used to adaptively select the number of convolution kernels t contained in Conv1. In addition, the Dropout layer and L2 regularization (Conv2-Conv4) are added to the model. Similarly, the drop rate and regularization coefficient are also selected by the Bayesian algorithm. The Bayesian algorithm will update the posterior distribution of the objective function to be optimized according to the new samples until the posterior distribution basically fits the actual distribution. The objective function to be optimized is a function in a general sense, and only the input and output of the function need to be specified. In the present invention, the objective function is set as the accuracy of the model on the validation set. The calculation process of the Bayesian optimization algorithm is as follows: (1) Obtain training data set and validation data set; (2) Setting initial values: setting the hyperparameters (non-adjustable part) of the recognition model, such as the number of iterations epoch, batch size (Batch_Size), the depth of other layers in the model except the first layer, the size of the convolution kernel, etc.: setting the value range and initial value of the hyperparameter to be searched (the initial value in the present invention is a random value within the value range), the objective function (the accuracy of the model validation set), the number of Bayesian optimizations, and the number of random explorations; (3) When the termination condition is not reached, the following steps are executed continuously: Use the data in the training set to construct the Gaussian regression model GP: Based on GP, ​​the expression of the ACF function is obtained, and the hyperparameter set when the ACF function value is the largest is calculated. ;Will The values ​​of each hyperparameter in the model are put into the recognition model, and then training is performed. Finally, the validation set samples are recognized to obtain the accuracy of the model on the validation set. (4) The final output of the Bayesian optimization algorithm is: the searched hyperparameter group and its corresponding validation set accuracy.

[0034] In step 102, the outside-door fainting detection system is used to detect whether the elderly person faints outside the door, specifically: Collecting video information outside the door based on the camera outside the door; The fainting recognition module builds a graph convolutional action recognition model and trains it based on a preset data set; The collected video information outside the door is input into the trained graph convolution action recognition model to identify the fainting situation of the elderly.

[0035] The graph convolution action recognition model is introduced in detail. The present invention is based on graph convolution action recognition based on spatiotemporal enhancement and multi-stream feature fusion; First, the two-stream adaptive graph convolutional network is introduced: 2s-AGCN builds a two-stream framework for action recognition. First, the joint points and bone information in the human skeleton graph are extracted, and then they are input into the same AGCN to form separate joint streams and bone streams. Finally, the recognition scores of the two branches are added together to obtain the fusion score and predict the action results. The advantage of the 2s-AGCN model is that it adaptively learns the graph topology of the sample through the adaptive graph convolutional network, and makes full use of the second-order information of the skeleton, that is, the bone information supplements the dynamic information of the human action. The adaptive graph convolution network AGCN is composed of 9 adaptive graph convolution modules in cascade. The adaptive graph convolution module is composed of adaptive graph convolution (Convs) and temporal convolution (Convt) in series. After extracting the spatial and temporal information of the human skeleton graph, batch normalization (BN) operation is performed, and the ReLU function is used as the activation function to improve the speed and generalization ability of the entire network model. The adaptive graph convolution block adds a Dropout function before the temporal graph convolution to reduce the connection between neurons, enhance the generalization ability of the network and the speed of parameter transfer; Next, the human skeleton spatiotemporal graph is introduced. When using skeleton sequence modality to analyze human body movements in a video, the skeleton sequence is usually designed as a human skeleton spatiotemporal graph and input into GCN for feature extraction. The human skeleton graph is composed of a series of points and lines that simulate the human skeleton structure. The information of these points is represented by 2D or 3D coordinates. Each coordinate point in the skeleton is called a joint point. The effective connection between two points becomes a bone. The human skeleton graph is constructed using the natural connection of the human body in the spatial dimension. The same nodes in the skeleton graphs of adjacent frames are connected in the time dimension, thereby forming a human skeleton spatiotemporal graph from the human skeleton graph. Next, the improved two-stream adaptive graph convolutional network is introduced, in which the weighted adaptive graph convolutional layer is: Adjacency matrix in ordinary graph convolution It can only represent a single human body structure graph and lacks the flexibility to model all action category samples. In order to construct different graph structures for different samples, 2s-AGCN proposes an adaptive graph convolutional layer AGCL. On this basis, an increase and Two matrices, such as Figure 5 As shown; Defined by the physical structure of the human body, it represents the natural connection relationship between the joints of the human skeleton in the spatial domain; yes The weight matrix, the matrix elements are completely learned from the input data, and can flexibly reflect the correlation strength between the joints of the human body's motion posture; Represents the attention adjacency matrix between joints. The attention strength values ​​of two joints are calculated through the motion trajectory of a joint, and a unique graph structure is generated for each sample. It is a key parameter that reflects the adaptability of the entire adaptive graph convolution layer. The calculation process of the adaptive graph convolution layer is shown in the following formula:

[0036] in, represents the input of the adaptive graph convolutional layer, represents the output of the adaptive graph convolutional layer, represents the weight of the kth neighborhood subset, Represents the division method of the neighborhood subset of human joint points, where ; By introducing parameters and AGCL adaptively learns the topological structure of skeleton joints in an end-to-end manner. This network layer allows adding connections other than the natural connections of the human body to dynamically adjust the graph structure to better suit the recognition task. Based on AGCL, this paper proposes a weighted adaptive graph convolution layer to independently capture the spatial information of the joints, such as Figure 6 As shown in the figure, the weighted adaptive graph convolution layer makes two improvements: in the adjacency matrix Add dynamic weight coefficients , this coefficient can adaptively adjust the value according to the increase of training process, thereby increasing the flexibility of the network; add one-dimensional convolution after the Gaussian embedding function normalization operation Utilize CNN to fuse channel dimension information and enhance the network's mapping capability to feature information; Output features of weighted adaptive graph convolutional layer The calculation steps are as follows: First, the nth layer input feature f in The dimension is , mapping it to the Gaussian embedding function and The joint correlation is calculated in , and the two embedding graphs are rearranged as and The calculation process of the two matrices is as follows:

[0037] Secondly, multiply the two matrices and perform normalization to obtain The similarity matrix is ​​then convolved with an output channel of 1 to aggregate the feature information of the channel dimension to obtain the following formula:

[0038] Finally, , and Add and fuse to form the adjacency matrix of the adaptive graph convolutional network, the output features of the nth layer of the adaptive graph convolutional network It can be expressed as the following formula:

[0039] The core of weighted AGCL is the dynamic weight coefficient , As the number of training rounds increases, the adjacency matrix representing the physical structure of the human body decreases. The effect weakened in the later stages of the experiment, thus highlighting the adaptive generation of matrices In order to improve the flexibility of human skeleton feature extraction, the number of iterations set in the experiment of the present invention is 50. The calculation formula is as follows:

[0040]

[0041] The spatial-temporal enhancement attention module is introduced. The skeleton sequence contains information in both time and space dimensions. Adding an attention mechanism to the action recognition model can help the network focus on the most recognizable spatial-temporal regions in complex sequences, while ignoring those interference regions that are irrelevant to the task, thereby improving the accuracy and efficiency of network recognition. The present invention proposes a spatial-temporal enhancement (STE) attention module, which is composed of a cascade of spatial attention and temporal attention. It can capture effective information in both spatial and temporal dimensions. The module can be embedded in any position of the network without affecting the original structure of the network.

[0042] The STE module structure is as follows Figure 7 As shown in the figure, the implementation process of the STE module can be divided into the following four steps: Step 1: Extract features from the input skeleton sequence to obtain the input feature map , and then transform the multi-frame images into a feature map composed of single-frame images The change in feature dimension during the conversion process is shown in the following formula:

[0043] Where C represents the number of channels, T represents the number of frames, W and H represent the width and height of the feature map respectively.

[0044] Step 2: For each frame, perform global average pooling AvgPool in its channel dimension to obtain the feature map of the nth frame image as shown in the following formula: , the role of AvgPool is to compress channel information in order to focus on spatial information. After channel pooling is performed on the feature maps of all frames, convolution operation is performed on the feature maps. To ensure that the size of the feature maps remains unchanged, the stride of the convolution kernel is set to 1. After the activation function Sigmoid is calculated, the spatial attention feature is obtained. , as shown below:

[0045]

[0046] In the formula, represents the Sigmoid activation function, Represents the convolution operation; Step 3: Spatial Attention Features After the spatial attention module, it becomes , input into the temporal attention module, and transform its feature shape into , as shown below,

[0047] Then, the converted features are input into the global average pooling layer to compress their spatial information. At this time, the calculation of temporal attention is converted into the calculation of channel attention. The calculation process is shown in the following formula. Input into the network layer consisting of two fully connected layers FC to capture temporal context information. After activation, the temporal attention feature is obtained , as shown below:

[0048]

[0049] In the formula, Indicates the first FC, the parameter is set to Indicates the second FC, the parameter is set to , after two fully connected layers, the dimensions of the feature map input before and after can be guaranteed to be the same; Step 4: Get the temporal attention features and All feature maps of each frame are weighted to obtain the final output feature , as shown below, input feature After spatial attention and temporal attention, the feature changes are as follows:

[0050]

[0051] The multi-stream feature fusion strategy is introduced. Most of the previous action recognition methods use human skeleton joint information alone, and rarely extract bone information in the skeleton sequence for fusion, which greatly weakens the utilization rate of features. In order to obtain as much information as possible in the skeleton sequence and improve the recognition ability of the network, people began to study the fusion of high-dimensional information in the human skeleton to obtain better recognition effects. There are two main types of feature fusion methods, divided into early fusion and late fusion. Early fusion usually fuses feature maps during feature extraction, and inputs the fused features into the network to learn and obtain the results. Late fusion is to train data of different modalities on each model separately, and fuse the classification probability output by each model using the corresponding fusion function; The feature correlation of early fusion is difficult to extract, and it will increase the computational complexity of the network. The late fusion process is independent of the features and will not affect the feature dimension. Therefore, the present invention uses the splicing fusion in the late fusion and proposes a multi-stream feature fusion strategy (Multi-Stream Feature Fusion, MSFF), which weights the classification probability on the tributaries to obtain the fusion result. The fused feature vector retains all the information of the three tributaries, and at the same time realizes the complementary information difference of the three tributaries under the current action. The fusion process is shown in the following formula:

[0052] In the formula, X, Y and Z represent the output characteristics of the three tributaries respectively. , Represents the fusion weight, and the weight range is [0, 1]. Represents matrix multiplication.

[0053] The spatiotemporal enhanced attention module is combined with the weighted adaptive graph convolution block to construct the backbone network STE-GCN, as shown in Figure 8 As shown in the figure, STE-GCN is composed of weighted adaptive graph convolution blocks, STE, BN layer, global average pooling (GAP) layer and Softmax output layer. Among them, the adaptive graph convolution block (AGCB) corresponds to B1~B9 in the figure respectively. The three numbers of each block correspond to the number of input channels, the number of output channels and the stride. Therefore, the output channels of each AGCB are 64, 64, 64, 128, 128, 128, 256, 256 and 256; The input data is first normalized by the BN layer, and then passes through three adaptive graph convolution blocks in sequence. The feature value with 64 channels is output from B3 and passed to the STE attention module to capture effective spatiotemporal information. The feature size output from STE remains unchanged, so the number of channels input to B4 is still 64. The stride of the convolution kernel is set to 2 in B4 and B7, and the number of frames is downsampled once to reduce the number of parameters and increase the network training speed. After the data is output from B9, global average pooling is performed to pool the feature maps of different samples to the same size. Finally, after Softmax activation, the prediction result is obtained.

[0054] The original skeleton sequence only contains the joint information of the skeleton data, that is, the two-dimensional or three-dimensional coordinates of the joint points.

[0055] In order to enhance the accuracy and recognition ability of the action recognition model, the present invention extracts the joint point information, bone information and bone movement information from the human skeleton features, and inputs them into the backbone network STE-GCN for training and learning respectively. The multi-stream feature fusion strategy is used to cross-fuse the semantic information of the three branches to form information complementarity, and the overall network STE-MSFF-GCN action recognition model is constructed. Fig. 9 as shown. Define the joint closest to the skeleton's center of mass as the source joint , the joint far away from the center of gravity is defined as the target joint , joints of frame t The skeleton information is represented as a vector pointing from its source joint to the target joint Assign the central joint to the self-built empty bone with a value of 0, and the graphics and networks of bones and joints can correspond one to one. The skeleton movement information is expressed by the formula: In the figure, J-Stream represents the branch trained using joint features, and B-Stream and M-Stream represent the skeleton branch and skeleton motion branch respectively.

[0056] During the training process, the three branches have different focuses on information collection and processing. The joint stream processes the relationship between the joints of the human skeleton in the video frame, the bone stream collects the strong and weak connections between the bones in the frame, and the bone motion stream captures the correlation between the bones in the frame and the bone motion between frames. The three branch networks are trained separately to obtain the corresponding features, and the fused features are obtained by splicing the adaptive feature fusion module, and finally the recognition results are obtained through the Softmax layer.

[0057] In step 4, the abnormal behavior of the elderly is identified based on the abnormal detection system according to the set abnormal warning rules, specifically: Obtain the number of times the elderly open their doors every day, the duration of time they go out, and the high-frequency time periods; The number of times the elderly open the door on that day is collected. If it drops by 70% compared to the average value within 7 days, it is judged that the elderly have an abnormality; Collect whether the elderly have gone out during the high-frequency time period of the day. If they have not gone out for three consecutive days, it is considered that something is wrong with the elderly. Collect the time the elderly have been out. If they have not returned after more than 12 hours, it is considered that the elderly have an abnormality. If it is determined that an abnormality has occurred in the elderly, the control module will call the family number and the platform emergency number through the emergency communication module.

[0058] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0059] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An aging-friendly smart door lock, characterized in that: include: A door lock body, a control module, a fainting detection system, an abnormality detection system, an emergency communication module and an interactive module. The control module, the fainting detection system, the abnormality detection system, the emergency communication module and the interactive module are arranged on the door lock body, and the fainting detection system, the abnormality detection system, the emergency communication module and the interactive module are connected to the control module; The door lock body is a semi-automatic door lock, the door handle is set to lock by lifting up and open quickly by pressing down, and the safety door plug is cancelled. The door lock body is provided with a password unlocking module, a mechanical key unlocking module and a fingerprint unlocking module; The fainting detection system includes an inside-door fainting detection system and an outside-door fainting detection system. The inside-door fainting detection system includes an infrared pyroelectric sensor and a sound recognition module. The infrared pyroelectric sensor and the sound recognition module are arranged on the door inner panel of the door lock body. The infrared pyroelectric sensor is used to monitor human micro-motion signals. The sound recognition module is used to recognize falling and impact sounds, cries for help, and long-term groans. The outside-door fainting detection system includes an outside-door camera and a fainting recognition module. The outside-door camera is used to collect video information of the elderly. The fainting recognition module is used to determine whether the elderly faint based on the video information of the elderly. The infrared pyroelectric sensor, the sound recognition module, the outside-door camera, and the fainting recognition module are all connected to the control module. The outside-door camera captures a picture when the door is opened for the first time every day, and sends it to the family WeChat account through the emergency communication module, so as to facilitate the daily status of the elderly. The emergency communication module includes an eSIM card module and a WIFI module. When there is WIFI, it is connected to the server through the WIFI module. Otherwise, it is connected through the eSIM card module. The eSIM card module and the WIFI module are both connected to the control module. The eSIM card module and the WIFI module are used to realize information transmission, and the door lock related information is transmitted to the server, and the server displays the door lock related information to the family WeChat account for viewing. The interactive module includes an emergency button, a display screen and a voice call module. The emergency button is set on the inner door panel and the outer door panel of the door lock body. The inner door panel of the door lock body is set with the display screen and the voice call module. The emergency button, the display screen and the voice call module are all connected to the control module. The voice call module is provided with three buttons. The first two buttons are bound to the family number first, and the third button is fixedly bound to the 24-hour service telephone of the smart platform. Based on the voice call module, the family number and the 24-hour service telephone of the smart platform can be called with one button.

2. The smart door lock according to claim 1, characterized in that: The anomaly detection system is used to identify abnormal behaviors of the elderly through set anomaly warning rules.

3. A method for using an aging-friendly smart door lock, characterized in that: include: Step 1: The fainting detection system monitors whether the elderly person faints in real time; Step 2: If the elderly person faints, the control module controls the emergency communication module to call the family number and the platform emergency number; Step 3: If the elderly person feels unwell at the door, he or she can dial the family number and the platform emergency number through the emergency button or voice call module based on the emergency communication module; Step 4: Based on the anomaly detection system, the abnormal behavior of the elderly is identified according to the set anomaly warning rules.

4. The method according to claim 3, characterized in that In step 1, the fainting detection system monitors whether the elderly person faints in real time, specifically: Step 101: Detecting whether an elderly person faints inside the door based on a fainting detection system inside the door; Step 102: Detect whether the elderly person faints outside the door based on the outside fainting detection system.

5. The method according to claim 4, characterized in that In step 101, the fainting detection system inside the door detects whether the elderly person faints inside the door, specifically: The infrared pyroelectric sensor detects human body micro-movement signals. If relevant signals are detected, the sound recognition module is activated. Based on the sound recognition module, it is determined whether there is a fall, impact sound, cry for help, or long groaning sound. If so, it is determined that the elderly person has fainted inside the door.

6. The method according to claim 5, characterized in that In step 102, the outside-door fainting detection system is used to detect whether the elderly person faints outside the door, specifically: Collecting video information outside the door based on the camera outside the door; The fainting recognition module builds a graph convolutional action recognition model and trains it based on a preset data set; The collected video information outside the door is input into the trained graph convolution action recognition model to identify the fainting situation of the elderly.

7. The method according to claim 6, characterized in that In step 4, the abnormal behavior of the elderly is identified based on the abnormal detection system according to the set abnormal warning rules, specifically: Obtain the number of times the elderly open their doors every day, the duration of time they go out, and the high-frequency time periods; The number of times the elderly open the door on that day is collected. If it drops by 70% compared to the average value within 7 days, it is judged that the elderly have an abnormality; Collect whether the elderly have gone out during the high-frequency time period of the day. If they have not gone out for three consecutive days, it is considered that something is wrong with the elderly. Collect the time the elderly have been out. If they have not returned after more than 12 hours, it is considered that the elderly have an abnormality. If it is determined that an abnormality has occurred in the elderly, the control module will call the family number and the platform emergency number through the emergency communication module.

Images