Intelligent lock management system and method based on behavior recognition

Abstract

The present invention discloses a smart lock management system and method based on behavior recognition, and specifically relates to the technical field of smart lock management, and is used to solve the problem of poor management of smart locks under high traffic. The present invention effectively responds to changes in different environments and user approach speeds by intelligently adjusting the acquisition frequency and range of smart lock sensors, and preprocesses and screens data collected by multiple sensors, and performs multimodal feature fusion on these data, so as to accurately extract comprehensive feature vectors, thereby improving the accuracy of preliminary user behavior recognition, and then dynamically evaluates the security status of the smart lock by analyzing and identifying accurate information, so as to decide whether to omit subsequent verification steps. When the security status is abnormal, it can automatically trigger additional verification processes, respond to potential security threats in a timely manner, optimize user experience, and improve the convenience and reliability of smart lock management.

Description

Technical Field

[0001] The present invention relates to the field of smart lock management technology, and more specifically, to a smart lock management system and method based on behavior recognition. Background Art

[0002] With the rapid development of technology in the smart field, smart locks, as an important part of the smart field, have been widely used. Smart locks not only improve the security and convenience of the house, but also provide a more intelligent experience through linkage with other smart devices. The existing smart lock system mainly relies on traditional identity authentication methods, such as passwords, fingerprints, cards or mobile phone APPs. Although these methods have improved security to a certain extent, there are still some limitations and potential security risks.

[0003] Deficiencies in existing technologies: Although some smart locks have the ability to link with other smart devices, their intelligence level is still limited and they are unable to fully realize personalized and automated security management, especially in periods of multiple users or high traffic. Smart locks need to identify and process each user's information and cannot reduce the pressure of these high-traffic periods by optimizing the verification process. They cannot dynamically adjust demand based on user behavior habits, which can cause congestion and even create safety hazards. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a smart lock management system and method based on behavior recognition to solve the problem of low adjustment efficiency of smart locks in optimizing verification requirements during high traffic periods proposed in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The smart lock management method based on behavior recognition includes the following steps:

[0007] Configure sensor devices for smart locks and obtain sensor data. The sensor intelligently adjusts the collection frequency and range according to the environment and user approach speed;

[0008] Behavior recognition is performed on the process of users moving from far to near. The data collected by sensors in each process is pre-processed. After filtering the features extracted by different sensors, multi-modal data features are fused. After feature fusion, the comprehensive feature vector is obtained and the classifier is used to perform preliminary user behavior recognition.

[0009] If the preliminary user behavior recognition is to skip the subsequent verification steps, the preliminary user behavior recognition process is subjected to security verification analysis to obtain accurate recognition information generated during the recognition process and determine the security status of the smart lock management process;

[0010] If the security status of the smart lock management process is stable, a management stability signal is generated and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated and the subsequent identity authentication process is started for management.

[0011] In a preferred embodiment, a sensor device is configured for the smart lock and the sensor data is acquired. The sensor intelligently adjusts the acquisition frequency and range according to the environment and the user's approach speed. The specific process is as follows:

[0012] Sensors include cameras, microphones, accelerometers, gyroscopes, infrared sensors, and depth cameras;

[0013] The sensor dynamically adjusts the acquisition frequency based on the user’s approach speed and distance;

[0014] Depending on the ambient light and noise level, the sensor automatically adjusts acquisition parameters.

[0015] In a preferred embodiment, the data collected by the sensors in each process from far to near are preprocessed, and the specific process is as follows:

[0016] When the user enters the long-distance sensing range of the smart lock, the infrared sensor and camera are activated, and the camera captures image data and motion trajectory;

[0017] Add microphone sensors to collect auditory data at medium distances, and use all sensors to collect data at close distances;

[0018] Preprocess the collected sensor data, including image denoising and feature extraction;

[0019] Use Gaussian filter to denoise the image data collected by the camera and depth camera;

[0020] According to the light intensity and noise level of the environmental conditions, the filter parameters of the Gaussian filter are adjusted to perform adaptive filtering;

[0021] Use convolutional neural networks to extract features from image and depth data, and use autoencoders to extract features from speech and motion data.

[0022] In a preferred embodiment, after the features extracted by different sensors are screened, multimodal data features are fused, and a classifier is used to perform preliminary user behavior recognition on the comprehensive feature vector obtained after feature fusion. The specific process is as follows:

[0023] Through the fusion of Bayesian theorem, the prior probability of the category and the marginal probability of the feature are obtained, and the likelihood probability of the corresponding feature under different categories is calculated based on the prior probability of the category and the marginal probability of the feature;

[0024] Calculate the posterior probability of the class given the features;

[0025] The features that are greater than the set posterior probability threshold are fused into a comprehensive feature vector;

[0026] Use the classifier to perform preliminary user behavior identification, select a machine learning model, and input the fused comprehensive feature vector into the selected machine learning model;

[0027] The classifier comprehensively evaluates the comprehensive feature vector and predicts the preliminary user behavior recognition results.

[0028] In a preferred embodiment, if the preliminary user behavior recognition is to skip the subsequent verification step, the preliminary user behavior recognition process is subjected to security verification analysis, accurate recognition information generated during the recognition process is obtained, and the security status of the smart lock management process is determined. The specific process is as follows:

[0029] Set the confidence level. If the confidence level of the prediction result of the classifier exceeds the preset confidence level threshold, the preliminary user behavior recognition will skip the subsequent verification steps.

[0030] Acquire accurate identification information generated during the identification process, including behavior pattern information and identification stability information;

[0031] The behavior pattern information includes the behavior recognition accuracy index, and the recognition stability information includes the composite stability index;

[0032] The obtained behavior recognition accuracy index and composite stability index are subjected to grey correlation analysis.

[0033] In a preferred embodiment, the obtained behavior recognition accuracy index and composite stability index are subjected to grey correlation analysis, and the specific process is as follows:

[0034] The composite stability index is set as the reference series, and the behavior recognition accuracy index is set as the comparison series;

[0035] The correlation coefficient was calculated using grey correlation analysis, and the correlation coefficient was averaged to obtain the average correlation degree in the entire analysis period.

[0036] In a preferred embodiment, if the security status of the smart lock management process is stable, a management stability signal is generated and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated and a subsequent identity authentication process is started for management. The specific steps are as follows:

[0037] The average correlation was compared to the identification management threshold;

[0038] If the average correlation is greater than or equal to the identification management threshold, a management stability signal is generated and no subsequent verification management measures are taken;

[0039] If the average correlation is less than the identification management threshold, an intelligent verification signal is generated to start the subsequent identity authentication process management.

[0040] The smart lock management system based on behavior recognition is used to implement the above-mentioned smart lock management method based on behavior recognition, including:

[0041] The data acquisition module is used to configure the sensor device for the smart lock and obtain the sensor collection data. The sensor intelligently adjusts the collection frequency and range according to the environment and the user's approach speed;

[0042] The behavior recognition module is used to recognize the behavior of users in the process of moving from far to near. The data collected by sensors in each process of moving from far to near is pre-processed, and the features extracted by different sensors are filtered, and then multi-modal data features are fused. The comprehensive feature vector obtained after feature fusion is used by a classifier to perform preliminary user behavior recognition.

[0043] The security analysis module is used to perform security verification analysis on the preliminary user behavior identification process, obtain accurate identification information generated during the identification process, and determine the security status of the smart lock management process;

[0044] The management module generates a management stability signal if the security status of the smart lock management process is stable, and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated to start the subsequent identity authentication process for management.

[0045] Technical effects and advantages of the present invention:

[0046] The present invention effectively responds to changes in different environments and user approach speeds by intelligently adjusting the acquisition frequency and range of smart lock sensors, thereby optimizing energy efficiency and increasing the efficiency of data acquisition. By preprocessing and screening data collected by multiple sensors and fusing these data with multimodal features, the present invention can accurately extract comprehensive feature vectors, thereby improving the accuracy of preliminary user behavior recognition. This process reduces the possibility of misrecognition and improves the quality of data processing.

[0047] In terms of security verification, the present invention dynamically evaluates the security status of the smart lock by analyzing and identifying accurate information, thereby deciding whether to omit subsequent verification steps. When the security status is abnormal, it can automatically trigger additional verification processes to respond to potential security threats in a timely manner. This flexible security strategy not only enhances the smart lock's ability to handle abnormal conditions, but also optimizes the user experience, avoids unnecessary operation delays, and improves the convenience and reliability of smart lock management. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of the smart lock management method based on behavior recognition of the present invention.

[0049] Figure 2 It is a structural diagram of the smart lock management system based on behavior recognition of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example 1: Figure 1 As shown, the smart lock management method based on behavior recognition includes the following steps:

[0052] Configure sensor devices for smart locks and obtain sensor data. The sensor intelligently adjusts the collection frequency and range according to the environment and user approach speed;

[0053] Behavior recognition is performed on the process of users moving from far to near. The data collected by sensors in each process is pre-processed. After filtering the features extracted by different sensors, multi-modal data features are fused. After feature fusion, the comprehensive feature vector is obtained and the classifier is used to perform preliminary user behavior recognition.

[0054] If the preliminary user behavior recognition is to skip the subsequent verification steps, the preliminary user behavior recognition process is subjected to security verification analysis to obtain accurate recognition information generated during the recognition process and determine the security status of the smart lock management process;

[0055] If the security status of the smart lock management process is stable, a management stability signal is generated and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated and the subsequent identity authentication process is started for management.

[0056] Install sensor devices, configure edge computing nodes and smart lock management software, set initial parameters and user account information, and implement sensor fusion and intelligent collection strategies to improve data accuracy. The specific steps are as follows:

[0057] Acquire sensor data. When a user approaches the smart lock, the sensor (camera, microphone, accelerometer, gyroscope, infrared sensor, depth camera) is activated. The sensor intelligently adjusts the acquisition frequency and range according to the environment and the user's approach speed.

[0058] The camera captures the user's gait, facial expressions and specific movements and is installed near the door frame or smart lock;

[0059] The microphone captures the user's voice for voiceprint recognition and command reception. It is installed near the smart lock to clearly receive the user's voice input.

[0060] The accelerometer is integrated in the smart lock or door handle to sense the user's motion characteristics when operating the door handle, detect the user's movement and acceleration changes, and is used to analyze gait and motion characteristics. Similarly, the gyroscope is integrated with the accelerometer, usually in the smart lock or door handle, to detect the user's rotation and tilt movements, and combine the accelerometer data for motion analysis;

[0061] Infrared sensors are used to detect the user's approach and departure, and thermal sensing is used to achieve recognition at night or in low-light environments; depth cameras capture the user's three-dimensional movements and gait, and improve recognition accuracy through depth data;

[0062] The camera captures the user's facial features and uses image processing technology to perform facial recognition. The camera and depth camera jointly capture the user's walking style and gait profile as image data for gait recognition.

[0063] The microphone captures the user's voice, uses voiceprint recognition technology to authenticate the user, and receives the user's voice commands as voice data;

[0064] The accelerometer and gyroscope detect the user's hand movements and overall motion for analyzing door opening and other operating behaviors as motion data;

[0065] The infrared sensor detects the user's approach and departure, triggering the smart lock to enter standby or activation state.

[0066] The sensor dynamically adjusts the acquisition frequency based on the user's approach speed and distance. For example, when the infrared sensor detects that the user is approaching, the camera and depth camera begin to collect image and depth data at a high frequency; when the user moves away, the acquisition frequency is reduced to save energy.

[0067] The sensor automatically adjusts the acquisition parameters according to the ambient light and noise level. For example, the camera automatically adjusts the exposure settings according to the light intensity, the microphone automatically adjusts the gain according to the background noise level, and uses different sensor devices according to different distances.

[0068] In the smart lock management based on behavior recognition, a series of intelligent operations are implemented by identifying the user's behavior process from far to near, thereby improving the convenience and security of the system. The general steps of intelligently selecting functions and performing behavior recognition according to the user's approach process are as follows:

[0069] When a user enters the long-distance sensing range of the smart lock, the infrared sensor or other sensors used to detect human approach are activated first, and the camera begins to capture image data. However, at this stage, it is mainly used to determine whether someone is approaching and observe the user's general behavior or movement trajectory. Starting from a long distance, the user's intention can be predicted by analyzing the user's behavior pattern (such as walking speed, whether carrying items, etc.). For example, if the camera and other sensors observe that the user approaches quickly and walks in a straight line towards the location of the smart lock, it can be predicted that the user is likely to enter the room;

[0070] At medium distances, the visual and auditory data collection is enhanced. When the user gets closer, the smart lock will enhance the collection of visual and auditory data, such as high-frequency video capture through the camera and possible voice commands or calls through the microphone. At this stage, facial recognition or voiceprint recognition technology can be used for more sophisticated identity authentication.

[0071] At close range, when the user approaches the door (close distance), the smart lock will activate all advanced functions, such as high-precision facial recognition, fingerprint recognition or voiceprint recognition combined with accelerometer and gyroscope for unlocking action recognition. At this time, if the smart lock has not confirmed the user's identity, it will require the user to perform explicit identity authentication, such as through facial, voiceprint or fingerprint.

[0072] It should be noted that in the process from far to near, the specific far and near distance division is determined based on the performance and usage of each sensor, and is not limited one by one here.

[0073] Data preprocessing is performed on the data collected by sensors from far to near. Data preprocessing is edge computing of the collected multi-source sensor data. By integrating edge computing on sensor devices, data preprocessing and preliminary analysis can be performed in real time, such as image denoising, feature extraction, and data compression, to reduce transmission delays and computing loads.

[0074] Use Gaussian filter to denoise the image. , where G(x, y) is the Gaussian function value, which is used to smooth the filter weight of the image, σ is the standard deviation of the Gaussian filter, which controls the width and smoothness of the filter, x, y are the image coordinates, which are used to calculate the distance from the center point, and e represents the base of the natural logarithm, which is approximately 2.71;

[0075] Automatically adjust the filtering parameters according to environmental conditions (such as light intensity and noise level) to perform adaptive filtering, thereby improving the data preprocessing effect. The specific steps are as follows:

[0076] The standard deviation σ of the Gaussian filter is adjusted according to the light intensity L and the noise level N, and the expression is: , where σ is the standard deviation of the Gaussian filter, , It is the weight coefficient that controls the influence of light and noise on the filtering parameters. is the reference light intensity, used to normalize the light intensity, The reference noise level is used to normalize the noise level and filter the image using the adjusted Gaussian filter to improve the data preprocessing effect;

[0077] Feature extraction: Convolutional neural networks (CNNs) and autoencoders are used to automatically extract complex behavioral features from multimodal data, improve the accuracy of feature extraction, and fuse data from different sensors in real time. Through multimodal fusion algorithms (such as Bayesian fusion, weighted voting, etc.), the user's behavioral characteristics are comprehensively analyzed to improve the accuracy of recognition;

[0078] Use convolutional neural networks to extract features from image and depth data, , where represents the i-th, j-th eigenvalue in the feature map, Represents the pixel value of the input image at position i+m, j+n, is the weight of the convolution kernel at position m and n, M and N are the sizes of the convolution kernel, and ReLU is a nonlinear activation function used to increase the expressive power of the network;

[0079] Use an automatic encoder to extract speech and motion data features. The specific encoder form is: h=f(Wx+b), where h is the hidden layer feature representation, x is the input data, W is the weight matrix, b is the bias, and f is the activation function, such as ReLU. The decoder form is: ,in, is the reconstructed input data, is the decoding weight matrix, is the decoding bias, g is the activation function;

[0080] Use data enhancement techniques (such as image rotation, cropping, noise addition, etc.) to enrich training data and enhance the generalization ability of the model. At the same time, use feature selection and dimensionality reduction techniques to extract the most discriminative features and reduce computational complexity.

[0081] After feature extraction, the features obtained from different sensors can be fused to form a comprehensive feature vector, which will be used for classification and recognition tasks;

[0082] Perform multimodal data fusion to fuse data from different sensors, that is, connect and combine feature vectors extracted from different sensors;

[0083] Combining the probability distribution of data features of different modalities, we fuse them through Bayesian theorem to obtain the likelihood probability P(x|y) of feature x in case of category y, obtain the prior probability P(y) of category y, obtain the marginal probability P(x) of feature x, and calculate the posterior probability of category y given feature x: ;

[0084] P(y|x) is the posterior probability, i.e. the probability of assuming that class y is true after observing feature x; P(x|y) is the likelihood probability, i.e. the probability of observing feature x if assuming that class y is true; P(y) is the prior probability, i.e. the probability of assuming that class y is true before observing the feature; P(x) is the marginal probability, also called evidence, which is the probability of observing feature x under all possible hypotheses;

[0085] For each feature, the posterior probability is calculated based on its relationship with the output category, and the posterior probability of each feature is compared with the preset posterior probability threshold. Only when the posterior probability of the feature is higher than this posterior probability threshold, the corresponding feature is selected for further fusion analysis.

[0086] Make accurate classification decisions based on observed data features according to the posterior probability. In smart lock management, this can help decide whether to allow user access or require further identity authentication. By continuously updating the prior probability and likelihood probability, smart lock management can learn and adapt to changes in user behavior, improve the overall intelligence and adaptability of management, and based on the analysis results of the posterior probability, continuously optimize the management strategy of smart locks, such as adjusting the strictness of the user verification process, to achieve the best balance between security and user convenience.

[0087] When performing multimodal data fusion, different features or modalities may have different impacts on the final decision. The posterior probability can be used to dynamically adjust the weight of each feature in the decision model. For example, in smart lock management, if the posterior probability of a feature (such as facial recognition) shows that it is highly correlated with a specific user category, the weight of this feature can be increased, thereby enhancing its influence in future feature fusion processes.

[0088] The posterior probability can guide the selection or adjustment of the feature fusion algorithm that best suits the current dataset. For example, if the posterior probability shows that there is a high correlation between certain features, you can choose fusion algorithms that can effectively handle highly correlated feature data, such as principal component analysis (PCA) or other forms of dimensionality reduction methods to avoid multicollinearity problems.

[0089] The features extracted by different sensors that meet their posterior probability thresholds are screened, and the screened feature vectors are fused into a comprehensive feature vector to form a fused comprehensive feature vector. , n is a positive integer, is the feature vector of the i-th sensor. The comprehensive feature vector may involve simple concatenation of feature vectors or may use more complex fusion techniques such as feature weighting and fusion networks. The specific fusion method used is determined according to the actual intelligent lock management requirements;

[0090] Use a classifier for preliminary user behavior recognition. Select a machine learning model (such as support vector machine, decision tree, random forest, or deep learning model) and input the fused comprehensive feature vector into this model, which can be expressed by a simple classification decision formula. The expression is: , where y is the predicted class or behavior, Classifier is the selected machine learning model, and z is the input fused feature vector;

[0091] The classifier evaluates the feature vector and gives the predicted user behavior recognition result, which is the user's identity or a set of possible user identities. If the output of the classifier is a single user identity, this information can be directly used to unlock, or the unlocking can be refused and subsequent verification can be performed. The specific steps and logic are as follows:

[0092] When the feature vector is input into the classifier, the classifier matches the current input feature through the patterns learned in the training stage and gives a prediction result. If the result points to a single user identity, it means that the classifier is relatively confident in identifying the specific user;

[0093] Set the confidence level. If the prediction result of the classifier is that the confidence level exceeds the preset confidence threshold, the intelligent lock management system can skip the subsequent verification steps and directly perform the intelligent lock unlocking operation to allow the user to enter. On the contrary, if the classifier identifies a user identity but the confidence level is insufficient, the intelligent lock management system requires the user to perform additional identity verification steps, such as entering a password at close range, reconfirming fingerprints or face, etc., to enhance security;

[0094] Furthermore, the classifier also gives a probability distribution indicating the possibility of each user identity. By setting a threshold, the intelligent lock can be unlocked only when the highest probability exceeds this set threshold;

[0095] Specifically, the administrator can set a threshold, for example, 70%, which means that the intelligent lock management system can perform the unlocking operation only when the prediction probability of the classifier for a certain user identity exceeds 70%; if the prediction probabilities of all user identities do not exceed this threshold, the intelligent lock management system will consider that the current recognition result is not sufficient for secure unlocking. In this case, the intelligent lock management system can require additional identity verification, prompt the user to try again or provide other identity verification methods.

[0096] For skipping the subsequent verification steps, you can directly perform security verification on the predicted results of the smart lock unlocking operation. The specific steps are as follows:

[0097] Conduct security verification and analysis on the preliminary user behavior identification process, obtain accurate identification information generated during the behavior identification process, and determine the security status of the smart lock management process. The accurate identification information includes behavior pattern information and identification stability information.

[0098] The behavior pattern information includes the behavior recognition accuracy index and is calibrated as XWS, and the recognition stability information includes the composite stability index and is calibrated as SJF;

[0099] The behavior recognition accuracy index in the behavior pattern information is used to indicate the recognition accuracy of the smart lock system on the user's behavior after comprehensively considering multiple factors, including the consistency of the user's behavior data with the historical pattern, and the adaptability of the user's behavior under specific environmental conditions. It has the following functions:

[0100] The consistency between behavior and historical data is used to evaluate the degree to which the user's current behavior matches their historical behavior data. This includes walking style, facial expressions, voice, etc. The higher the degree of match, the better the consistency of the user's behavior and the higher the recognition accuracy;

[0101] Environmental adaptability is a measure of whether the user's behavior is adapted to the current environmental conditions. For example, the behavior of a user talking in a low voice in a quiet library should be different from that of a user talking loudly on a noisy street. The stronger the adaptability, the more accurate the user's behavior recognition is in different environments; based on the comprehensive consideration of behavioral data and environmental factors, it reflects the overall behavior recognition accuracy of the smart lock system under current conditions.

[0102] The behavior recognition accuracy index is obtained as follows:

[0103] Collect user behavior data at different time points, and integrate the features extracted from different sources into a multi-dimensional behavior feature vector. Represents the behavior characteristics at time point t, obtains the current time T, and calculates the behavior attenuation value. The calculation expression is: , where T is the current time, e is a constant with a value of 2.71, and the vector cosine similarity between the current behavior feature vector and the historical behavior feature vector is calculated. The calculation expression is: , where Represents the behavior feature vector at the current time point T, and calculates the historical behavior consistency value. The calculation expression is: ;

[0104] Get the current environment data that occurs at the same time as the user behavior, construct the environment feature vector from the current environment data, and calculate the vector cosine similarity between the current environment feature vector and the historical environment feature vector: , where represents the behavior feature vector at the current time point T, Representing the behavior feature vector at time point t, obtain and calculate the behavior recognition accuracy index: .

[0105] It should be noted that behavioral data includes data such as video (gait analysis), audio (voiceprint analysis) and biometrics (such as facial expressions) collected by sensors; feature vectors are extracted from the behavioral data of each instance, such as facial feature vectors, gait feature sequences, sound spectrum features, etc.; when calculating the similarity between the current feature vector and the historical feature vector, the data collected by the current feature is matched one-to-one with the historical feature. For example, if the current collected features are long-distance gait and medium-distance facial data, the features collected by the historical data are also long-distance gait and medium-distance facial data; environmental data, including but not limited to light, temperature, humidity, noise level, time (time of day), location (indoors and outdoors), etc.

[0106] The composite stability index in the identification stability information is used to evaluate the stability of user behavior over a certain period of time, and is used to more accurately determine whether the user's behavior pattern is continuous and reliable, thereby providing more effective user verification and behavior monitoring for security systems (such as smart lock systems).

[0107] Evaluate the consistency of user behavior in consecutive recognition events over time, determine long-term patterns and short-term changes in user behavior, analyze whether the user's behavior is adapted to the current environmental conditions, such as whether the appropriate sound level is maintained in a noisy environment, or whether appropriate behavior is exhibited in public places, pay attention to the diversity and complexity of user behavior, help identify different patterns of user behavior, and increase sensitivity to atypical or novel behaviors.

[0108] The composite stability index is obtained as follows:

[0109] Obtain the measured values ​​of each environmental parameter during the behavior recognition process and calculate the environmental variability. The calculation expression is: , where is the measured value of an environmental parameter at the i-th identification, is the average of these measurements;

[0110] Collect user behavior data when interacting with the smart lock, classify the user behavior data by behavior type, and calculate the relative frequency of each interaction method in all interactions. The calculation expression is: ,in, is the number of occurrences of behavior type j, N is the total number of behaviors, and the composite stability index is calculated using the following expression: , where k is the number of behavior types.

[0111] It should be noted that the classification of user behavior data such as entry method (face, fingerprint, password, RFID, etc.), entry time, residence time, etc. refers to treating facial recognition, fingerprint recognition, password entry, etc. as different behavior types.

[0112] The obtained behavior recognition accuracy index and composite stability index are subjected to grey correlation analysis to evaluate the strength and pattern of the relationship between the two indexes. The collected data are processed in the same time period and the same user behavior scenario to eliminate the possible dimensional effects and differences in numerical ranges between different indexes, for example, normalization processing;

[0113] The composite stability index is set as the reference series, and the behavior recognition accuracy index is set as the comparison series. For each pair of processed data points, the absolute difference between the reference series and the comparison series is calculated. The calculation expression is: ,in, is the value of the reference sequence at time t, is the value of the comparison sequence at time t;

[0114] The correlation coefficient is calculated using the formula in grey correlation analysis: , where ρ is the resolution coefficient, which is used to adjust the sensitivity of the difference and usually takes a smaller value, such as 0.1. represents the maximum value among all differences, Represents the minimum value among all differences;

[0115] Get the average correlation, that is, average the correlation coefficients at a single time point to get the average correlation over the entire analysis period;

[0116] A high average correlation indicates that there is a strong correlation between the behavior recognition accuracy index and the composite stability index. According to the results of the correlation analysis, decision makers can better understand and evaluate the interaction of different security indicators in the smart lock system to optimize the configuration of the smart lock management system and improve the user verification process;

[0117] A high correlation means that the behavior recognition accuracy index can reliably predict or reflect the composite stability index, that is, the user's behavior pattern is highly consistent with its stability. This consistency can help the smart lock management system more accurately identify normal and abnormal behaviors and promptly detect potential security threats, such as attempts to impersonate or steal identities. When the two indices are highly correlated, the smart lock management system can interpret user behavior more accurately, effectively reduce false positives and false positives, and improve the overall security performance of the smart lock management system.

[0118] Strong correlation provides reliable data support, enabling the smart lock management system to use historical and real-time data for trend analysis and behavior prediction, foresee potential security issues and prepare in advance.

[0119] Compare the generated average correlation with the defined identification management threshold, generate different management signals, and adjust the corresponding smart lock management strategy according to the generated management signals;

[0120] After obtaining the average correlation, the average correlation is compared with the identification management threshold;

[0121] If the average correlation is greater than or equal to the identification management threshold, a management stability signal is generated, no additional operations or adjustments are required, the existing security settings are maintained, and users do not need to perform subsequent verification;

[0122] If the average correlation is less than the identification management threshold, an intelligent verification signal is generated to initiate additional subsequent identity authentication processes, such as requiring secondary confirmation of the user's identity or sending a security alert to the relevant management system.

[0123] Once the smart verification signal is triggered, the direct unlocking permission allowed by the user's previous analysis will be suspended, and the user will be required to perform additional verification, requiring the user to complete multi-factor authentication (MFA), including but not limited to biometrics, PIN code, security questions, or verification through pairing with the user's device (such as smartphone app notification);

[0124] At the same time, the smart lock management system should record in detail all relevant activities of this incident, including the behavioral data that triggers verification, the user's response, and the system's operation. Subsequently, the security team will evaluate the handling effect of such incidents and their impact on the user experience, and adjust the trigger threshold and process of smart verification when necessary to optimize the balance between security and convenience;

[0125] The smart lock management system can effectively handle smart verification events triggered by low correlation while ensuring high security, ensuring that all user operations are fully verified, maintaining security and user trust.

[0126] It should be noted that the setting of the recognition management threshold can be determined according to specific scenarios and requirements, and is usually adjusted and optimized based on factors such as historical data and real-time data.

[0127] The present invention effectively responds to changes in different environments and user approaching speeds by intelligently adjusting the acquisition frequency and range of the intelligent lock sensor, realizes the optimization of energy efficiency and the high efficiency of data acquisition. By preprocessing and screening the data collected by multiple sensors, and performing multi-modal feature fusion on these data, the comprehensive feature vector can be accurately extracted, thereby improving the accuracy of preliminary user behavior recognition. This process reduces the possibility of misrecognition and improves the quality of data processing.

[0128] In terms of security verification, the present invention dynamically evaluates the security status of the intelligent lock by analyzing the accurate recognition information, so as to decide whether to omit subsequent verification steps. When the security status is abnormal, it can automatically trigger an additional verification process to respond to potential security threats in a timely manner. This flexible security strategy not only enhances the intelligent lock's ability to handle abnormal states, but also optimizes the user experience, avoids unnecessary operation delays, and improves the convenience and reliability of intelligent lock management.

[0129] Embodiment 2: This embodiment is a system embodiment of Embodiment 1, used to implement the intelligent lock management method based on behavior recognition introduced in Embodiment 1, as Figure 2 shown, specifically including:

[0130] A data acquisition module, which is used to configure sensor devices for the intelligent lock and obtain the sensor acquisition data, and the sensor intelligently adjusts the acquisition frequency and range according to the environment and the user approaching speed;

[0131] A behavior recognition module, which is used to recognize the user's behavior during the process from far to near, preprocess the data collected by the sensor during each process from far to near, screen the features extracted by different sensors, and then perform multi-modal data feature fusion, and use a classifier to perform preliminary user behavior recognition on the comprehensive feature vector obtained after feature fusion;

[0132] A security analysis module, which is used to perform security verification analysis on the preliminary user behavior recognition process, obtain the accurate recognition information generated during the recognition process, and determine the security status of the intelligent lock management process;

[0133] A management module, if the security status of the intelligent lock management process is stable, generates a management stable signal and does not perform subsequent verification; if the security status of the intelligent lock management process is abnormal, generates an intelligent verification signal and starts the subsequent identity verification process for management.

[0134] The above formulas are all dimensionless and calculated numerically. Specific dimension removal can be achieved by various means such as standardization, which will not be elaborated here. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0135] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, an ATA hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state ATA hard disk.

[0136] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0137] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0138] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. The smart lock management method based on behavior recognition is characterized by: The steps include: Configure sensor devices for smart locks and obtain sensor data. The sensor intelligently adjusts the collection frequency and range according to the environment and user approach speed; Behavior recognition is performed on the process of users moving from far to near. The data collected by sensors in each process is pre-processed. After filtering the features extracted by different sensors, multi-modal data features are fused. After feature fusion, the comprehensive feature vector is obtained and the classifier is used to perform preliminary user behavior recognition. If the preliminary user behavior recognition is to skip the subsequent verification steps, the preliminary user behavior recognition process is subjected to security verification analysis to obtain accurate recognition information generated during the recognition process and determine the security status of the smart lock management process; If the security status of the smart lock management process is stable, a management stability signal is generated and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated and the subsequent identity authentication process is started for management; The data collected by the sensors in each process from far to near are preprocessed. The specific process is as follows: When the user enters the long-distance sensing range of the smart lock, the infrared sensor and camera are activated, and the camera captures image data and motion trajectory; Add microphone sensors to collect auditory data at medium distances, and use all sensors to collect data at close distances; Preprocess the collected sensor data, including image denoising and feature extraction; Use Gaussian filter to denoise the image data collected by the camera and depth camera; According to the light intensity and noise level of the environmental conditions, the filter parameters of the Gaussian filter are adjusted to perform adaptive filtering; Use convolutional neural networks to extract features from image and depth data, and use autoencoders to extract features from speech and motion data; If the preliminary user behavior recognition is to skip the subsequent verification steps, the preliminary user behavior recognition process is subjected to security verification analysis to obtain accurate recognition information generated during the recognition process and determine the security status of the smart lock management process. The specific process is as follows: Set the confidence level. If the confidence level of the prediction result of the classifier exceeds the preset confidence level threshold, the preliminary user behavior recognition will skip the subsequent verification steps. Acquire accurate identification information generated during the identification process, including behavior pattern information and identification stability information; The behavior pattern information includes the behavior recognition accuracy index, and the recognition stability information includes the composite stability index; The obtained behavior recognition accuracy index and composite stability index are subjected to grey correlation analysis; The obtained behavior recognition accuracy index and composite stability index are subjected to grey correlation analysis. The specific process is as follows: The composite stability index is set as the reference series, and the behavior recognition accuracy index is set as the comparison series; The correlation coefficient was calculated using grey correlation analysis, and the correlation coefficient was averaged to obtain the average correlation degree in the entire analysis period; The composite stability index is obtained as follows: Obtain the measured values ​​of each environmental parameter during the behavior recognition process and calculate the environmental variability. The calculation expression is: In the formula, y i is the measured value of an environmental parameter at the i-th identification, is the average of these measurements; Collect user behavior data when interacting with the smart lock, classify the user behavior data by behavior type, and calculate the relative frequency of each interaction method in all interactions. The calculation expression is: p j =n j / N, where n j is the number of occurrences of behavior type j, N is the total number of behaviors, and the composite stability index is calculated using the following expression: Where k is the number of behavior types; The behavior recognition accuracy index is obtained as follows: Collect user behavior data at different time points, and integrate the features extracted from different sources into a multi-dimensional behavior feature vector. t Represents the behavior characteristics at time point t, obtains the current time T, calculates the behavior attenuation value, and the calculation expression is: w(t) = e -2*(T-t) , where T is the current time, e is a constant with a value of 2.71, and the vector cosine similarity between the current behavior feature vector and the historical behavior feature vector is calculated. The calculation expression is: XW = (F T ·F t ) / (||F T ||||F t ||), where F T Represents the behavior feature vector at the current time point T, and calculates the historical behavior consistency value. The calculation expression is: Get the current environment data that occurs at the same time as the user behavior, construct the environment feature vector based on the current environment data, and calculate the vector cosine similarity between the current environment feature vector and the historical environment feature vector: HJ = (H T ·H t ) / (||H T ||||H t ||), where H T represents the behavior feature vector at the current time point T, H t Representing the behavior feature vector at time point t, obtain and calculate the behavior recognition accuracy index:

2. The smart lock management method based on behavior recognition according to claim 1 is characterized in that: Configure the sensor device for the smart lock and obtain the sensor collection data. The sensor intelligently adjusts the collection frequency and range according to the environment and the user's approach speed. The specific process is as follows: Sensors include cameras, microphones, accelerometers, gyroscopes, infrared sensors, and depth cameras; The sensor dynamically adjusts the acquisition frequency based on the user’s approach speed and distance; Depending on the ambient light and noise level, the sensor automatically adjusts acquisition parameters.

3. The smart lock management method based on behavior recognition according to claim 1 is characterized in that: After screening the features extracted by different sensors, the multimodal data features are fused, and the comprehensive feature vector obtained after feature fusion is used by the classifier to perform preliminary user behavior recognition. The specific process is as follows: Through the Bayesian theorem, the likelihood probability of corresponding features in different categories is obtained; Get the prior probability of the category and the marginal probability of the feature, and calculate the posterior probability of the category given the feature; The features that are greater than the set posterior probability threshold are fused into a comprehensive feature vector; Use the classifier to perform preliminary user behavior identification, select a machine learning model, and input the fused comprehensive feature vector into the selected machine learning model; The classifier comprehensively evaluates the comprehensive feature vector and predicts the preliminary user behavior recognition results.

4. The smart lock management method based on behavior recognition according to claim 3 is characterized in that: If the security status of the smart lock management process is stable, a management stability signal is generated and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated and the subsequent identity authentication process is started for management. The specific steps are as follows: The average correlation was compared to the identification management threshold; If the average correlation is greater than or equal to the identification management threshold, a management stability signal is generated and no subsequent verification management measures are taken; If the average correlation is less than the identification management threshold, an intelligent verification signal is generated to start the subsequent identity authentication process management.

5. A smart lock management system based on behavior recognition, used to implement the smart lock management method based on behavior recognition as described in any one of claims 1 to 4, characterized in that: include: The data acquisition module is used to configure the sensor device for the smart lock and obtain the sensor collection data. The sensor intelligently adjusts the collection frequency and range according to the environment and the user's approach speed; The behavior recognition module is used to recognize the behavior of users in the process of moving from far to near. The data collected by sensors in each process of moving from far to near is pre-processed, and the features extracted by different sensors are filtered, and then multi-modal data features are fused. The comprehensive feature vector obtained after feature fusion is used by a classifier to perform preliminary user behavior recognition. The security analysis module is used to perform security verification analysis on the preliminary user behavior identification process, obtain accurate identification information generated during the identification process, and determine the security status of the smart lock management process; The management module generates a management stability signal if the security status of the smart lock management process is stable, and no subsequent verification is performed; if the security status of the smart lock management process is abnormal, a smart verification signal is generated to start the subsequent identity authentication process for management.

Images