Abnormal shielding detection method and system for intelligent door lock, terminal and storage medium

Abstract

The invention discloses an abnormal shielding detection method and system for an intelligent door lock, a terminal and a storage medium. The method comprises the following steps: acquiring image data of a user, analyzing and processing the image data to obtain a face image, and identifying and processing the face image to obtain a face shielding result; performing shielding evaluation on the face image according to the face shielding result to obtain shielding degree information, and performing distinguishing processing on behaviors of the user according to the shielding degree information to obtain a behavior distinguishing result; and if the behavior distinguishing result is an abnormal behavior, generating early warning information according to the behavior distinguishing result, and performing early warning according to the early warning information. According to the invention, the recognition capability of the intelligent door lock on the intentional shielding behavior is improved, the safety is enhanced, and the corresponding measures can be taken quickly when the abnormal behavior occurs, so that the risks of misrecognition and illegal invasion caused by face shielding are reduced.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, system, terminal and computer-readable storage medium for detecting abnormal occlusion of a smart door lock. Background Art

[0002] With the development of technology and the improvement of people's living standards, the popularity of smart door locks is getting higher and higher. At present, smart door locks generally use face recognition technology as a means of security authentication.

[0003] However, the existing face recognition system of smart door locks may not be able to effectively identify the user's identity when the user's face is partially blocked by objects such as hands, hats, scarves, etc., resulting in security vulnerabilities. It is also unable to distinguish between the user's normal wear (for example, glasses, masks, etc.) and objects deliberately used to cover the face, making it difficult for the system to balance security and convenience; and when abnormal behaviors such as face blocking are detected, preventive measures cannot be taken in time.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a method, system, terminal and storage medium for detecting abnormal obstruction of a smart door lock, aiming to solve the problem that the prior art cannot effectively distinguish objects that obstruct the user's face, resulting in difficulty in balancing the security and convenience of the system, and cannot take preventive measures in time when abnormal behavior is detected.

[0006] To achieve the above object, the present invention provides a method for detecting abnormal obstruction of a smart door lock, the method comprising the following steps:

[0007] Acquire image data of the user, analyze and process the image data to obtain a face image, and perform recognition processing on the face image to obtain a face occlusion result;

[0008] Performing an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and distinguishing the user's behavior according to the occlusion degree information to obtain a behavior distinction result;

[0009] If the behavior distinction result is an abnormal behavior, warning information is generated according to the behavior distinction result, and a warning is issued according to the warning information.

[0010] Optionally, the abnormal occlusion detection method for the smart door lock, wherein the acquiring of the image data of the user and analyzing and processing the image data to obtain the face image specifically includes:

[0011] When it is detected that the current state of the user meets the preset requirements, the image data of the user is collected, wherein the preset requirements include a preset distance and a preset position;

[0012] Preprocessing the image data to obtain target image data, and performing face detection on the target image data to obtain a detection result, wherein the preprocessing includes scaling and normalization;

[0013] If there is face information in the detection result, the target image data is segmented according to the detection result to obtain a face image.

[0014] Optionally, the abnormal occlusion detection method for the smart door lock, wherein the recognition processing of the face image to obtain the face occlusion result specifically includes:

[0015] Extracting feature points from the face image to obtain a plurality of face feature points, and analyzing the spatial relationship between all the face feature points to obtain a target spatial relationship;

[0016] Analyze the facial texture of the facial image to obtain texture information, and determine whether the user's face is blocked according to the target spatial relationship and the texture information;

[0017] If the target spatial relationship is incomplete or the texture information is unclear, it is determined that the user's face is blocked, and a face blocking result is obtained.

[0018] Optionally, the abnormal occlusion detection method of the smart door lock, wherein the occlusion evaluation is performed on the face image according to the face occlusion result to obtain the occlusion degree information, specifically includes:

[0019] Generating a segmentation instruction according to the face occlusion result, and performing segmentation processing on the face image according to the segmentation instruction to obtain an unoccluded face area image and an occlusion object image;

[0020] Calculating the number of pixels of the facial region image and the occlusion image respectively to obtain a first number of pixels and a second number of pixels, and obtaining an occlusion area ratio according to the first number of pixels and the second number of pixels;

[0021] Performing a position analysis on the occlusion in the occlusion image to obtain occlusion position information, performing object classification on the occlusion image to obtain object type, and evaluating the occlusion degree of the face image based on the occlusion area ratio, the occlusion position information and the object type to obtain occlusion degree information.

[0022] Optionally, the abnormal obstruction detection method for the smart door lock, wherein the user's behavior is distinguished according to the obstruction degree information to obtain a behavior distinction result, specifically includes:

[0023] Acquire a training image set, and train the created network training model according to the training image set to obtain a behavior classification model, wherein the training image set includes multiple groups of training samples, and each group of training samples includes an occlusion image of an object and a corresponding occlusion behavior;

[0024] The occlusion degree information is input into the behavior classification model, and a behavior distinction result is output.

[0025] Optionally, the abnormal occlusion detection method for the smart door lock, wherein the step of acquiring a training image set and training a created network training model according to the training image set to obtain a behavior classification model, specifically includes:

[0026] Acquire a training image set, create a network training model, and input a group of training samples in the training image set into the network training model;

[0027] Extracting features from the occlusion image to obtain facial features and occlusion features, and analyzing the occlusion area ratio and occlusion position of the occlusion image according to the facial features and the occlusion features to obtain corresponding analysis results;

[0028] Performing behavior prediction on the occlusion image according to the analysis result to obtain a predicted occlusion behavior, calculating a loss value between the predicted occlusion behavior and the occlusion behavior according to a cross entropy loss function, and adjusting parameters of the network training model according to the loss value;

[0029] The next set of training samples is input into the network training model until the training status of the network training model meets the preset conditions, thereby obtaining a trained behavior classification model.

[0030] Optionally, the abnormal obstruction detection method of the smart door lock, wherein if the behavior distinction result is an abnormal behavior, generating warning information according to the behavior distinction result, and issuing a warning according to the warning information, specifically includes:

[0031] If the behavior distinction result is an abnormal behavior, generating voice reminder information according to the behavior distinction result, and broadcasting the voice reminder information to the user to obtain a broadcast result;

[0032] If the user is not detected to have removed the obstruction within a preset time according to the broadcast result, a corresponding warning message is generated, and a target user is reminded to process according to the warning message, and the target user is a user associated with the smart door lock.

[0033] Optionally, in the abnormal cover detection method of the smart door lock, the abnormal cover detection system of the smart door lock comprises:

[0034] If the behavior differentiation result is an abnormal behavior, generating warning information according to the behavior differentiation result, and issuing a warning according to the warning information, specifically includes:

[0035] If the behavior distinction result is an abnormal behavior, generating voice reminder information according to the behavior distinction result, and broadcasting the voice reminder information to the user to obtain a broadcast result;

[0036] If the user is not detected to have removed the obstruction within a preset time according to the broadcast result, a corresponding warning message is generated, and a target user is reminded to process according to the warning message, and the target user is a user associated with the smart door lock.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an abnormal obstruction detection program for a smart door lock stored in the memory and runnable on the processor, and when the abnormal obstruction detection program for a smart door lock is executed by the processor, the steps of the abnormal obstruction detection method for a smart door lock as described above are implemented.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an abnormal obstruction detection program for a smart door lock, and when the abnormal obstruction detection program for a smart door lock is executed by a processor, the steps of the abnormal obstruction detection method for a smart door lock as described above are implemented.

[0039] In the present invention, the image data of the user is obtained, the image data is analyzed and processed to obtain a face image, and the face image is recognized and processed to obtain a face occlusion result; the face image is evaluated for occlusion according to the face occlusion result to obtain occlusion degree information, and the user's behavior is distinguished and processed according to the occlusion degree information to obtain a behavior distinction result; if the behavior distinction result is an abnormal behavior, an early warning message is generated according to the behavior distinction result, and an early warning is issued according to the early warning message. The present invention not only improves the smart door lock's ability to identify intentional occlusion behavior and enhances security, but also can quickly take countermeasures when abnormal behavior occurs, reducing the risk of misidentification and illegal intrusion caused by face occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of a preferred embodiment of the abnormal blocking detection method of the intelligent door lock of the present invention;

[0041] Figure 2 It is a schematic diagram of the principle of a preferred embodiment of the abnormal blocking detection system of the intelligent door lock of the present invention;

[0042] Figure 3 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0045] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0046] The abnormal occlusion detection method of the smart door lock described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the abnormal occlusion detection method of the smart door lock includes the following steps:

[0047] Step S10: Acquire image data of the user, analyze and process the image data to obtain a face image, and perform recognition processing on the face image to obtain a face occlusion result.

[0048] The step S10 comprises:

[0049] Step S11: when it is detected that the current state of the user meets the preset requirements, collecting the image data of the user, wherein the preset requirements include a preset distance and a preset position;

[0050] Step S12, preprocessing the image data to obtain target image data, and performing face detection on the target image data to obtain a detection result, wherein the preprocessing includes scaling and normalization;

[0051] Step S13: if there is face information in the detection result, segment the target image data according to the detection result to obtain a face image;

[0052] Step S14, extracting feature points from the face image to obtain a plurality of facial feature points, and analyzing the spatial relationship between all the facial feature points to obtain a target spatial relationship;

[0053] Step S15, analyzing the facial texture of the facial image to obtain texture information, and determining whether the user's face is blocked according to the target spatial relationship and the texture information;

[0054] Step S16: If the target spatial relationship is incomplete or the texture information is unclear, it is determined that the user's face is blocked, and a face blocking result is obtained.

[0055] Specifically, when a user tries to unlock the smart door lock, the sensor component of the smart door lock detects the user approaching, that is, recognizes the user at a preset distance and a preset position, and then automatically activates the camera component to collect the facial image of the user to obtain corresponding image data, wherein the camera component uses a high-resolution camera that can adapt to different lighting conditions to ensure image quality, and the camera component can be fixed or adjustable to adapt to people of different heights, and the corresponding focal length is also adjustable to ensure that a clear image is captured within an appropriate distance; thereafter, the image data is preprocessed, that is, the image data is scaled and normalized to meet the input requirements of the subsequent model to obtain target image data, and face detection is performed on the target image data to obtain a detection result, specifically, the target image data is input into a trained deep learning model, and the corresponding detection result is output after the deep learning model is processed.

[0056] The training process of the deep learning model is as follows: first, a data set containing a large number of facial images needs to be collected and organized. This training set is used for subsequent model training. These facial images should cover different angles, lighting conditions, expressions and occlusion conditions, etc. Commonly used facial image data sets include LFW (Labeled Faces in the Wild), CelebA (CelebFaces Attributes), and FDDB (FaceDetection Data Set and Benchmark, a data set for evaluating face detection methods). After the data set is prepared, the facial images need to be annotated to mark the position and size of the face, usually represented by a bounding box; then, a suitable deep learning model architecture is selected, for example, CNN (Convolutional Neural Networks, convolutional neural networks); then, using the pre-trained model parameters as a starting point can accelerate the training process and improve the performance of the model. In order to improve the generalization ability of the model, the training set is enhanced, such as rotation, scaling, cropping and flipping, and then the input training set is input into the corresponding deep learning model, and the prediction result calculated by the deep learning model output includes a predicted bounding box and a predicted category probability. Then, the prediction result is compared with the corresponding true result to obtain a corresponding loss function, such as a positioning loss and a classification loss, and the gradient of the loss function with respect to the network parameters is calculated. According to the gradient, the learning rate, batch size, number of iterations and other parameters of the deep learning model are adjusted to obtain the best training effect; then, the performance of the trained deep learning model is evaluated on an independent validation set, and indicators such as accuracy, recall rate, F1 score and the like are used to measure the effect of the model; finally, according to the validation result, the model structure or training strategy is adjusted to further optimize the model performance.

[0057] If there is facial information in the detection result, the target image data is segmented according to the detection result to obtain a facial image, and feature points are extracted from the facial image to obtain multiple facial feature points, such as eye feature points, nose feature points, and mouth feature points; and the spatial relationship between all the facial feature points is analyzed to obtain a target spatial relationship, and the facial texture of the facial image is analyzed to obtain texture information, and whether the user's face is obscured is determined according to the target spatial relationship and the texture information; if the target spatial relationship is complete and the texture information is clear, it is determined that the user's face is not obscured; if the target spatial relationship is complete and the texture information is unclear, or if the target spatial relationship is incomplete and the texture information is clear, or if the target spatial relationship is incomplete and the texture information is clear, it is determined that the user's face is obscured, and a face occlusion result is obtained.

[0058] Step S20: performing an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and performing differentiation processing on the user's behavior according to the occlusion degree information to obtain a behavior differentiation result.

[0059] The step S20 comprises:

[0060] Step S21, generating a segmentation instruction according to the face occlusion result, and performing segmentation processing on the face image according to the segmentation instruction to obtain an unoccluded face area image and an occlusion object image;

[0061] Step S22, respectively calculating the number of pixels of the facial region image and the occlusion image to obtain a first number of pixels and a second number of pixels, and obtaining an occlusion area ratio according to the first number of pixels and the second number of pixels;

[0062] Step S23, performing position analysis on the occluder in the occluder image to obtain occluder position information, performing object classification on the occluder image to obtain object types, and evaluating the occlusion degree of the face image according to the occlusion area ratio, the occluder position information and the object type to obtain occlusion degree information;

[0063] Step S24, obtaining a training image set, and training the created network training model according to the training image set to obtain a behavior classification model, wherein the training image set includes multiple groups of training samples, and each group of training samples includes an occlusion image of an object and a corresponding occlusion behavior;

[0064] Step S25: input the occlusion degree information into the behavior classification model, and output the behavior classification result.

[0065] Specifically, after obtaining that the user's face is obstructed, it is necessary to determine whether the user's obstruction behavior is abnormal. First, it is necessary to segment the unobstructed facial area and the obstructed part of the user's facial image, that is, generate a segmentation instruction according to the face occlusion result, and segment the facial image according to the segmentation instruction to obtain an unobstructed facial area image and an obstructed image, wherein the facial area image is determined by using a facial detection algorithm to determine the boundary of the facial area, and the facial area is segmented from the entire facial image according to this boundary, and the obstructed image is identified by using image segmentation technology (for example, threshold segmentation, edge detection, and region growth, etc.) to separate the obstruction; after segmenting the facial area image and the obstructed image, it is necessary to calculate the area percentage of the obstruction. and the corresponding positions, the specific process is, respectively calculating the number of pixels of the facial area image and the occlusion image to obtain a first number of pixels and a second number of pixels, and calculating the area of ​​the first number of pixels and the second number of pixels according to a calculation formula to obtain the occlusion area ratio, wherein the calculation formula is occlusion area ratio percentage = (second number of pixels / first number of pixels) x 100%; performing position analysis on the occlusion of the occlusion image to obtain occlusion position information, and the purpose of analyzing the occlusion position is to subsequently determine whether the user has intentional occlusion behavior; performing object classification on the occlusion image to obtain object type, and evaluating the occlusion degree of the face image according to the occlusion area ratio, the occlusion position information and the object type to obtain occlusion degree information.

[0066] Afterwards, the user is distinguished whether the wearer is wearing the device normally or intentionally covering the device according to the occlusion degree information, and the behavior distinction result is output by inputting the occlusion degree information into the trained behavior classification model, wherein the training process of the behavior classification model is to obtain a training image set, wherein the training image set includes multiple groups of training samples, each group of the training samples includes an occlusion image of an object and a corresponding occlusion behavior; create a network training model, and input a group of training samples in the training image set into the network training model; perform feature extraction on the occlusion image to obtain facial features and occlusion features, and analyze the occlusion area ratio and occlusion position of the occlusion image according to the facial features and the occlusion features to obtain corresponding analysis results; perform behavior prediction on the occlusion image according to the analysis results to obtain predicted occlusion behavior, and calculate the predicted occlusion behavior according to the cross entropy loss function. Measure the error between the occlusion behavior and the occlusion behavior, that is, the loss value; and adjust the parameters of the network training model according to the loss value using the corresponding optimization algorithm (for example, Adam optimizer or SGD); input the next set of training samples into the network training model, and repeat the above process, which will not be repeated here, until the training status of the network training model meets the preset conditions, and the preset conditions include that the loss value meets the preset requirements or the number of training times reaches the preset number of times, wherein the preset requirements can be determined according to the accuracy of the fuzzy image type recognition model, which will not be described in detail here, and the preset number of times can be the maximum number of training times for the fuzzy type training network model, for example, 2000 times, etc., to obtain a trained behavior classification model; in addition, in an embodiment of the present invention, the behavior classification model can also be optimized according to the user's usage habits, so as to better distinguish the user's behavior.

[0067] Step S30: If the behavior distinction result is abnormal behavior, generate warning information according to the behavior distinction result, and issue a warning according to the warning information.

[0068] The step S30 comprises:

[0069] Step S31: if the behavior distinction result is abnormal behavior, generate voice reminder information according to the behavior distinction result, and broadcast the voice reminder information to the user to obtain a broadcast result;

[0070] Step S32: If the user is not detected to have removed the obstruction within a preset time according to the broadcast result, a corresponding warning message is generated, and a target user is reminded to process according to the warning message, and the target user is a user associated with the smart door lock.

[0071] Specifically, if the behavior distinction result is abnormal behavior, a voice reminder message is generated according to the behavior distinction result, for example, please remove the obstruction or obstruction is detected, please confirm the identity, etc., and the voice reminder message is broadcast to the user to obtain the broadcast result; after completing the broadcast, wait for the user to respond, if the user removes the corresponding obstruction after hearing the voice reminder message, that is, according to the broadcast result, it is detected that the user removes the obstruction within a preset time (for example, 5 seconds), then the identity of the user is re-identified; if the user does not remove the corresponding obstruction after hearing the voice reminder message, That is, if the user is not detected to have removed the obstruction within a preset time according to the broadcast result, or if obstruction is detected for multiple consecutive times, a preset early warning mechanism needs to be triggered, thereby effectively improving the home safety protection capability, generating corresponding early warning information according to the broadcast result, and reminding the target user to process according to the early warning information. The target user is the user associated with the smart door lock, wherein the early warning reminder method includes sound warning (for example, issuing a warning sound through the built-in speaker of the smart door lock), visual warning (for example, reminding on-site personnel through flashing LED lights) and notifying the target user through mobile phone applications, text messages, emails, etc.

[0072] Furthermore, while alerting the target user according to the early warning information, the camera component is automatically activated to start recording, so as to record possible illegal behavior or intrusion, and provide video evidence for subsequent investigations.

[0073] Furthermore, if Figure 2 As shown, based on the above-mentioned abnormal blocking detection method of the smart door lock, the present invention also provides an abnormal blocking detection system of the smart door lock, wherein the abnormal blocking detection system of the smart door lock includes:

[0074] The data processing module 51 is used to obtain the image data of the user, analyze and process the image data to obtain a face image, and perform recognition processing on the face image to obtain a face occlusion result;

[0075] An abnormality detection module 52 is used to perform an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and to distinguish the user's behavior according to the occlusion degree information to obtain a behavior distinction result;

[0076] The early warning reminder module 53 is used to generate early warning information according to the behavior distinction result if the behavior distinction result is abnormal behavior, and issue an early warning according to the early warning information.

[0077] The abnormal occlusion detection system for a smart door lock provided by the present invention not only improves the smart door lock's ability to recognize intentional occlusion behavior and enhances security, but also can quickly take countermeasures when abnormal behavior occurs, reducing the risk of misidentification and illegal intrusion due to facial occlusion.

[0078] Furthermore, if Figure 3 As shown, based on the abnormal obstruction detection method of the above-mentioned smart door lock, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 3 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0079] The memory 20 may be an internal storage unit of the terminal in some embodiments, such as a hard disk or memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores an abnormal occlusion detection program 40 of a smart door lock, and the abnormal occlusion detection program 40 of the smart door lock can be executed by the processor 10, thereby realizing the abnormal occlusion detection method of the smart door lock in the present application.

[0080] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the abnormal occlusion detection method of the smart door lock.

[0081] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0082] In one embodiment, when the processor 10 executes the abnormal blocking detection program 40 of the smart door lock in the memory 20, the following steps are implemented:

[0083] Acquire image data of the user, analyze and process the image data to obtain a face image, and perform recognition processing on the face image to obtain a face occlusion result;

[0084] Performing an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and distinguishing the user's behavior according to the occlusion degree information to obtain a behavior distinction result;

[0085] If the behavior distinction result is an abnormal behavior, warning information is generated according to the behavior distinction result, and a warning is issued according to the warning information.

[0086] The step of acquiring the image data of the user and analyzing and processing the image data to obtain the face image specifically includes:

[0087] When it is detected that the current state of the user meets the preset requirements, the image data of the user is collected, wherein the preset requirements include a preset distance and a preset position;

[0088] Preprocessing the image data to obtain target image data, and performing face detection on the target image data to obtain a detection result, wherein the preprocessing includes scaling and normalization;

[0089] If there is face information in the detection result, the target image data is segmented according to the detection result to obtain a face image.

[0090] The step of performing recognition processing on the face image to obtain a face occlusion result specifically includes:

[0091] Extracting feature points from the face image to obtain a plurality of face feature points, and analyzing the spatial relationship between all the face feature points to obtain a target spatial relationship;

[0092] Analyze the facial texture of the facial image to obtain texture information, and determine whether the user's face is blocked according to the target spatial relationship and the texture information;

[0093] If the target spatial relationship is incomplete or the texture information is unclear, it is determined that the user's face is blocked, and a face blocking result is obtained.

[0094] The step of performing occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information specifically includes:

[0095] Generating a segmentation instruction according to the face occlusion result, and performing segmentation processing on the face image according to the segmentation instruction to obtain an unoccluded face area image and an occlusion object image;

[0096] Calculating the number of pixels of the facial region image and the occlusion image respectively to obtain a first number of pixels and a second number of pixels, and obtaining an occlusion area ratio according to the first number of pixels and the second number of pixels;

[0097] Performing a position analysis on the occlusion in the occlusion image to obtain occlusion position information, performing object classification on the occlusion image to obtain object type, and evaluating the occlusion degree of the face image based on the occlusion area ratio, the occlusion position information and the object type to obtain occlusion degree information.

[0098] The performing of differentiation processing on the user's behavior according to the occlusion degree information to obtain a behavior differentiation result specifically includes:

[0099] Acquire a training image set, and train the created network training model according to the training image set to obtain a behavior classification model, wherein the training image set includes multiple groups of training samples, and each group of training samples includes an occlusion image of an object and a corresponding occlusion behavior;

[0100] The occlusion degree information is input into the behavior classification model, and a behavior distinction result is output.

[0101] The step of acquiring a training image set and training the created network training model according to the training image set to obtain a behavior classification model specifically includes:

[0102] Acquire a training image set, create a network training model, and input a group of training samples in the training image set into the network training model;

[0103] Extracting features from the occlusion image to obtain facial features and occlusion features, and analyzing the occlusion area ratio and occlusion position of the occlusion image according to the facial features and the occlusion features to obtain corresponding analysis results;

[0104] Performing behavior prediction on the occlusion image according to the analysis result to obtain a predicted occlusion behavior, calculating a loss value between the predicted occlusion behavior and the occlusion behavior according to a cross entropy loss function, and adjusting parameters of the network training model according to the loss value;

[0105] The next set of training samples is input into the network training model until the training status of the network training model meets the preset conditions, thereby obtaining a trained behavior classification model.

[0106] If the behavior differentiation result is an abnormal behavior, generating warning information according to the behavior differentiation result, and issuing a warning according to the warning information, specifically includes:

[0107] If the behavior distinction result is an abnormal behavior, generating voice reminder information according to the behavior distinction result, and broadcasting the voice reminder information to the user to obtain a broadcast result;

[0108] If the user is not detected to have removed the obstruction within a preset time according to the broadcast result, a corresponding warning message is generated, and a target user is reminded to process according to the warning message, and the target user is a user associated with the smart door lock.

[0109] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an abnormal obstruction detection program for a smart door lock, and when the abnormal obstruction detection program for a smart door lock is executed by a processor, the steps of the abnormal obstruction detection method for a smart door lock as described above are implemented.

[0110] In summary, the present invention provides an abnormal occlusion detection method, system, terminal and storage medium for a smart door lock, the method comprising: obtaining image data of a user, analyzing and processing the image data to obtain a face image, and identifying and processing the face image to obtain a face occlusion result; performing an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and distinguishing and processing the user's behavior according to the occlusion degree information to obtain a behavior distinction result; if the behavior distinction result is an abnormal behavior, generating warning information according to the behavior distinction result, and issuing a warning according to the warning information. The present invention not only improves the smart door lock's ability to identify intentional occlusion behavior and enhances security, but also can quickly take countermeasures when abnormal behavior occurs, reducing the risk of misidentification and illegal intrusion caused by face occlusion.

[0111] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0112] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0113] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting abnormal occlusion of a smart door lock, characterized in that: The abnormal occlusion detection method of the smart door lock includes: Acquire image data of the user, analyze and process the image data to obtain a face image, and perform recognition processing on the face image to obtain a face occlusion result; Performing an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and distinguishing the user's behavior according to the occlusion degree information to obtain a behavior distinction result; If the behavior distinction result is an abnormal behavior, warning information is generated according to the behavior distinction result, and a warning is issued according to the warning information.

2. The abnormal occlusion detection method of the smart door lock according to claim 1 is characterized in that: The acquiring of the image data of the user and analyzing and processing the image data to obtain the face image specifically includes: When it is detected that the current state of the user meets the preset requirements, the image data of the user is collected, wherein the preset requirements include a preset distance and a preset position; Preprocessing the image data to obtain target image data, and performing face detection on the target image data to obtain a detection result, wherein the preprocessing includes scaling and normalization; If there is face information in the detection result, the target image data is segmented according to the detection result to obtain a face image.

3. The abnormal occlusion detection method of the smart door lock according to claim 1 is characterized in that: The performing recognition processing on the face image to obtain a face occlusion result specifically includes: Extracting feature points from the face image to obtain a plurality of face feature points, and analyzing the spatial relationship between all the face feature points to obtain a target spatial relationship; Analyze the facial texture of the facial image to obtain texture information, and determine whether the user's face is blocked according to the target spatial relationship and the texture information; If the target spatial relationship is incomplete or the texture information is unclear, it is determined that the user's face is blocked, and a face blocking result is obtained.

4. The abnormal occlusion detection method of the smart door lock according to claim 1 is characterized in that: The step of performing occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information specifically includes: Generating a segmentation instruction according to the face occlusion result, and performing segmentation processing on the face image according to the segmentation instruction to obtain an unoccluded face area image and an occlusion object image; Calculating the number of pixels of the facial region image and the occlusion image respectively to obtain a first number of pixels and a second number of pixels, and obtaining an occlusion area ratio according to the first number of pixels and the second number of pixels; Performing a position analysis on the occlusion in the occlusion image to obtain occlusion position information, performing object classification on the occlusion image to obtain object type, and evaluating the occlusion degree of the face image based on the occlusion area ratio, the occlusion position information and the object type to obtain occlusion degree information.

5. The abnormal occlusion detection method of the smart door lock according to claim 1 is characterized in that: The distinguishing process of the user's behavior according to the occlusion degree information to obtain a behavior distinction result specifically includes: Acquire a training image set, and train the created network training model according to the training image set to obtain a behavior classification model, wherein the training image set includes multiple groups of training samples, and each group of training samples includes an occlusion image of an object and a corresponding occlusion behavior; The occlusion degree information is input into the behavior classification model, and a behavior distinction result is output.

6. The abnormal occlusion detection method of the smart door lock according to claim 5 is characterized in that: The acquiring of the training image set and training of the created network training model according to the training image set to obtain the behavior classification model specifically includes: Acquire a training image set, create a network training model, and input a group of training samples in the training image set into the network training model; Extracting features from the occlusion image to obtain facial features and occlusion features, and analyzing the occlusion area ratio and occlusion position of the occlusion image according to the facial features and the occlusion features to obtain corresponding analysis results; Performing behavior prediction on the occlusion image according to the analysis result to obtain a predicted occlusion behavior, calculating a loss value between the predicted occlusion behavior and the occlusion behavior according to a cross entropy loss function, and adjusting parameters of the network training model according to the loss value; The next set of training samples is input into the network training model until the training status of the network training model meets the preset conditions, thereby obtaining a trained behavior classification model.

7. The abnormal occlusion detection method of the smart door lock according to claim 1 is characterized in that: If the behavior differentiation result is an abnormal behavior, generating warning information according to the behavior differentiation result, and issuing a warning according to the warning information, specifically includes: If the behavior distinction result is an abnormal behavior, generating voice reminder information according to the behavior distinction result, and broadcasting the voice reminder information to the user to obtain a broadcast result; If the user is not detected to have removed the obstruction within a preset time according to the broadcast result, a corresponding warning message is generated, and a target user is reminded to process according to the warning message, and the target user is a user associated with the smart door lock.

8. An abnormal occlusion detection system for a smart door lock, characterized in that: The abnormal occlusion detection system of the intelligent door lock includes: A data processing module is used to obtain image data of the user, analyze and process the image data to obtain a face image, and perform recognition processing on the face image to obtain a face occlusion result; an abnormality detection module, used to perform an occlusion evaluation on the face image according to the face occlusion result to obtain occlusion degree information, and to distinguish the user's behavior according to the occlusion degree information to obtain a behavior distinction result; The early warning reminder module is used to generate early warning information according to the behavior distinction result if the behavior distinction result is abnormal behavior, and to issue an early warning according to the early warning information.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and an abnormal obstruction detection program for a smart door lock stored in the memory and executable on the processor. When the abnormal obstruction detection program for a smart door lock is executed by the processor, the steps of the abnormal obstruction detection method for a smart door lock as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an abnormal obstruction detection program for a smart door lock, and when the abnormal obstruction detection program for a smart door lock is executed by a processor, the steps of the abnormal obstruction detection method for a smart door lock as described in any one of claims 1 to 7 are implemented.

Images