Abnormal pushing method of intelligent door lock, storage medium and electronic equipment

Abstract

The invention relates to the technical field of intelligent door locks, and discloses an abnormity pushing method of an intelligent door lock, a storage medium and electronic equipment, the method comprises the following steps: executing image feature detection, the image feature detection comprises determining whether human body features exist in acquired first video data; when the image feature detection meets a first image condition, motion trail detection is executed, the first image condition includes that no human body feature is detected in the first video data, and the motion trail detection includes determining whether an abnormal movement condition exists or not based on the motion trail data; and executing abnormal event pushing corresponding to the condition that the motion trail detection meets a first trail condition, the first trail condition comprising determining that an abnormal movement condition exists based on the motion trail data. The method can improve the accuracy of abnormal condition detection of the intelligent door lock.

Description

Technical Field

[0001] The present invention relates to the technical field of smart door locks, and in particular to an abnormal push method, storage medium and electronic device for a smart door lock. Background Art

[0002] At present, smart door locks generally use abnormal stay detection to determine whether there are abnormal situations such as abnormal personnel to ensure user safety. Among them, the abnormal stay detection of smart door locks refers to detecting whether there are one or more targets walking back and forth within the visual range of the smart door lock. For example, if someone is found to be wandering back and forth within a period of time within the detection range of the smart door lock, this abnormal situation can be reported to the user's mobile phone and other devices so that the user can know, thereby improving the security within the monitoring environment of the smart door lock.

[0003] At present, the abnormal stay detection method of smart door locks usually adopts the method of using artificial intelligence (AI) algorithm to detect whether there is a continuous appearance of human faces or human figures in the monitoring area outside the door to determine whether there is an abnormal stay of the target. However, in some special scenarios, if someone wears a mask, the light is weak or the standing position is not good, the face or human figure will not be detected, and the abnormal stay cannot be accurately detected. Summary of the invention

[0004] The embodiment of the present invention provides a method for abnormal push notification of a smart door lock, a storage medium and an electronic device.

[0005] In a first aspect, the present invention provides an abnormal push method for a smart door lock, comprising: performing image feature detection, wherein the image feature detection includes determining that there are no human features in the acquired first video data; corresponding to the image feature detection satisfying a first image condition, performing motion trajectory detection, wherein the first image condition includes: no human features are detected in the first video data, and the motion trajectory detection includes determining that there is an abnormal movement based on motion trajectory data; corresponding to the motion trajectory detection satisfying a first trajectory condition, performing abnormal event push, wherein the first trajectory condition includes: the motion trajectory data determines that there is an abnormal movement.

[0006] The above-mentioned abnormal push method of the smart door lock combines motion trajectory detection with image feature detection to obtain more accurate detection results of abnormal situations, and pushes abnormal events based on the more accurate detection results of abnormal situations, thereby improving the security of the smart door lock monitoring area.

[0007] In a possible implementation of the first aspect above, determining the presence of an abnormal movement situation based on the motion trajectory data includes: determining the presence of an abnormal moving target based on an average distance and / or variance of the motion trajectory data; and determining the presence of an abnormal movement situation corresponding to the presence of the abnormal moving target.

[0008] The above-mentioned abnormal push method of the smart door lock can obtain a more accurate analysis result based on the motion trajectory data by analyzing the average distance and / or variance of the motion trajectory data.

[0009] In a possible implementation of the first aspect above, determining the presence of abnormal movement based on motion trajectory data also includes: determining whether the abnormal moving target includes human features based on the second video data, wherein the second video data includes video data collected within a time range from the start time of the abnormal detection process to the current time; corresponding to the abnormal moving target including human features, determining the presence of abnormal movement.

[0010] The above-mentioned abnormal push method of the smart door lock can further determine whether there is an abnormality in the motion trajectory data based on the video data collected within the time range from the start of the abnormal detection process to the current moment, thereby improving the accuracy of detecting abnormalities through motion trajectory data.

[0011] In a possible implementation of the first aspect above, corresponding to the abnormal moving target including human features, determining the existence of an abnormal movement situation includes: corresponding to the determined abnormal moving target being an abnormal staying target, and based on the second video data, determining that the abnormal staying target includes human features, and determining the existence of an abnormal staying situation; corresponding to the determined abnormal moving target being a passing target, and based on the second video data, determining that the passing target includes human features, and determining that someone is passing.

[0012] The abnormal push method of the above-mentioned smart door lock can more accurately distinguish abnormal movement situations into abnormal stay situations and people passing by situations, thereby improving the accuracy of abnormal movement detection.

[0013] In a possible implementation of the first aspect above, the presence of an abnormal moving target is confirmed in the following manner: first motion trajectory data is acquired, and the first motion trajectory data is divided into motion trajectory data of a plurality of first detection cycles according to the acquisition time, wherein the first motion trajectory data is the motion trajectory data collected within a time range from the start time of the abnormal detection process to the start time of the motion trajectory detection; an average distance of the motion trajectory data of each first detection cycle is calculated; the average distances of the motion trajectory data corresponding to each first detection cycle all satisfy the first distance condition, and the presence of an abnormal lingering target is confirmed.

[0014] It can be understood that the first distance condition can be less than or equal to a preset distance threshold. The abnormal push method of the above-mentioned smart door lock can obtain a more accurate detection result of the abnormal staying target by analyzing the average distance in the first motion trajectory data.

[0015] In a possible implementation of the first aspect, the method further includes: when the variance of the average distance of the motion trajectory data corresponding to each first detection period satisfies a first variance condition, it is confirmed that there is an abnormal lingering target.

[0016] It can be understood that the first variance condition can be less than or equal to a preset variance threshold. The above-mentioned abnormal push method of the smart door lock can obtain a more accurate detection result of the abnormal staying target by analyzing the variance of the average distance in the first motion trajectory data.

[0017] In a possible implementation of the first aspect above, the method further includes: the average distance of the motion trajectory data corresponding to each first detection period satisfies the first distance condition, and the variance of the average distance satisfies the first variance condition, confirming the existence of an abnormal lingering target.

[0018] The above-mentioned abnormal push method of the smart door lock can obtain a more accurate detection result of the abnormal staying target by analyzing the average distance in the first motion trajectory data and the variance of the average distance.

[0019] In a possible implementation of the first aspect, the method further includes: when the average distance of the motion trajectory data corresponding to each first detection period does not satisfy the first distance condition, it is confirmed that there is a passing target.

[0020] The above-mentioned abnormal push method of the smart door lock can obtain a more accurate detection result of the passing target by analyzing the average distance in the first motion trajectory data.

[0021] In a possible implementation of the first aspect, the method further includes: when variances of average distances of motion trajectory data corresponding to each first detection period do not satisfy a first variance condition, it is confirmed that there is a passing target.

[0022] The above-mentioned abnormal push method of the smart door lock can obtain a more accurate detection result of the passing target by analyzing the variance of the average distance in the first motion trajectory data.

[0023] In a possible implementation of the first aspect above, the method further includes: the average distance of the motion trajectory data corresponding to each first detection period does not satisfy the first distance condition, and the variance of the average distance does not satisfy the first variance condition, confirming the presence of a passing target.

[0024] The above-mentioned abnormal push method of the smart door lock can obtain a more accurate detection result of the passing target by analyzing the average distance in the first motion trajectory data and the variance of the average distance.

[0025] In a possible implementation of the first aspect, when the motion trajectory detection satisfies the first trajectory condition, abnormal event push is performed, including: when the motion trajectory detection determines that there is an abnormal stay situation, an abnormal stay message is pushed; or when the motion trajectory detection determines that there is a person passing by, a person passing by message is pushed.

[0026] The above-mentioned abnormal push method of the smart door lock can push different abnormal situation messages according to the motion trajectory detection results.

[0027] In a possible implementation of the first aspect above, the method further includes: obtaining a preset time for an abnormality detection process; corresponding to detection of human features in the first video data, determining that the abnormality detection process has not ended based on the preset time, and performing image feature detection corresponding to the second video data.

[0028] The above-mentioned abnormal push method of the smart door lock can detect human features in the first video data, and when the abnormal detection process is not completed, continue to perform the next image feature detection, so as to obtain more video data and improve the user experience.

[0029] In a possible implementation of the first aspect above, corresponding to image feature detection satisfying a first image condition, motion trajectory detection is performed, including: corresponding to image feature detection satisfying the first image condition, and determining according to a preset time that the abnormality detection process has not ended, and performing motion trajectory detection.

[0030] The above-mentioned abnormal push method of the smart door lock can detect human features in the first video data, and perform motion trajectory detection when the abnormal detection process is not completed, thereby improving the accuracy of the abnormal detection process.

[0031] In a possible implementation of the first aspect above, image feature detection also includes: determining that the difference between each image frame in the acquired first video data and the background frame does not satisfy the difference condition; the first image condition also includes: no human features are detected in the first video data, and the difference between each image frame in the first video data and the background frame does not satisfy the difference condition.

[0032] It can be understood that the difference condition can be greater than a preset difference threshold. The above-mentioned abnormal push method of the smart door lock combines the detection of the difference between each image frame in the first video data and the background frame with the human feature detection, thereby improving the accuracy of image feature detection.

[0033] In the second aspect, an embodiment of the present invention provides an abnormal push device for a smart door lock, comprising one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the device executes any one of the abnormal push methods for a smart door lock provided in the first aspect and various possible implementations of the first aspect.

[0034] In a third aspect, an embodiment of the present invention provides a chip, the chip including a processor, the processor executing computer instructions stored in a computer storage medium, so that an electronic device implements any one of the abnormal push methods for smart door locks provided in various possible implementations of the first aspect mentioned above.

[0035] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device implements any one of the possible implementations of the first aspect to implement the abnormal push method of the smart door lock.

[0036] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, for executing the instructions stored in the memory to implement any one of the abnormal push methods of a smart door lock provided in various possible implementations of the first aspect above.

[0037] In a sixth aspect, an embodiment of the present invention provides a program product, which includes instructions, which when executed by an electronic device can enable the electronic device to implement any one of the smart door lock abnormal push methods provided by various possible implementations of the above-mentioned first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of a usage scenario of a smart door lock in an embodiment of the present application is shown;

[0039] Figure 2 A schematic diagram of a method flow for abnormal stay detection based on AI in an embodiment of the present application is shown;

[0040] Figure 3 A schematic diagram of another method flow for abnormal stay detection based on AI in an embodiment of the present application is shown;

[0041] Figure 4A A schematic diagram showing a process of a method for pushing abnormal conditions of a smart door lock in an embodiment of the present application;

[0042] Figure 4BA schematic diagram showing a detection process of an abnormal situation of a smart door lock in an embodiment of the present application;

[0043] Figure 5 A schematic flow chart of a method for determining an abnormal situation based on motion trajectory data in a motion trajectory detection phase in an embodiment of the present application is shown;

[0044] Figure 6 A schematic flow chart of a method for determining an abnormal situation based on motion trajectory data in a motion trajectory detection phase in an embodiment of the present application is shown;

[0045] Figure 7 A schematic flow chart of another method for determining an abnormal situation based on motion trajectory data in a motion trajectory detection phase in an embodiment of the present application is shown;

[0046] Figure 8 A schematic flow chart of another method for determining an abnormal situation based on motion trajectory data in a motion trajectory detection phase in an embodiment of the present application is shown;

[0047] Fig. 9 A schematic flow chart of another method for determining an abnormal situation based on motion trajectory data in a motion trajectory detection phase in an embodiment of the present application is shown;

[0048] Fig.10 A schematic diagram showing a flow chart of another method for pushing abnormal conditions of a smart door lock in an embodiment of the present application;

[0049] Fig.11 A functional flow diagram of an abnormality detection system for a smart door lock in an embodiment of the present application is shown;

[0050] Fig.12 A schematic diagram showing the relationship between functional modules of an abnormality detection system for a smart door lock in an embodiment of the present application is shown;

[0051] Fig.13 A schematic diagram of the hardware structure of a smart door lock in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0053] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0054] Figure 1 The usage scenario diagram of the smart door lock is shown in FIG. Figure 1 As shown, the smart door lock 200 may include an image acquisition unit 201 and a communication chip 202, and the smart door lock 200 may communicate with the terminal device 100 through the communication chip 202. Among them, the image acquisition unit 201 may include a camera (not shown in the figure) and a processor (not shown in the figure). After the camera captures the real-time picture in front of the door, the camera transmits the data of the real-time picture to the abnormal stay detection module. If the processor determines that the detection result of this time meets the abnormal stay through the push method of the abnormal situation of the smart door lock mentioned in this application, the processor will send the detection result of the abnormal stay to the cloud server through the communication chip 202, and then the cloud server will send the detection result of the abnormal stay to the terminal device 100 through information prompts and other methods, which is used to prompt the user that there may be abnormal stays and improve the safety of the detection environment.

[0055] It is understood that the smart door lock may include other components installed on the panel outside the door, such as a fingerprint sensor, a button, a microphone, a speaker, etc. The camera may include a face recognition camera, a cat's eye camera, etc. The smart door lock may also include a lock body, and components installed on the panel inside the door, etc. In the embodiments of the present application, the composition structure of the smart door lock is not specifically limited.

[0056] As described in the background technology, the current method for pushing abnormal situations of smart door locks can detect human faces and shapes through AI means to determine whether there is abnormal stay. For example, Figure 2 A histogram-based AI detection method is shown, which can be performed by a smart door lock. The method may include:

[0057] S201: Targets that appear in the monitoring area more than a first preset number of times are regarded as candidate targets.

[0058] For example, if the number of times that a target face or human figure is detected to appear in the monitoring area exceeds a first preset number within a preset time, these identified target faces or human figures can be used as candidate targets.

[0059] S202: Detect candidate targets by using a histogram matching tracking algorithm, and regard candidate targets that appear more than a second preset number of times as wandering targets.

[0060] For example, a histogram matching tracking algorithm can be used to track and detect candidate targets, that is, to precisely track candidate targets. Specifically, the number of candidate target appearances can be further updated by comparing the similarity between the histogram corresponding to the image taken when the candidate target is at the initial position and the histogram corresponding to the image taken when the candidate target is at each subsequent moving position. For example, if the similarity of the histograms of the corresponding areas of the two positions is less than 70%, the number of candidate target appearances can be reduced by 1, that is, the candidate target is not considered to appear in the monitoring area this time; conversely, if the similarity of the histograms of the corresponding areas of the two positions is greater than or equal to 70%, the number of appearances of the current candidate target can continue to be maintained. After the detection process is completed, the candidate target that has been presented more than the second preset number of times will be regarded as a wandering target, that is, an abnormal lingering object.

[0061] For example, Figure 3 A detection method based on an AI model is shown, which can be performed by a smart door lock, and includes:

[0062] S301: constructing a feature vector of the model based on the environmental background area and the motion trajectory of the abnormal wandering.

[0063] For example, the corresponding environmental background area and the motion track of abnormal wandering can be extracted based on multiple sample video data, and the obtained multiple environmental background areas and the motion track of abnormal wandering can be converted into feature vectors as sample data of the model. Wherein, the sample video data adopts human shooting objects.

[0064] S302: Construct a detection model through feature vectors.

[0065] For example, the constructed feature vector can be used as sample data of the model to train the model, so as to obtain a model for detecting abnormal wandering.

[0066] S303: Detect abnormal behavior based on the detection model and calculate the probability of abnormal situation occurring.

[0067] For example, if a face or a human figure is detected in the monitoring area, the motion trajectory of the detection target in the surrounding environment and the environmental background area can be input into the model to obtain the probability of abnormal situation.

[0068] However, for the above-mentioned AI-based detection method, in some special scenarios, such as when the detection target wears a mask, stands sideways or with its back to the camera, or in scenes with poor lighting, it will be difficult to recognize the face or human figure, and it will be impossible to obtain candidate targets, and then it will be impossible to obtain abnormal stay detection results through the AI ​​analysis algorithm.

[0069] In other abnormal stay detection methods, whether there is abnormal stay can be determined by detecting whether the length of the target's motion trajectory is greater than a threshold, without the need to accurately identify faces or human shapes. For example, if it is found that a target is moving back and forth within a period of time in the monitoring area, and the total length of the motion trajectory is greater than the threshold, this situation will be reported as an abnormal stay. However, in some special scenarios, if the target is deliberately stationary, the target's motion trajectory will not be obtained. Therefore, in order to improve the accuracy of detection and avoid the inability to obtain the target's motion trajectory, a longer detection time is often required to obtain the motion trajectory of more targets. For lightweight smart door locks, in order not to increase the computing burden of the processor and reduce power consumption, it is often impossible to set a longer detection time, which makes the detection accuracy low.

[0070] In view of this, the present invention proposes a method for pushing abnormal situations of smart door locks, which can comprehensively judge whether there is an abnormal situation outside the door based on the judgment result of whether there is a face or a human figure in the video and the movement trajectory of the target.

[0071] Specifically, for example, in some embodiments, after triggering the detection process of the smart door lock, the video detection stage can be entered first. In the video detection stage, the acquired video data can be detected by AI face and human figure recognition technology. If the acquired video data is detected to have human features, such as face or human figure, the human body detection stage is entered again before the end of the detection process (such as the duration of the detection process set by the user). If no human features are detected in the human body detection stage, the motion trajectory detection stage is entered to judge the abnormal situation based on the motion trajectory data. For example, in the motion trajectory detection stage, the motion trajectory data obtained by ToF ranging technology can be analyzed to determine whether there is an abnormal lingering target or a passing target passing through the monitoring area of ​​the smart door lock. If there is an abnormal lingering target or a passing target passing through the monitoring area, the detection process is terminated, and the situation of the abnormal lingering target or the passing target is reported to the user. If no abnormal lingering target or passing target is detected, the detection process is terminated.

[0072] In the above-mentioned method for pushing abnormal situations of smart door locks, AI detection is used as the main means, combined with ToF technology detection, to push more accurate detection results. Moreover, before determining the final detection result of this detection process, it will analyze whether there are faces or human figures in all previous video data at the current moment, so that more accurate detection results can be obtained based on face and human figure recognition technology. Moreover, the detection cycle can be stopped before the final detection result of the abnormal situation is determined to save power consumption of electronic equipment.

[0073] Combine the following FIG. 4A to FIG. 12 The technical solutions of some embodiments of the present invention are described.

[0074] Combine the following FIG. 4A to FIG. 12 The method for pushing abnormal situations of the smart door lock in the embodiment of the present application is explained.

[0075] It can be understood that in the embodiment of the present application, the abnormal conditions detected by the smart door lock may include abnormal stay and someone passing by.

[0076] It can be understood that all methods and steps in the embodiments of the present application are executed by the smart door lock.

[0077] Figure 4A The following is a method for notifying an abnormal situation of a smart door lock in an embodiment of the present application. Figure 4A As shown, a method for pushing abnormal conditions of a smart door lock may include the following steps:

[0078] S401, a triggering event of a detection process is detected.

[0079] It can be understood that the triggering event of the abnormal situation detection process in the embodiment of the present application can be as follows: Fig.11 As shown. The monitoring area outside the door can be detected by a passive infrared (PIR) detector 1120. If the PIR senses that a target is active in the monitoring area, the main control (microcontroller unit, MCU) 1110 will wake up the camera 1141 in the image acquisition module 1140 and the ToF sensor 1131 in the ToF ranging module 1130, and start video recording and ranging, that is, start the detection process. Among them, the main control 1110 can be set in a smart door lock. It can be understood that the detection process can also be started by other types of sensors or methods. In the embodiment of the present application, the triggering event of the detection process is not specifically limited. It can be understood that the duration of the detection process can be set according to the specific situation, for example, it can be set to 30s. In the embodiment of the present application, the duration of the detection process is not specifically limited.

[0080] Figure 4B A schematic diagram of a detection process of an abnormal situation of a smart door lock is shown in FIG. Figure 4B As shown, in the embodiment of the present application, after the external conditions trigger the detection process, each detection process will be divided into at least one video detection stage. At the beginning of each video detection stage, the abnormal situation in the monitoring area will be detected based on the AI ​​face and figure recognition technology or the mobile monitoring technology. If the face or figure cannot be detected by the AI ​​face and figure recognition technology in a certain video detection stage, and the mobile monitoring technology recognizes that the image frame and the background frame of the video have not changed, and the current detection process has not ended, it will enter the motion trajectory detection stage, and the detection results of the abnormal situation will be further obtained based on the ToF ranging technology.

[0081] It can be understood that the motion monitoring technology can also be used to identify whether there are changes between the current image frame of the video and the adjacent frames. The scope of application of the motion monitoring technology is not specifically limited in the embodiments of the present application.

[0082] It can be understood that the video detection stage can be set according to the specific scenario. For example, it can be set to be within 30 seconds of the detection process, with each 3 seconds being a video detection stage, that is, the face and human figure within the 3s time range and the changes of all image frames in the video within the 3s time range are detected once every 3 seconds. In the embodiment of the present application, the specific setting of the video detection stage is not specifically limited.

[0083] In the embodiment of the present application, a combination of AI face and shape detection technology and ToF ranging technology is used to determine abnormal conditions in the monitored area. When an abnormal condition is found, the smart door lock will push an abnormal condition message to the terminal device.

[0084] S402, confirm whether there are human features based on the video data of the current video detection stage. If yes, go to step S403, detect human features in the video data of the next video detection stage. If not, go to step S404, determine abnormal conditions based on the motion trajectory data of the motion trajectory detection stage.

[0085] In one possible implementation, this can be accomplished by Fig.11 The algorithm in the AI ​​recognition module 1150 detects whether there are human features, such as faces or human figures, in the video data in the current video detection stage. It is understood that the AI ​​algorithm can be a face recognition algorithm or a human figure recognition algorithm. In the embodiment of the present application, there is no specific limitation on the AI ​​algorithm used to detect faces or human figures. It is understood that if a face or a human figure is detected in the current video detection stage, it means that an abnormal target has appeared in the monitoring data of the current video detection stage.

[0086] In a possible implementation, step S403 may also be: confirm whether there are human features based on the video data of the current video detection stage, or whether the image frame has changed compared to the background image frame. If yes, go to step S403 to detect human features in the video data of the next video detection stage. If not, go to step S404 to determine the abnormal situation based on the motion trajectory data of the motion trajectory detection stage. That is, the video detection stage can be divided into a human feature detection stage and a motion detection stage. Among them, the motion detection stage is used to detect whether the image frame has changed compared to the background image frame in the video data of the current video detection stage.

[0087] In one possible implementation, this can be accomplished by Fig.11 The motion detection module 1160 in the video detection stage is used to determine whether all the image frames of the video in the current video detection stage are changed compared with the background image frames. It can be understood that the background image frame can be an image preset in the motion detection module 1160, for example, an image of the surrounding environment outside the door including certain valuable objects, or an image of the area outside the door to be detected can be set. Then, by comparing all the image frames of the video in the current video detection stage with the background image frames, it can be determined whether there are abnormal targets in the monitoring area outside the door. In the embodiment of the present application, the algorithm used to detect the image frames and the background image frames is not specifically limited.

[0088] It can be understood that in certain scenarios, such as when a suspicious person outside the door wears a mask and deliberately stands with his back to the camera, or when the ambient light is poor and it is impossible to take a clear picture, the motion detection module 1160 can be used to detect changes in the current image frame, and combined with the AI ​​recognition module 1150 to perform anomaly detection, abnormal situations can be judged more accurately.

[0089] Understandably, Fig.11 The main control 1110 is mainly used to control the ToF ranging module 1130 and the image acquisition module 1140 for ranging and recording. The processor of Maoyan 3516 is mainly used to determine the final detection result of abnormal situations based on AI face and figure recognition algorithm (including face and figure recognition), motion detection algorithm, and motion trajectory detection algorithm.

[0090] S403, detecting human features in the video data in the next video detection stage.

[0091] It can be understood that, after determining that there is a human face or a human figure in the video data in the current video detection stage, the detection in the next video detection stage can be continued.

[0092] In some possible implementations, when it is determined in step S402 that there is a face or a human figure in the video data in the current video detection stage, it is determined whether the detection process is finished, such as whether the set time of 30 seconds is completed, and after determining that the detection process is not finished, the detection of the next video detection stage is started, that is, the recording program of the camera does not stop, and it is necessary to continue to perform face and human figure recognition within this 3s time range every 3 seconds until the detection process is finished. It is understandable that if the detection process is not finished, the detection of the video detection stage can be repeated to make the actual detection time more consistent with the set duration of the detection process, thereby improving the user experience.

[0093] In some possible implementations, when it is determined in step S402 that there is a face or a human figure in the video data in the current video detection stage, and when it is determined that the detection process is finished, Fig.11 The processor of Maoyan 3516 sends the message of abnormal stay and all video data of the current detection process to the cloud, and sends the abnormal stay message and all video data of the current detection process to the user's terminal device through the cloud to remind the user that there is abnormal stay in the monitoring area outside the door, so that relevant early warning measures can be taken.

[0094] S404, determining an abnormal situation based on the motion trajectory data in the motion trajectory detection phase.

[0095] In a possible implementation, if it is determined in step S402 that no face or human figure is detected in the video data in the current video detection stage, the motion trajectory data of the target movement in the monitoring area in front of the door acquired by the ToF sensor 1131 can be analyzed instead.

[0096] It is understandable that in a scenario where human features cannot be obtained at a certain video detection stage, the distance data of the motion trajectory can be analyzed instead, so there is no need to analyze all video data of the set time of the detection process, which can improve the efficiency and accuracy of the detection process.

[0097] It can be understood that the ToF sensor 1131 uses the ToF ranging method to obtain the motion trajectory data of the target movement in the monitoring area in front of the door. The ToF ranging method belongs to a two-way ranging technology, which mainly uses the flight time of the signal between two asynchronous transceivers (Transceiver) to measure the distance between nodes. The combination of ToF ranging technology and AI face and shape recognition technology can further improve the detection accuracy of abnormal situations. It can be understood that the motion trajectory data may include distance characteristics within a set time, variance characteristics of the distance, moving range and moving speed of the target in front of the door, etc. In the embodiment of the present application, the specific parameter type in the motion trajectory data is not specifically limited.

[0098] In a possible implementation, if it is determined in step S402 that there is no face or human figure in the video data of the current video detection stage, it can be further determined whether this detection process is terminated. And when it is determined that this detection process is terminated, it is further determined whether there is a face or human figure in all the video data in this detection process. If it is determined that there is a face or human figure in all the video data in this detection process, it can be determined that there is an abnormal stay, and an abnormal stay message can be pushed. If it is determined that there is no face or human figure in all the video data in this detection process, no message is pushed. It can be understood that the abnormal situation in this detection process can be determined by analyzing whether there is a face or human figure in all the video data of the detection process.

[0099] In one possible implementation, if it is determined in step S402 that there is no face or human figure in the video data of the current video detection stage, it can be further determined that this detection process has not ended, and then go to step S404 to determine the abnormal situation based on the motion trajectory data of the motion trajectory detection stage, and determine the abnormal situation of the current detection process by further analyzing the motion trajectory data to obtain a more accurate detection result.

[0100] In one possible implementation, after analyzing the motion trajectory data, human features can also be detected in the video data at the current moment and before, and the final abnormal situation can be determined based on the analysis results of the motion trajectory data and the detection results of the human features in the video data at the current moment and before.

[0101] S405, end the detection process.

[0102] It can be understood that in step S404, by analyzing the motion trajectory data, abnormal conditions in the smart door lock monitoring area are determined, such as abnormal stay, someone passing by, or no abnormal conditions, etc., and the detection process can be terminated to save the power consumption of the smart door lock.

[0103] S406, push a message based on the abnormal situation.

[0104] It is understood that after the detection process is completed, corresponding messages can be pushed according to the determined abnormal conditions. For example, according to the abnormal stay condition, an abnormal stay message is pushed; according to the situation that someone passes by, a message that someone passes by is pushed; and according to the situation that there is no abnormal condition, no message is pushed.

[0105] Specifically, in step S404, a method for determining an abnormality based on the motion trajectory data in the motion trajectory detection phase, such as a method for determining an abnormality based on the average distance of the motion trajectory data, the variance of the average distance, etc., will be described below in conjunction with Figure 5 , Figure 6 , Figure 7 , Figure 8 and Fig. 9 Describe in detail.

[0106] The above-mentioned abnormal stay detection method uses ToF ranging technology in conjunction with AI face and shape detection technology to identify abnormal situations outside the door. It can not only determine the two abnormal situations including abnormal stay and someone passing by, but also improve the detection accuracy of abnormal situation detection using only a single detection technology. In addition, it can stop the detection process when the detection result is determined, thereby saving power consumption of the smart door lock and improving user experience.

[0107] The following will be combined Figure 5 , Figure 6 , Figure 7 , Figure 8 and Fig. 9 , a method for determining abnormal conditions based on motion trajectory data in the motion trajectory detection phase in step S404 is described in detail.

[0108] Figure 5 A method is shown in which, in the detection process, if no human face or human figure is detected in a certain video detection stage, the final detection result is obtained based on the motion trajectory data of the motion trajectory detection stage. In this method, if Figure 4A If it is determined in step S402 that no face or human figure is detected in the current video detection stage, the final detection result will be further obtained by analyzing the average distance of the motion trajectory data of the moving target in the monitoring area in front of the door obtained by the ToF sensor 1131.

[0109] The specific steps include:

[0110] S501, acquiring motion trajectory data from the start of the detection process to the start of detecting the motion trajectory data.

[0111] S502, determine whether the average distance between the start of the detection process and the start of the motion trajectory data detection is greater than the first distance. If yes, go to step S503, determine whether there are human features in the video data at time t1 and before. If not, go to step S506, determine whether there are human features in the video data at time t1 and before.

[0112] The time t1 is the time at which the video data is analyzed again after step S502.

[0113] It is understandable that after the information of the target cannot be obtained through the AI ​​recognition module 1150, the relationship between the average distance of the motion trajectory data from the start of the detection process to the start of the detection of the motion trajectory data and the first distance can be used to determine whether there is an abnormal situation. It is understandable that the duration from the start of the detection process to the start of the detection of the motion trajectory data can be divided into m parts, each of which is a first detection cycle, and m is a natural number greater than or equal to 1, and then the average distance within the time length of all first detection cycles is analyzed. It is understandable that the specific value of the first distance can be set according to the actual situation. In the embodiment of the present application, for the setting rules of the first detection cycle, the specific value of the first distance is not specifically limited.

[0114] In one possible implementation, after determining that the average distance in all first detection cycles is greater than the first distance, it means that the distance of the target's motion trajectory is large and the probability of an abnormal situation is small. The camera recording process can be stopped to save power consumption of the smart door lock.

[0115] S503, determine whether there are human features in the video data at time t1 and before. If yes, go to step S504 to determine whether there is a person passing by. If not, go to step S505 to determine whether there is no abnormal situation.

[0116] It is understandable that after determining that the average distance of the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data is greater than the first distance, the situation of this detection process can be further determined by further analyzing the video data at and before time t1.

[0117] S504, determining whether someone is passing by.

[0118] It can be understood that if the average distance in all first detection cycles is greater than the first distance, but a face or a human figure is detected in the video data at time t1 and before, it can be determined that there is a high possibility that someone has passed through this detection process.

[0119] S505, determining that there is no abnormal situation.

[0120] It can be understood that if the average distance in all first detection cycles is greater than the first distance, and no face or human figure is detected in the video data at time t1 and before, it can be said that there is no abnormality from the start of the detection process to the current moment.

[0121] S506, determine whether there are human features in the video data at time t1 and before. If yes, go to step S507 to determine whether there is abnormal stay. If not, go to step S505 to determine whether there is no abnormal situation.

[0122] It is understandable that after determining that the average distance of the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data is less than or equal to the first distance, the situation of this detection process can be further determined by further analyzing the video data at and before time t1.

[0123] It can be understood that if the average distance in all the first detection cycles is less than or equal to the first distance, but no face or human figure is detected in the video data at time t1 and before, it can be explained that all the moving targets in the first detection cycle may not be people, that is, there are no abnormalities from the beginning of the detection process to the current moment.

[0124] S507, determining whether there is an abnormal stay situation.

[0125] It can be understood that if the average distance in all first detection cycles is less than or equal to the first distance, and a face or a human figure is detected in the video data at time t1 and before, it means that there is a high possibility that someone has stayed abnormally in this detection process.

[0126] When the above method determines that no human face or human figure is detected in a certain video detection stage, the final detection result is further obtained by analyzing the average distance of the motion trajectory data, and before determining the final detection result, the human body features in the current and previous video data are analyzed again to determine a more accurate final detection result. In addition, after determining that the average distance in all first detection cycles is greater than the first distance, the camera recording process is stopped, which can save the power consumption of the smart door lock and improve the user experience.

[0127] Figure 6 A method is shown in which, after determining that no human face or human figure is detected in a certain video detection stage, and all image frames and background image frames of the video have not changed, the final detection result is obtained through motion trajectory data, and a message of the final abnormal situation is pushed. Figure 5 The difference between the above methods is that when it is determined that no face or human figure is detected in a certain video detection stage, and all image frames and background image frames of the video do not change, a more accurate abnormal situation is further determined by analyzing the variance of the average distance in the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data.

[0128] The specific steps include:

[0129] S601, acquiring motion trajectory data from the start of the detection process to the start of detecting the motion trajectory data.

[0130] S602, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the variance threshold. If yes, go to step S603 to determine whether there are human features in the video data at time t2 and before. If not, go to step S606 to determine whether there are human features in the video data at time t2 and before. Among them, time t2 is the time when the video data is analyzed again after step S602.

[0131] It can be understood that, similar to step S502, after the target information cannot be obtained through the AI ​​recognition module 1150 or the motion detection module 1160, the relationship between the variance of the average distance of the motion trajectory data from the start of the detection process to the start of the detection of the motion trajectory data and the variance threshold can be analyzed to determine whether there is an abnormal situation. It can be understood that the time between the start of the detection process and the start of the detection of the motion trajectory data can be divided into n parts, each of which is a second detection cycle, n is a natural number greater than or equal to 1, and then the variance of the average distance within the time length of all second detection cycles is analyzed. It can be understood that the specific value of the variance threshold can be set according to the actual situation. In the embodiment of the present application, for the setting rules of the second detection cycle, the specific value of the variance threshold is not specifically limited.

[0132] In one possible implementation, after determining that the variance of the average distance in all second detection cycles is greater than the variance threshold, it means that the distance of the target's motion trajectory is large and the probability of an abnormal situation is small. The camera recording process can be stopped to save power consumption of the smart door lock.

[0133] In one possible implementation, in step S602, after determining that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data is greater than the variance threshold, it means that the distance of the target's motion trajectory is large and the probability of an abnormal situation is small. At this time, the camera's recording process can be stopped to save power consumption of the smart door lock.

[0134] S603, determine whether there are human features in the video data at time t2 and before. If yes, go to step S604 to determine whether there is a person passing by. If yes, go to step S605 to determine whether there is no abnormal situation.

[0135] It can be understood that if the variance of the average distance of all the motion trajectory data in the second detection period is greater than the variance threshold, the situation of this detection process can be further determined by further analyzing the video data at time t2 and before.

[0136] S604, determining whether a person is passing by.

[0137] It can be understood that if the variance of the average distance in all second detection cycles is greater than the variance threshold, but a face or a human figure is detected in the video data at time t2 and before, it means that there is a high possibility that someone has passed through this detection process.

[0138] S605, determining that there is no abnormal situation.

[0139] It can be understood that if the variance of the average distance in all the second detection cycles is greater than the variance threshold, but no face or human figure is detected in the video data at time t2 and before, it can be explained that all the moving targets in the second detection cycle may not be people, that is, there is no abnormality from the beginning of the detection process to the current moment.

[0140] S606, determine whether there are human features in the video data at time t2 and before. If yes, go to step S607 to determine whether there is abnormal stay. If yes, go to step S605 to determine whether there is no abnormal situation.

[0141] It can be understood that if the variance of the average distance of all motion trajectory data in the second detection period is less than or equal to the variance threshold, the situation of this detection process can be further determined by further analyzing the video data at time t2 and before.

[0142] S607, determining whether there is an abnormal stay situation.

[0143] It can be understood that if the variance of the average distance in all second detection cycles is less than or equal to the variance threshold, and a face or human figure is detected in the video data at time t2 and before, it means that there is a high possibility that someone has stayed abnormally in this detection process.

[0144] When the above method determines that no human face or human figure is detected in a certain video detection stage, the final detection result is further obtained by analyzing the variance of the average distance of the motion trajectory data, and before determining the final detection result, the human body features are analyzed again in the video data at the time of the second detection and in the previous video data, so as to determine a more accurate final detection result. In addition, after determining that the variance of the average distance in all the second detection cycles is greater than the variance threshold, the camera recording process is stopped, which can save the power consumption of the smart door lock and improve the user experience.

[0145] Figure 7 Another method is shown in which, when it is determined that no human face or human figure is detected in a certain video detection stage, the final detection result is obtained by analyzing the motion trajectory data. Figure 5The difference between the above methods is that when it is determined that no face or human figure is detected in a certain video detection stage, and the average distance in the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data is less than or equal to the first distance, a more accurate abnormal situation is further determined by analyzing the variance of the average distance in the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data.

[0146] The specific steps include:

[0147] S701, acquiring motion trajectory data from the start of the detection process to the start of detecting the motion trajectory data.

[0148] S702, determine whether the average distance between the start of the detection process and the start of the motion trajectory data detection is greater than the first distance. If yes, go to step S703, determine whether there are human features in the video data at time t3 and before. If not, go to step S706, determine whether there are human features in the video data at time t3 and before.

[0149] The time t3 is the time at which the video data is analyzed again after step S702.

[0150] S703, determine whether there are human features in the video data at time t3 and before. If yes, go to step S704 to determine whether there is a person passing by. If not, go to step S705 to determine whether there is no abnormal situation.

[0151] S704, determining whether someone is passing by.

[0152] S705, determine that there is no abnormal situation.

[0153] It can be understood that steps S701 to S705 correspond to steps S501 to S505 and are therefore not described again.

[0154] S706, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the variance threshold. If yes, go to step S707 to determine whether there are human features in the video data at time t4 and before. If not, go to step S710 to determine whether there are human features in the video data at time t4 and before.

[0155] The time t4 is the time at which the video data is analyzed again after step S706.

[0156] It can be understood that after determining that the average distance of the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data is less than or equal to the first distance, the variance characteristics of the average distance can be further analyzed to judge the movement state of the target appearing in front of the door. If the variance of the average distance is greater than the variance threshold, then it means that there is a high probability that someone has passed by. If the variance of the average distance is less than or equal to the variance threshold, then it means that there is a high probability that there is an abnormal stay.

[0157] In one possible implementation, after determining that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of detecting the motion trajectory data is greater than the variance threshold, it means that the distance of the target's motion trajectory is large and the probability of an abnormal situation is small. The camera's recording process can be stopped to save power consumption of the smart door lock.

[0158] S707, determine whether there are human features in the video data at time t4 and before. If yes, go to step S708 to determine whether there is a person passing by. If not, go to step S709 to determine whether there is no abnormal situation.

[0159] It can be understood that if the variance of the average distance is determined to be greater than the variance threshold, it means that the moving range of the target in front of the door is large, then the video data at time t4 and before can be further analyzed. If the video data at time t4 and before does not recognize a face or a human figure, then it means that there is a high probability that there is no abnormality.

[0160] S708, determining whether someone is passing by.

[0161] It can be understood that if the variance of the average distance is greater than the variance threshold, and the video data at time t4 and before recognizes a face or a human figure, it means that there is a high probability that someone has passed by.

[0162] S709, determine that there is no abnormal situation.

[0163] It can be understood that if the variance of the average distance is determined to be less than or equal to the variance threshold, it means that the moving range of the target in front of the door is small, then the video data at time t4 and before can be further analyzed. If the video data at time t4 and before does not recognize a face or a human figure, then it means that the probability of an abnormal situation where there is no person is high.

[0164] S710, determine whether there are human features in the video data at time t4 and before. If yes, go to step S711 to determine whether there is abnormal stay. If not, go to step S709 to determine whether there is no abnormal situation.

[0165] It can be understood that if the variance of the average distance is determined to be less than or equal to the variance threshold, it means that the moving range of the target in front of the door is small, then the video data at time t4 and before can be further analyzed. If the video data at time t4 and before does not recognize a face or a human figure, then it means that there is a high probability that there is no abnormal situation.

[0166] S711, determining that there is an abnormal stay situation.

[0167] It can be understood that if the variance of the average distance is less than or equal to the variance threshold, and the video data at time t4 and before recognizes a face or a human figure, it means that there is a high probability that someone is staying abnormally.

[0168] When the above method determines that no human face or human figure is detected in a certain video detection stage, the final detection result is further obtained by analyzing the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data and the variance of the average distance, thereby improving the detection accuracy. Moreover, before determining the final detection result, a more accurate final detection result is further determined by re-detecting the human body features in the video data moment and the previous video data. Moreover, after determining that the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the first distance, or after determining that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the variance threshold, the recording process of the camera is stopped, which can also save the power consumption of the smart door lock and improve the user experience.

[0169] Figure 8 A method is shown in which, after determining that no human face or human figure is detected in a certain video detection stage, and all image frames and background image frames of the video have not changed, the final detection result is obtained through motion trajectory data, and a message of the final abnormal situation is pushed. Figure 6 The difference between the above methods is that after determining that no face or human figure is detected in a certain video detection stage, and the variance of the average distance in the motion trajectory data is less than or equal to the variance threshold, a more accurate abnormal situation is further determined by starting to detect the motion trajectory data after the moment of motion trajectory data detection.

[0170] The specific steps include:

[0171] S801, acquiring motion trajectory data from the start of the detection process to the start of detecting the motion trajectory data.

[0172] S802, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the variance threshold. If yes, go to step S803 to determine whether there are human features in the video data at time t2 and before. If not, go to step S806 to determine whether there are human features in the video data at time t2 and before.

[0173] The time t5 is the time at which the video data is analyzed again after step S802.

[0174] S803, determine whether there are human features in the video data at time t2 and before. If yes, go to step S804 to determine whether there is a person passing by. If yes, go to step S805 to determine whether there is no abnormal situation.

[0175] S804, determining whether someone is passing by.

[0176] S805, determine that there is no abnormal situation.

[0177] It can be understood that steps S801 to S805 correspond to steps S601 to S605, and time t5 corresponds to time t2, which will not be repeated here.

[0178] In a possible implementation, if it is determined in step S802 that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the variance threshold, it means that the movement range of the target outside the door is large during the time between the start of the detection process and the start of the motion trajectory data detection. At this time, the recording can be stopped to save power consumption of the smart door lock.

[0179] S806, acquiring the motion trajectory data after the moment when the motion trajectory data detection starts.

[0180] It can be understood that if it is determined in step S802 that the variance of the average distance of the motion trajectory data between the start of the detection process and the moment when the motion trajectory data is started to be detected is less than or equal to the variance threshold, then the abnormal situation can be further determined based on the average distance of the motion trajectory data after the moment when the motion trajectory data is started to be detected, thereby further improving the accuracy of abnormal situation detection.

[0181] S807, determine whether the average distance of the motion trajectory data after the start of motion trajectory data detection is greater than the first distance. If yes, go to step S808, determine whether there are human features in the video data at time t6 and before. Alternatively, go to step S811, determine whether the detection process is completed.

[0182] The time t6 is the time when the video data is analyzed again after step S807.

[0183] It can be understood that the detection time after the start of the motion trajectory data detection can be divided into p parts, p is a natural number, each part is a third detection cycle, and the setting rules of the third detection cycle and the specific value of the first distance are not specifically limited. The detection accuracy can be further improved by analyzing the average distance of the motion trajectory data after the start of the motion trajectory data detection.

[0184] It can be understood that the detection time after the start of detecting the motion trajectory data can be any period of time between the start of detecting the motion trajectory data and the end of the current detection process. In the embodiment of the present application, the detection time after the start of detecting the motion trajectory data is not specifically limited.

[0185] In a possible implementation, if it is determined in step S807 that the average distance of the motion trajectory data after the start of the motion trajectory data detection is greater than the first distance, it means that the moving range of the target outside the door is large within the time period after the start of the motion trajectory data detection. At this time, the recording can be stopped to save power consumption of the smart door lock, and a more accurate detection result can be obtained by detecting whether there are human features in the video data at the time of the video data and before the third time.

[0186] S808, determine whether there are human features in the video data at time t6 and before. If yes, go to step S809 to determine whether there is abnormal stay. If not, go to step S810 to determine whether there is no abnormal situation.

[0187] In a possible implementation, if it is determined in step S807 that the average distance of the motion trajectory data of all the third detection cycles is less than or equal to the first distance, then it can be further determined whether the current detection process is finished. If it is confirmed that the current detection process is finished, a more accurate final detection result can be determined by further analyzing whether there is a face or a human figure in the video data at the time of the third detection and before. If it is confirmed that the current detection process has not ended, then it can continue to go to step S809 and continue to obtain the motion trajectory data after the moment of starting to detect the motion trajectory data, so as to obtain as much motion trajectory data as possible after the current moment to improve the detection accuracy.

[0188] S809, determining that there is an abnormal stay situation.

[0189] It can be understood that if it is determined in step S807 that the average distance of the motion trajectory data of all the third detection cycles is greater than the first distance, and it is determined that a face or a human figure is recognized in the video data at time t6 and before, then it can be determined that the probability of abnormal stay is high.

[0190] S810, determining that there is no abnormal situation.

[0191] It can be understood that if it is determined in step S807 that the average distance of the motion trajectory data of all the third detection cycles is greater than the first distance, and no face or human figure is recognized in the video data at time t6 and before, then it can be determined that there is a high probability that there is no abnormality.

[0192] S811, determine whether the detection process is finished. If yes, go to step S812, judge the video at time t7 and before. If no, go to step S806, obtain the motion track data after the start of motion track data detection.

[0193] Here, time t7 is the time when the video data is analyzed again after step S811.

[0194] In a possible implementation, if it is determined in step S807 that the average distance of all the motion trajectory data of the third detection period is less than or equal to the first distance, it can be further determined whether the current detection process is terminated. If it is confirmed that the current detection process is not terminated, it can continue to step S806 to obtain the motion trajectory data after the moment of starting to detect the motion trajectory data, so as to obtain as much motion trajectory data after the current moment as possible to improve the detection accuracy.

[0195] S812, determine whether there are human features in the video data at time t7 and before. If yes, go to step S813 to determine whether there is abnormal stay. If not, go to step S814 to determine whether there is no abnormal situation.

[0196] It is understandable that if it is confirmed in step S811 that the current detection process is finished, a more accurate final detection result can be determined by further analyzing whether there is a face or a human figure in the video data at the time of the third detection and before the video data.

[0197] S813, determining that there is an abnormal stay situation.

[0198] It can be understood that if it is determined that a human face or a human figure is recognized in the video data at time t7 and before, then it can be determined that there is a high probability that an abnormal stay situation exists.

[0199] S814, determine that there is no abnormal situation.

[0200] It can be understood that if it is determined that no human face or human figure is recognized in the video data at time t7 and before, then it can be determined that there is a high probability that there is no abnormal situation.

[0201] When the above method determines that no human face or human figure is detected in a certain video detection stage, the final detection result is further obtained by analyzing the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data, and the average distance of the motion trajectory data after the start of the detection of the motion trajectory data, and before determining the final detection result, the human body features are analyzed in the video data before and after the third detection of the video data, so as to determine a more accurate final detection result. In addition, after determining that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the variance threshold, or after determining that the average distance of the motion trajectory data after the start of the detection of the motion trajectory data is greater than the first distance, the recording process of the camera is stopped, which can also save the power consumption of the smart door lock and improve the user experience. In addition, after determining that the average distance of the motion trajectory data after the start of the detection of the motion trajectory data is less than or equal to the first distance, it will be further determined whether the current detection process is ended, and when it is determined that the detection is not ended, the motion trajectory data will continue to be acquired to obtain a more accurate detection result through more motion trajectory data.

[0202] Fig. 9 A method for obtaining the final detection result by using motion trajectory data, etc., when determining that no human face or human figure is detected in a certain video detection stage is shown. Figure 7 The difference between the methods is that after determining that no face or human figure is detected in a certain video detection stage, and the average distance in the motion trajectory data from the start of the detection process to the moment when the motion trajectory data is started to be detected is less than or equal to the first distance, and the variance of the average distance in the motion trajectory data from the start of the detection process to the moment when the motion trajectory data is started to be detected is less than or equal to the variance threshold, a more precise abnormal situation is further determined through the motion trajectory data after the moment when the motion trajectory data is started to be detected.

[0203] The specific steps include:

[0204] S901, acquiring motion trajectory data from the start of the detection process to the start of detecting the motion trajectory data.

[0205] S902, determine whether the average distance between the start of the detection process and the start of the motion trajectory data detection is greater than the first distance. If yes, go to step S903, determine whether there are human features in the video data at time t8 and before. If not, go to step S906, determine whether there are human features in the video data at time t8 and before.

[0206] The time t8 is the time at which the video data is analyzed again after step S902.

[0207] S903, determine whether there are human features in the video data at time t8 and before. If yes, go to step S904 to determine whether there is a person passing by. If not, go to step S905 to determine whether there is no abnormal situation.

[0208] S904, determining whether someone is passing by.

[0209] S905, determine that there is no abnormal situation.

[0210] S906, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the variance threshold. If yes, go to step S907 to determine whether there are human features in the video data at time t9 and before. If not, go to step S910 to obtain the motion trajectory data after the start of the motion trajectory data detection.

[0211] The time t9 is the time when the video data is analyzed again after step S906.

[0212] S907, determine whether there are human features in the video data at time t9 and before. If yes, go to step S908 to determine whether there is a person passing by. If not, go to step S909 to determine whether there is no abnormal situation.

[0213] S908, determining whether someone is passing by.

[0214] S909, determine that there is no abnormal situation.

[0215] It can be understood that steps S901 to S909 correspond to steps S701 to S709 and are identical, and are not described in detail herein.

[0216] S910, acquiring motion trajectory data after the moment of starting to detect the motion trajectory data.

[0217] S911, determine whether the average distance of the motion trajectory data after the start of motion trajectory data detection is greater than the first distance. If yes, go to step S912, determine whether there are human features in the video data at time t10 and before. Alternatively, go to step S915, determine whether the detection process is completed.

[0218] The time t10 is the time at which the video data is analyzed again after step S911.

[0219] S912, determine whether there are human features in the video data at time t10 and before. If yes, go to step S913 to determine whether there is abnormal stay. If not, go to step S914 to determine whether there is no abnormal situation.

[0220] S913, determining that there is an abnormal stay situation.

[0221] S914, determine that there is no abnormal situation.

[0222] S915, determine whether the detection process is finished. If yes, go to step S916, judge the video before time t11. If no, go to step S910, obtain the motion track data after the start of motion track data detection.

[0223] The time t11 is the time at which the video data is analyzed again after step S915.

[0224] S916, determine whether there are human features in the video data at time t11 and before. If yes, go to step S917 to determine whether there is abnormal stay. If not, go to step S918 to determine whether there is no abnormal situation.

[0225] S917, determine that there is an abnormal stay situation.

[0226] S918, determine that there is no abnormal situation.

[0227] It can be understood that steps S910 to S918 correspond to steps S806 to S814 and are identical, and are not described again here.

[0228] When the above method determines that no human face or human figure is detected in a certain video detection stage, the final detection result is further obtained by analyzing the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data, the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data, and the average distance of the motion trajectory data after the start of the detection of the motion trajectory data, and before determining the final detection result, the human body features are detected for the third time in the video data and before the video data, so as to determine a more accurate final detection result. In addition, after determining that the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the first distance, or after determining that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the variance threshold, or after determining that the average distance of the motion trajectory data after the start of the detection of the motion trajectory data is greater than the first distance, the recording process of the camera is stopped, which can also save the power consumption of the smart door lock. Moreover, after determining that the average distance of the motion trajectory data after the start of detecting the motion trajectory data is less than or equal to the first distance, it will be further determined whether the current detection process is ended, and if it is determined that the detection is not ended, the motion trajectory data will continue to be acquired to obtain a more accurate detection result through more motion trajectory data.

[0229] Fig.10Another method for pushing abnormal conditions of a smart door lock in an embodiment of the present application is shown. Fig.10 As shown, a method for pushing abnormal conditions of a smart door lock may include the following steps:

[0230] S1001, a triggering event of a detection process is detected.

[0231] S1002, confirming whether there are human features based on the video data in the current video detection stage.

[0232] It can be understood that step S1001 and step S1002 correspond to step S401 and step S402, and are not repeated here.

[0233] S1003, confirm whether the detection process is finished. If yes, go to step S1004, determine whether there is abnormal stay, and push abnormal stay message. If not, go to step S1002, confirm whether there is human body feature based on the video data of the current video detection stage.

[0234] In some possible implementations, when it is determined in step S1002 that there is a face or a human figure in the video data in the current video detection stage, it is determined whether the detection process is finished, and after it is determined that the detection process is not finished, the detection of the next video detection stage is started, that is, the video recording program of the camera does not stop until the detection process is finished. It is understandable that if the detection process is not finished, the detection of the video detection stage can be repeated to make the actual detection time more consistent with the set detection process duration, thereby improving the user experience.

[0235] S1004, determining that there is an abnormal stay situation, and pushing an abnormal stay message.

[0236] It is understandable that if the presence of human features is confirmed in a certain video detection stage and the detection process is completed, it can be determined that there is an abnormal stay and an abnormal stay message is pushed.

[0237] S1005, confirm whether the detection process is finished. If yes, go to step S1006, determine whether there is a human body feature in the video data at time t12 and before.

[0238] Here, time t12 is the time when the video data is analyzed again after step S1005.

[0239] In a possible implementation, if it is determined in step S1002 that there is no human face or human figure in the video data of the current video detection stage, it can be further determined whether this detection process is terminated. When it is determined that this detection process is terminated, it is further determined whether there is a human face or human figure in all the video data in this detection process, so as to determine the abnormal situation in this detection process.

[0240] S1006, determine whether there are human features in the video data at time t12 and before. If yes, go to step S1007, determine whether there is an abnormal stay, and push an abnormal stay message. If not, go to step S1008, determine whether there is no abnormal situation, and do not push any message.

[0241] S1007, determining that there is an abnormal stay situation, and pushing an abnormal stay message.

[0242] S1008: Determine that there is no abnormal situation and do not push any message.

[0243] S1009, acquiring the motion trajectory data from the start of the detection process to the start of detecting the motion trajectory data.

[0244] S1010, determine whether the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the first distance. If yes, go to step S1011 to determine whether there are human features in the video data at time t13 and before. If not, go to step S1014 to determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the detection of the motion trajectory data is greater than the variance threshold.

[0245] Here, time t13 is the time when the video data is analyzed again after step S1010.

[0246] S1011, determine whether there are human features in the video data at time t13 and before. If yes, go to step S1012, determine whether there is a person passing by, and push a message of someone passing by. If not, go to step S1013, determine whether there is no abnormal situation, and do not push any message.

[0247] S1012, determining that someone is passing by, and pushing a message indicating that someone is passing by.

[0248] S1013: Determine that there is no abnormal situation and do not push any message.

[0249] S1014, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the variance threshold. If yes, go to step S1015 to determine whether there are human features in the video data at time t14 and before. If not, go to step S1018 to obtain the motion trajectory data after the start of the motion trajectory data detection.

[0250] Here, time t14 is the time when the video data is analyzed again after step S1014.

[0251] S1015, determine whether there are human features in the video data at time t14 and before. If yes, go to step S1016, determine whether there is a person passing by, and push a message of someone passing by. If not, go to step S1017, determine whether there is no abnormal situation, and do not push any message.

[0252] S1016, determining that someone has passed by, and pushing a message indicating that someone has passed by.

[0253] S1017: Determine that there is no abnormal situation and do not push any message.

[0254] S1018, acquiring the motion trajectory data after the moment when the motion trajectory data detection starts.

[0255] S1019, determine whether the average distance of the motion trajectory data after the start of motion trajectory data detection is greater than the first distance. If yes, go to step S1020, determine whether there are human features in the video data at time t15 and before, determine whether there is an abnormal stay, and push an abnormal stay message. If no, go to step S1023, confirm whether the detection process is completed.

[0256] Here, time t15 is the time when the video data is analyzed again after step S1019.

[0257] S1020, determine whether there are human features in the video data at time t15 and before, determine whether there is an abnormal stay, and push an abnormal stay message. If yes, go to step S1021, determine whether there is an abnormal stay, and push an abnormal stay message. If not, go to step S1022, determine whether there is no abnormal situation, and do not push any message.

[0258] S1021, determining that there is an abnormal stay situation, and pushing an abnormal stay message.

[0259] S1022: Determine that there is no abnormal situation and do not push any message.

[0260] S1023, confirm whether the detection process is finished. If yes, go to step S1024, determine whether there are human features in the video data at time t16 and before. If no, go to step S1018, obtain the motion trajectory data after the moment of starting to detect the motion trajectory data.

[0261] Here, time t16 is the time when the video data is analyzed again after step S1023.

[0262] S1024, determine whether there are human features in the video data at time t16 and before. If yes, go to step S1025, determine whether there is an abnormal stay, and push an abnormal stay message. If not, go to step S1026, determine whether there is no abnormal situation, and do not push any message.

[0263] In a possible implementation, if it is determined in step S1016 that the average distance of the motion trajectory data after the moment of starting to detect the motion trajectory data is less than or equal to the first distance, then it can be further determined whether the current detection process is finished. If it is confirmed that the current detection process is finished, then a more accurate final detection result can be determined by further analyzing whether there is a face or a human figure in the video data at the third detection moment and before. If it is confirmed that the current detection process is not finished, then it can continue to go to step S1015 to continue to obtain the motion trajectory data after the moment of starting to detect the motion trajectory data, so as to obtain as much motion trajectory data as possible after the current moment to improve the detection accuracy.

[0264] S1025, determining that there is an abnormal stay situation, and pushing an abnormal stay message.

[0265] S1026: Determine that there is no abnormal situation and do not push any message.

[0266] It can be understood that steps S1009 to S1026 are similar to step S918 of step S901, and the difference lies in that in steps S1009 to S1026, if the final abnormal situation is confirmed, the corresponding abnormal situation message will be pushed, which will not be repeated here.

[0267] The above-mentioned abnormal stay detection method uses ToF ranging technology in conjunction with AI face and human shape detection technology to identify abnormal situations outside the door, which can not only determine two abnormal situations including abnormal stay and someone passing by, but also improve the detection accuracy of abnormal situation detection using only a single detection technology, and can stop the detection process when the detection result is determined, saving the power consumption of the smart door lock and improving the user experience. In addition, before determining the final detection result, a more accurate final detection result is determined by the presence of human features in the previous video data. In addition, when it is determined that the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the first distance, or when it is determined that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the variance threshold, or when it is determined that the average distance of the motion trajectory data after the start of the motion trajectory data detection is greater than the first distance, the camera recording process is stopped, which can also save the power consumption of the smart door lock. Moreover, after determining that the average distance of the motion trajectory data after the start of detecting the motion trajectory data is less than or equal to the first distance, it will be further determined whether the current detection process is ended, and if it is determined that the detection is not ended, the motion trajectory data will continue to be acquired to obtain a more accurate detection result through more motion trajectory data.

[0268] By way of example and not limitation, Fig.12 The schematic diagram of the relationship between the functional modules of the abnormal stay detection system of the smart door lock is shown in FIG. Fig.12 As shown, the smart door lock may include an image acquisition module 1140, a time of flight (ToF) ranging module, an AI recognition module 1150, a motion detection module 1160 and a joint judgment module. Among them, the camera in the image acquisition module 1140 can shoot a video of the range in front of the door, and the processor in the image acquisition module 1140 can execute the push method of the abnormal situation of the smart door lock in this application; the ToF ranging module 1130 can extract the variance characteristics of the moving distance of the object in front of the door within the set time according to the video data of the range in front of the door, and determine the moving range and moving speed of the object in front of the door; the AI ​​recognition module 1150 can identify whether there is a face or human figure in front of the door according to the video data of the range in front of the door; the motion detection module 1160 can perform a differential operation based on the set background image frame and the image frame in the video of the range in front of the door according to the built-in algorithm, so as to determine whether there is an object moving in front of the door; finally, the joint judgment module can comprehensively determine the abnormal situation in the video taken by the camera in front of the door according to the built-in algorithm based on the analysis results of the ToF ranging module 1130, the AI ​​recognition module 1150 and the motion detection module 1160, such as determining whether someone stays abnormally, whether someone passes by, and whether the image acquisition module 1140 stops shooting video.

[0269] Combine the following Fig.13 right Figure 1 The hardware structure of the smart door lock 200 in the illustrated scenario is exemplarily introduced.

[0270] like Fig.13 As shown, the smart door lock 200 may include a processor 110, an external memory interface 120, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 130, a wireless communication module 140, a sensor module 150, a camera 160, a display screen 170, etc. The sensor module 150 may include a pressure sensor 150A, a gyroscope sensor 150B, an acceleration sensor 150C, a distance sensor 150D, a fingerprint sensor 150E, a touch sensor 150F, etc.

[0271] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. Among them, the controller can be the nerve center and command center of the smart door lock 200. The controller can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory can also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. This memory can save instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use this instruction or data again, it can be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. The wireless communication function of the smart door lock 200 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the smart door lock 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0272] The mobile communication module 130 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the smart door lock 200. The mobile communication module 130 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 130 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 130 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 can be set in the same device as at least some of the modules of the processor 110.

[0273] The wireless communication module 140 can provide wireless communication solutions for application in the smart door lock 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), NFC, infrared technology (IR), etc.

[0274] The wireless communication module 140 may be one or more devices integrating at least one communication processing module. The wireless communication module 140 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 140 may also receive a signal to be sent from the processor 110, modulate the frequency of the signal, amplify the signal, and convert it into an electromagnetic wave for radiation through the antenna 2. Exemplarily, in an embodiment of the present application, a communication connection may be established between different electronic devices via BT or WLAN.

[0275] In some embodiments, the antenna 1 of the smart door lock 200 is coupled to the mobile communication module 130, and the antenna 2 is coupled to the wireless communication module 140, so that the smart door lock 200 can communicate with the network and other devices through wireless communication technology. Wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TDSCDMA), long term evolution (LTE), the fifth generation (5G) mobile communication system, future communication systems such as the sixth generation (6G) system, BT, GNSS, WLAN, NFC, FM and / or IR technology, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0276] The display screen 170 is used to display the display interface of the application, etc. The display screen 170 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (AMOLED), a flexible light emitting diode (FLED), Mini-led, Micro-Led, Micro-oLed, quantum dot light emitting diodes (QLED), etc. In some embodiments, the smart door lock 200 may include 1 or N display screens 170, where N is a positive integer greater than 1. In an embodiment of the present application, the display screen 170 can be used to display multiple application interfaces simultaneously.

[0277] The camera 160 is used to capture still images or videos. The camera 160 may include a front camera and a rear camera.

[0278] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the smart door lock 200 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and the software code of at least one application program, etc. The data storage area can store data (such as images, videos, etc.) generated during the use of the smart door lock 200. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0279] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the smart door lock 200. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as pictures and videos are saved in the external memory card.

[0280] The pressure sensor 150A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 150A can be provided on the display screen 170. The gyroscope sensor 150B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the smart door lock 200 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 150B.

[0281] The gyro sensor 150B can be used to determine the motion posture of the smart door lock 200. In some embodiments, the angular velocity of the smart door lock 200 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 150B. The gyro sensor 150B can be used for anti-shake shooting. Exemplarily, when the shutter is pressed, the gyro sensor 150B detects the angle of the shaking of the smart door lock 200, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the smart door lock 200 through reverse movement to achieve anti-shake. The gyro sensor 150B can also be used for navigation and somatosensory game scenes.

[0282] The acceleration sensor 150C can detect the magnitude of the acceleration of the smart door lock 200 in all directions (generally three axes). When the smart door lock 200 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices and applied to applications such as horizontal and vertical screen switching, pedometers, etc.

[0283] The distance sensor 150D is used to measure the distance. The smart door lock 200 can measure the distance by infrared or laser. In some embodiments, the smart door lock 200 can use the distance sensor 150D to obtain the moving distance data of the moving object in the monitoring area outside the door.

[0284] The fingerprint sensor 150E is used to collect fingerprints. The smart door lock 200 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc.

[0285] The touch sensor 150F is also called a "touch panel". The touch sensor 150F can be arranged on the display screen 170, and the touch sensor 150F and the display screen 170 form a touch screen, also called a "touch screen". The touch sensor 150F is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 170. In other embodiments, the touch sensor 150F can also be arranged on the surface of the smart door lock 200, which is different from the position of the display screen 170.

[0286] Understandably, Fig.12The components shown do not constitute a specific limitation on the smart door lock 200. The smart door lock 200 may also include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.

[0287] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0288] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0289] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the aforementioned elements.

[0290] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A method for abnormal push notification of a smart door lock, characterized in that: include: Performing image feature detection, wherein the image feature detection includes determining that no human features exist in the acquired first video data; Corresponding to the image feature detection satisfying a first image condition, performing motion trajectory detection, wherein the first image condition includes: no human features are detected in the first video data, and the motion trajectory detection includes determining the presence of abnormal movement based on the motion trajectory data; Corresponding to the motion trajectory detection satisfying a first trajectory condition, abnormal event push is performed, wherein the first trajectory condition includes: determining the presence of abnormal movement based on the motion trajectory data.

2. The method according to claim 1, characterized in that Determining the presence of abnormal movement based on the motion trajectory data includes: Determining the presence of an abnormal moving target based on an average distance and / or variance of the motion trajectory data; Corresponding to the existence of the abnormal moving target, it is determined that an abnormal movement situation exists.

3. The method according to claim 2, characterized in that Determining the presence of abnormal movement based on the motion trajectory data also includes: Determining, based on second video data, that the abnormal moving target includes human features, wherein the second video data includes video data collected within a time range from the start time of the abnormality detection process to the current time; Corresponding to the abnormal moving target including human features, it is determined that there is an abnormal movement situation.

4. The method according to claim 3, characterized in that The abnormal movement target includes human features, and determining that there is abnormal movement includes: Corresponding to the determination that the abnormal moving target is an abnormal lingering target, and based on the second video data, determining that the abnormal lingering target includes human features, determining that an abnormal lingering situation exists; The abnormal moving target is determined to be a passing target, and based on the second video data, it is determined that the passing target includes human features, and it is determined that there is a person passing by.

5. The method according to claim 2, characterized in that: Confirm the presence of an abnormal moving target by: Acquire first motion trajectory data, and divide the first motion trajectory data into motion trajectory data of a plurality of first detection cycles according to the acquisition time, wherein the first motion trajectory data is the motion trajectory data collected within a time range from the start time of the abnormality detection process to the start time of the motion trajectory detection; Calculating the average distance of the motion trajectory data of each of the first detection cycles; The average distances of the motion trajectory data corresponding to each of the first detection cycles all meet the first distance condition, confirming the presence of an abnormal lingering target.

6. The method according to claim 5, characterized in that The method further comprises: The variances of the average distances of the motion trajectory data corresponding to each of the first detection cycles satisfy the first variance condition, confirming the presence of an abnormal lingering target.

7. The method according to claim 5, characterized in that The method further comprises: The average distances of the motion trajectory data corresponding to each of the first detection cycles all satisfy the first distance condition, and the variances of the average distances all satisfy the first variance condition, confirming the presence of an abnormal lingering target.

8. The method according to claim 5, characterized in that The method further comprises: The average distance of the motion trajectory data corresponding to each of the first detection cycles does not meet the first distance condition, confirming that there is a passing target.

9. The method according to claim 5, characterized in that The method further comprises: The variances of the average distances of the motion trajectory data corresponding to each of the first detection cycles do not satisfy the first variance condition, confirming that there is a passing target.

10. The method according to claim 5, characterized in that The method further comprises: The average distances of the motion trajectory data corresponding to each of the first detection cycles do not satisfy the first distance condition, and the variances of the average distances do not satisfy the first variance condition, confirming that there is a passing target.

11. The method according to claim 1, characterized in that: When the motion trajectory detection satisfies the first trajectory condition, executing abnormal event push includes: In response to the motion trajectory detection determining that there is an abnormal stay situation, an abnormal stay message is pushed; or In response to the motion trajectory detection determining that someone has passed by, a message indicating that someone has passed by is pushed.

12. The method according to claim 1, characterized in that Also includes: Obtaining a preset time for the anomaly detection process; Corresponding to the detection of human features in the first video data, it is determined according to the preset time that the abnormality detection process has not ended, and image feature detection corresponding to the second video data is performed.

13. The method according to claim 12, characterized in that The performing motion trajectory detection corresponding to the image feature detection satisfying the first image condition includes: Corresponding to the image feature detection satisfying the first image condition and determining, according to the preset time, that the abnormality detection process has not ended, motion trajectory detection is performed.

14. The method according to claim 12, characterized in that The image feature detection further includes: determining that the difference between each image frame and the background frame in the acquired first video data does not satisfy a difference condition; The first image condition also includes: no human features are detected in the first video data, and the difference between each image frame and the background frame in the first video data does not meet the difference condition.

15. A computer storage medium, characterized in that: The storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to implement the method according to any one of claims 1 to 14.

16. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, configured to execute instructions stored in the memory to implement the method according to any one of claims 1 to 14.

Images