Methods for sending abnormal notifications to smart door locks, storage media, and electronic devices

Abstract

This invention relates to the field of smart lock technology, and discloses a method for anomaly detection in smart locks, a storage medium, and an electronic device. The method includes: performing image feature detection, wherein the image feature detection includes determining whether human features exist in acquired first video data; corresponding to the image feature detection satisfying a first image condition, performing motion trajectory detection, wherein the first image condition includes: no human features were detected in the first video data, and the motion trajectory detection includes determining whether abnormal movement exists based on the motion trajectory data; corresponding to the motion trajectory detection satisfying a first trajectory condition, performing anomaly event push notification, wherein the first trajectory condition includes: determining that abnormal movement exists based on the motion trajectory data. This method can improve the accuracy of anomaly detection in smart locks.

Description

Technical Field

[0001] This invention relates to the field of smart door lock technology, specifically to a method for abnormal push notifications for a smart door lock, a storage medium, and an electronic device. Background Technology

[0002] Currently, smart locks typically use abnormal loitering detection to determine the presence of suspicious individuals or other abnormal situations, thus ensuring user safety. Abnormal loitering detection in smart locks refers to detecting one or more targets moving back and forth within the lock's field of vision. For example, if the smart lock detects someone loitering within its detection range for a period of time, this abnormality can be reported to the user's mobile phone or other devices, allowing the user to be aware and thereby improving the security of the smart lock's monitoring environment.

[0003] Currently, the detection method for abnormal loitering in smart door locks usually uses artificial intelligence (AI) algorithms to detect whether a face or human shape continuously appears in the monitoring area outside the door to determine whether there is a target loitering abnormally. However, in some special scenarios, if someone is wearing a mask, the light is weak, or the position is not good, it will be impossible to detect a face or human shape, thus making it impossible to accurately detect abnormal loitering. Summary of the Invention

[0004] This invention provides a method for sending abnormal notifications for a smart door lock, a storage medium, and an electronic device.

[0005] In a first aspect, the present invention provides an abnormal push notification method for a smart door lock, comprising: performing image feature detection, wherein the image feature detection includes determining that no human features exist in the acquired first video data; corresponding to the image feature detection satisfying a first image condition, performing motion trajectory detection, wherein the first image condition includes: no human features were detected in the first video data, and the motion trajectory detection includes determining that there is an abnormal movement based on the motion trajectory data; corresponding to the motion trajectory detection satisfying a first trajectory condition, performing an abnormal event push notification, wherein the first trajectory condition includes: the motion trajectory data determines that there is an abnormal movement.

[0006] The above-mentioned abnormal push method for smart door locks combines motion trajectory detection with image feature detection, which can obtain more accurate detection results of abnormal situations. Based on the more accurate detection results of abnormal situations, abnormal events are pushed, thereby improving the security of the smart door lock monitoring area.

[0007] In one possible implementation of the first aspect above, determining the existence of abnormal movement based on motion trajectory data includes: determining the existence of an abnormally moving target based on the average distance and / or variance of the motion trajectory data; and determining the existence of abnormal movement in relation to the existence of an abnormally moving target.

[0008] The above-mentioned abnormal push method for smart door locks can obtain relatively accurate analysis results based on motion trajectory data by analyzing the average distance and / or variance of motion trajectory data.

[0009] In one possible implementation of the first aspect above, determining the existence of abnormal movement based on motion trajectory data further includes: determining whether the abnormal moving target includes human features based on second video data, wherein the second video data includes video data collected from the start of the anomaly detection process to the current time; and determining the existence of abnormal movement is consistent with the fact that the abnormal moving target includes human features.

[0010] The above-mentioned abnormal push method for smart door locks can further determine whether there are abnormalities in motion trajectory data based on video data collected from the start of the abnormality detection process to the current time, thereby improving the accuracy of detecting abnormalities through motion trajectory data.

[0011] In one possible implementation of the first aspect above, determining the existence of abnormal movement corresponding to the abnormal moving target including human features includes: corresponding to the determined abnormal moving target being an abnormal loitering target, and based on the second video data, determining that the abnormal loitering target includes human features, thus determining the existence of abnormal loitering; corresponding to the determined abnormal moving target being a passing target, and based on the second video data, determining that the passing target includes human features, thus determining the existence of someone passing by.

[0012] The above-mentioned abnormal notification method for smart door locks can accurately distinguish between abnormal movement situations and abnormal loitering situations and situations where someone passes by, thereby improving the accuracy of abnormal movement detection.

[0013] In one possible implementation of the first aspect above, the existence of an abnormal moving target is confirmed by: acquiring first motion trajectory data and dividing the first motion trajectory data into multiple first detection cycle motion trajectory data according to the acquisition time, wherein the first motion trajectory data is motion trajectory data acquired within the time range from the start time of the abnormal detection process to the start time of the motion trajectory detection; calculating the average distance of the motion trajectory data of each first detection cycle; if the average distance of the motion trajectory data corresponding to each first detection cycle satisfies the first distance condition, the existence of an abnormal lingering target is confirmed.

[0014] It is understandable that the first distance condition can be less than or equal to a preset distance threshold. The above-mentioned abnormal push method of smart door lock can obtain a more accurate detection result of abnormal loitering targets by analyzing the average distance in the first motion trajectory data.

[0015] In one possible implementation of the first aspect above, the method further includes: the variance of the average distance of the motion trajectory data corresponding to each first detection cycle satisfies the first variance condition, confirming the existence of an abnormally loitering target.

[0016] It is understandable that the first variance condition can be less than or equal to a preset variance threshold. The above-mentioned abnormal push method of smart door lock can obtain a more accurate detection result of abnormal loitering targets by analyzing the variance of the average distance in the first motion trajectory data.

[0017] In one possible implementation of the first aspect above, the method further includes: the average distance of the motion trajectory data corresponding to each first detection cycle satisfies the first distance condition, and the variance of the average distance satisfies the first variance condition, thus confirming the existence of an abnormally loitering target.

[0018] The above-mentioned abnormal push method for smart door locks can obtain relatively accurate detection results of abnormal loitering targets by analyzing the average distance and the variance of the average distance in the first motion trajectory data.

[0019] In one possible implementation of the first aspect above, the method further includes: if the average distance of the motion trajectory data corresponding to each first detection cycle does not meet the first distance condition, it is confirmed that a target has passed through.

[0020] The above-mentioned abnormal push method for smart door locks can obtain relatively accurate detection results of passing targets by analyzing the average distance in the first motion trajectory data.

[0021] In one possible implementation of the first aspect above, the method further includes: if the variance of the average distance of the motion trajectory data corresponding to each first detection cycle does not satisfy the first variance condition, it is confirmed that there is a target.

[0022] The above-mentioned abnormal push method for smart door locks can obtain a relatively accurate detection result of the passing target by analyzing the variance of the average distance in the first motion trajectory data.

[0023] In one possible implementation of the first aspect above, the method further includes: if the average distance of the motion trajectory data corresponding to each first detection cycle does not meet the first distance condition, and the variance of the average distance does not meet the first variance condition, it is confirmed that there is a target.

[0024] The above-mentioned abnormal push method for smart door locks can obtain relatively accurate detection results of passing targets by analyzing the average distance and the variance of the average distance in the first motion trajectory data.

[0025] In one possible implementation of the first aspect above, when the motion trajectory detection satisfies the first trajectory condition, an abnormal event push is executed, including: pushing an abnormal stay message when the motion trajectory detection determines that there is an abnormal stay; or pushing a person pass message when the motion trajectory detection determines that someone has passed.

[0026] The above-mentioned abnormal notification method for smart door locks can push different abnormal situation messages based on the motion trajectory detection results.

[0027] In one possible implementation of the first aspect above, the method further includes: obtaining a preset time for the anomaly detection process; corresponding to the detection of human features in the first video data, determining that the anomaly detection process has not ended based on the preset time, and performing image feature detection corresponding to the second video data.

[0028] The above-mentioned abnormal push method for smart door locks can detect human features in the first video data, and continue to perform the next image feature detection before the abnormal detection process ends, thereby obtaining more video data and improving the user experience.

[0029] In one possible implementation of the first aspect above, the motion trajectory detection is performed in accordance with the first image condition, which includes: in accordance with the first image condition, and after determining that the anomaly detection process has not ended according to a preset time, the motion trajectory detection is performed.

[0030] The above-mentioned abnormal push method for smart door locks can detect human features in the first video data and perform motion trajectory detection before the abnormal detection process ends, thereby improving the accuracy of the abnormal detection process.

[0031] In one possible implementation of the first aspect above, the image feature detection further includes: determining that the difference between each image frame in the acquired first video data and the background frame does not satisfy the difference condition; the first image condition further includes: no human features are detected in the first video data, and the difference between each image frame in the first video data and the background frame does not satisfy the difference condition.

[0032] It is understood that the difference condition can be greater than the preset difference threshold. The above-mentioned abnormal push method of the smart door lock combines the detection of the difference between each image frame and the background frame in the first video data with human feature detection, thereby improving the accuracy of image feature detection.

[0033] In a second aspect, embodiments of the present invention provide an abnormal push device for a smart door lock, comprising one or more processors and one or more memories; the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the device to perform any of the abnormal push methods for a smart door lock provided by the first aspect and various possible implementations of the first aspect described above.

[0034] Thirdly, embodiments of the present invention provide a chip, the chip including a processor, the processor executing computer instructions stored in a computer storage medium, causing the electronic device to implement any of the abnormal push methods of a smart lock provided by various possible implementations of the first aspect above.

[0035] Fourthly, embodiments of the present invention provide a readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to implement any of the abnormal push methods for a smart lock provided in various possible implementations of the first aspect.

[0036] Fifthly, embodiments of the present invention provide an electronic device comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, one of the processors of the electronic device, for executing the instructions stored in the memory to implement any of the abnormal push methods for a smart lock provided by various possible implementations of the first aspect above.

[0037] In a sixth aspect, embodiments of the present invention provide a program product that includes instructions that, when executed by an electronic device, enable the electronic device to implement any of the abnormal push methods for a smart lock provided in the various possible implementations of the first aspect. Attached Figure Description

[0038] Figure 1 This application shows a schematic diagram illustrating a usage scenario of a smart door lock according to an embodiment of the present application;

[0039] Figure 2 This paper illustrates a flowchart of an AI-based method for detecting abnormal loitering in an embodiment of this application.

[0040] Figure 3 A schematic diagram of another AI-based method for detecting abnormal loitering is shown in an embodiment of this application;

[0041] Figure 4A A flowchart illustrating a method for sending notifications of abnormal situations to a smart door lock, as shown in an embodiment of this application, is provided.

[0042] Figure 4BA schematic diagram of an abnormal condition detection process for a smart door lock according to an embodiment of this application is shown;

[0043] Figure 5 A flowchart illustrating a method for determining abnormal situations based on motion trajectory data during the motion trajectory detection stage, as shown in an embodiment of this application, is presented.

[0044] Figure 6 A flowchart illustrating a method for determining abnormal situations based on motion trajectory data during the motion trajectory detection stage, as shown in an embodiment of this application, is presented.

[0045] Figure 7 A flowchart illustrating another method for determining abnormal situations based on motion trajectory data during the motion trajectory detection stage, as shown in an embodiment of this application, is presented.

[0046] Figure 8 A flowchart illustrating another method for determining abnormal situations based on motion trajectory data during the motion trajectory detection stage, as shown in an embodiment of this application, is presented.

[0047] Figure 9 A flowchart illustrating another method for determining abnormal situations based on motion trajectory data during the motion trajectory detection stage, as shown in an embodiment of this application, is presented.

[0048] Figure 10 A flowchart illustrating another method for sending notifications of abnormal situations in a smart door lock, as shown in an embodiment of this application, is illustrated.

[0049] Figure 11 A functional flowchart of an abnormal situation detection system for a smart door lock according to an embodiment of this application is shown;

[0050] Figure 12 A schematic diagram showing the functional module relationship of an abnormal situation detection system for a smart door lock according to an embodiment of this application is illustrated.

[0051] Figure 13 A schematic diagram of the hardware structure of a smart door lock according to an embodiment of this application is shown. Detailed Implementation

[0052] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0053] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of these features, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0054] Figure 1 The image shows a usage scenario diagram of a smart door lock. For example... Figure 1 As shown, the smart lock 200 may include an image acquisition unit 201 and a communication chip 202. The smart lock 200 can communicate with the terminal device 100 through the communication chip 202. The image acquisition unit 201 may include a camera (not shown) and a processor (not shown). After the camera captures a real-time image of the door, it transmits the data to the abnormal loitering detection module. If the processor determines that the detection result matches abnormal loitering using the abnormal situation push method for smart locks mentioned in this application, the processor will send the abnormal loitering detection result to the cloud server through the communication chip 202. The cloud server then sends the abnormal loitering detection result to the terminal device 100 through information prompts or other means to alert the user to the potential abnormal loitering situation and improve the security of the detection environment.

[0055] It is understood that a smart door lock may include other components installed on the panel outside the door, such as fingerprint sensors, buttons, microphones, speakers, etc. Cameras may include facial recognition cameras, peephole cameras, etc. A smart door lock may also include a lock body and components installed on the panel inside the door. In this embodiment, the specific structure of the smart door lock is not limited.

[0056] As described in the background section, current methods for sending alerts about abnormal situations to smart door locks can use AI to detect facial features and human shapes to determine if there is any unusual loitering. For example, Figure 2 This paper demonstrates a histogram-based AI detection method that can be executed by a smart door lock. The method may include:

[0057] S201: Select targets within the monitored area that are displayed more than the first preset number of times as candidate targets.

[0058] For example, if the number of times a target face or human figure appears in the monitored area within a preset time exceeds a first preset number, these identified target faces or human figures can be used as candidate targets.

[0059] S202: Detect candidate targets using a histogram matching and tracking algorithm, and identify candidate targets that have been presented more than a second preset number of times as hovering targets.

[0060] For example, a histogram matching tracking algorithm can be used to track and detect candidate targets, i.e., to perform fine-grained tracking of candidate targets. Specifically, the similarity between the histograms of images taken when the candidate target is at its initial position and the histograms of images taken when the candidate target is at each subsequent movement position can be used to further update the frequency of candidate target occurrences. For example, if the similarity between the histograms of corresponding regions at two positions is less than 70%, the frequency of candidate target occurrences can be reduced by 1, meaning the candidate target is not considered to be present in the monitored area. Conversely, if the similarity between the histograms of corresponding regions at two positions is greater than or equal to 70%, the current frequency of candidate target occurrences can be maintained. After the detection process is completed, candidate targets that have appeared more than a second preset number of times are designated as loitering targets, i.e., abnormally loitering objects.

[0061] For example, Figure 3 An AI-based detection method is shown, which can be executed by a smart door lock. This method includes:

[0062] S301: Feature vectors of the model constructed based on the environmental background area and the abnormal wandering motion trajectory.

[0063] For example, corresponding environmental background areas and abnormal loitering motion trajectories can be extracted from multiple sample video data, and the acquired environmental background areas and abnormal loitering motion trajectories can be transformed into feature vectors to serve as sample data for the model. The sample video data uses human-shot subjects.

[0064] S302: Construct a detection model using feature vectors.

[0065] For example, the constructed feature vectors can be used as sample data for model training to obtain a model for detecting abnormal wandering.

[0066] S303: Detect abnormal behavior based on the detection model and calculate the probability of abnormal situations occurring.

[0067] For example, if a face or human figure is detected in the monitored area, the motion trajectory of the detected target in the surrounding environment and the background area can be input into the model to obtain the probability of the anomaly.

[0068] However, in some special scenarios, such as when the target is wearing a mask, standing sideways or with its back to the camera, or in poor lighting conditions, it will be difficult to identify a face or human shape, and it will be impossible to obtain candidate targets. Consequently, it will be impossible to obtain abnormal loitering detection results through AI analysis algorithms.

[0069] In other methods for detecting abnormal loitering, the presence of abnormal loitering can be determined by checking if the length of the target's motion trajectory exceeds a threshold, without requiring precise facial or human shape recognition. For example, if a target is found to be moving back and forth within a monitored area for a period of time, and the total length of its motion trajectory exceeds a threshold, this will be reported as abnormal loitering. However, in some special scenarios, if the target deliberately remains stationary, its motion trajectory cannot be obtained. Therefore, to improve detection accuracy and avoid missing target motion trajectories, a longer detection time is often required to acquire more target motion trajectories. However, for lightweight smart door locks, to avoid increasing the processor's computational burden and reducing power consumption, a longer detection time is often not feasible, resulting in lower detection accuracy.

[0070] In view of this, the present invention proposes a method for pushing abnormal situations of smart door locks, which can comprehensively judge whether there are abnormal situations outside the door based on the judgment results of whether there is a face or human figure in the video, as well as the movement trajectory of the target.

[0071] Specifically, for example, in some embodiments, after triggering the smart lock's detection process, a video detection phase can be initiated first. In this phase, AI facial and human recognition technology can be used to detect the acquired video data. If human features, such as a face or human shape, are detected in the video data, the human detection phase resumes before the detection process ends (e.g., the duration of the detection process set by the user). If no human features are detected in the human detection phase, the motion trajectory detection phase begins, where anomalies are assessed based on motion trajectory data. For example, in the motion trajectory detection phase, motion trajectory data acquired using ToF ranging technology can be analyzed to determine if there are any abnormally loitering or passing targets within the smart lock's monitoring area. If abnormally loitering or passing targets are found, the detection process ends, and the user is notified of the situation. If no abnormally loitering or passing targets are detected, the detection process ends.

[0072] The aforementioned method for sending notifications regarding abnormal situations in smart locks primarily utilizes AI detection, combined with ToF technology, to deliver relatively accurate detection results. Furthermore, before determining the final detection result of the current detection process, it analyzes all video data from the current moment and all previous moments to identify faces or human figures, thereby obtaining more precise detection results based on face and human figure recognition technology. Additionally, the detection cycle can be stopped before determining the final detection result of the abnormal situation, thus saving power consumption of the electronic device.

[0073] The following is combined with Figures 4A to 12 The technical solutions of some embodiments of the present invention are described below.

[0074] The following is combined with Figures 4A to 12 The method for sending notifications regarding abnormal situations of the smart door lock in this application embodiment is described.

[0075] It is understood that in this embodiment of the application, the abnormal situations detected by the smart door lock may include two situations: abnormal loitering and someone passing by.

[0076] It is understood that all methods and steps in the embodiments of this application are executed by the smart door lock.

[0077] Figure 4A This application illustrates a method for sending notifications in case of an abnormal situation with a smart door lock, as shown in an embodiment of this application. Figure 4A As shown, a method for sending notifications about abnormal situations in a smart door lock may include the following steps:

[0078] S401, a trigger event for the detection process has been detected.

[0079] It is understood that the triggering event of the abnormal situation detection process in the embodiments of this application can be as follows: Figure 11 As shown, the monitoring area outside the door can be detected by a passive infrared (PIR) detector 1120. If the PIR detects a target moving within the monitoring area, the microcontroller unit (MCU) 1110 will wake up the camera 1141 in the image acquisition module 1140 and the ToF sensor 1131 in the ToF ranging module 1130 to start video recording and ranging, thus initiating the detection process. The MCU 1110 can be located within the smart lock. It is understood that the detection process can also be initiated by other types of sensors or methods; however, in this embodiment, the triggering event for the detection process is not specifically limited. It is understood that the duration of the detection process can be set according to specific circumstances, such as 30 seconds; however, in this embodiment, the duration of the detection process is not specifically limited.

[0080] Figure 4B The diagram illustrates a detection process for an abnormal situation in a smart door lock. For example... Figure 4B As shown in the embodiments of this application, after the detection process is triggered by external conditions, each detection process will be divided into at least one video detection stage. At the beginning of each video detection stage, AI-based face and human figure recognition technology or motion detection technology will be used to detect abnormal situations in the monitoring area. If a face or human figure cannot be detected by AI-based face and human figure recognition technology in a certain video detection stage, and the image frames and background frames of the video identified by motion detection technology have not changed, and the current detection process has not ended, the motion trajectory detection stage will be entered, and the detection results of abnormal situations will be further obtained based on ToF ranging technology.

[0081] It is understood that motion detection technology can also be used to identify whether there are changes between the current image frame and several adjacent frames in a video. In this application embodiment, the scope of application of motion detection technology is not specifically limited.

[0082] It is understood that the video detection phase can be set according to the specific scenario. For example, it can be set to be a video detection phase every 3 seconds within the 30-second detection process, that is, every 3 seconds, the face and human figure within this 3-second time range and the changes of all image frames in the video within this 3-second time range are detected. In this embodiment of the application, the specific setting of the video detection phase is not specifically limited.

[0083] In the embodiments of this application, AI face and human figure detection technology and ToF ranging technology are combined to determine abnormal situations in the monitored area. When an abnormal situation is detected, the smart door lock will push an abnormal situation message to the terminal device.

[0084] S402, confirm the presence of human features based on the video data of the current video detection stage. If yes, proceed to step S403 to detect human features in the video data of the next video detection stage. If no, proceed to step S404 to determine anomalies based on the motion trajectory data of the motion trajectory detection stage.

[0085] In one possible implementation, it can be achieved through... Figure 11 The algorithm in the AI ​​recognition module 1150 detects whether human features, such as faces or human shapes, exist in the video data during the current video detection phase. It is understood that the AI ​​algorithm can be a face recognition algorithm or a human shape recognition algorithm; however, this embodiment does not specifically limit the AI ​​algorithm used to detect faces or human shapes. It is understood that if a face or human shape is detected during the current video detection phase, it indicates that an abnormal target has appeared in the monitoring data during the current video detection phase.

[0086] In one possible implementation, step S403 can also be: confirming the presence of human features based on the video data of the current video detection stage, or whether there are changes in the image frames compared to the background image frames. If yes, proceed to step S403 to detect human features in the video data of the next video detection stage. If no, proceed to step S404 to determine anomalies based on the motion trajectory data of the motion trajectory detection stage. That is, the video detection stage can be divided into a human feature detection stage and a motion detection stage. The motion detection stage is used to detect whether there are changes in the image frames compared to the background image frames in the video data of the current video detection stage.

[0087] In one possible implementation, it can be achieved through... Figure 11 The motion detection module 1160 in the video is used to determine whether there are any changes between all image frames of the video in the current video detection phase and the background image frames. It can be understood that the background image frames can be images preset in the motion detection module 1160, such as images of the surrounding environment outside the door including valuable objects, or images of the area outside the door to be detected. Then, by comparing all image frames of the video in the current video detection phase with the background image frames, it can be determined whether there are any abnormal targets in the monitoring area outside the door. In this embodiment, the algorithm used to detect the image frames and background image frames is not specifically limited.

[0088] Understandably, in certain scenarios, such as when a suspicious person outside the door is wearing a mask and deliberately stands with their back to the camera, or when the ambient light is poor and a clear image cannot be captured, the motion detection module 1160 can detect changes in the current image frame, and combined with the AI ​​recognition module 1150 for anomaly detection, anomalies can be more accurately identified.

[0089] Understandable. Figure 11 The main controller 1110 is mainly used to control the ToF ranging module 1130 and the image acquisition module 1140 to perform ranging and recording. The processor of the Maoyan 3516 is mainly used to determine the final detection result of abnormal situations based on AI face and human figure recognition algorithms (including face and human figure recognition), motion detection algorithms, and motion trajectory detection algorithms.

[0090] S403 detects human features in the video data of the next video detection stage.

[0091] It is understandable that if a face or human figure is found in the video data within the current video detection phase, the next video detection phase can proceed.

[0092] In some possible implementations, if a face or human figure is detected in the video data within the current video detection phase in step S402, it can be determined whether the detection process has ended. For example, whether the set time of 30 seconds has been completed. If it is determined that the detection process has not ended, the next video detection phase can begin. That is, the camera recording program does not stop and needs to continue to perform face and human figure recognition every 3 seconds within this 3-second time range until the detection process ends. It can be understood that if the detection process has not ended, the video detection phase can be repeated to make the actual detection time more consistent with the set detection process duration, thereby improving the user experience.

[0093] In some possible implementations, if it is determined in step S402 that a face or human figure exists in the video data during the current video detection phase, and if it is determined that the detection process has ended, then... Figure 11 The processor of the Maoyan 3516 sends the message of someone loitering abnormally and all video data of the current detection process to the cloud. The cloud then sends the message of abnormal loitering and all video data of the current detection process to the user's terminal device to remind the user that there is an abnormal loitering phenomenon in the monitoring area outside the door and to take relevant early warning measures.

[0094] S404, Determine abnormal situations based on motion trajectory data from the motion trajectory detection phase.

[0095] In one possible implementation, if it is determined in step S402 that no face or human figure is detected in the video data during the current video detection phase, the analysis can then proceed to the analysis of the motion trajectory data of the target movement in the monitoring area in front of the door acquired by the ToF sensor 1131.

[0096] It is understandable that in scenarios where human features cannot be obtained at a certain stage of video detection, analyzing the distance data of motion trajectories can be used instead. This eliminates the need to analyze all video data within the set time of the detection process, thereby improving the efficiency and accuracy of the detection process.

[0097] It is understood that the ToF sensor 1131 uses the ToF ranging method to acquire the motion trajectory data of the target moving in the monitoring area in front of the door. The ToF ranging method is a two-way ranging technology, which mainly uses the time of flight of the signal between two asynchronous transceivers to measure the distance between nodes. Combining ToF ranging technology with AI face and human recognition technology can further improve the detection accuracy of abnormal situations. It is understood that the motion trajectory data may include distance features within a set time period, distance variance features, the movement range and movement speed of the target in front of the door, etc. In this embodiment of the application, the specific parameter types in the motion trajectory data are not specifically limited.

[0098] In one possible implementation, if step S402 determines that no face or human figure is present in the video data of the current video detection stage, it can be further determined whether the detection process has ended. Upon determining that the detection process has ended, it is further determined whether any face or human figure is present in all the video data from the detection process. If a face or human figure is present in all the video data from the detection process, an abnormal loitering situation can be identified, and an abnormal loitering message can be pushed. If no face or human figure is present in all the video data from the detection process, no message is pushed. It can be understood that abnormal situations in the detection process can be determined by analyzing whether any face or human figure appears in all the video data from the detection process.

[0099] In one possible implementation, if it is determined in step S402 that there is no face or human figure in the video data of the current video detection stage, it can be further determined that the detection process has not ended, and the process proceeds to step S404. Based on the motion trajectory data of the motion trajectory detection stage, anomalies are determined. By further analyzing the motion trajectory data, the anomalies of the current detection process are determined in order to obtain more accurate detection results.

[0100] In one possible implementation, after analyzing motion trajectory data, human features can also be detected in video data from the current moment and previous moments. Based on the analysis results of motion trajectory data and the detection results of human features in video data from the current moment and previous moments, the final anomaly can be determined.

[0101] S405, End of testing process.

[0102] It is understandable that in step S404, by analyzing the motion trajectory data, an abnormal situation in the monitoring area of ​​the smart door lock is determined, such as abnormal loitering, someone passing by, or no abnormal situation, and the detection process can be terminated, thereby saving the power consumption of the smart door lock.

[0103] S406, push messages based on abnormal situations.

[0104] It's understandable that after the detection process ends, appropriate messages can be pushed based on the identified anomalies. For example, an abnormal stay message can be pushed based on abnormal loitering; a passerby message can be pushed based on someone passing by; and no message can be pushed if there are no anomalies.

[0105] Specifically, in step S404, the method for determining anomalies based on motion trajectory data from the motion trajectory detection phase, such as methods for determining anomalies based on the average distance and variance of the average distance of the motion trajectory data, will be discussed in detail below. Figure 5 , Figure 6 , Figure 7 , Figure 8 and Figure 9 Provide a detailed description.

[0106] The above-mentioned abnormal loitering detection method uses ToF ranging technology in conjunction with AI face and human detection technology to identify abnormal situations outside the door. It can not only identify two abnormal situations, including abnormal loitering and people passing by, but also improve the detection accuracy of abnormal situation detection using only a single detection technology. Furthermore, it can stop the detection process after determining the detection result, saving the power consumption of the smart door lock and improving the user experience.

[0107] The following will combine Figure 5 , Figure 6 , Figure 7 , Figure 8 and Figure 9 The method for determining abnormal situations based on motion trajectory data in the motion trajectory detection stage in step S404 is described in detail.

[0108] Figure 5 This paper demonstrates a method for obtaining the final detection result based on motion trajectory data from the motion trajectory detection stage when no face or human figure is detected in a certain video detection stage within the detection process. In this method, if no face or human figure is detected in a certain video detection stage... Figure 4A If it is determined in step S402 that no face or human figure is detected in the current video detection stage, the final detection result will be obtained by analyzing the average distance of the motion trajectory data of the moving target in the monitoring area in front of the door obtained by the ToF sensor 1131.

[0109] The specific steps include:

[0110] S501, acquire motion trajectory data from the start of the detection process to the moment when motion trajectory data detection begins.

[0111] S502, determine whether the average distance between the start of the detection process and the start of motion trajectory data detection is greater than the first distance. If yes, proceed to step S503, determine whether there are human features in the video data at time t1 and before. If no, proceed to step S506, determine whether there are human features in the video data at time t1 and before.

[0112] Here, time t1 is the moment when the video data is analyzed again after step S502.

[0113] It is understandable that, after failing to obtain target information through the AI ​​recognition module 1150, anomalies can be determined by detecting the relationship between the average distance of the motion trajectory data from the start of the detection process to the start of motion trajectory data detection and the first distance. It is understandable that the duration from the start of the detection process to the start of motion trajectory data detection can be divided into m parts, each part being a first detection cycle, where m is a natural number greater than or equal to 1. Then, the average distance within the duration of all first detection cycles is analyzed. It is understandable that the specific value of the first distance can be set according to actual conditions. In this embodiment, the specific value of the first distance is not specifically limited in terms of the rules for setting the first detection cycle.

[0114] In one possible implementation, after determining that the average distance in all first detection cycles is greater than the first distance, it indicates that the target's trajectory distance is large and the probability of an anomaly is small. The camera recording process can then be stopped to save power consumption of the smart lock.

[0115] S503, determine whether there are human features in the video data at and before time t1. If yes, proceed to step S504 to confirm that someone has passed by. If no, proceed to step S505 to confirm that there are no abnormalities.

[0116] It is understandable that after determining that the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is greater than the first distance, the situation of this detection process can be further determined by further analyzing the video data at time t1 and before.

[0117] S504, It has been confirmed that someone passed by.

[0118] It is understandable that if the average distance in all the first detection cycles is greater than the first distance, but a face or human figure is detected in the video data at time t1 and before, then it can be determined that there is a high probability that someone passed through this detection process.

[0119] S505, confirming that there are no abnormalities.

[0120] It is understandable that if the average distance in all the first detection cycles is greater than the first distance, and no face or human figure is detected in the video data at time t1 and before, it can be said that there are no abnormalities from the start of the detection process to the current time.

[0121] S506, determine whether there are human features in the video data at and before time t1. If yes, proceed to step S507 to confirm the existence of abnormal loitering. If no, proceed to step S505 to confirm the absence of abnormalities.

[0122] It is understandable that after determining that the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is less than or equal to the first distance, the situation of this detection process can be further determined by further analyzing the video data at time t1 and before.

[0123] It is understandable that if the average distance in all first detection cycles is less than or equal to the first distance, but no face or human figure is detected in the video data at time t1 and before, it can be said that the moving target in all first detection cycles may not be a person, that is, there is no abnormality from the start of the detection process to the current time.

[0124] S507, an abnormal stay has been identified.

[0125] It is understandable that if the average distance in all first detection cycles is less than or equal to the first distance, and a face or human figure is detected in the video data at time t1 and before, then it is likely that someone is abnormally loitering in this detection process.

[0126] If the above method determines that no face or human figure is detected in a certain video detection phase, it further obtains the final detection result by analyzing the average distance of the motion trajectory data. Before determining the final detection result, it re-analyzes the video data at the current moment and previous moments to determine the presence of human features, thereby determining a more accurate final detection result. Furthermore, after determining that the average distance in all first detection cycles is greater than the first distance, the camera recording process is stopped, which can save power consumption of the smart lock and improve the user experience.

[0127] Figure 6 This paper demonstrates a method for obtaining the final detection result based on motion trajectory data, etc., after determining that no human face or human figure was detected in a certain video detection stage, and that all image frames and background image frames of the video remained unchanged, and then pushing a message indicating the final anomaly. This method is similar to... Figure 5 The difference between the two methods is that if no face or human figure is detected in a certain video detection stage, and all image frames and background image frames of the video do not change, then the variance of the average distance in the motion trajectory data between the start of the detection process and the start of motion trajectory data detection can be analyzed to further determine more accurate anomalies.

[0128] The specific steps include:

[0129] S601, acquire motion trajectory data from the start of the detection process to the moment when motion trajectory data detection begins.

[0130] S602, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is greater than the variance threshold. If yes, proceed to step S603, determine whether there are human features in the video data at and before time t2. If no, proceed to step S606, determine whether there are human features in the video data at and before time t2. Here, time t2 is the time after step S602 when the video data is analyzed again.

[0131] It is understood that, similar to step S502, after failing to obtain target information through the AI ​​recognition module 1150 or the motion detection module 1160, the relationship between the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection can be analyzed to determine whether there are any abnormalities. It is understood that the time between the start of the detection process and the start of motion trajectory data detection can be divided into n parts, each part being a second detection cycle, where n is a natural number greater than or equal to 1. Then, the variance of the average distance within the time length of all second detection cycles is analyzed. It is understood that the specific value of the variance threshold can be set according to the actual situation. In this embodiment, the specific value of the variance threshold is not specifically limited for the setting rules of the second detection cycle.

[0132] In one possible implementation, after determining that the variance of the average distance in all second detection cycles is greater than the variance threshold, it indicates that the target's trajectory has a large distance and the probability of it being an anomaly is small. In this case, the camera's recording process can be stopped to save power consumption of the smart lock.

[0133] In one possible implementation, in step S602, after determining that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is greater than the variance threshold, it indicates that the distance of the target's motion trajectory is large and the probability of it being an abnormal situation is small. At this time, the recording process of the camera can be stopped to save the power consumption of the smart door lock.

[0134] S603, determine whether there are human features in the video data at and before time t2. If yes, proceed to step S604 to confirm that someone has passed by. If yes, proceed to step S605 to confirm that there are no abnormalities.

[0135] It is understandable that if the variance of the average distance of all motion trajectory data in the second detection period is greater than the variance threshold, the situation of this detection process can be further determined by further analyzing the video data at time t2 and before.

[0136] S604, It has been confirmed that someone passed by.

[0137] It is understandable that if the variance of the average distance in all second detection periods is greater than the variance threshold, but a face or human figure is detected in the video data at time t2 and before, then it is likely that someone passed through this detection process.

[0138] S605, confirming that there are no abnormalities.

[0139] It is understandable that if the variance of the average distance in all second detection cycles is greater than the variance threshold, but no face or human figure is detected in the video data at time t2 and before, it can be said that the moving target in all second detection cycles may not be a person, that is, there is no abnormality from the start of the detection process to the current time.

[0140] S606, determine whether there are human features in the video data at and before time t2. If yes, proceed to step S607 to confirm the existence of abnormal loitering. If yes, proceed to step S605 to confirm that there are no abnormalities.

[0141] It is understandable that if the variance of the average distance of all motion trajectory data in the second detection period is less than or equal to the variance threshold, the situation of this detection process can be further determined by further analyzing the video data at time t2 and before.

[0142] S607, an abnormal stay has been identified.

[0143] It is understandable that if the variance of the average distance in all second detection cycles is less than or equal to the variance threshold, and a face or human figure is detected in the video data at time t2 and earlier, then it is likely that someone is abnormally loitering in this detection process.

[0144] If the above method determines that no face or human figure is detected in a certain video detection phase, it further obtains the final detection result by analyzing the variance of the average distance of the motion trajectory data. Before determining the final detection result, it re-analyzes the video data at the time of the next detection and the video data before that time to determine a more accurate final detection result. Furthermore, after determining that the variance of the average distance in all second detection cycles is greater than a variance threshold, the camera recording process is stopped, which can save power consumption of the smart lock and improve the user experience.

[0145] Figure 7 This paper demonstrates another method for obtaining the final detection result by analyzing motion trajectory data, etc., after determining that no face or human figure was detected in a certain video detection stage. This method is similar to... Figure 5The difference between the two methods is that when no face or human figure is detected in a certain video detection stage, and when the average distance in the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is less than or equal to the first distance, the variance of the average distance in the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is analyzed to further determine more accurate anomalies.

[0146] The specific steps include:

[0147] S701, acquire motion trajectory data from the start of the detection process to the moment when motion trajectory data detection begins.

[0148] S702, determine whether the average distance between the start of the detection process and the start of motion trajectory data detection is greater than the first distance. If yes, proceed to step S703, determine whether there are human features in the video data at time t3 and before. If no, proceed to step S706, determine whether there are human features in the video data at time t3 and before.

[0149] Here, time t3 is the moment when the video data is analyzed again after step S702.

[0150] S703, determine whether there are human features in the video data at and before time t3. If yes, proceed to step S704 to confirm that someone has passed by. If no, proceed to step S705 to confirm that there are no abnormalities.

[0151] S704, it has been confirmed that someone passed by.

[0152] S705, confirming that there are no abnormalities.

[0153] It is understood that steps S701 to S705 correspond to the same steps S501 to S505, and will not be repeated here.

[0154] S706, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than the variance threshold. If yes, proceed to step S707, determine whether there are human features in the video data at and before time t4. If no, proceed to step S710, determine whether there are human features in the video data at and before time t4.

[0155] Here, time t4 is the moment when the video data is analyzed again after step S706.

[0156] It is understandable that after determining that the average distance of the motion trajectory data between the start of the detection process and the start of the motion trajectory data detection is less than or equal to the first distance, the variance characteristics of the average distance can be further analyzed to determine the movement status of the target appearing in front of the door. If the variance of the average distance is greater than the variance threshold, it means that there is a high probability that someone has passed by. If the variance of the average distance is less than or equal to the variance threshold, it means that there is a high probability that there is an abnormal stay.

[0157] In one possible implementation, if the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is greater than the variance threshold, it indicates that the distance of the target's motion trajectory is large and the probability of an abnormal situation is small. The camera recording process can be stopped to save the power consumption of the smart door lock.

[0158] S707, determine whether there are human features in the video data at and before time t4. If yes, proceed to step S708 to confirm that someone passed by. If no, proceed to step S709 to confirm that there are no abnormalities.

[0159] It is understandable that if the variance of the average distance is greater than the variance threshold, it means that the target in front of the door has a large range of movement. Then, we can further analyze the video data at time t4 and before. If no face or human figure is identified in the video data at time t4 and before, it means that there is a high probability that there is no abnormality.

[0160] S708, it has been confirmed that someone passed by.

[0161] It is understandable that if the variance of the average distance is greater than the variance threshold, and a face or human figure is identified in the video data at time t4 and before, then it means that there is a high probability that someone has passed by.

[0162] S709, confirming that there are no abnormalities.

[0163] It is understandable that if the variance of the average distance is less than or equal to the variance threshold, it means that the target in front of the door has a small range of movement. Then, we can further analyze the video data at time t4 and before. If no face or human figure is identified in the video data at time t4 and before, it means that the probability of an abnormal situation with no people is relatively high.

[0164] S710, determine whether there are human features in the video data at and before time t4. If yes, proceed to step S711 to confirm the existence of abnormal loitering. If no, proceed to step S709 to confirm that there are no abnormalities.

[0165] It is understandable that if the variance of the average distance is less than or equal to the variance threshold, it means that the target in front of the door has a small range of movement. Then, we can further analyze the video data at time t4 and before. If no face or human figure is identified in the video data at time t4 and before, it means that there is a high probability that there is no abnormality.

[0166] S711, an abnormal stay has been confirmed.

[0167] It is understandable that if the variance of the average distance is less than or equal to the variance threshold, and a face or human figure is identified in the video data at time t4 and earlier, then it indicates that there is a high probability that someone is lingering abnormally.

[0168] The above method, upon determining that no face or human figure was detected during a certain video detection phase, further improves detection accuracy by analyzing the average distance and variance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection. Furthermore, before determining the final detection result, the presence of human features in the video data at the previous time point is re-detected to obtain a more accurate final result. Moreover, the camera recording process is stopped after determining that the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is greater than a first distance, or that the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is greater than a variance threshold. This also saves power consumption for the smart lock and improves the user experience.

[0169] Figure 8 This paper demonstrates a method for obtaining the final detection result based on motion trajectory data, etc., after determining that no face or human figure was detected in a certain video detection phase, and that all image frames and background image frames of the video remained unchanged, and then pushing a message indicating the final anomaly. This method is similar to... Figure 6 The difference between the two methods is that, after determining that no face or human figure was detected in a certain video detection phase, and that the variance of the average distance in the motion trajectory data is less than or equal to the variance threshold, the anomalies are further determined by using motion trajectory data after the start of motion trajectory data detection.

[0170] The specific steps include:

[0171] S801, acquire motion trajectory data from the start of the detection process to the moment when motion trajectory data detection begins.

[0172] S802, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than the variance threshold. If yes, proceed to step S803, determine whether there are human features in the video data at and before time t2. If no, proceed to step S806, determine whether there are human features in the video data at and before time t2.

[0173] Here, time t5 is the moment when the video data is analyzed again after step S802.

[0174] S803, determine whether there are human features in the video data at and before time t2. If yes, proceed to step S804 to confirm that someone has passed by. If yes, proceed to step S805 to confirm that there are no abnormalities.

[0175] S804, It has been confirmed that someone passed by.

[0176] S805, confirming that there are no abnormalities.

[0177] It is understood that steps S801 to S805 correspond to steps S601 to S605, and time t5 corresponds to time t2, which will not be elaborated here.

[0178] In one possible implementation, if the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is greater than a variance threshold in step S802, it indicates that the target outside the door has a large range of movement during the time between the start of the detection process and the start of motion trajectory data detection. In this case, recording can be stopped to save power consumption of the smart lock.

[0179] S806, acquire motion trajectory data after the start of motion trajectory data detection.

[0180] It is understandable that if the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is less than or equal to the variance threshold in step S802, then the abnormal situation can be further determined based on the average distance of the motion trajectory data after the start of motion trajectory data detection, thereby further improving the accuracy of abnormal situation detection.

[0181] S807, determine whether the average distance of the motion trajectory data after the start time of motion trajectory data detection is greater than the first distance. If so, proceed to step S808 to determine whether human features exist in the video data at and before time t6. Alternatively, proceed to step S811 to determine whether the detection process has ended.

[0182] Here, time t6 is the moment when the video data is analyzed again after step S807.

[0183] It is understandable that the detection time after the start of motion trajectory data detection can be divided into p parts, where p is a natural number. Each part constitutes a third detection cycle, and the rules for setting the third detection cycle and the specific value of the first distance are not specifically limited. Detection accuracy can be further improved by analyzing the average distance of the motion trajectory data after the start of motion trajectory data detection.

[0184] It is understood that the detection time after the start of motion trajectory data detection can be any duration between the start of motion trajectory data detection and the end of the current detection process. In this embodiment of the application, no specific limitation is made on the detection time after the start of motion trajectory data detection.

[0185] In one possible implementation, if the average distance of the motion trajectory data after the start of motion trajectory data detection in step S807 is greater than the first distance, it indicates that the target outside the door has moved a large range within the time period after the start of motion trajectory data detection. At this point, recording can be stopped to save power consumption of the smart lock, and a more accurate detection result can be obtained by detecting the presence of human features in the video data at the third detection time and in the preceding video data.

[0186] S808, determine if human features are present in the video data at and before time t6. If so, proceed to step S809 to confirm the existence of abnormal loitering. If not, proceed to step S810 to confirm the absence of abnormalities.

[0187] In one possible implementation, if it is determined in step S807 that the average distance of all motion trajectory data in the third detection cycle is less than or equal to the first distance, then it can be further determined whether the current detection process has ended. If it is confirmed that the current detection process has ended, a more accurate final detection result can be determined by further analyzing whether there are faces or human figures in the video data at the time of the third detection and before. If it is confirmed that the current detection process has not ended, then it can continue to step S809 to continue acquiring motion trajectory data after the time when the motion trajectory data detection started, thereby acquiring as much motion trajectory data after the current time as possible to improve detection accuracy.

[0188] S809, an abnormal stay has been confirmed.

[0189] It is understandable that if in step S807 it is determined that the average distance of all motion trajectory data of the third detection cycle is greater than the first distance, and it is determined that a face or human figure is identified in the video data at time t6 and before, then it can be determined that there is a high probability of abnormal loitering.

[0190] S810, confirming that there are no abnormalities.

[0191] It is understandable that if in step S807 it is determined that the average distance of all motion trajectory data in the third detection cycle is greater than the first distance, and no face or human figure is identified in the video data at time t6 and before, then it can be determined that there is a high probability that there is no abnormality.

[0192] S811, determine whether the detection process has ended. If yes, proceed to step S812 to determine the video at time t7 and before. If no, proceed to step S806 to obtain the motion trajectory data after the time when the motion trajectory data detection started.

[0193] Here, time t7 is the moment when the video data is analyzed again after step S811.

[0194] In one possible implementation, if it is determined in step S807 that the average distance of all motion trajectory data in the third detection cycle is less than or equal to the first distance, then it can be further determined whether the current detection process has ended. If it is confirmed that the current detection process has not ended, then it can continue to step S806 to obtain motion trajectory data after the start time of motion trajectory data detection, thereby obtaining as much motion trajectory data after the current time as possible to improve detection accuracy.

[0195] S812, determine if human features are present in the video data at and before time t7. If so, proceed to step S813 to confirm the presence of abnormal loitering. If not, proceed to step S814 to confirm the absence of abnormalities.

[0196] It is understandable that if the current detection process is confirmed to be over in step S811, a more accurate final detection result can be determined by further analyzing whether there are faces or human figures in the video data at the time of the third detection and in the video data before.

[0197] S813, an abnormal stay has been identified.

[0198] It is understandable that if a face or human figure is identified in the video data at time t7 and earlier, then it is highly likely that there is an abnormal loitering situation.

[0199] S814, confirming that there are no abnormal situations.

[0200] It is understandable that if no face or human figure is identified in the video data at time t7 and before, then it is highly likely that there is no abnormality.

[0201] The above method, upon determining that no face or human figure was detected in a certain video detection stage, further obtains the final detection result by analyzing the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection, as well as the average distance of the motion trajectory data after the start time of motion trajectory data detection. Before determining the final detection result, it analyzes the presence of human features in the video data at the third detection time and before to determine a more accurate final detection result. Furthermore, if the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than a variance threshold, or if the average distance of the motion trajectory data after the start time of motion trajectory data detection is greater than a first distance, the camera recording process is stopped, which can save power consumption of the smart lock and improve the user experience. Moreover, if the average distance of the motion trajectory data after the start time of motion trajectory data detection is less than or equal to the first distance, it will further determine whether the current detection process has ended. If it is determined that the detection has not ended, motion trajectory data will continue to be acquired to obtain a more accurate detection result through more motion trajectory data.

[0202] Figure 9 This paper demonstrates a method for obtaining the final detection result by using motion trajectory data, etc., after determining that no face or human figure was detected in a certain video detection stage. This method is similar to... Figure 7 The difference between the two methods is that, after determining that no face or human figure was detected in a certain video detection stage, and that the average distance in the motion trajectory data from the start of the detection process to the start of motion trajectory data detection is less than or equal to the first distance, and that the variance of the average distance in the motion trajectory data from the start of the detection process to the start of motion trajectory data detection is less than or equal to the variance threshold, more precise anomalies are determined by using motion trajectory data after the start of motion trajectory data detection.

[0203] The specific steps include:

[0204] S901, acquire motion trajectory data from the start of the detection process to the moment when motion trajectory data detection begins.

[0205] S902, determine whether the average distance between the start of the detection process and the start of motion trajectory data detection is greater than the first distance. If yes, proceed to step S903, determine whether there are human features in the video data at time t8 and before. If no, proceed to step S906, determine whether there are human features in the video data at time t8 and before.

[0206] Here, time t8 is the moment after step S902 when the video data is analyzed again.

[0207] S903, determine whether there are human features in the video data at and before time t8. If yes, proceed to step S904 to confirm that someone passed by. If no, proceed to step S905 to confirm that there are no abnormalities.

[0208] S904, it has been confirmed that someone passed by.

[0209] S905, confirming that there are no abnormalities.

[0210] S906, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than the variance threshold. If yes, proceed to step S907, determine whether there are human features in the video data at and before time t9. If no, proceed to step S910, obtain the motion trajectory data after the start time of motion trajectory data detection.

[0211] Here, time t9 is the moment when the video data is analyzed again after step S906.

[0212] S907, determine whether there are human features in the video data at and before time t9. If yes, proceed to step S908 to confirm that someone has passed by. If no, proceed to step S909 to confirm that there are no abnormalities.

[0213] S908, it has been confirmed that someone passed by.

[0214] S909, confirming that there are no abnormalities.

[0215] It is understood that steps S901 to S909 correspond to steps S701 to S709, and will not be repeated here.

[0216] S910: Obtain motion trajectory data after the start of motion trajectory data detection.

[0217] S911, determine whether the average distance of the motion trajectory data after the start time of motion trajectory data detection is greater than the first distance. If so, proceed to step S912 to determine whether human features exist in the video data at and before time t10. Alternatively, proceed to step S915 to determine whether the detection process has ended.

[0218] Wherein, time t10 is the time when the video data is analyzed again after step S911.

[0219] S912, determine if human features are present in the video data at and before time t10. If yes, proceed to step S913 to confirm the presence of abnormal loitering. If no, proceed to step S914 to confirm the absence of abnormalities.

[0220] S913, an abnormal stay has been confirmed.

[0221] S914, confirming that there are no abnormalities.

[0222] S915, determine whether the detection process has ended. If yes, proceed to step S916 to determine the video at time t11 and before. If no, proceed to step S910 to obtain the motion trajectory data after the time when the motion trajectory data detection started.

[0223] Here, time t11 is the moment when the video data is analyzed again after step S915.

[0224] S916, determine if human features are present in the video data at and before time t11. If yes, proceed to step S917 to confirm the existence of abnormal loitering. If no, proceed to step S918 to confirm the absence of abnormalities.

[0225] S917, an abnormal stay has been confirmed.

[0226] S918, confirming that there are no abnormalities.

[0227] It is understood that steps S910 to S918 correspond to the same steps S806 to S814, and will not be repeated here.

[0228] The above method, upon determining that no face or human figure was detected during a particular video detection phase, further analyzes the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection, the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection, and the average distance of the motion trajectory data after the start of motion trajectory data detection to obtain the final detection result. Before determining the final detection result, a more accurate final detection result is obtained by detecting human features in the video data before and after the third detection time. Furthermore, the camera recording process is stopped when the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is determined to be greater than a first distance, or when the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection is determined to be greater than a variance threshold, or when the average distance of the motion trajectory data after the start of motion trajectory data detection is determined to be greater than the first distance, thus saving power consumption of the smart lock. Furthermore, after determining that the average distance of the motion trajectory data after the start of motion trajectory data detection is less than or equal to the first distance, it will further determine whether the current detection process has ended. If it is determined that the detection has not ended, it will continue to acquire motion trajectory data in order to obtain more accurate detection results through more motion trajectory data.

[0229] Figure 10This application illustrates another method for sending notifications in case of an abnormal situation with a smart door lock, as shown in an embodiment of this application. Figure 10 As shown, a method for sending notifications about abnormal situations in a smart door lock may include the following steps:

[0230] S1001, a trigger event for the detection process has been detected.

[0231] S1002, Based on the video data of the current video detection stage, confirm whether human features exist.

[0232] It is understood that steps S1001 and S1002 correspond to the same steps S401 and S402, and will not be repeated here.

[0233] S1003, confirm whether the detection process has ended. If yes, proceed to step S1004, confirm the existence of abnormal loitering, and push an abnormal loitering message. If no, proceed to step S1002, confirm the presence of human features based on the video data of the current video detection stage.

[0234] In some possible implementations, if a face or human figure is detected in the video data within the current video detection phase in step S1002, it can be determined whether the detection process has ended. If it is determined that the detection process has not ended, the next video detection phase can begin, meaning the camera's recording program continues until the detection process ends. It can be understood that if the detection process has not ended, the video detection phase can be repeated to make the actual detection time more consistent with the set detection process duration, thus improving the user experience.

[0235] S1004, an abnormal stay has been confirmed, and an abnormal stay message is pushed.

[0236] It is understandable that if human features are confirmed at a certain video detection stage and the detection process ends, then an abnormal loitering situation can be identified, and an abnormal loitering message will be pushed.

[0237] S1005, confirm whether the detection process is complete. If so, proceed to step S1006 to determine whether human features exist in the video data at time t12 and earlier.

[0238] Here, time t12 is the time when the video data is analyzed again after step S1005.

[0239] In one possible implementation, if step S1002 determines that no face or human figure is present in the video data of the current video detection stage, it can be further determined whether the detection process has ended. Upon determining that the detection process has ended, it can be further determined whether any face or human figure exists in all the video data from the detection process, thereby identifying any abnormalities in the detection process.

[0240] S1006, determine whether there are human features in the video data at and before time t12. If yes, proceed to step S1007, confirm the existence of abnormal loitering, and push an abnormal loitering message. If no, proceed to step S1008, confirm that there is no abnormality, and do not push any message.

[0241] S1007, an abnormal stay has been confirmed, and an abnormal stay message is pushed.

[0242] S1008, after confirming that there are no abnormalities, no messages will be pushed.

[0243] S1009, Obtain motion trajectory data from the start of the detection process to the moment when motion trajectory data detection begins.

[0244] S1010, determine whether the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than a first distance. If yes, proceed to step S1011, determine whether there are human features in the video data at and before time t13. If no, proceed to step S1014, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than a variance threshold.

[0245] Wherein, time t13 is the time when the video data is analyzed again after step S1010.

[0246] S1011, determine whether there are human features in the video data at and before time t13. If yes, proceed to step S1012, confirm that someone has passed by, and push a message indicating that someone has passed by. If no, proceed to step S1013, confirm that there are no abnormalities, and do not push any messages.

[0247] S1012, confirming that someone has passed by, push a message indicating that someone has passed by.

[0248] S1013, confirming there are no abnormalities, do not push any messages.

[0249] S1014, determine whether the variance of the average distance of the motion trajectory data between the start of the detection process and the start time of motion trajectory data detection is greater than the variance threshold. If yes, proceed to step S1015, determine whether there are human features in the video data at and before time t14. If no, proceed to step S1018, obtain the motion trajectory data after the start time of motion trajectory data detection.

[0250] Here, time t14 is the moment when the video data is analyzed again after step S1014.

[0251] S1015, determine whether there are human features in the video data at and before time t14. If yes, proceed to step S1016, confirm that someone has passed by, and push a message indicating that someone has passed by. If no, proceed to step S1017, confirm that there are no abnormalities, and do not push any messages.

[0252] S1016, confirming that someone has passed by, push a message indicating that someone has passed by.

[0253] S1017, after confirming that there are no abnormalities, no messages will be pushed.

[0254] S1018, Obtain motion trajectory data after the start of motion trajectory data detection.

[0255] S1019: Determine whether the average distance of the motion trajectory data after the start time of motion trajectory data detection is greater than the first distance. If yes, proceed to step S1020: Determine whether there are human features in the video data at and before time t15, confirm the existence of abnormal loitering, and push an abnormal loitering message. If no, proceed to step S1023: Confirm whether the detection process has ended.

[0256] Here, time t15 is the moment when the video data is analyzed again after step S1019.

[0257] S1020: Determine if human features are present in the video data at time t15 and earlier, confirming an abnormal loitering situation, and push an abnormal loitering message. If yes, proceed to step S1021, confirming an abnormal loitering situation, and push an abnormal loitering message. If no, proceed to step S1022, confirming no abnormal situation, and do not push any message.

[0258] S1021, an abnormal stay has been confirmed, and an abnormal stay message is pushed.

[0259] S1022, after confirming that there are no abnormalities, no messages will be pushed.

[0260] S1023, Confirm whether the detection process has ended. If yes, proceed to step S1024 to determine whether human features exist in the video data at and before time t16. If no, proceed to step S1018 to obtain motion trajectory data after the start time of motion trajectory data detection.

[0261] Here, time t16 is the moment when the video data is analyzed again after step S1023.

[0262] S1024, determine whether there are human features in the video data at and before time t16. If yes, proceed to step S1025, confirm the existence of abnormal loitering, and push an abnormal loitering message. If no, proceed to step S1026, confirm that there is no abnormality, and do not push any message.

[0263] In one possible implementation, if, in step S1016, it is determined that the average distance of the motion trajectory data after the start of motion trajectory data detection is less than or equal to the first distance, then it can be further determined whether the current detection process has ended. If it is confirmed that the current detection process has ended, then a more accurate final detection result can be determined by further analyzing whether there are faces or human figures in the video data before the third detection time. If it is confirmed that the current detection process has not ended, then it can continue to step S1015 to continue acquiring motion trajectory data after the start of motion trajectory data detection, thereby acquiring as much motion trajectory data after the current time as possible to improve detection accuracy.

[0264] S1025, an abnormal stay has been confirmed, and an abnormal stay message is pushed.

[0265] S1026, after confirming that there are no abnormalities, no messages will be pushed.

[0266] It is understood that steps S1009 to S1026 are similar to steps S918 of S901. The difference lies in the fact that in steps S1009 to S1026, if the final abnormal situation is confirmed, the corresponding abnormal situation message will be pushed. This will not be elaborated here.

[0267] The aforementioned abnormal loitering detection method uses ToF ranging technology combined with AI face and human detection technology to identify abnormal situations outside the door. It can identify both abnormal loitering and human passage, improving the detection accuracy compared to using only a single detection technology. Furthermore, it can stop the detection process after determining the detection result, saving power for the smart lock and improving the user experience. Before determining the final detection result, it uses the presence of human features in previous video data to determine a more accurate final result. Additionally, the camera recording process is stopped when the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection exceeds a first distance, or when the variance of the average distance of the motion trajectory data between the start of the detection process and the start of motion trajectory data detection exceeds a variance threshold, or when the average distance of the motion trajectory data after the start of motion trajectory data detection exceeds a first distance, further saving power for the smart lock. Furthermore, after determining that the average distance of the motion trajectory data after the start of motion trajectory data detection is less than or equal to the first distance, it will further determine whether the current detection process has ended. If it is determined that the detection has not ended, it will continue to acquire motion trajectory data in order to obtain more accurate detection results through more motion trajectory data.

[0268] As an example rather than a limitation, Figure 12 The diagram illustrates the relationship between the functional modules of an abnormal loitering detection system for smart door locks. (For example...) Figure 12 As shown, the smart door lock may include an image acquisition module 1140, a time-of-flight (ToF) ranging module, an AI recognition module 1150, a motion detection module 1160, and a joint judgment module. The image acquisition module 1140 includes a camera that captures video of the area in front of the door, and a processor that executes the method for pushing notifications of abnormal situations of the smart door lock as described in this application. The ToF ranging module 1130 can extract the variance features of the distance an object moves in front of the door within a set time period based on the video data of the area in front of the door, and determine the range and speed of the object's movement. The AI ​​recognition module 1150 can identify whether a face or human figure exists in front of the door based on the video data of the area in front of the door. The motion detection module 1160 can perform differential calculations based on a built-in algorithm using a set background image frame and image frames in the video of the area in front of the door to determine whether an object is moving in front of the door. Finally, the joint judgment module can comprehensively determine abnormal situations in the video captured by the camera in front of the door based on the analysis results of the ToF ranging module 1130, the AI ​​recognition module 1150, and the motion detection module 1160, such as determining whether someone is lingering abnormally, whether someone is passing by, and whether the image acquisition module 1140 has stopped capturing video.

[0269] The following is combined with Figure 13 right Figure 1 The hardware structure of the smart door lock 200 in the scenario shown is described as an example.

[0270] like Figure 13 As shown, the smart door lock 200 may include a processor 110, an external memory interface 120, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 130, a wireless communication module 140, a sensor module 150, a camera 160, a display screen 170, etc. The sensor module 150 may include a pressure sensor 150A, a gyroscope sensor 150B, an accelerometer sensor 150C, a proximity sensor 150D, a fingerprint sensor 150E, a touch sensor 150F, etc.

[0271] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the smart lock 200. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. Processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has recently used or is repeatedly used. If processor 110 needs to reuse this instruction or data, it can directly retrieve it from memory. This avoids repeated access, reduces the waiting time of processor 110, and thus improves system efficiency. The wireless communication function of the smart door lock 200 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the smart door lock 200 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0272] The mobile communication module 130 can provide wireless communication solutions including 2G / 3G / 4G / 5G for use in the smart door lock 200. The mobile communication module 130 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 130 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 130 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 130 and at least some modules of the processor 110 may be housed in the same device.

[0273] The wireless communication module 140 can provide solutions for wireless communication applications on the smart door lock 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), NFC, infrared (IR) technology, etc.

[0274] The wireless communication module 140 may be one or more devices integrating at least one communication processing module. The wireless communication module 140 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 140 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. Exemplarily, in this embodiment, different electronic devices can establish communication connections via BT or WLAN.

[0275] In some embodiments, antenna 1 of the smart lock 200 is coupled to mobile communication module 130, and antenna 2 is coupled to wireless communication module 140, enabling the smart lock 200 to communicate with networks and other devices via wireless communication technology. Wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TDSCDMA), Long Term Evolution (LTE), the fifth generation (5G) mobile communication system, future communication systems such as the sixth generation (6G) system, BT, GNSS, WLAN, NFC, FM and / or IR technologies, etc. GNSS can include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0276] The display screen 170 is used to display the application's interface, etc. The display screen 170 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the smart door lock 200 may include one or N display screens 170, where N is a positive integer greater than 1. In this embodiment, the display screen 170 can be used to simultaneously display multiple application interfaces.

[0277] Camera 160 is used to capture still images or videos. Camera 160 may include a front-facing camera and a rear-facing camera.

[0278] The internal memory 121 can be used to store computer executable program code, including instructions. The processor 110 executes various functional applications and data processing of the smart lock 200 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and software code for at least one application. The data storage area may store data generated during the use of the smart lock 200 (such as images, videos, etc.). Furthermore, the internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0279] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the smart lock 200. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, images, videos, and other files can be saved on the external memory card.

[0280] Pressure sensor 150A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, pressure sensor 150A may be disposed on display screen 170. Gyroscope sensor 150B can be used to determine the motion posture of electronic device 100. In some embodiments, the angular velocity of smart door lock 200 about three axes (i.e., x, y, and z axes) can be determined by gyroscope sensor 150B.

[0281] The gyroscope sensor 150B can be used to determine the motion posture of the smart lock 200. In some embodiments, the gyroscope sensor 150B can determine the angular velocity of the smart lock 200 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 150B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 150B detects the angle of the smart lock 200's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the smart lock 200 through reverse movement, thus achieving image stabilization. The gyroscope sensor 150B can also be used in navigation and motion-sensing game scenarios.

[0282] The accelerometer 150C can detect the magnitude of acceleration of the smart lock 200 in various directions (generally three axes). When the smart lock 200 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices, and is applied to applications such as screen orientation switching and pedometers.

[0283] A distance sensor 150D is used to measure distance. The smart door lock 200 can measure distance using infrared or laser. In some embodiments, the smart door lock 200 can utilize the distance sensor 150D to acquire data on the movement distance of moving objects in the monitored area outside the door.

[0284] The fingerprint sensor 150E is used to collect fingerprints. The smart door lock 200 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, access to the lock app, fingerprint photography, fingerprint answering of calls, etc.

[0285] Touch sensor 150F, also known as a "touch panel," can be located on display screen 170. The touch sensor 150F and display screen 170 together form a touchscreen, also known as a "touch screen." Touch sensor 150F detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 170. In other embodiments, touch sensor 150F may also be located on the surface of smart lock 200, in a different position than display screen 170.

[0286] Understandable, Figure 12The components shown do not constitute a specific limitation on the smart lock 200. The smart lock 200 may also include more or fewer components than shown, or combine some components, or split some components, or have different component arrangements.

[0287] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0288] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0289] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the aforementioned element.

[0290] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A method for sending abnormal notifications for a smart door lock, characterized in that, include: Perform image feature detection, wherein the image feature detection includes determining that no human features are present in the acquired first video data; When the image feature detection satisfies the first image condition, motion trajectory detection is performed, wherein the first image condition includes: no human features are detected in the first video data, and the motion trajectory detection includes determining the existence of abnormal movement based on the motion trajectory data; When the motion trajectory detection satisfies the first trajectory condition, an abnormal event is pushed out, wherein the first trajectory condition includes: determining that there is an abnormal movement based on the motion trajectory data.

2. The method according to claim 1, characterized in that, Based on the motion trajectory data, it is determined that there is abnormal movement, including: Based on the average distance and / or variance of the motion trajectory data, it is determined that there is an abnormal moving target; Based on the existence of the abnormally moving target, an abnormal movement situation is determined.

3. The method according to claim 2, characterized in that, Determining the existence of abnormal movement based on the motion trajectory data also includes: Based on the second video data, it is determined that the abnormal moving target includes human features, wherein the second video data includes video data collected from the start time of the anomaly detection process to the current time. The presence of abnormal movement is determined by identifying human characteristics corresponding to the abnormal moving target.

4. The method according to claim 3, characterized in that, The abnormal movement target includes human characteristics, and determining the existence of abnormal movement includes: The abnormally moving target is identified as an abnormally loitering target, and based on the second video data, the abnormally loitering target is determined to include human features, thus confirming the existence of an abnormal loitering situation. The abnormal moving target is identified as a passing target, and based on the second video data, the passing target is determined to include human features, thus confirming that someone has passed by.

5. The method according to claim 2, characterized in that, The presence of an abnormally moving target can be confirmed using the following methods: Acquire first motion trajectory data, and divide the first motion trajectory data into multiple first detection cycle motion trajectory data according to the acquisition time. The first motion trajectory data is the motion trajectory data collected from the start time of the anomaly detection process to the start time of motion trajectory detection. Calculate the average distance of the motion trajectory data for each of the first detection cycles; The average distance of the motion trajectory data corresponding to each of the first detection cycles all meet the first distance condition, confirming the existence of an abnormally loitering target.

6. The method according to claim 5, characterized in that, The method further includes: The variance of the average distance corresponding to the motion trajectory data of each of the first detection cycles satisfies the first variance condition, confirming the existence of an abnormally loitering target.

7. The method according to claim 5, characterized in that, The method further includes: If the average distance of the motion trajectory data corresponding to each of the first detection cycles meets the first distance condition, and the variance of the average distance meets the first variance condition, it is confirmed that there is an abnormally loitering target.

8. The method according to claim 5, characterized in that, The method further includes: The average distance of the motion trajectory data corresponding to each of the first detection cycles does not meet the first distance condition, confirming that a target has passed through.

9. The method according to claim 5, characterized in that, The method further includes: The variance of the average distance of the motion trajectory data corresponding to each of the first detection cycles does not meet the first variance condition, confirming that the target has passed through.

10. The method according to claim 5, characterized in that, The method further includes: If the average distance of the motion trajectory data corresponding to each of the first detection cycles does not meet the first distance condition, and the variance of the average distance does not meet the first variance condition, it is confirmed that the target has passed through.

11. The method according to claim 1, characterized in that, When the motion trajectory detection satisfies the first trajectory condition, the abnormal event push is executed, including: Corresponding to the motion trajectory detection determining an abnormal stay, an abnormal stay message is pushed; or If the motion trajectory detection determines that someone has passed by, a message indicating that someone has passed by is pushed out.

12. The method according to claim 1, characterized in that, Also includes: Obtain the preset time for the anomaly detection process; If a human feature is detected in the first video data, and the anomaly detection process is determined to be incomplete according to the preset time, image feature detection corresponding to the second video data is performed.

13. The method according to claim 12, characterized in that, The step of performing motion trajectory detection corresponding to the image feature detection satisfying the first image condition includes: If the image feature detection satisfies the first image condition, and the anomaly detection process is determined not to have ended according to the preset time, motion trajectory detection is performed.

14. The method according to claim 12, characterized in that, The image feature detection further includes: determining that the differences between each image frame and the background frame in the acquired first video data do not meet the difference condition; The first image condition further includes: no human features are detected in the first video data, and the differences between each image frame and the background frame in the first video data do not meet the difference condition.

15. A computer storage medium, characterized in that, The storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 14.

16. An electronic device, characterized in that, include: Memory is used to store instructions executed by one or more processors of an electronic device; And a processor, one of the processors of the electronic device, for executing instructions stored in the memory to implement the method of any one of claims 1 to 14.

Images