Smart doorbell monitoring method, device, equipment and storage medium

Abstract

The present invention relates to a monitoring method for a smart doorbell, comprising the following steps: obtaining a video captured by a camera of the doorbell, and identifying an object in the captured video to obtain an identified object video; eliminating pets in the identified object video to obtain a human identified object video, if the object in the human identified object video does not match an object preset in a graph knowledge base, extracting spatial parameters of the object in the human identified object video to obtain human behavior features; judging whether the human behavior features have abnormal behavior features; if there are potential intrusion behavior features, performing in-depth analysis on the potential intrusion behavior features through a preset human posture estimation algorithm to assist in judging human intentions and behaviors to obtain in-depth analysis results; if the in-depth analysis result shows that there are dangerous behaviors, pushing the video corresponding to the dangerous behaviors to a user's terminal; ensuring that the user can quickly obtain potential risk information and improve the level of home safety.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security technology, and in particular to a monitoring method, device, equipment and storage medium for an intelligent doorbell. Background Art

[0002] With the rapid development of smart home technology, people's requirements for home safety and convenience are constantly increasing. As an important part of the smart home system, the main function of the smart doorbell is to quickly identify visitors and convey relevant information to family members when someone approaches or presses the doorbell, thereby increasing the safety and convenience of the family. However, in the face of complex and changing life scenes, traditional smart doorbell systems can only provide basic visitor notification functions and cannot well identify specific behavior patterns, such as potential intrusions or emergency situations such as accidental falls, which limits the application effect of smart doorbells in the field of home security. Summary of the invention

[0003] The main purpose of the present invention is to provide a monitoring method, device, equipment and storage medium for a smart doorbell, which uses a preset human posture estimation algorithm to deeply analyze behavioral characteristics with potential risks to identify specific human intentions and behaviors.

[0004] To achieve the above object, the present invention provides a monitoring method for a smart doorbell, comprising the following steps:

[0005] Obtaining a video captured by a camera of the doorbell, and identifying an object in the captured video to obtain a video of the identified object; wherein the object includes humans and pets;

[0006] Eliminate pets from the object recognition video to obtain a human recognition object video, and determine whether the object in the human recognition object video matches the preset object in the graph knowledge base on the smart doorbell;

[0007] If the object in the human recognition object video does not match the preset object in the atlas knowledge base, the spatial parameters of the object in the human recognition object video are extracted to obtain human behavior features;

[0008] Determine whether the human behavior features have abnormal behavior features; wherein the abnormal behavior features include fall features or potential intrusion behavior features. If there are potential intrusion behavior features, perform in-depth analysis on the potential intrusion behavior features through a preset human posture estimation algorithm to assist in determining human intentions and behaviors and obtain in-depth analysis results;

[0009] If the in-depth analysis result shows that there is dangerous behavior, the video corresponding to the dangerous behavior will be pushed to the user's terminal.

[0010] As a further solution of the present invention, a video captured by a camera of a doorbell is obtained, and an object in the captured video is identified to obtain an object identification video, including:

[0011] The doorbell's camera captures the area in front of the door to obtain real-time video;

[0012] The real-time video is quality evaluated by a preset quality analysis algorithm to obtain a quality evaluation result; wherein the quality evaluation includes video resolution evaluation, video frame rate evaluation and video stability evaluation; the quality evaluation result includes low-level recognition quality, medium-level recognition quality and high-level recognition quality;

[0013] If it is determined that the quality assessment result is low-level recognition quality or medium-level recognition quality, quality mapping is performed on the quality assessment result to obtain a first abnormal parameter of the real-time video;

[0014] Adjusting the first abnormal parameter of the real-time video according to the parameter corresponding to the advanced recognition quality to obtain the adjusted parameter of the real-time video;

[0015] Based on the adjusted parameters of the real-time video, a video of the recognized object is obtained.

[0016] As a further solution of the present invention, the extraction of spatial parameters of objects in the human recognition object video to obtain human behavior features includes:

[0017] Extracting spatial parameters of the object in the human recognition object video to obtain corresponding object behavior parameters; wherein the spatial parameters include human coordinates, human angles, and the distance between the human and the doorbell;

[0018] Calculating the object behavior parameters to obtain a behavior characteristic curve; wherein the behavior characteristic curve includes a change in moving distance and a change in path;

[0019] Fitting the behavior characteristic curve to obtain a behavior characteristic fitting curve, and determining an inflection point of the behavior characteristic fitting curve and a slope corresponding to the inflection point position; wherein the inflection point represents a change or an important event in the behavior, and the slope corresponding to the inflection point position represents a rate of change in the behavior of the object;

[0020] The inflection point and the slope corresponding to the inflection point position are analyzed through a preset behavior characteristic analysis model to obtain human behavior characteristics.

[0021] As a further solution of the present invention, the human behavior characteristics include human posture characteristics, action duration characteristics and relative movement characteristics between body parts.

[0022] As a further solution of the present invention, the potential intrusion behavior characteristics are deeply analyzed by a preset human posture estimation algorithm to assist in judging human intentions and behaviors, and obtain a deep analysis result, including

[0023] Performing vector conversion on the potential intrusion behavior feature through a preset human posture estimation algorithm to obtain an intrusion behavior feature vector;

[0024] The intrusion behavior feature vector is feature-weighted by a weight layer in a preset human posture estimation algorithm to obtain a key feature vector;

[0025] Introducing a feedback mechanism into a preset human posture estimation algorithm, dynamically adjusting the key feature vector through the feedback mechanism to obtain an adjustment vector; wherein the feedback mechanism has a pre-input standard key feature vector;

[0026] Transposing the adjustment vector to obtain a transposed vector;

[0027] constructing an intrusion behavior matrix based on the transposed vector;

[0028] The intrusion behavior matrix is ​​deeply analyzed through the analysis layer in the preset human posture estimation algorithm to obtain a deep analysis result.

[0029] As a further solution of the present invention, the training process of the preset behavior feature analysis model includes:

[0030] Obtaining behavior characteristic parameters, and inputting the behavior characteristic parameters into a preliminary behavior characteristic analysis model; wherein the behavior characteristic parameters include inflection point training parameters and slope training parameters corresponding to the inflection point position; the preliminary behavior characteristic analysis model includes a preliminary inflection point model and a slope model corresponding to the preliminary inflection point position;

[0031] Obtaining actual labels pre-calibrated in the behavior characteristic parameters; wherein the actual labels include inflection point analysis results corresponding to the inflection point training parameters and slope analysis results corresponding to the slope;

[0032] Inputting the inflection point training parameters into the preliminary inflection point model to obtain a first prediction analysis result;

[0033] Inputting the slope training parameter corresponding to the inflection point position into the slope model corresponding to the preliminary inflection point position to obtain a second prediction analysis result;

[0034] Comparing the inflection point analysis result with the first prediction analysis result to obtain an inflection point loss value;

[0035] Comparing the slope analysis result with the second prediction analysis result to obtain a slope loss value;

[0036] The model parameters of the preliminary inflection point model and the model parameters of the slope model corresponding to the preliminary inflection point position are iteratively adjusted, and the inflection point loss value and the slope loss value are minimized through loss-based optimization to obtain the trained behavior feature analysis model.

[0037] As a further solution of the present invention, the behavior characteristic curve is calculated using the following formula, including:

[0038] ;

[0039] in, A time function representing the angular parameter of human orientation; The time function representing the distance from the human to the doorbell, Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents the position coordinates of human objects Over time The derivative of , i.e. the velocity component; Indicates the rate of change of human heading angle over time; is the speed weight; is the human angular velocity weight, is the distance weight from human to doorbell; represents the attenuation constant; represents the initial time point; represents the base of natural logarithms; Represents the behavioral characteristic curve.

[0040] The present invention also provides a monitoring device for a smart doorbell, comprising:

[0041] An acquisition module is used to acquire a video captured by a camera of the doorbell, and identify an object in the captured video to obtain a video of the identified object; wherein the object includes humans and pets;

[0042] The recognition module is used to remove pets from the recognition object video to obtain the human recognition object video, and determine whether the object in the human recognition object video matches the preset object in the atlas knowledge base on the smart doorbell;

[0043] A matching module is used to extract spatial parameters of the object in the human recognition object video to obtain human behavior features if the object in the human recognition object video does not match the object preset in the atlas knowledge base;

[0044] A judgment module, used to judge whether the human behavior characteristics have abnormal behavior characteristics; wherein the abnormal behavior characteristics include fall characteristics or potential intrusion behavior characteristics. If there are potential intrusion behavior characteristics, the potential intrusion behavior characteristics are deeply analyzed by a preset human posture estimation algorithm to assist in judging human intentions and behaviors and obtain a deep analysis result;

[0045] The analysis module is used to push the video corresponding to the dangerous behavior to the user's terminal if the deep analysis result shows that there is dangerous behavior.

[0046] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0047] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0048] The monitoring method, device, computer equipment and storage medium of the smart doorbell provided by the present invention include the following steps: obtaining a video captured by a camera of the doorbell, and identifying objects in the captured video to obtain an identified object video; wherein the objects include humans and pets; removing pets from the identified object video to obtain a human identified object video, and judging whether the object in the human identified object video matches the preset object in the graph knowledge base on the smart doorbell; if the object in the human identified object video does not match the preset object in the graph knowledge base, extracting spatial parameters of the object in the human identified object video to obtain human behavior features; judging whether the human behavior features are whether there are abnormal behavior characteristics; wherein, the abnormal behavior characteristics include fall characteristics or potential intrusion behavior characteristics. If there are potential intrusion behavior characteristics, the potential intrusion behavior characteristics are deeply analyzed through a preset human posture estimation algorithm to assist in judging human intentions and behaviors, and obtain a deep analysis result; if the deep analysis result is that there are dangerous behaviors, the video corresponding to the dangerous behavior is pushed to the user's terminal; through the deep behavior analysis technology, potential intrusion behaviors or emergencies such as falls can be identified in a timely and accurate manner, which solves the problem that traditional smart doorbells cannot well identify specific behavior patterns, and enables users to quickly obtain potential risk information and improve home safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the steps of a monitoring method of a smart doorbell in one embodiment of the present invention;

[0050] Figure 2It is a structural block diagram of a monitoring device of a smart doorbell in one embodiment of the present invention;

[0051] Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] like Figure 1 As shown, Figure 1 It is a schematic diagram of the steps of a monitoring method for a smart doorbell in one embodiment of the present invention;

[0055] An embodiment of the present invention provides a method for monitoring a smart doorbell, comprising the following steps:

[0056] Step S1, obtaining a video captured by a camera of a doorbell, and identifying an object in the captured video to obtain a video of the identified object; wherein the object includes humans and pets.

[0057] Specifically, choose and install a smart doorbell with a high-resolution camera, night vision, and motion detection. Set the video capture parameters, including video quality, capture interval, monitoring time, etc. through the smart doorbell's application or software interface. Ensure that the video data captured by the camera is transmitted in an encrypted manner to prevent the data from being intercepted or tampered with during transmission. The captured video data can be uploaded to a cloud server for storage or stored in a local memory device, depending on actual needs and the function of the doorbell. Use an artificial intelligence algorithm with object recognition capabilities, which can be a deep learning model such as a convolutional neural network (CNN). If it is a custom recognition system, the model needs to be trained and a large amount of image data of humans and pets needs to be provided for learning and recognition. The intelligent analysis system analyzes the video stream in real time when the video communication arrives, and uses the AI ​​model to identify humans and pets in the video. When the recognition system finds the target object, it can notify the homeowner through the app, and can also make specific responses to specific objects, such as automatically unlocking the door lock when it is recognized as a family member.

[0058] The above steps can bring the following technical effects: Automatic identification of people and animals at the door is of great significance for preventing strangers from invading. Through automatic identification, users can quickly understand who is outside the door, improving the convenience of daily life. For suspicious behavior or unauthorized intrusion, real-time alarms can be issued to improve the level of home security protection. Identifying pets and recording their activities is helpful for pet management and prevention of loss. Through such steps, not only the safety and convenience of the family are improved, but also the development and application of smart home technology are promoted.

[0059] Step S2, remove the pets in the object recognition video to obtain the human recognition object video, and determine whether the object in the human recognition object video matches the preset object in the graph knowledge base on the smart doorbell.

[0060] Specifically, a video stream is analyzed using a trained deep learning model (such as a convolutional neural network (CNN)) that can recognize and distinguish different objects in the video, including humans and pets. The pets in the video are automatically identified by the algorithm, and this part of the data is removed, leaving only the video data of the human object. The smart doorbell system needs to have a built-in or cloud-based graph knowledge base to store the facial data or body feature data authorized by the user for identification. The facial or body features of the human are extracted from the video of the identified object, and these features are compared and matched with the data in the graph knowledge base. The system calculates the matching degree through the algorithm, and if the identified human object matches a preset object feature in the knowledge base, the identity of the object is determined. Once the system identifies and matches the object, it sends a notification to the user through the smart doorbell application or other contact methods (such as mobile phone text messages, emails, etc.). According to user settings, different actions can be performed based on different recognition results, such as automatically unlocking when it is identified as a family member, or recording a video and sending a warning when it is identified as a stranger.

[0061] The above steps can bring the following technical effects: It can effectively identify whether the visitor is a preset family member or authorized person, thereby increasing family safety. Through intelligent identification of visitors, users can more conveniently manage family visitors without having to confirm in person at the door. For family members, quick access unlocking can be achieved, improving the convenience and comfort of life. Through accurate identification, false alarms for pets or unrelated objects can be effectively reduced, improving the accuracy of the system and user experience. The above steps can not only effectively improve family safety, but also significantly improve the convenience and intelligence of daily life through intelligent identification technology.

[0062] Step S3: If the object in the human recognition object video does not match the preset object in the atlas knowledge base, spatial parameters of the object in the human recognition object video are extracted to obtain human behavior features.

[0063] Specifically, the human recognition object video is subjected to frame extraction, denoising and normalization processing to prepare for subsequent feature extraction. Using models such as convolutional neural networks (CNN), recurrent neural networks (RNN) or long short-term memory networks (LSTM), the above models can learn and extract spatial features and time series features from video frames. Human key points (such as hand and foot positions) and their motion trajectories are extracted from each frame, and spatial parameters of human posture and action, such as speed, direction, posture change, etc., are calculated. Based on the extracted spatial parameters, machine learning or deep learning algorithms are applied to analyze human behavior patterns, such as walking, running, knocking on the door, etc. Abnormal or suspicious behavior characteristics are identified by comparing normal behavior patterns with current behaviors. Based on the results of the behavior feature analysis, some predefined security measures can be automatically executed, such as activating an alarm system or notifying family members. Notifications about detected behaviors and possible abnormalities are sent to users through the smart doorbell application or other set channels.

[0064] The above steps can bring the following technical effects: By automatically extracting behavioral features and performing behavioral analysis, abnormal behavior can be identified more accurately, reducing the pressure of manual monitoring. Timely identification of suspicious behavior and taking measures greatly improves the security of home and property. Using advanced spatial parameter extraction and behavioral analysis, normal behavior and abnormal behavior can be distinguished more accurately, thereby reducing the false alarm rate. Users can not only receive timely security alerts, but also understand the events happening in front of the door in more detail through the behavioral analysis information provided by the system, improving the user experience. In summary, the above technical strategies can significantly enhance the functionality of the smart doorbell system, not only can it identify people, but also can intelligently analyze people's behavior, and improve the ability and efficiency of home security protection.

[0065] Step S4, determining whether the human behavior characteristics include abnormal behavior characteristics; wherein the abnormal behavior characteristics include fall characteristics or potential intrusion behavior characteristics. If there are potential intrusion behavior characteristics, a preset human posture estimation algorithm is used to perform an in-depth analysis of the potential intrusion behavior characteristics to assist in determining human intentions and behaviors and obtain in-depth analysis results.

[0066] Specifically, deep learning algorithms, such as long short-term memory (LSTM) networks or 3D convolutional neural networks, are applied to analyze the movement patterns and behavioral characteristics of people in the video. It is pre-defined what kind of behavior is abnormal, such as falling, wandering, climbing, etc. By comparing the behavioral characteristics with the preset abnormal behavior patterns, it is determined in real time whether there is abnormal behavior in the current video. Once abnormal behavior is detected, it is further determined whether the above behavior meets the characteristics of potential intrusion. Human posture estimation algorithms, such as OpenPose or AlphaPose, are applied to conduct a deeper analysis of the detected potential intrusion behavior. The above algorithms can identify the key posture nodes of people in the video (such as hands, feet, head, etc.). Based on the results of posture estimation, pattern recognition and behavior analysis techniques are used to infer the behavioral intentions of people, such as whether they attempt to forcibly intrude. Based on the results of deep analysis, the potential security risk level is evaluated. If the analysis results show high-risk intrusion behavior, an early warning will be automatically activated and the user or security personnel will be notified through an application, SMS or email. The abnormal event and its analysis results will be recorded for subsequent review by the user or security personnel or as a basis for countermeasures.

[0067] The above steps can bring the following technical effects: through in-depth behavior analysis, false alarms can be reduced to ensure that alarms are only issued when there is a real potential risk. Accurate abnormal behavior detection and intention analysis can enable security personnel to take countermeasures more quickly and improve security protection. Provide users with more intelligent and accurate security monitoring services and increase users' trust in smart home security systems. By continuously collecting and analyzing abnormal behavior data, the algorithm can be continuously optimized to improve the accuracy and response speed of future event judgments. In summary, it can not only effectively identify potential security threats, but also deeply analyze people's behavioral intentions, providing more comprehensive and accurate protection for home safety.

[0068] Step S5: If the in-depth analysis result shows that there is a dangerous behavior, the video corresponding to the dangerous behavior is pushed to the user's terminal.

[0069] Specifically, after the system determines that a certain behavior is dangerous through deep behavioral analysis, it will automatically mark the time period and related video content of the behavior in the background. A brief summary of the marked video content is made, and key frames are captured for quick preview. Only the video segments before and after the dangerous behavior are captured to reduce unnecessary information. The captured video is compressed to ensure that it can be quickly transmitted over the network without losing too much image quality. The video data is encrypted to ensure security during transmission.

[0070] Select push channel: You can push the video by email, mobile app notification or SMS with a link to download the video. Send a notification containing video content through the channel selected by the user. This notification should contain a brief description and a link to view the video or a direct video attachment. Provide direct feedback options in the push notification, such as calling the police directly, ignoring the alarm or viewing more details.

[0071] The above steps can bring the following technical effects: Users can more intuitively understand potential threats and respond quickly by receiving videos containing dangerous behaviors in real time. Timely video push allows users or security personnel to make decisions faster, such as contacting the police in time or taking measures to avoid or reduce losses. The pushed video can serve as important evidence at the time of the incident and provide support for possible accountability later. Timely response and effective information push make users feel at ease and increase their trust in the smart security system. In summary, the smart security monitoring system can not only detect and analyze possible security threats in advance, but also let users understand the situation in the most direct and effective way, so that they can take appropriate countermeasures and significantly improve the safety of their homes and property.

[0072] In a specific implementation, a video captured by a camera of a doorbell is obtained, and an object in the captured video is identified to obtain an object identification video, including:

[0073] The doorbell's camera captures the area in front of the door to obtain real-time video;

[0074] The real-time video is quality evaluated by a preset quality analysis algorithm to obtain a quality evaluation result; wherein the quality evaluation includes video resolution evaluation, video frame rate evaluation and video stability evaluation; the quality evaluation result includes low-level recognition quality, medium-level recognition quality and high-level recognition quality;

[0075] If it is determined that the quality assessment result is low-level recognition quality or medium-level recognition quality, quality mapping is performed on the quality assessment result to obtain a first abnormal parameter of the real-time video;

[0076] Adjusting the first abnormal parameter of the real-time video according to the parameter corresponding to the advanced recognition quality to obtain the adjusted parameter of the real-time video;

[0077] Based on the adjusted parameters of the real-time video, a video of the recognized object is obtained.

[0078] Specifically, the real-time video of the area in front of the door is captured by the camera of the smart doorbell. The real-time monitoring and the accuracy of the coverage area are ensured, providing raw data for subsequent analysis. The quality of the captured real-time video is evaluated using a preset quality analysis algorithm, including three main aspects: video resolution, frame rate, and video stability. The above steps ensure that the basic standards of video quality can meet the subsequent object recognition and analysis requirements. Ensure that the image clarity is sufficient to identify details; ensure that the video is smooth and can capture fast-moving events; reduce jitter and ensure that the video picture is stable enough for effective recognition. If the video is evaluated as low or medium recognition quality, the first abnormal parameters will be extracted and adjusted through quality mapping. For example, the intensity of digital signal processing can be increased to improve image clarity or reduce jitter. According to the high-level recognition quality requirements, the parameters of the real-time video are fine-tuned, including increasing resolution, stabilizing the video picture, and optimizing the frame rate, to obtain an improved video. Based on the adjusted video parameters, the object recognition process is executed. This may include using machine learning or deep learning models to identify dynamic or static objects such as people, vehicles, and pets in the video.

[0079] The above steps can bring the following technical effects: By ensuring that the video quality meets advanced recognition standards, the accuracy of the recognition algorithm can be greatly improved, both during the day and at night. The quality-optimized video is easier to process and analyze, so that the target object can be identified faster and potential security threats can be responded to in a timely manner. Users can receive clearer and more stable video images, and have a better visual experience whether it is used for daily monitoring or viewing recordings of specific events. By optimizing the video quality, the misidentification caused by blurry and jittery images is reduced, such as mistakenly misjudging the fluttering of leaves as intrusion. In general, the above process improves the overall performance and application value of the smart doorbell system through advanced quality assessment and optimization technology, making it more reliable and effective in actual monitoring scenarios.

[0080] In a specific implementation manner, extracting spatial parameters of objects in a human recognition object video to obtain human behavior features includes:

[0081] Extracting spatial parameters of the object in the human recognition object video to obtain corresponding object behavior parameters; wherein the spatial parameters include human coordinates, human angles, and the distance between the human and the doorbell;

[0082] Calculating the object behavior parameters to obtain a behavior characteristic curve; wherein the behavior characteristic curve includes a change in moving distance and a change in path;

[0083] Fitting the behavior characteristic curve to obtain a behavior characteristic fitting curve, and determining an inflection point of the behavior characteristic fitting curve and a slope corresponding to the inflection point position; wherein the inflection point represents a change or an important event in the behavior, and the slope corresponding to the inflection point position represents a rate of change in the behavior of the object;

[0084] The inflection point and the slope corresponding to the inflection point position are analyzed through a preset behavior characteristic analysis model to obtain human behavior characteristics.

[0085] Specifically, image recognition technology such as convolutional neural network (CNN) in deep learning is used to identify human objects in the video and extract spatial parameters. The above parameters include: human coordinates (position in the video frame), human angle (direction or posture relative to the camera or doorbell), and the distance from the human to the doorbell (obtained through depth perception technology). The extracted object behavior parameters are analyzed in time series to obtain a behavior feature curve. The feature curve shows: changes in movement distance (such as the movement distance of the human object over a period of time) and changes in path (such as changes in the movement path of the human object).

[0086] Perform mathematical fitting on the behavior characteristic curve, such as using polynomial fitting or curve fitting technology, to obtain the fitted curve of the behavior characteristic. Identify the inflection point in the fitted curve, which may indicate a change in behavior or an important event (such as from walking to running). Calculate the slope of the inflection point position to quantify the rate of change of the object's behavior. Use a preset behavior characteristic analysis model (which may be based on a machine learning algorithm) to analyze the inflection point and its corresponding slope. Through model analysis, comprehensively evaluate human behavior characteristics, such as identifying specific behavior patterns or abnormal behaviors.

[0087] The above steps can bring the following technical effects: Through the detailed analysis of spatial parameters, the system can more accurately determine when the behavior has changed abnormally, thereby reducing false alarms and missed alarms. Through in-depth analysis of behavioral characteristic curves, it is possible to understand the potential intentions behind the behavior, such as distinguishing between simple stops in front of the door and stays with potential security threats. Automated behavioral analysis reduces the need for manual monitoring and improves the efficiency and response speed of security monitoring. Users can obtain more accurate and timely security alerts, which improves their satisfaction and trust in the smart security system. Through such an implementation method, the smart doorbell or security monitoring system can achieve effective early warning of abnormal behavior through in-depth analysis of human objects in the video, greatly improving the safety of living and working environments.

[0088] In a specific implementation, the human behavior characteristics include human body posture characteristics, action duration characteristics, and relative movement characteristics between body parts.

[0089] Specifically, using human posture estimation technology, such as OpenPose, it is possible to identify key parts of the human body (such as head, shoulders, arms, knees, etc.) and postures in video images in real time. By analyzing posture changes, the system can identify specific behavior patterns, such as falling, climbing, etc. Combined with time series analysis methods, monitor the duration of specific actions or postures to determine whether they are beyond the normal range. For example, a person wandering or standing in a certain place for a long time may trigger a security alarm. Using vectorization methods, describe the relative movement between various parts of the human body, including speed, direction, and rate of change of movement. Application: By analyzing the movement of the arm relative to the torso, behavioral patterns such as raising hands and waving can be identified.

[0090] The above steps can bring the following technical effects: Combining the above detailed features, the recognition accuracy of behavior and the response sensitivity of the system can be improved, especially in identifying atypical behaviors (such as specific types of intrusions or emergency situations). By analyzing these behavioral characteristics in real time, potential abnormal events can be predicted, so as to achieve advance warnings and avoid possible hazards. Through an in-depth understanding of human posture and its changes, people's behavioral intentions in specific environments can be better understood, and the situational awareness of security monitoring systems can be improved. The analyzed behavioral characteristics can help formulate more targeted security measures or strategies, such as launching different response procedures for specific behavior patterns. In summary, by integrating human posture characteristics, action duration characteristics, and relative motion characteristics between body parts, the intelligent monitoring system's understanding of human behavior and the effectiveness of response strategies can be significantly improved, greatly enhancing the security performance of the monitoring system.

[0091] In a specific implementation, the potential intrusion behavior characteristics are deeply analyzed by a preset human posture estimation algorithm to assist in judging human intentions and behaviors, and obtain a deep analysis result, including

[0092] Performing vector conversion on the potential intrusion behavior feature through a preset human posture estimation algorithm to obtain an intrusion behavior feature vector;

[0093] The intrusion behavior feature vector is feature-weighted by a weight layer in a preset human posture estimation algorithm to obtain a key feature vector;

[0094] Introducing a feedback mechanism into a preset human posture estimation algorithm, dynamically adjusting the key feature vector through the feedback mechanism to obtain an adjustment vector; wherein the feedback mechanism has a pre-input standard key feature vector;

[0095] Transposing the adjustment vector to obtain a transposed vector;

[0096] constructing an intrusion behavior matrix based on the transposed vector;

[0097] The intrusion behavior matrix is ​​deeply analyzed through the analysis layer in the preset human posture estimation algorithm to obtain a deep analysis result.

[0098] Specifically, a preset human posture estimation algorithm (such as OpenPose or other deep learning models) is used to process the video data, identify the human posture therein and convert it into a feature vector. The above steps involve converting the position information of the key points of the human posture (such as hands, feet, torso, etc.) into a mathematical vector representation. The obtained intrusion behavior feature vector is weighted by the weight layer inside the posture estimation algorithm to highlight those features that are particularly important for the judgment of intrusion behavior and form a key feature vector. The above process is similar to the weight adjustment in the neural network, which is used to enhance the sensitivity of intrusion behavior recognition. The key feature vector is further adjusted by a preset feedback mechanism. This feedback mechanism is based on the learning of historical data and can automatically adjust according to the pre-input standard key feature vector to optimize the accuracy of the feature vector. The adjusted key feature vector is transposed to obtain a data structure suitable for subsequent analysis, and then an intrusion behavior matrix for specific human behavior analysis is constructed. Finally, the intrusion behavior matrix is ​​deeply analyzed by the analysis layer in the algorithm, and the specific behavior features are extracted and identified by combining machine learning models such as convolutional neural network (CNN) or recurrent neural network (RNN), so as to obtain in-depth analysis results for potential intrusion behaviors.

[0099] The above steps can bring the following technical effects: through the weighting of feature vectors and the adjustment of feedback mechanisms, it is possible to focus more accurately on the key features for intrusion behavior identification, thereby improving the accuracy of intrusion behavior identification. The introduced feedback mechanism enables the system to self-learn and adjust according to new data, thereby improving the accuracy and generalization of predictions of future intrusion behaviors. By constructing an intrusion behavior matrix and deep analysis, the system can perform more detailed and complex behavior analysis and identify more subtle differences in behavior patterns. By accurately identifying and predicting potential intrusion behaviors, the security protection capabilities of homes, public places and key facilities can be significantly improved, especially in the fields of intelligent monitoring, alarm systems and artificial intelligence security. In general, by deeply analyzing human postures and movements, the security monitoring system can identify potential threats more intelligently and proactively, greatly improving the efficiency and accuracy of protection.

[0100] In a specific implementation, the training process of the preset behavior feature analysis model includes:

[0101] Obtaining behavior characteristic parameters, and inputting the behavior characteristic parameters into a preliminary behavior characteristic analysis model; wherein the behavior characteristic parameters include inflection point training parameters and slope training parameters corresponding to the inflection point position; the preliminary behavior characteristic analysis model includes a preliminary inflection point model and a slope model corresponding to the preliminary inflection point position;

[0102] Obtaining actual labels pre-calibrated in the behavior characteristic parameters; wherein the actual labels include inflection point analysis results corresponding to the inflection point training parameters and slope analysis results corresponding to the slope;

[0103] Inputting the inflection point training parameters into the preliminary inflection point model to obtain a first prediction analysis result;

[0104] Inputting the slope training parameter corresponding to the inflection point position into the slope model corresponding to the preliminary inflection point position to obtain a second prediction analysis result;

[0105] Comparing the inflection point analysis result with the first prediction analysis result to obtain an inflection point loss value;

[0106] Comparing the slope analysis result with the second prediction analysis result to obtain a slope loss value;

[0107] The model parameters of the preliminary inflection point model and the model parameters of the slope model corresponding to the preliminary inflection point position are iteratively adjusted, and the inflection point loss value and the slope loss value are minimized through loss-based optimization to obtain the trained behavior feature analysis model.

[0108] Specifically, the behavior feature parameters are obtained, and the behavior feature parameters include inflection point training parameters (such as the time point and amplitude of the inflection point) and the slope training parameters corresponding to the inflection point position. A preliminary behavior feature analysis model is created, and the model consists of two sub-models: a preliminary inflection point model and a slope model corresponding to the preliminary inflection point position. The obtained behavior feature parameters are input into the two sub-models respectively, and the preliminary inflection point model receives the inflection point training parameters, and the slope model receives the slope training parameters corresponding to the inflection point position. The preliminary inflection point model outputs a first prediction analysis result, that is, the predicted inflection point position, according to the input inflection point training parameters. The slope model outputs a second prediction analysis result, that is, the predicted slope change, according to the input slope training parameters. The model prediction result is compared with the pre-calibrated actual label data, and the actual label contains the real inflection point analysis result and the slope analysis result. The difference between the first prediction analysis result (inflection point position) and the actual inflection point analysis result is calculated to obtain the inflection point loss value. The difference between the second prediction analysis result (slope) and the actual slope analysis result is calculated to obtain the slope loss value. Using gradient descent or similar optimization algorithms, the parameters of the preliminary inflection point model and the preliminary slope model are iteratively adjusted based on the inflection point loss value and the slope loss value. The goal is to continuously update the model parameters so that when the model processes new training samples, the predicted inflection point position and slope change are closer to the actual value, thereby reducing the loss value.

[0109] The above steps can bring the following technical effects: Through the above training process, the behavior feature analysis model can more accurately identify and predict the inflection point position and its corresponding slope change in the behavior data, and improve the accuracy of the prediction. The trained model can effectively capture the key features of the behavior data, which is of great significance for studying complex behavior patterns, trend prediction and anomaly detection. It realizes the automated analysis of a large amount of behavior data, reduces the workload of manual analysis, and improves work efficiency. When applied to real-time monitoring scenarios, the model can quickly respond and identify key behavior changes, providing timely and accurate reference for decision makers.

[0110] In a specific implementation, the behavior characteristic curve is calculated using the following formula, including:

[0111] ;

[0112] in, A time function representing the angular parameter of human orientation; The time function representing the distance from the human to the doorbell, Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents the position coordinates of human objects Over time The derivative of , i.e. the velocity component; Indicates the rate of change of human heading angle over time; is the speed weight; is the human angular velocity weight, is the distance weight from human to doorbell; represents the attenuation constant; represents the initial time point; The base of natural logarithms; Represents the behavioral characteristic curve.

[0113] Specifically, the behavior characteristic curve is calculated using the following formula, including:

[0114] ;

[0115] in, A time function representing the angular parameter of human orientation; The time function representing the distance from the human to the doorbell, Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents the position coordinates of human objects Over time The derivative of , i.e. the velocity component; Indicates the rate of change of human heading angle over time; is the speed weight; is the human angular velocity weight, is the distance weight from human to doorbell; represents the attenuation constant; represents the initial time point; The base of natural logarithms; Represents the behavioral characteristic curve.

[0116] The above steps can bring the following technical effects: Through in-depth analysis of the interaction between people and smart doorbells, people's behavioral intentions can be more accurately identified and understood. The above analysis can reveal subtle changes in behavior, such as hesitation, rapid approach, etc., which is of great significance in security monitoring. Using the behavioral characteristic curve, abnormal behavior (such as unauthorized intrusion attempts or potential fall events) can be effectively identified. It can help to respond quickly to possible security threats or emergencies in real time or after the event. By analyzing the inflection point, slope and other mathematical characteristics of the behavioral characteristic curve, people's behavioral intentions can be understood more deeply. For example, the above pattern may indicate curiosity, malice or accidental approach to the doorbell. The above analysis method can be used as part of the intelligent security monitoring system to improve the system's ability to identify potential security threats. Combined with other technologies such as video monitoring, more comprehensive and efficient security monitoring can be achieved. In a smart home environment, it can be used to adjust the response strategy of the smart doorbell to provide a more personalized and comfortable user experience based on the user's specific behavior and habits. By distinguishing between regular and abnormal behaviors in the data analysis stage, the amount of video that needs to be manually checked can be reduced, thereby improving the processing efficiency and response speed of the security monitoring system. By comprehensively using parameters such as speed, angle change and distance, applications in the fields of smart doorbells and security monitoring can greatly enhance the system's ability to understand human behavior, thereby improving home safety while also increasing the transparency and explainability of user behavior.

[0117] The above describes the monitoring method of the smart doorbell in the embodiment of the present invention. The following describes the monitoring method device of the smart doorbell in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a monitoring method and device for a smart doorbell includes:

[0118] The acquisition module 21 is used to acquire the video captured by the camera of the doorbell, and identify the object in the captured video to obtain the identified object video; wherein the object includes humans and pets;

[0119] The recognition module 22 is used to remove pets from the recognition object video to obtain the human recognition object video, and determine whether the object in the human recognition object video matches the preset object in the atlas knowledge base on the smart doorbell;

[0120] A matching module 23 is used to extract spatial parameters of the object in the human recognition object video to obtain human behavior features if the object in the human recognition object video does not match the object preset in the atlas knowledge base;

[0121] The judging module 24 is used to judge whether the human behavior features have abnormal behavior features; wherein the abnormal behavior features include fall features or potential intrusion behavior features. If there are potential intrusion behavior features, the potential intrusion behavior features are deeply analyzed by a preset human posture estimation algorithm to assist in judging human intentions and behaviors and obtain a deep analysis result;

[0122] The analysis module 25 is used to push the video corresponding to the dangerous behavior to the user's terminal if the in-depth analysis result shows that there is a dangerous behavior. In this embodiment, the specific implementation of each unit in the above device embodiment is described in the above method embodiment, which will not be repeated here.

[0123] Reference Figure 3 The present invention also provides a computer device in an embodiment, wherein the internal structure of the computer device can be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0124] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0125] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0127] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0128] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A monitoring method for a smart doorbell, characterized in that: The following steps are involved: Obtaining a video captured by a camera of the doorbell, and identifying an object in the captured video to obtain a video of the identified object; wherein the object includes humans and pets; Eliminate pets from the object recognition video to obtain a human recognition object video, and determine whether the object in the human recognition object video matches the preset object in the graph knowledge base on the smart doorbell; If the object in the human recognition object video does not match the preset object in the atlas knowledge base, the spatial parameters of the object in the human recognition object video are extracted to obtain human behavior features; Determine whether the human behavior features have abnormal behavior features; wherein the abnormal behavior features include fall features or potential intrusion behavior features. If there are potential intrusion behavior features, perform in-depth analysis on the potential intrusion behavior features through a preset human posture estimation algorithm to assist in determining human intentions and behaviors and obtain in-depth analysis results; If the in-depth analysis result shows that there is dangerous behavior, the video corresponding to the dangerous behavior will be pushed to the user's terminal; The step of extracting spatial parameters of objects in the human recognition object video to obtain human behavior features includes: Extracting spatial parameters of the object in the human recognition object video to obtain corresponding object behavior parameters; wherein the spatial parameters include human coordinates, human angles, and the distance between the human and the doorbell; Calculating the object behavior parameters to obtain a behavior characteristic curve; wherein the behavior characteristic curve includes a change in moving distance and a change in path; Fitting the behavior characteristic curve to obtain a behavior characteristic fitting curve, and determining an inflection point of the behavior characteristic fitting curve and a slope corresponding to the inflection point position; wherein the inflection point represents a change or an important event in the behavior, and the slope corresponding to the inflection point position represents a rate of change in the behavior of the object; The inflection point and the slope corresponding to the inflection point position are analyzed by a preset behavior characteristic analysis model to obtain human behavior characteristics; The training process of the preset behavior feature analysis model includes: Obtaining behavior characteristic parameters, and inputting the behavior characteristic parameters into a preliminary behavior characteristic analysis model; wherein the behavior characteristic parameters include inflection point training parameters and slope training parameters corresponding to the inflection point position; the preliminary behavior characteristic analysis model includes a preliminary inflection point model and a slope model corresponding to the preliminary inflection point position; Obtaining actual labels pre-calibrated in the behavior characteristic parameters; wherein the actual labels include inflection point analysis results corresponding to the inflection point training parameters and slope analysis results corresponding to the slope; Inputting the inflection point training parameters into a preliminary inflection point model to obtain a first prediction analysis result; Inputting the slope training parameter corresponding to the inflection point position into the slope model corresponding to the preliminary inflection point position to obtain a second prediction analysis result; Comparing the inflection point analysis result with the first prediction analysis result to obtain an inflection point loss value; Comparing the slope analysis result with the second prediction analysis result to obtain a slope loss value; The model parameters of the preliminary inflection point model and the model parameters of the slope model corresponding to the preliminary inflection point position are iteratively adjusted, and the inflection point loss value and the slope loss value are minimized through loss-based optimization to obtain the trained behavior feature analysis model.

2. The monitoring method of the smart doorbell according to claim 1, characterized in that: Obtaining a video captured by a camera of a doorbell, and identifying an object in the captured video to obtain an identified object video, including: The doorbell's camera captures the area in front of the door to obtain real-time video; The real-time video is quality evaluated by a preset quality analysis algorithm to obtain a quality evaluation result; wherein the quality evaluation includes video resolution evaluation, video frame rate evaluation and video stability evaluation; the quality evaluation result includes low-level recognition quality, medium-level recognition quality and high-level recognition quality; If it is determined that the quality assessment result is low-level recognition quality or medium-level recognition quality, quality mapping is performed on the quality assessment result to obtain a first abnormal parameter of the real-time video; Adjusting the first abnormal parameter of the real-time video according to the parameter corresponding to the advanced recognition quality to obtain the adjusted parameter of the real-time video; Based on the adjusted parameters of the real-time video, a video of the recognized object is obtained.

3. The monitoring method of the smart doorbell according to claim 1, characterized in that: The human behavior characteristics include human posture characteristics, action duration characteristics, and relative movement characteristics between body parts.

4. The monitoring method of the smart doorbell according to claim 1, characterized in that: The potential intrusion behavior characteristics are deeply analyzed by a preset human posture estimation algorithm to assist in judging human intentions and behaviors, and obtain in-depth analysis results, including Performing vector conversion on the potential intrusion behavior feature through a preset human posture estimation algorithm to obtain an intrusion behavior feature vector; The intrusion behavior feature vector is feature-weighted by a weight layer in a preset human posture estimation algorithm to obtain a key feature vector; Introducing a feedback mechanism into a preset human posture estimation algorithm, dynamically adjusting the key feature vector through the feedback mechanism to obtain an adjustment vector; wherein the feedback mechanism has a pre-input standard key feature vector; Transposing the adjustment vector to obtain a transposed vector; constructing an intrusion behavior matrix based on the transposed vector; The intrusion behavior matrix is ​​deeply analyzed through the analysis layer in the preset human posture estimation algorithm to obtain a deep analysis result.

5. The monitoring method of the smart doorbell according to claim 1, characterized in that: The behavior characteristic curve is calculated using the following formula, including: ； in, A time function representing the angular parameter of human orientation; represents the time function of the distance from the human to the doorbell, Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents human position coordinates Over time The derivative of , i.e. the velocity component; Represents the position coordinates of human objects Over time The derivative of , i.e. the velocity component; Indicates the rate of change of human orientation angle over time; is the speed weight; is the human angular velocity weight, is the distance weight from human to doorbell; represents the attenuation constant; represents the initial time point; represents the base of natural logarithms; Represents the behavioral characteristic curve.

6. A monitoring device for a smart doorbell, characterized in that: include: An acquisition module is used to acquire a video captured by a camera of the doorbell, and identify an object in the captured video to obtain a video of the identified object; wherein the object includes humans and pets; The recognition module is used to remove pets from the recognition object video to obtain the human recognition object video, and determine whether the object in the human recognition object video matches the preset object in the atlas knowledge base on the smart doorbell; A matching module is used to extract spatial parameters of the object in the human recognition object video to obtain human behavior features if the object in the human recognition object video does not match the object preset in the atlas knowledge base; A judgment module, used to judge whether the human behavior characteristics have abnormal behavior characteristics; wherein the abnormal behavior characteristics include fall characteristics or potential intrusion behavior characteristics. If there are potential intrusion behavior characteristics, the potential intrusion behavior characteristics are deeply analyzed by a preset human posture estimation algorithm to assist in judging human intentions and behaviors and obtain a deep analysis result; The analysis module is used to push the video corresponding to the dangerous behavior to the user's terminal if the in-depth analysis result shows that there is dangerous behavior; The step of extracting spatial parameters of objects in the human recognition object video to obtain human behavior features includes: Extracting spatial parameters of the object in the human recognition object video to obtain corresponding object behavior parameters; wherein the spatial parameters include human coordinates, human angles, and the distance between the human and the doorbell; Calculating the object behavior parameters to obtain a behavior characteristic curve; wherein the behavior characteristic curve includes a change in moving distance and a change in path; Fitting the behavior characteristic curve to obtain a behavior characteristic fitting curve, and determining an inflection point of the behavior characteristic fitting curve and a slope corresponding to the inflection point position; wherein the inflection point represents a change or an important event in the behavior, and the slope corresponding to the inflection point position represents a rate of change in the behavior of the object; The inflection point and the slope corresponding to the inflection point position are analyzed by a preset behavior characteristic analysis model to obtain human behavior characteristics; The training process of the preset behavior feature analysis model includes: Obtaining behavior characteristic parameters, and inputting the behavior characteristic parameters into a preliminary behavior characteristic analysis model; wherein the behavior characteristic parameters include inflection point training parameters and slope training parameters corresponding to the inflection point position; the preliminary behavior characteristic analysis model includes a preliminary inflection point model and a slope model corresponding to the preliminary inflection point position; Obtaining actual labels pre-calibrated in the behavior characteristic parameters; wherein the actual labels include inflection point analysis results corresponding to the inflection point training parameters and slope analysis results corresponding to the slope; Inputting the inflection point training parameters into a preliminary inflection point model to obtain a first prediction analysis result; Inputting the slope training parameter corresponding to the inflection point position into the slope model corresponding to the preliminary inflection point position to obtain a second prediction analysis result; Comparing the inflection point analysis result with the first prediction analysis result to obtain an inflection point loss value; Comparing the slope analysis result with the second prediction analysis result to obtain a slope loss value; The model parameters of the preliminary inflection point model and the model parameters of the slope model corresponding to the preliminary inflection point position are iteratively adjusted, and the inflection point loss value and the slope loss value are minimized through loss-based optimization to obtain the trained behavior feature analysis model.

7. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images