Terminal for intelligent doorplate management system

Abstract

The invention relates to the technical field of image recognition, in particular to a terminal for an intelligent doorplate management system. The system comprises a door lock monitoring unit, a doorplate management unit, a user terminal and a cloud platform, and the cloud platform comprises a visitor behavior analysis module, a resident behavior analysis module and an abnormal signal transmission module. According to the terminal for the intelligent doorplate management system, the light adjustment factor is introduced into the covariance matrix, so that the system can dynamically adapt to the change of the environment illumination intensity, false detection or missing detection caused by sudden light change is reduced, the robustness under different illumination conditions is improved, and the user experience is improved. The matching degree adjustment mechanism enables the model to dynamically adjust the weight of each Gaussian component according to the similarity between the current scene and the historical model, so that the system can more accurately distinguish dynamic visiting target guests and static backgrounds, can maintain high detection precision even in complex and changeable scenes, and reduces the false alarm rate.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a terminal for an intelligent house number management system. Background Art

[0002] With the development of society, people have higher and higher requirements for the safety of living and working environments. Traditional door locks and doorplate systems are difficult to meet the needs of modern security management, especially for places with frequent visitors. How to effectively monitor and manage visitor behavior and prevent illegal intrusion has become an urgent problem to be solved. The rapid development of the Internet of Things, cloud computing and artificial intelligence technologies has promoted the concept of smart home and smart office space to be deeply rooted in people's hearts. Users expect to remotely control home security, understand the situation at home, and even improve the quality of indoor environment through smartphones or other smart terminals. This pursuit of intelligent life quality has led to the birth of smart doorplate management systems. When traditional door locks monitor visitors, the video analysis and target tracking algorithms may not fully consider the complexity of light changes in design, lack sufficient adaptive mechanisms to cope with these changes, resulting in a decrease in overall system performance, and when the targets occlude each other or overlap with static objects in the background, the simple tracking algorithm may lose the target, or mistakenly identify the occluder as part of the target, resulting in tracking failure or a significant decrease in accuracy. Therefore, a terminal for a smart doorplate management system is designed. Summary of the invention

[0003] The object of the present invention is to provide a terminal for an intelligent door number management system to solve the problem in the above background technology that the complexity of light changes and the decrease in tracking accuracy are not fully considered.

[0004] To achieve the above object, the present invention provides a terminal for an intelligent house number management system, comprising:

[0005] Door lock monitoring unit: used to monitor the visitor's behavior data and environmental data, and transmit the monitored data to the cloud platform in real time. The door lock monitoring unit includes at least a communication module, a sensor, a camera, a speaker and a microphone;

[0006] Doorplate management unit: used for intelligent display control, information reception and indoor air quality detection;

[0007] User terminal: used to receive data from the door lock monitoring unit and the cloud platform to realize remote control function, record the entry and exit time of residents, and transmit the entry and exit time of residents to the cloud platform;

[0008] Cloud platform: used to collect monitoring data from door lock monitoring units and user terminals, process the monitoring data, transmit the processed images to the user terminals, integrate and analyze the collected monitoring data, identify the behavior patterns of residents and visitors, generate abnormal signals for the identified abnormal visitor behaviors and transmit them to the door lock monitoring unit, house number management unit and user terminals according to the set alarm rules;

[0009] Among them, the cloud platform includes visitor behavior analysis module, resident behavior analysis module and abnormal signal transmission module.

[0010] As a further improvement of the technical solution, the house number management unit includes a health management module, an alarm response module and an interactive page; wherein,

[0011] The health management module is used to detect indoor gas quality indicators and issue an early warning if it exceeds the safety threshold;

[0012] The alarm response module is used to sound an alarm when receiving a signal from the abnormal signal transmission module. At the same time, the alarm response module also has a built-in emergency call button. If an abnormality occurs, the user can press the button to quickly send an alarm to the preset security service center and emergency contact;

[0013] The interactive page is used to display resident information, dynamic welcome information, weather forecast and property notifications.

[0014] As a further improvement of this technical solution, in the cloud platform,

[0015] The visitor behavior analysis module is used to identify abnormal behavior patterns of visitors through video analysis algorithms;

[0016] The resident behavior analysis module is used to retrieve the user's entry and exit records in the cloud platform when an abnormal signal is generated, and determine whether the user is at home or away from home when the visitor appears;

[0017] The abnormal signal transmission module is used to generate an abnormal signal when the visitor behavior analysis module detects abnormal behavior, and transmit the abnormal signal to the corresponding alarm module according to the rules based on the judgment result of the resident behavior analysis module at the time when the abnormal signal occurs.

[0018] As a further improvement of this technical solution, the workflow of the visitor behavior analysis module is as follows:

[0019] S1: De-noise and enhance the original video transmitted from the door lock monitoring unit;

[0020] S2: Establish a background model of the space within the camera's imaging range through a mixed Gaussian model;

[0021] S3: Use Kalman filtering and optical flow tracking technology to continuously track visitor targets;

[0022] S4: Extract trajectory, speed and direction changes from the tracked targets to identify visitor-specific behavior patterns.

[0023] As a further improvement of the technical solution, in S2, the mixed Gaussian model is based on the expectation maximization algorithm, and the calculation steps of establishing the background model using the expectation maximization algorithm are:

[0024]

[0025] Among them, w ik For each data point x i The posterior probability of belonging to the kth Gaussian component; π k is the proportion of the kth Gaussian component in the entire mixture model; N(x i ∣μ k ,∑ k ) represents the density function of Gaussian distribution; x i is the probability from the kth Gaussian distribution; μ k is the mean vector of the kth Gaussian component; ∑ k is the covariance matrix of the kth Gaussian component; K is the number of Gaussian components; j = 1, 2, .., K; π j is the proportion of the jth Gaussian component in the entire mixture model; μ j is the mean vector of the jth Gaussian component; ∑ j is the covariance matrix of the jth Gaussian component;

[0026] The light adjustment factor is introduced into the covariance matrix of the Gaussian component for optimization. The expression involved in the optimized covariance matrix is ​​as follows:

[0027] Monitor the global brightness and contrast in the video frame to assess the current lighting conditions:

[0028]

[0029] Among them, L avg is the average brightness; W is the width of the image; H is the height of the image; I(i,j) is the position of the pixel position (i,j); i=1,2,…,W; j=1,2,…,H;

[0030]

[0031] Among them, C std is the contrast;

[0032] λ(L)=α·(L avg ―L base)+β·(C std ―C base )+1

[0033] Among them, λ(L) is the light adjustment factor; α and β are adjustment coefficients; L base is the reference brightness; C base is the reference contrast value;

[0034] ∑′ k =λ(L)·∑ k

[0035] Among them, ∑′ k is the covariance matrix of the kth Gaussian component after optimization by introducing the light adjustment factor;

[0036] The matching degree between the current frame and the historical model is introduced into the proportion of the Gaussian component in the entire mixed model for adjustment. The adjusted expression is:

[0037] τ′ π =π k ·(1+δ k ·sim k )

[0038] Among them, π′ k The proportion of the kth Gaussian component in the entire mixed model after adjusting the matching degree between the current frame and the historical model; δ k is the gain factor for adjusting the mixing coefficient according to the similarity; sim k is the similarity between the kth Gaussian component in the current frame and the historical model;

[0039] but,

[0040]

[0041] Among them, w ik ′ is each data point x after optimization adjustment i The posterior probability of belonging to the kth Gaussian component; π′ j The proportion of the jth Gaussian component in the entire mixed model after adjusting the matching degree between the current frame and the historical model; ∑′ j is the covariance matrix of the jth Gaussian component after optimization by introducing the light adjustment factor;

[0042] Update the blending coefficients:

[0043]

[0044] Where N is the number of pixels; i = 1, 2, ..., N;

[0045] Update the mean:

[0046]

[0047] Update the covariance matrix:

[0048]

[0049] Where, T is the transpose operation;

[0050] Repeat the above steps until the change in the parameter is less than a preset small threshold or the maximum number of iterations is reached.

[0051] As a further improvement of the technical solution, the calculation steps in S3 are:

[0052] Perform Kalman position prediction:

[0053]

[0054] in, is the position prediction value at time t; F t―1 is the position transfer matrix; x t―1∣t―1 is the estimated position at the last moment; B t―1 is the control input matrix; u t―1 It is the external control input;

[0055] Perform Kalman's covariance forecast:

[0056]

[0057] Among them, P t∣t―1 is the predicted covariance matrix; P t―1∣t―1 is the posterior covariance of the previous moment; is the transpose of the position transfer matrix; Q t―1 is the process noise covariance;

[0058] Minimize the brightness change of the image patch by calculating the optical flow:

[0059] I x (x,y)u+I y (x,y)v+I t (x,y)=0

[0060] Among them, I x is the gradient of the image in the x direction; u is the horizontal velocity component; I y is the gradient of the image in the y direction; v is the velocity component in the vertical direction; I t is the temporal gradient of the image; (x, y) is the coordinate of the video image block;

[0061] The optical flow information is converted into a prediction of the position space, and then combined with the Kalman prediction to obtain the fused and predicted position:

[0062]

[0063] Among them, x fusion is the predicted position; α is the fusion weight; is the position obtained by optical flow analysis;

[0064] Adaptive weights are introduced into the calculation formula of the predicted position for optimization. The optimized expression is:

[0065] w kalman =f1(D flow ,C detect ,S history )

[0066] w flow =1―w kalman

[0067]

[0068] Among them, w kalman is the weight of Kalman prediction; w flow The weight predicted by the optical flow data; D flow is the reliability index of optical flow data; C detect is the confidence of the target detection in the current frame; S history For historical tracking stability indicators; is the position obtained by optical flow analysis; The predicted position after optimization by introducing adaptive weights;

[0069] Perform Kalman gain calculation:

[0070]

[0071] Among them, T t is the Kalman gain; is the transpose of the observation model matrix; R is the observation noise covariance;

[0072] To perform a location update:

[0073]

[0074] Among them, x t∣t is the updated position.

[0075] As a further improvement of the technical solution, in S4, the judgment rule for identifying the specific behavior pattern of the visitor includes:

[0076] As time changes, if the target visitor's trajectory moves along a repeated path and the duration exceeds a threshold, it is judged as wandering behavior;

[0077] As time changes, if the target visitor’s coverage background model suddenly increases with speed, it is judged as a forced intrusion behavior;

[0078] As time changes, if the number of target visitors in the background model increases rapidly and their stay time is long, it is judged as crowd gathering behavior.

[0079] As a further improvement of the technical solution, the rule transmitted from the abnormal signal transmission module to the corresponding alarm module is:

[0080] When it is determined that the resident is at home, the abnormal signal is transmitted to the door lock monitoring unit and the alarm response module;

[0081] When it is determined that the resident is not at home, an abnormal signal is transmitted to the door lock monitoring unit and the user terminal; a reminder of pending tasks appears on the APP in the user terminal. If the resident responds within the specified time, the reminder disappears. If there is no response within the preset time, an alarm message is sent to the user terminal.

[0082] As a further improvement of the present technical solution, the alarm response module and the door lock monitoring unit both alarm in the form of issuing a voice alarm.

[0083] As a further improvement of the technical solution, when the cloud platform identifies an abnormal visitor, it can capture the video of the visitor and automatically save it to the user terminal.

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] 1. In the terminal for intelligent door number management system, a light adjustment factor is introduced into the covariance matrix, so that the system can dynamically adapt to the changes in ambient light intensity, reduce false detection or missed detection caused by sudden changes in light, and improve the robustness under different lighting conditions. The matching adjustment mechanism enables the model to dynamically adjust the weight of each Gaussian component according to the similarity between the current scene and the historical model, so that the system can more accurately distinguish between dynamic visitors and static backgrounds, and maintain high detection accuracy even in complex and changing scenes, reducing the false alarm rate;

[0086] 2. In the terminal for the intelligent door number management system, the Kalman filter can effectively estimate and predict the position of the target based on historical motion information and current observation values, and reduce noise interference. Combined with optical flow tracking technology, the dynamic changes of the target can be accurately captured by analyzing the pixel motion between consecutive frames. The combination of the two significantly improves the tracking accuracy of the visitor's movement trajectory, and introduces the calculation formula for adaptive weight optimization prediction position, which means that the system can dynamically adjust the weight distribution according to factors such as the stability of the target motion and the complexity of the environment. When the target motion is stable and there is little environmental interference, the weight of historical information is increased to maintain smooth tracking; when the target changes rapidly or the environment is complex and changeable, the weight of the current observation information is increased to respond to changes quickly, which enhances the algorithm's adaptability and robustness to different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is a flowchart of the whole process of the present invention;

[0088] Figure 2 is a system block diagram of the house number management unit in the present invention;

[0089] Figure 3 This is a system block diagram of the cloud platform in the present invention.

[0090] The meaning of each number in the figure is:

[0091] 1. Door lock monitoring unit; 2. House number management unit; 21. Health management module; 22. Alarm response module; 23. Interactive page; 3. User terminal; 4. Cloud platform; 41. Visitor behavior analysis module; 42. Resident behavior analysis module; 43. Abnormal signal transmission module. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] Example

[0094] See also Figure 1-3As shown, a terminal for an intelligent house number management system is provided, including a door lock monitoring unit 1, which is installed outside the door of the resident and has a built-in doorbell button. When the visitor presses the doorbell button, the user terminal 3 and the house number management unit 2 will receive a visitor reminder, and the resident can choose to open the door or refuse. At the same time, the time points of opening the door and refusing will be recorded by the user terminal and transmitted to the cloud platform 4 for storage. The door lock monitoring unit 1 is mainly used to monitor the behavior data and environmental data of the visitor, and the environmental data includes noise and light environment, and the monitored behavior data and environmental data are transmitted to the cloud platform 4 in real time, wherein the door lock monitoring unit 1 at least includes a communication module, Sensors, cameras, speakers and microphones; the sensors can collect environmental data around the door lock, the cameras can capture video images within the monitoring range in real time, and the speakers are used to sound alarms; the microphones are used to communicate and interact with visitors, the door lock monitoring unit 1 is wirelessly connected to the house number management unit 2, the user terminal 3 and the cloud platform 4 through the communication module, and the door lock monitoring unit 1 can transmit the monitored images to the cloud platform 4 in real time; further, the door lock monitoring unit 1 and the house number management unit 2 also include a power detection module, and when the power detection module detects that the battery power is lower than the preset value, it can promptly send a power exhaustion reminder to the user terminal 3.

[0095] It also includes a house number management unit 2, which is used for intelligent display control, information reception and indoor air quality detection; it includes a health management module 21, an alarm response module 22 and an interactive page 23; among them, the health management module 21 is used to detect indoor gas quality indicators, and issue an early warning if it exceeds the safety threshold; for example, it monitors the concentration of air pollutants such as carbon dioxide, formaldehyde, PM2.5, etc. that may affect human health. It uses built-in high-sensitivity sensors to collect data in real time and compares it with the preset safety threshold. Once any gas indicator is detected to exceed the safety range, the health management module 21 immediately issues an early warning through the house number interactive page 23 and the connected user terminal 3, reminding residents to pay attention to indoor air quality and take necessary measures, such as opening windows for ventilation or starting air purifiers, to ensure the health of the living environment;

[0096] The alarm response module 22 is used to sound an alarm when receiving a signal from the abnormal signal transmission module 43. At the same time, the alarm response module 22 also has a built-in emergency call button. If an abnormality occurs, the user can press the button to quickly send an alarm to the preset security service center and emergency contact. The specific process is as follows: after receiving the alarm signal, the alarm response module 22 will immediately activate the local alarm device, such as a sound alarm or a flashing alarm, and provide an emergency call button as a manual trigger point. When the button is pressed in a critical situation, the system will automatically send a request for help to the preset security service center and emergency contact to ensure that assistance can be quickly obtained in an emergency.

[0097] The interactive page 23 is used to display resident information, dynamic welcome information, weather forecast and property notification, as well as a prompt page after the visitor presses the doorbell. Specifically: automatically generate a welcome message according to time, date or specific events to improve user experience; display the current or next few days' weather conditions to facilitate residents to prepare before traveling; timely push announcements or important notifications issued by the property to maintain effective communication between residents and the property; when a visitor uses the doorbell, display visitor-related information and answer options, and the resident can choose whether to open the door or send a message to the visitor, which enhances safety and convenience.

[0098] It also includes a user terminal 3, which is usually a mobile phone of the resident, and is used to receive data from the door lock monitoring unit 1 and the cloud platform 4 to realize the remote control function, and at the same time record the entry and exit time of the resident, and transmit the entry and exit time of the resident to the cloud platform 4 for storage and processing; specifically: the user terminal 3 receives the real-time status information sent by the door lock monitoring unit 1 through Wi-Fi and mobile network communication methods, such as the door lock switch status, abnormal intrusion warning, etc. When a visitor presses the doorbell integrated in the door lock monitoring unit 1, the information is quickly transmitted to the user terminal 3, and the user can receive the notification immediately no matter where he is. The user terminal 3 can start two-way communication with the camera and microphone at the door lock, so that the user can see the situation outside the door, and communicate with the visitor through voice dialogue to decide whether to open the door lock;

[0099] Therefore, the user can remotely control the door lock through the user terminal 3 application, such as remotely opening the door, setting a temporary password, viewing the door lock log, etc., which greatly improves convenience and safety. The door lock monitoring unit 1 records the time and events each time the door lock is opened and closed, such as residents entering and exiting, visitors visiting, etc. This information is instantly transmitted to the user terminal 3. The user terminal 3 collects these entry and exit records and uploads them to the cloud platform 4 regularly or in real time. The cloud platform is responsible for long-term storage and analysis of this data. The cloud platform 4 can provide users with value-added services such as entry and exit habit analysis and security assessment reports based on this data to help users better understand and manage home security.

[0100] It also includes a cloud platform 4, which is used to collect monitoring data from the door lock monitoring unit 1 and the user terminal 3, process the monitoring data, transmit the processed images to the user terminal 3, and then integrate and analyze the collected monitoring data to identify the behavior patterns of residents and visitors, generate abnormal signals for the identified abnormal visitor behavior, and transmit them to the speaker of the door lock monitoring unit 1, the alarm response module 22 of the house number management unit 2, and the information reminder of the user terminal 3 according to the set alarm rules;

[0101] The cloud platform 4 includes a visitor behavior analysis module 41, a resident behavior analysis module 42 and an abnormal signal transmission module 43;

[0102] The visitor behavior analysis module 41 is used to identify the abnormal behavior patterns of visitors through video analysis algorithms; the working process is as follows:

[0103] S1: De-noise and enhance the original video transmitted from the door lock monitoring unit 1;

[0104] Denoising is to eliminate random noise in the video, such as electromagnetic interference and compression distortion, which may affect the accuracy of subsequent analysis; enhancement is to improve the video quality, increase contrast, brightness and clarity, and make the details in the video more obvious, especially for objects in shadow areas and low-light environments; this can provide clean and clear video materials for subsequent background modeling and target detection, which is the basis for accurately identifying visitor behavior;

[0105] S2: Establish a background model of the space within the camera's camera range through a mixed Gaussian model; the mixed Gaussian model assumes that the background is composed of multiple Gaussian distributions, each of which represents a color or brightness pattern in the background. By fitting these distributions to the data of the input frame, the model can identify which pixels belong to the background and which are visitor targets. Therefore, its main purpose is to achieve adaptive background subtraction, that is, to separate dynamic foreground objects from video frames;

[0106] The mixed Gaussian model is based on the expectation maximization algorithm. The calculation steps of using the expectation maximization algorithm to establish the background model are:

[0107]

[0108] Among them, w ik For each data point x i The posterior probability of belonging to the kth Gaussian component; π k is the proportion of the kth Gaussian component in the entire mixture model; N(x i ∣μ k ,∑ k ) represents the density function of Gaussian distribution; x i is the probability from the kth Gaussian distribution; μ k is the mean vector of the jth Gaussian component; Σ k is the covariance matrix of the kth Gaussian component; K is the number of Gaussian components; j = 1, 2, .., K; π j is the proportion of the jth Gaussian component in the entire mixture model; μ j is the mean vector of the jth Gaussian component; Σ j is the covariance matrix of the jth Gaussian component;

[0109] Introducing the light adjustment factor into the covariance matrix of the Gaussian component for optimization allows the model to dynamically adapt to slight changes in background color or brightness caused by changes in lighting conditions, so that the background model can remain accurate under different times and lighting conditions, reducing false positives caused by environmental changes. The expression involved in the optimized covariance matrix is ​​as follows:

[0110] Monitor the global brightness and contrast in the video frame to assess the current lighting conditions:

[0111]

[0112] Among them, L avg is the average brightness, reflecting the overall illumination level; W is the width of the image; H is the height of the image; I(i,j) is the position of the pixel position (i,j); i=1,2,…,W; j=1,2,…,H;

[0113]

[0114] Among them, C std Contrast is a measure of the dispersion of pixel brightness in an image, reflecting the richness of scene details. When modeling backgrounds, high contrast may mean that there are more details or uneven lighting in the scene. Adding contrast factors to the adjustment factor can make the model more sensitive to these changes, thereby improving the adaptability of the model.

[0115] λ(L)=α·(L avg ―L base )+β·(C std ―C base )+1

[0116] Among them, λ(L) is the light adjustment factor; α and β are adjustment coefficients; L base is the reference brightness; C base is the baseline contrast value; the linear effect of brightness and contrast changes is added to the baseline value, ensuring that the variance adjustment has a certain degree of flexibility when the lighting conditions change; by linearly combining the deviation of the average brightness and contrast of the current frame from the baseline value, an adjustment factor is generated, which reflects the degree of change of the lighting conditions relative to a certain reference state, allowing the model to dynamically adjust the covariance matrix, thereby maintaining the accuracy of the background model under different lighting conditions and reducing false positives;

[0117] Σ′ k =λ(L)·∑ k

[0118] Among them, Σ′ kThe covariance matrix of the kth Gaussian component after optimization by introducing the light adjustment factor; the light adjustment factor is directly multiplied by the original covariance matrix to achieve dynamic adjustment of the covariance matrix, so that the model can enlarge or reduce the variance of each Gaussian component according to the current lighting conditions, so as to better accommodate the pixel value fluctuations caused by lighting changes, maintain an accurate description of the background, and improve the robustness and adaptability of the model;

[0119] The matching degree between the current frame and the historical model is introduced into the proportion of the Gaussian component in the entire mixed model to adjust the adaptive update of the model. In this way, the model will continuously learn and update itself according to the new frame data, while maintaining a good memory of the long-term stable background, avoiding the excessive impact of short-term changes on the model, and ensuring the stability and robustness of the background model. The adjusted expression is:

[0120] π′ k =π k ·(1+δ k ·sim k )

[0121] Among them, π′ k The proportion of the kth Gaussian component in the entire mixed model after adjusting the matching degree between the current frame and the historical model; δ k is the gain factor for adjusting the mixing coefficient according to the similarity; sim k is the similarity between the kth Gaussian component in the current frame and the historical model, which is the matching degree between the pixel points in the area covered by the component and the corresponding component of the historical model. When a component still matches the background well under the current lighting conditions, its sim k Close to 1, π′ k will increase, otherwise it will decrease;

[0122] but,

[0123]

[0124] Among them, w ik ′ is each data point x after optimization adjustment i The posterior probability of belonging to the kth Gaussian component; π′ j The proportion of the jth Gaussian component in the entire mixed model after adjusting the matching degree between the current frame and the historical model; ∑′ j is the covariance matrix of the jth Gaussian component after optimization by introducing the light adjustment factor;

[0125] Update the blending coefficients:

[0126]

[0127] Where N is the number of pixels; i = 1, 2, ..., N; the mixing coefficient represents the proportion of the kth Gaussian component in all pixels. The purpose of updating the mixing coefficient is to reflect the relative importance of each Gaussian component under the current model. By recalculating the weighted sum of each component, it ensures that the model can accurately reflect the probability distribution in the data set and helps distinguish which components are more frequently used to describe the data, that is, which patterns are more consistent with the background characteristics.

[0128] Update the mean:

[0129]

[0130] The mean represents the center position of the Gaussian distribution. The purpose of updating the mean is to make the Gaussian components gather more closely around the actual data points, so as to more accurately capture the characteristics of the background area. By using weighted average, it is ensured that the data points with larger weights have a greater impact on the mean, which helps to reduce noise interference and make the background model more able to reflect the statistical characteristics of the real background.

[0131] Update the covariance matrix:

[0132]

[0133] Where T is the transposition operation; the covariance matrix describes the distribution of data points around the mean and the correlation between dimensions. Updating the covariance matrix is ​​intended to adapt to the spatial changes of data points, including the fluctuation range of attributes such as brightness and color. By calculating the outer product of the deviation vector of each data point with the mean and weighted summing it, and then dividing it by the total weight, the latest shape and direction reflecting the data distribution can be obtained, which helps to more accurately simulate the natural changes of background pixels and distinguish normal changes from abnormal changes, such as slight light and shadow movement and visitor movement;

[0134] Repeat the above steps until the change in parameters is less than a preset small threshold or the maximum number of iterations is reached. The expectation maximization algorithm optimizes the model parameters through an iterative process. Each iteration attempts to maximize the likelihood of the model for the data, which can help filter out random noise and temporary interference, ensuring that the background model can more accurately reflect the true background state, thereby reducing false detection and missed detection in subsequent foreground detection; so the final background model is composed of the mixed Gaussian model parameters after multiple iterations of optimization, including:

[0135] Mixing coefficient π k , represents the proportion of each Gaussian component in the total, and together they constitute 1, reflecting the relative importance of different regions or features in the background;

[0136] Mean vector μ k, for each Gaussian component, it defines the center position of the distribution, that is, the average value of a certain type of pixel value or color in the background;

[0137] Covariance matrix∑ k , describes the shape of each Gaussian distribution and the correlation between dimensions, determines the range and direction of background pixel value changes, and the illumination-adjusted covariance matrix is ​​more adaptable to background changes under different lighting conditions;

[0138] The final background model can be seen as a combination of a set of Gaussian distributions defined by these parameters. Each distribution represents a sub-model of the background, and together they cover the distribution of pixel values ​​in the entire background space. This model can be used in real-time video processing to distinguish moving visitor targets from the background by calculating the degree of match between the incoming pixel values ​​and the background model. Pixels with a high degree of mismatch are considered foreground, while pixels with a high degree of match are considered to be part of the background, thereby achieving effective segmentation and behavior analysis of dynamic objects in video scenes.

[0139] S3: Use Kalman filtering and optical flow tracking technology to continuously track visitor targets; Kalman filtering uses two steps, prediction and update, combined with previous state estimates and new measurements, to provide continuous and smooth tracking of target positions, which is particularly suitable for processing noisy data. Prediction is based on the state of the previous moment and the system model to predict the current state, and update is to correct the predicted state based on new observation data to reduce error accumulation; optical flow tracking is to estimate the direction and speed of the object by analyzing the motion information of pixels between adjacent frames, which is suitable for accurate tracking of targets in scenes with rich details; combined with the prediction accuracy of Kalman filtering and the immediacy of optical flow, it can efficiently and accurately follow the movement of visitors in the video, and can quickly resume tracking even after the target is occluded or disappears briefly; the calculation steps are:

[0140] Perform Kalman position prediction:

[0141]

[0142] in, is the position prediction value at time t; F t―1 is the position transfer matrix; x t―1∣t―1 is the estimated position at the last moment; B t―1 is the control input matrix; u t―1 It is the external control input;

[0143] It is used to predict the target position at the current time t based on the estimated value of the target position at the previous time and external control input, such as the visitor's autonomous motion control or the system's external force, through the position transfer matrix and the control input matrix. It can help the system estimate the possible position of the visitor in advance when there is no new observation data, and provide useful information for the next update step;

[0144] Perform Kalman's covariance forecast:

[0145]

[0146] Among them, P t∣t―1 is the predicted covariance matrix; P t―1∣t―1 is the posterior covariance of the previous moment; is the transpose of the position transfer matrix; Q t―1 is the process noise covariance;

[0147] The covariance matrix used to predict the current time t, that is, the uncertainty of the predicted state estimate. It is based on the posterior covariance matrix of the previous time, transfers uncertainty through the position transfer matrix, and takes into account process noise, that is, sources of uncertainty within the system, such as erratic behavior of visitors or measurement errors, which helps to assess the reliability of the predicted position, so that when new observations are received, the state estimate can be updated more accurately and the covariance can be adjusted;

[0148] Minimize the brightness change of the image patch by calculating the optical flow:

[0149] I x (x,y)u+I y (x,y)v+I t (x,y)=0

[0150] Among them, I x is the gradient of the image in the x direction; u is the horizontal velocity component; I y is the gradient of the image in the y direction; v is the velocity component in the vertical direction; I t is the temporal gradient of the image; (x, y) is the coordinate of the video image block; by solving this set of equations, the optical flow field of each pixel, i.e., the motion vector, can be obtained, which reflects the movement direction and speed of the visitor in the image;

[0151] The optical flow information is converted into a prediction of the position space, and then combined with the Kalman prediction to obtain the fused and predicted position:

[0152]

[0153] Among them, x fusion is the predicted position; α is the fusion weight; The position obtained by optical flow analysis; Kalman filtering is good at processing noisy measurements and predictions, providing statistical estimates of target states, while optical flow directly extracts dynamic information from image motion and is more sensitive to fast motion. By adjusting the contribution of the two through weight α, the stability and sensitivity of prediction can be balanced in different scenarios, optimizing visitor tracking performance; when the target moves quickly or the environment is complex, increasing the weight of optical flow may be more conducive to capturing dynamic changes; when stability and long-term prediction are required, increase the weight of Kalman filtering;

[0154] Adaptive weights are introduced into the calculation formula of the predicted position for optimization. The optimized expression is:

[0155] w kalman =f1(D flow ,C detect ,S history )

[0156] w flow =1―w kalman

[0157]

[0158] Among them, w kalman is the weight of Kalman prediction; w flow The weight predicted by the optical flow data; D flow is the reliability index of optical flow data; C detect is the confidence of the target detection in the current frame; S history For historical tracking stability indicators; is the position obtained by optical flow analysis; The predicted position after optimization by introducing adaptive weights;

[0159] By adapting the weight w kalman and w kalman , the model can dynamically adjust the contribution of Kalman filtering and optical flow prediction according to the current environmental conditions and historical performance. In this way, the system can more flexibly optimize the prediction performance when facing different motion characteristics such as fast or slow, lighting changes, target occlusion, etc.;

[0160] D flow As a reliability indicator of optical flow data, it reflects the credibility of optical flow estimation. High reliability means that the optical flow estimation is more accurate and should be given more trust. detect Represents the confidence of target detection in the current frame. High confidence indicates the accuracy of target detection and helps to confirm the reliability of Kalman filter prediction.

[0161] S history The stability of historical tracking is evaluated, and if past forecasts have been stable and reliable, the current strategy tends to be maintained.

[0162] Perform Kalman gain calculation:

[0163]

[0164] Among them, T t is the Kalman gain, which is used to balance the trust between the predicted state and the new observation data; is the transpose of the observation model matrix; R is the observation noise covariance; describes the uncertainty in the observation data. By calculating the gain, the system decides how to combine the new observation information to correct the predicted state, so as to more accurately reflect the true position of the target;

[0165] To perform a location update:

[0166]

[0167] Among them, x t∣t is the updated position. The updated position x t∣t It accurately reflects the real position of the target at the current moment, taking into account the uncertainty of the latest observation information and model prediction;

[0168] S4: Extracting the trajectory, speed and direction change from the tracked target, thereby identifying the specific behavior pattern of the visitor. The judgment rules for identifying the specific behavior pattern of the visitor include:

[0169] As time changes, if the target visitor's trajectory moves along a repeated path and the duration exceeds a threshold, it is judged as wandering behavior;

[0170] As time changes, if the target visitor’s coverage background model suddenly increases with speed, it is judged as a forced intrusion behavior;

[0171] As time changes, if the number of target visitors in the background model increases rapidly and their stay time is long, it is judged as crowd gathering behavior.

[0172] Furthermore, the resident behavior analysis module 42 is used to retrieve the entry and exit records of users in the cloud platform 4 when an abnormal signal is generated, and to determine whether the user is at home or away from home when the visitor appears; the abnormal signal transmission module 43 is used to generate an abnormal signal when the visitor behavior analysis module 41 detects abnormal behavior, and transmits the abnormal signal to the corresponding alarm module according to the rules in combination with the judgment result of the resident behavior analysis module 42 at the time of the abnormal signal appearance. The abnormal signal transmission module 43 is responsible for intelligently selecting the alarm path according to the status of the resident at home or away from home when abnormal visitor behavior is detected, which can reduce false alarms, improve the effectiveness of the alarm and the response efficiency of the user, and by sending signals to different receiving ends in a targeted manner, the system ensures the immediacy and suitability of the alarm and improves safety.

[0173] The rule for the abnormal signal transmission module 43 to transmit to the corresponding alarm module is:

[0174] First, when it is determined that the resident is at home, the abnormal signal is transmitted to the door lock monitoring unit 1 and the alarm response module 22; the alarm response module 22 and the door lock monitoring unit 1 both issue voice alarms. Directly activate the voice warning and alarm response modules near the door lock to quickly alert the resident, which is suitable for situations that require immediate response;

[0175] Second, when it is determined that the resident is not at home, an abnormal signal is transmitted to the door lock monitoring unit 1 and the user terminal 3; the APP in the user terminal 3 displays a reminder of pending tasks. If the resident responds within the specified time, the reminder disappears. If the resident does not respond after the preset time, an alarm message is sent to the user terminal 3. The reminder is sent through the user terminal 3, giving the resident a certain amount of time to respond. If there is no response, it is upgraded to an alarm message. This mechanism balances safety and inconvenience of disturbing, and is suitable for situations where the resident is not present.

[0176] Furthermore, when the cloud platform 4 identifies an abnormal visitor, it can capture the video of the visitor and automatically save it to the user terminal 3. The cloud platform 4 automatically saves the video clips when the abnormal visitor appears to the user terminal 3, which not only provides the user with evidence review to facilitate understanding of the details of the incident, but also retains key information for possible subsequent security investigations or legal proceedings, which helps residents to assess the severity of the situation themselves. It is also an important security log, which improves the reliability and practicality of the overall system.

[0177] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A terminal for an intelligent house number management system, characterized in that: include: A door lock monitoring unit (1): used to monitor the behavior data and environmental data of visitors and transmit the monitored data to the cloud platform (4) in real time, wherein the door lock monitoring unit (1) at least includes a communication module, a sensor, a camera, a speaker and a microphone; Doorplate management unit (2): used for intelligent display control, information reception and indoor air quality detection; User terminal (3): used to receive data from the door lock monitoring unit (1) and the cloud platform (4), realize remote control function, record the entry and exit time of residents, and transmit the entry and exit time of residents to the cloud platform (4); The cloud platform (4) is used to collect monitoring data from the door lock monitoring unit (1) and the user terminal (3), process the monitoring data, transmit the processed images to the user terminal (3), integrate and analyze the collected monitoring data, identify the behavior patterns of residents and visitors, generate abnormal signals for the identified abnormal visitor behavior, and transmit them to the door lock monitoring unit (1), the door number management unit (2) and the user terminal (3) according to the set alarm rules; The cloud platform (4) includes a visitor behavior analysis module (41), a resident behavior analysis module (42) and an abnormal signal transmission module (43).

2. A terminal for an intelligent door number management system according to claim 1, characterized in that: The house number management unit (2) comprises a health management module (21), an alarm response module (22) and an interactive page (23); wherein: The health management module (21) is used to detect indoor gas quality indicators and issue an early warning if the safety threshold is exceeded; The alarm response module (22) is used to issue an alarm when receiving a signal from the abnormal signal transmission module (43). At the same time, the alarm response module (22) also has a built-in emergency call button. If an abnormality occurs, the user can press the button to quickly send an alarm to a preset security service center and an emergency contact; The interactive page (23) is used to display resident information, dynamic welcome information, weather forecast and property notification.

3. A terminal for intelligent door number management system according to claim 2, characterized in that: In the cloud platform (4), The visitor behavior analysis module (41) is used to identify abnormal behavior patterns of visitors through video analysis algorithms; The resident behavior analysis module (42) is used to retrieve the user's entry and exit records in the cloud platform (4) when an abnormal signal is generated, and determine whether the user is in a home time period or a non-home time period when the visitor appears; The abnormal signal transmission module (43) is used to generate an abnormal signal when the visitor behavior analysis module (41) detects abnormal behavior, and transmit the abnormal signal to the corresponding alarm module according to the rules in combination with the judgment result of the resident behavior analysis module (42) at the time when the abnormal signal occurs.

4. The terminal for intelligent door number management system according to claim 3, characterized in that: The workflow of the visitor behavior analysis module (41) is as follows: S1: De-noising and enhancing the original video transmitted from the door lock monitoring unit (1); S2: Establish a background model of the space within the camera's imaging range through a mixed Gaussian model; S3: Use Kalman filtering and optical flow tracking technology to continuously track visitor targets; S4: Extract trajectory, speed and direction changes from the tracked targets to identify visitor-specific behavior patterns.

5. The terminal for intelligent door number management system according to claim 4, characterized in that: In S2, the mixed Gaussian model is based on the expectation maximization algorithm, and the calculation steps of establishing the background model using the expectation maximization algorithm are: Among them, w ik For each data point x i The posterior probability of belonging to the kth Gaussian component; π k is the proportion of the kth Gaussian component in the entire mixture model; N(x i ∣μ k ,∑ k ) represents the density function of Gaussian distribution; x i is the probability from the kth Gaussian distribution; μ k is the mean vector of the kth Gaussian component; ∑ k is the covariance matrix of the kth Gaussian component; K is the number of Gaussian components; j = 1, 2, .., K; π j is the proportion of the jth Gaussian component in the entire mixture model; μ j is the mean vector of the jth Gaussian component; ∑ j is the covariance matrix of the jth Gaussian component; The light adjustment factor is introduced into the covariance matrix of the Gaussian component for optimization. The expression involved in the optimized covariance matrix is ​​as follows: Monitor the global brightness and contrast in the video frame to assess the current lighting conditions: Among them, L avg is the average brightness; W is the width of the image; H is the height of the image; I(i,j) is the position of the pixel position (i,j); i=1,2,…,W; j=1,2,…,H; Among them, C std is the contrast; λ(L)=α·(L avg ―L base )+β·(C std ―C base )+1 Among them, λ(L) is the light adjustment factor; β and β are adjustment coefficients; L base is the reference brightness; C base is the reference contrast value; In k =λ(L)·∑ k Among them, ∑ ′ k is the covariance matrix of the kth Gaussian component after optimization by introducing the light adjustment factor; The matching degree between the current frame and the historical model is introduced into the proportion of the Gaussian component in the entire mixed model for adjustment. The adjusted expression is: p′ k =π k *(1+d k ·sim k ) Among them, π ′ k The proportion of the kth Gaussian component in the entire mixed model after adjusting the matching degree between the current frame and the historical model; δ k is the gain factor for adjusting the mixing coefficient according to the similarity; sim k is the similarity between the kth Gaussian component in the current frame and the historical model; but, Among them, w ik ′ is each data point x after optimization adjustment i The posterior probability of belonging to the kth Gaussian component; π ′ j The proportion of the jth Gaussian component in the entire mixed model after adjusting the matching degree between the current frame and the historical model; ∑′ j is the covariance matrix of the jth Gaussian component after optimization by introducing the light adjustment factor; Update the blending coefficients: Where N is the number of pixels; i = 1, 2, ..., N; Update the mean: Update the covariance matrix: Where, T is the transpose operation; Repeat the above steps until the change in the parameter is less than a preset small threshold or the maximum number of iterations is reached.

6. A terminal for an intelligent door number management system according to claim 5, characterized in that: The calculation steps in S3 are: Perform Kalman position prediction: in, is the position prediction value at time t; F t―1 is the position transfer matrix; x t―1∣t―1 is the estimated position at the last moment; B t―1 is the control input matrix; u t―1 It is the external control input; Perform Kalman's covariance forecast: Among them, P t∣t―1 is the predicted covariance matrix; P t―1∣t―1 is the posterior covariance of the previous moment; is the transpose of the position transfer matrix; Q t―1 is the process noise covariance; Minimize the brightness change of the image patch by calculating the optical flow: I x (x,y)u+I y (x,y)v+I t (x,y)=0 Among them, I x is the gradient of the image in the x direction; u is the horizontal velocity component; I y is the gradient of the image in the y direction; v is the velocity component in the vertical direction; I t is the temporal gradient of the image; (x, y) is the coordinate of the video image block; The optical flow information is converted into a prediction of the position space, and then combined with the Kalman prediction to obtain the fused and predicted position: Among them, x fusion is the predicted position; α is the fusion weight; is the position obtained by optical flow analysis; Adaptive weights are introduced into the calculation formula of the predicted position for optimization. The optimized expression is: w kalman =f1(D flow ,C detect ,S history ) In flow =1―in kalman Among them, w kalman is the weight of Kalman prediction; w flow The weight predicted by the optical flow data; D flow is the reliability index of optical flow data; C detect is the confidence of the target detection in the current frame; S history For historical tracking stability indicators; is the position obtained by optical flow analysis; The predicted position after optimization by introducing adaptive weights; Perform Kalman gain calculation: Among them, T t is the Kalman gain; is the transpose of the observation model matrix; R is the observation noise covariance; To perform a location update: Among them, x t∣t is the updated position.

7. The terminal for intelligent door number management system according to claim 6, characterized in that: In S4, the judgment rules for identifying the specific behavior patterns of visitors include: As time changes, if the target visitor's trajectory moves along a repeated path and the duration exceeds a threshold, it is judged as wandering behavior; As time changes, if the target visitor’s coverage background model suddenly increases with speed, it is judged as a forced intrusion behavior; As time changes, if the number of target visitors in the background model increases rapidly and their stay time is long, it is judged as crowd gathering behavior.

8. The terminal for intelligent door number management system according to claim 7, characterized in that: The rule transmitted from the abnormal signal transmission module (43) to the corresponding alarm module is: When it is determined that the resident is at home, an abnormal signal is transmitted to the door lock monitoring unit (1) and the alarm response module (22); When it is determined that the resident is not at home, an abnormal signal is transmitted to the door lock monitoring unit (1) and the user terminal (3); an APP in the user terminal (3) displays a reminder of pending tasks, and if the resident responds within the specified time, the reminder disappears; if the resident fails to respond within the preset time, an alarm message is sent to the user terminal (3).

9. The terminal for intelligent door number management system according to claim 8, characterized in that: The alarm response module (22) and the door lock monitoring unit (1) both alarm in the form of issuing a voice alarm.

10. The terminal for intelligent door number management system according to claim 9, characterized in that: When the cloud platform (4) identifies an abnormal visitor, it is able to capture a video of the visitor and automatically save it to the user terminal (3).

Images