Open fan non-locking recognition method and system based on deep neural network and background modeling

Abstract

The application provides an opening fan unlocking recognition method and system based on a deep neural network and background modeling, comprising: a camera for capturing a video stream of a door and window area in real time; a deep neural network module for automatically identifying the position and contour of a window and outputting the boundary box coordinates and classification label of the window area; a background modeling and motion detection module for detecting the motion state of the door and window; a geometric analysis module for calculating the intersection-over-union and Euclidean distance between the window boundary box and the motion area to evaluate and verify the detection result; a locking state judgment module for judging whether the door and window are unlocked according to the detection results of consecutive frames; and an alarm and feedback module for triggering an alarm and sending specific position and state information to managers. The application realizes the automatic detection of the unlocking state of large-scale glass curtain wall building doors and windows by combining the deep neural network and background modeling technology, replaces the traditional manual inspection, and greatly improves the detection efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of building safety, in particular to an open sash non-locking identification method and system based on deep neural network and background modeling. BACKGROUND

[0002] The existing glass curtain wall building door and window non-locking detection method mainly relies on manual inspection, and workers need to confirm whether each window is locked one by one. This method is particularly inconvenient in large buildings, as it takes a lot of time and labor to complete a comprehensive inspection. In addition, some existing methods focus on developing independent detection tools, but the application scenarios of these tools are usually limited to single window detection, and cannot realize real-time monitoring of the entire building. In the face of extreme weather (such as typhoon or heavy rain), manual inspection is not only inefficient, but also has certain risks.

[0003] Disadvantages of the prior art:

[0004] 1. Low efficiency: manual inspection is very inefficient, especially in large buildings, it is difficult to complete accurate detection of all door and window states in a short time.

[0005] 2. High cost: due to the need for a large number of manual participation, the cost of the entire detection process is relatively high.

[0006] 3. Obvious limitations: some existing automatic detection devices are usually only suitable for single window, and cannot realize comprehensive detection of the entire building, lacking a global view.

[0007] 4. Susceptible to environmental influences: existing technologies are susceptible to weather, light and other external environmental factors, which may lead to missed detection or false detection, especially in poor lighting conditions.

[0008] 5. Poor adaptability: with the increase of high-rise buildings in cities, especially the widespread use of glass curtain wall buildings, the limitations of existing technologies are more prominent, and they cannot effectively meet the actual needs.

[0009] Therefore, the existing technology has deficiencies and needs to be further improved. SUMMARY

[0010] In view of the problems existing in the prior art, the present application provides an open sash non-locking identification method and system based on deep neural network and background modeling.

[0011] To achieve the above purpose, the specific scheme of the present application is as follows:

[0012] The present application provides an open sash non-locking identification system based on deep neural network and background modeling, characterized in that the system comprises:

[0013] a camera for collecting video frames covering the door and window area;

[0014] a deep neural network module for locating the window in the video frames and outputting a bounding box B and a classification label C of the window area;

[0015] a background modeling and motion detection module for differentiating the video frames from the background based on K-Nearest Neighbor (KNN) background modeling and generating a foreground binary mask M t , and performing morphological opening and closing operations on M t to remove noise and small interference points to obtain a processed foreground mask;

[0016] a geometric analysis module for calculating the intersection over union (IoU) and the center point Euclidean distance (D) between B and the motion area indicated by the processed foreground mask in each frame based on the bounding box B and the processed foreground mask;

[0017] a locking state judgment module for determining whether the corresponding window sash is in an open / unlocked state according to the relationship between the IoU and the center point Euclidean distance D and the preset threshold, and combining the consistency criterion of consecutive n frames;

[0018] an alarm and feedback module for triggering an alarm and sending location and state information when it is determined to be unlocked.

[0019] Further, the deep neural network module outputs the window area bounding box coordinates and the classification label by performing a nonlinear transformation on the video frames.

[0020] Further, the background modeling and motion detection module uses K-Nearest Neighbor (KNN) algorithm for background subtraction, and optimizes the motion detection mask through morphological processing.

[0021] Further, the system further comprises a continuous state detection module for detecting the open state of the window in consecutive n frames, and determining that the window is unlocked, wherein n is a preset continuous frame threshold.

[0022] Further, the alarm and feedback module includes various alarm mechanisms, including SMS, email, or notification of a real-time monitoring system.

[0023] The present application also provides an open sash unlocking identification method based on deep neural network and background modeling, based on the above-mentioned system, the method comprising the following steps:

[0024] S1: collecting video frames covering the door and window area;

[0025] S2: locating the window using a deep neural network and outputting a bounding box B and a classification label C;

[0026] S3: Background subtraction using K-Nearest Neighbor background modeling to obtain foreground binary mask M t t M t Perform morphological opening and closing operations to remove noise and obtain the processed foreground mask.

[0027] S4: Input the bounding box B and the processed foreground mask, calculate the intersection over union IoU and the center point Euclidean distance D between the window bounding box and the motion region indicated by the processed mask.

[0028] S5: If the intersection over union IoU and the center point Euclidean distance D satisfy the preset threshold relationship, and in the continuous n frames, it is determined that the corresponding window sash is not locked.

[0029] S6: Trigger alarm and feedback position and state information.

[0030] Further, the deep neural network model obtains the feature representation of the window region by nonlinear transformation of the input video frame, and outputs the bounding box coordinates and classification label of the window region.

[0031] Further, the background modeling uses K-Nearest Neighbor algorithm (KNN) to detect the motion region in the video, and removes noise and small interference points through morphological operation, optimizing the motion detection mask.

[0032] Further, the continuous state detection and locked state judgment includes: detecting the window opening state in continuous n frames, then judging that the window is not locked, where n is a preset continuous frame number threshold.

[0033] Further, the alarm and feedback mechanism includes multiple ways, including SMS, email or real-time monitoring system notification.

[0034] The technical solution of the present application has the following advantages:

[0035] 1. Automatic detection: By combining deep neural network and background modeling technology, the automatic detection of large-scale glass curtain wall building window unsecured state is realized, replacing the traditional manual inspection, greatly improving the detection efficiency and accuracy.

[0036] 2. Real-time monitoring: The system can monitor the status of all windows in the building in real time, timely detect and report unsecured windows, especially before extreme weather, can give early warning to ensure the safety of the building.

[0037] 3. Comprehensive coverage: Unlike existing single window detection tools, the present application can comprehensively cover the entire building, realize the monitoring of all windows, and solve the problem of past partial detection.

[0038] 4. Strong anti-interference: through the feature extraction ability of neural network and the motion detection algorithm of background modeling, the system can still maintain good recognition performance under complex lighting conditions, reducing false detection or missed detection caused by environmental changes.

[0039] 5. High reliability: using continuous frame detection method, the stability of detection results is ensured, the error caused by single frame detection is reduced, and the reliability and accuracy of the whole system are improved.

[0040] 6. Timely feedback: the system has instant alarm function, once the door and window are detected to be not locked, the alarm can be sent to the management personnel, and the specific position information is provided, so that timely measures can be taken.

[0041] 7. Wide applicability: the present application is applicable to any form of glass curtain wall building, and is not limited by building height and window style, and has wide practicability. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a schematic diagram of an unlocked window of a building;

[0043] Figure 2 is an example diagram of the neural network of the present application identifying whether the window is locked;

[0044] Figure 3 is a photo of the unlocked window fed back to the building management personnel by the alarm feedback mechanism of the present application;

[0045] Figure 4 is a system block diagram of the present application;

[0046] Figure 5 is a whole flowchart of the present application. DETAILED DESCRIPTION

[0047] The present application will be further described in detail below in combination with the drawings and examples; it can be understood that the specific examples described herein are only used to explain the present application, but not to limit the present application; in addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all.

[0048] In combination with Figures 1-5 the present application provides an opening fan unlocked identification system based on deep neural network and background modeling, which comprises:

[0049] a camera installed at the bottom of the glass curtain wall building, for capturing video stream of the door and window area in real time;

[0050] a deep neural network module for automatically identifying the position and contour of the window, and outputting the boundary box coordinates and classification label of the window area;

[0051] The background modeling and motion detection module is used for detecting the motion state of the door and window, and removing noise and small interference points through morphological operation to optimize the motion detection mask.

[0052] The geometric analysis module is used for calculating the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion area to evaluate and verify the detection result.

[0053] The locked state judgment module is used for judging whether the door and window are unlocked according to the detection results of continuous frames.

[0054] The alarm and feedback module is used for triggering an alarm and sending specific location and state information to the manager when the unlocked state is detected.

[0055] The deep neural network module outputs the window region bounding box coordinates and classification labels through nonlinear transformation of the video frames.

[0056] The background modeling and motion detection module uses the K-nearest neighbor algorithm (KNN) for background subtraction, and optimizes the motion detection mask through morphological processing.

[0057] The system further comprises a continuous state detection module for detecting the window opening state in continuous n frames, and then judging that the window is unlocked, wherein n is a preset continuous frame threshold.

[0058] The alarm and feedback module comprises various alarm mechanisms, including short message, email or notification of the real-time monitoring system.

[0059] The application also provides an opening sash unlocked identification method based on deep neural network and background modeling, which is based on the above-mentioned system and comprises the following steps:

[0060] S1, a fixed camera is installed at the bottom of the glass curtain wall building to monitor the state of each window in real time.

[0061] S2, a deep neural network model is used to automatically identify the position and contour of the window, and output the bounding box coordinates and classification labels of the window region.

[0062] S3, background modeling and motion detection are performed on the video frames to detect the motion state of the door and window, and noise and small interference points are removed through morphological operation to optimize the motion detection mask.

[0063] S4, the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion area are calculated to evaluate whether the motion area overlaps with the window area, and the detection result is verified.

[0064] S5, whether the door and window are unlocked is judged according to the detection results of continuous frames.

[0065] S6, triggering an alarm and sending specific location and status information to the manager when detecting the unlocked state.

[0066] The deep neural network model obtains the feature representation of the window region by performing nonlinear transformation on the input video frame, and outputs the bounding box coordinates and classification label of the window region.

[0067] The background modeling adopts K-Nearest Neighbor algorithm (KNN) to detect the motion region in the video, and removes noise and small interference points through morphological operation to optimize the motion detection mask.

[0068] The continuous state detection and locked state judgment includes: if the window opening state is detected in continuous n frames, it is judged that the window is not locked, where n is a preset continuous frame threshold.

[0069] The alarm and feedback mechanism includes various ways, including SMS, email or real-time monitoring system notification.

[0070] Working principle overview:

[0071] Camera installation and video acquisition

[0072] Fixed cameras are installed at the bottom of the glass curtain wall building to ensure that all windows are within the monitoring range. The camera captures real-time video stream of the door and window area, providing data source for subsequent analysis.

[0073] Neural network automatic identification

[0074] The pre-trained deep neural network model is used to process video frames to automatically identify the location and contour of all windows. The neural network learns the feature representation of the window region through multiple layers of nonlinear transformation, and outputs the bounding box coordinates and classification label of the window region. The model adjusts its weights and bias terms by minimizing the loss function during training, ensuring accurate identification and positioning of each window even under complex lighting conditions.

[0075] Background modeling and motion detection

[0076] Based on K-Nearest Neighbor algorithm (KNN), the background modeling of video frames is performed, and the differences between the current frame and the background are analyzed frame by frame to generate a binary mask image, which identifies the possible motion region (such as open doors and windows). The system optimizes the motion detection mask through morphological operations (such as closing and opening operations) to remove noise and small interference points, ensuring that only the motion region related to the door and window is retained.

[0077] Window state geometric analysis

[0078] The system calculates the Intersection over Union (IoU) between the motion region detected by background modeling and the window bounding box identified by the neural network to assess whether the motion region overlaps with the window region. Additionally, the system calculates the Euclidean distance between the motion region and the center point of the window to further verify the accuracy of the detection results.

[0079] Continuous state detection and locked state determination

[0080] In each frame of video, the opening state of the window is detected, and the stability of the state is analyzed in combination with the detection results of consecutive frames. If the window is detected to be open in consecutive frames (usually 2 or 3 frames), it is determined that the window may not be locked.

[0081] Alarm and feedback mechanism

[0082] When the system detects that a window is in an unlocked state, an alarm is triggered immediately, and feedback information is sent to the administrator, indicating the specific window location and state. The system can also save images and videos of the detection results for the administrator to further verify and handle.

[0083] Process summary

[0084] Install cameras and set up monitoring areas: Ensure that the monitoring range covers all windows.

[0085] Neural network automatically identifies windows: Identify the location and outline of the window.

[0086] Background modeling and motion detection: Detect the motion state of the door and window.

[0087] Window state geometry analysis: Evaluate whether the motion region overlaps with the window region.

[0088] Continuous state detection and locked state determination: Determine whether the door and window are unlocked based on consecutive frames.

[0089] Alarm and feedback mechanism: Trigger an alarm when an unlocked state is detected and send specific location and state information to the administrator.

[0090] These steps collectively form a complete solution, ensuring efficient, real-time, and accurate detection of the unlocked state of the door and window of a glass curtain wall building.

[0091] Example 1:

[0092] 1. Install cameras and set up monitoring areas: Install fixed cameras at the bottom of the glass curtain wall building to ensure that the monitoring range covers all windows. The cameras will capture real-time video streams of the door and window areas for subsequent state analysis and detection.

[0093] 2. Neural network automatic window recognition: A neural network model is used to process video frames and automatically identify all window locations and contours of glass curtain walls. The goal of this neural network model is to output the classification results and bounding boxes of window regions based on the input image information. Assuming that the input image is □, the neural network learns the feature representation of the window region □(□) through several layers of nonlinear transformation. The basic structure of the neural network can be represented as:

[0094] F(X) = σ(W n (…σ(W2(σ(W1X + b1) + b2)…) + b n )X is the input image or image feature, W1, W2, … W n are the weight matrices of each layer, b1, b2, …, b n are the bias terms of each layer, and σ is the activation function. After processing by the neural network, the output is the bounding box coordinates B and the classification result C of the window region, i.e. [B, C] = F(X), where B = [x min , y min , x max , y max ] represents the bounding box coordinates of the window, and C represents the classification label of the window region. During training, the neural network adjusts its weights and bias terms by minimizing the loss function, which usually includes a bounding box regression loss L bbox and a classification loss L cls , i.e. L = L cls + λL bbox , where □ is a hyperparameter that balances the classification loss and the bounding box regression loss. This neural network model accurately segments the windows based on the characteristics of the building structure and outputs the bounding boxes of the windows to the subsequent geometric analysis module, ensuring that each window can be accurately identified and located even under complex lighting conditions or different viewing angles.

[0095] 3. Background modeling and motion detection: Based on the K-Nearest Neighbor algorithm (KNN), the motion regions in the video are detected and the background of the video frames is modeled. This background subtraction method analyzes the differences between the current frame and the background frame by frame to generate a binary mask image that identifies possible motion regions (such as open doors and windows). The system optimizes the motion detection mask through morphological operations (such as closing and opening operations) to remove noise and small interference points, ensuring that only motion regions related to doors and windows are retained. Assuming that the current frame is F t , the background frame is B, and the binary mask image M t of the motion region can be obtained through the formula:

[0096]

[0097] , where is an indicator function, which determines the foreground (moving region) when the pixel difference between the current frame and the background frame is greater than a preset threshold.

[0098] To eliminate noise and small non-relevant regions, the system performs morphological processing on the motion mask M t , including closing and opening operations:

[0099] M t = MorphClose(MorphOpen(M t , kernel1), kernel2)

[0100] where kernel1 and kernel2 are the structure elements used for morphological operations, ensuring the smoothness and integrity of the moving region.

[0101] 4. Geometric analysis of window state: The system uses the window bounding box B = [x min , y min , x max , y max ] detected from the neural network, combined with the moving region detected by background modeling, to evaluate whether the moving region overlaps with the window region by calculating the intersection over union (IoU). The formula for calculating IoU is:

[0102]

[0103] where B is the window bounding box identified by the neural network, M t is the motion detection mask, B∩M t is the intersection area of the window region and the moving region, and |B| and |M t | are the areas of the window region and the moving region, respectively. If IoU > Threshold IoU , the system will consider that the window is in a moving state (i.e., it may not be locked). In addition, the system also calculates the Euclidean distance D between the moving region and the center point of the window to further verify the accuracy of the detection. The formula for the Euclidean distance is:

[0104]

[0105] where (x B , y B ) is the center point coordinate of the window bounding box B, and (x M , y M ) is the center point coordinate of the moving region. If the Euclidean distance D is less than a certain threshold Threshold D , it is further confirmed that the moving region is related to the window.

[0106] 5. Continuous state detection and locked state determination: The system detects the opening state of the window in each frame of video, but in order to improve the accuracy of detection and reduce false positives, the system will further analyze the detection results of consecutive frames. Assuming that the system detects the opening state of the window in the t frame, the system will continue to detect the window in the next t+1 frame, t+2 frame, etc., to ensure the stability of the state. The state of each window is recorded as S t , if the window is detected to be open in two or more consecutive frames, it is determined that the window may not be locked. The specific process is as follows: first, assuming that the opening of the window is detected in the t frame, its state is recorded as 1 (indicating opening). If the opening state S t+1 =1 is still detected in the next frame t+1, and the state duration difference between the two frames satisfies the condition:

[0107] ΔT=t+1-t=1

[0108] , the window can be preliminarily judged to be unlocked. In order to further confirm the stability of the opening state, the system needs to detect more frames, if the opening state S t =1 is satisfied in the continuous n frames, it is considered that the window is not locked, the specific condition is:

[0109] S t =S t+1 =S t+2 =…=S t+n =1

[0110] , n is the preset continuous frame number threshold, usually set to 2 or 3 frames, to ensure the stability of the detection and reduce false detection. In the detection process, if one frame state changes, such as in the t+k frame state S t+n =0 (indicating that the opening is not detected), the system will start monitoring again, ignoring the current detection result, until the opening state is detected again. The final judgment condition can be expressed as:

[0111]

[0112] 6. Alarm and feedback mechanism: When detecting that a window is in an unlocked state, the system will immediately trigger an alarm and send feedback information to the building management personnel, indicating the specific window location and unlocked state. The system can save the images and videos of the detection results for the management personnel to verify and handle later.

[0113] By introducing the combination of deep neural networks and background modeling, the status of doors and windows can be automatically recognized and detected, enabling real-time monitoring of large-scale glass curtain wall buildings and solving the problem of low efficiency and easy to miss detection in traditional manual inspection. The core advantage lies in the efficient neural network processing of window recognition in complex environments, combined with motion detection and geometric analysis, ensuring accurate detection of unlocked doors and windows in various weather conditions, timely warning, and avoiding safety hazards.

[0114] By using the method of combining deep neural networks and background modeling, automatic recognition and unlocked state detection of doors and windows in glass curtain wall buildings are achieved. The innovation lies in the use of neural networks to detect and locate windows in glass curtain wall buildings, combined with techniques such as Intersection over Union (IoU) and Euclidean distance, to improve the accuracy and reliability of detection, especially in complex lighting conditions.

[0115] Example 2: Unlocked door and window detection under normal lighting conditions

[0116] Scene description:

[0117] A high-rise glass curtain wall building needs to ensure that all windows are locked before the arrival of a typhoon. Fixed cameras have been installed at the bottom of the building, covering all window areas.

[0118] Operation steps:

[0119] Install cameras and set up monitoring areas

[0120] The camera is installed at the bottom of the building to ensure that the monitoring range covers all windows.

[0121] The camera captures real-time video streams of the door and window areas.

[0122] Neural network automatic window recognition

[0123] Use the trained deep neural network model to process video frames to identify the location and contour of all windows.

[0124] Output the bounding box coordinates and classification labels of the window area.

[0125] Background modeling and motion detection

[0126] Use K-Nearest Neighbor algorithm (KNN) for background modeling of video frames to detect the motion state of doors and windows.

[0127] Remove noise and small interference points through morphological operations such as closing and opening operations to optimize the motion detection mask.

[0128] Window state geometric analysis

[0129] The intersection over union (IoU) and Euclidean distance between the window bounding box and the motion region are calculated to evaluate and verify the detection results.

[0130] If the IoU exceeds a set threshold and the Euclidean distance is less than a preset threshold, it is further confirmed that the motion region is related to the window.

[0131] Continuous state detection and locked state determination

[0132] If the window opening state is detected in consecutive n frames, it is determined that the window is not locked, where n is a preset threshold of consecutive frames.

[0133] If the window opening is detected in consecutive frames (e.g. 2 or 3 frames), it is determined that the window is not locked.

[0134] Alarm and feedback mechanism

[0135] When a window is detected in an unlocked state, the system immediately triggers an alarm and sends specific location and status information to the administrator.

[0136] After receiving the alarm information, the administrator can view the saved detection result images and videos for subsequent verification and processing.

[0137] Effect description

[0138] This embodiment shows that under normal lighting conditions, the system can efficiently and accurately identify and detect unlocked doors and windows, notify the administrator in time for processing, and ensure the safety of the building before extreme weather arrives.

[0139] Example 3: Unlocked door and window detection under complex lighting conditions

[0140] Scene description

[0141] A glass curtain wall building needs to check the locking status of all windows during the dusk period. At this time, the lighting conditions are relatively complex, and as the sun sets, the light gradually darkens.

[0142] Operation steps:

[0143] Installation of camera and monitoring area setting

[0144] Same as Example 1.

[0145] Neural network automatic window recognition

[0146] Use a specially trained deep neural network model to process video frames to adapt to complex lighting conditions.

[0147] Ensure that each window can be accurately recognized and located under different viewing angles.

[0148] Background modeling and motion detection

[0149] The same as example 1, but note the effect of background modeling under complex lighting conditions.

[0150] Window state geometry analysis

[0151] The same as example 1, but further improve the accuracy and reliability of detection under complex lighting conditions through techniques such as Intersection over Union (IoU) and Euclidean distance.

[0152] Continuous state detection and locked state judgment

[0153] The same as example 1, ensuring that the system can correctly determine that the window is not locked when the window opening state is detected in consecutive frames.

[0154] Alarm and feedback mechanism

[0155] The same as example 1, ensuring that even under complex lighting conditions, the management personnel can be notified in time for processing.

[0156] Effect description

[0157] This example shows that even under complex lighting conditions, the technology of the present application can ensure the accuracy of recognition, timely discovery and reporting of unlocked doors and windows, and protection of building safety.

[0158] These two examples show that the present application can effectively operate under different lighting conditions, ensuring that the unlocked state of doors and windows in glass curtain wall buildings is detected and processed in time.

[0159] The above is only a preferred embodiment of the present application, and does not limit the scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or direct / indirect application in other related technical fields within the inventive concept of the present application, as described in the present application and the drawings, are included in the protection scope of the present application.

Claims

1. An open fan unlatching recognition system based on deep neural network and background modeling, characterized in that, The system comprises: a camera for collecting video frames covering the door and window area; a deep neural network module for locating the window in the video frames and outputting the bounding box B and classification label C of the window area; a background modeling and motion detection module for differentiating the video frame from a background based on K-Nearest Neighbor background modeling background subtraction to generate a foreground binary mask M t and performing morphological opening and closing operations on M t to remove noise and small interference points to obtain a processed foreground mask; a geometric analysis module for calculating the intersection over union IoU and the center point Euclidean distance D between the bounding box B and the motion area indicated by the processed foreground mask in each frame as input; a locking state judgment module for determining whether the corresponding window sash is in an open / unlocked state according to the relationship between the intersection over union IoU and the center point Euclidean distance D and the preset threshold, and combining the consistency criterion of consecutive n frames, wherein n is a preset continuous frame number threshold; an alarm and feedback module for triggering an alarm and sending position and state information when it is determined to be unlocked.

2. The system of claim 1, wherein, The deep neural network module outputs the window area bounding box coordinates and classification label by performing nonlinear transformation on the video frames.

3. The system of claim 1, wherein, The alarm and feedback module includes multiple alarm mechanisms, including SMS, email or real-time monitoring system notifications.

4. An opening fan unlatching recognition method based on deep neural network and background modeling, based on the system of any one of claims 1-3, characterized in that, The method comprises the following steps: S1: Collecting video frames covering the door and window area; S2: Locating the window using a deep neural network and outputting the bounding box B and classification label C; S3: Background subtraction using K-Nearest Neighbors background modeling to obtain foreground binary mask M t and performing morphological opening and closing operations on M t to obtain a processed foreground mask; S4: Taking the bounding box B and the processed foreground mask as input, calculating the intersection over union IoU and the center point Euclidean distance D between the window bounding box and the motion area indicated by the processed mask; S5: Based on the intersection over union IoU and the center point Euclidean distance D satisfying the preset threshold relationship, and when the consecutive n frames all satisfy, it is determined that the corresponding window sash is unlocked, wherein n is a preset continuous frame number threshold; S6: Triggering an alarm and feeding back position and state information.

5. The method of claim 4, wherein, The deep neural network model obtains the feature representation of the window area by performing nonlinear transformation on the input video frames, and outputs the bounding box coordinates and classification label of the window area.

6. The method of claim 4, wherein, The alarm and feedback mechanism includes multiple ways, including SMS, email or real-time monitoring system notifications.

Images