Open sash unlocking identification method and system based on deep neural network and background modeling

Through the combination of deep neural network and background modeling, the automation and real-time monitoring of doors and windows of glass curtain wall buildings is achieved, solving the problems of low efficiency, high cost and limitations in the existing technology, and ensuring safety under extreme weather conditions and accuracy.

CN119296007AActive Publication Date: 2025-01-10DECORATION CO LTD OF CHINA CONSTR 3RD ENG BUREAU +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411568917.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-05
Publication Date
2025-01-10
Estimated Expiration
2044-11-05

AI Technical Summary

Technical Problem

The existing glass curtain wall building doors and windows detection methods are inefficient, costly, obvious limitations, and are susceptible to environmental impacts. Real-time monitoring of the entire building cannot be achieved, especially in extreme weather conditions.

Method used

The system based on deep neural network and background modeling is adopted to capture video streams in real time through the camera. The deep neural network module automatically recognizes the window position and outline, and the background modeling and motion detection module detects the motion state, combining geometric analysis and continuous frame detection to determine the failure. Locked state and trigger alarm and feedback.

Benefits of technology

It realizes automated and real-time monitoring of large-scale buildings, improves detection efficiency and accuracy, reduces false and missed inspections, has anti-interference and high reliability, and can accurately identify unlocked doors and windows under complex lighting conditions. , and report to the management personnel in a timely manner.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119296007A_ABST
    Figure CN119296007A_ABST
Patent Text Reader

Abstract

The invention provides an open sash unlocking identification method and system based on a deep neural network and background modeling, and the system comprises a camera which is used for capturing a video stream of a door and window region in real time; the deep neural network module is used for automatically identifying the position and contour of a window and outputting bounding box coordinates and classification labels of a window area; the background modeling and motion detection module is used for detecting motion states of doors and windows; the geometric analysis module is used for calculating the intersection-to-union ratio and the Euclidean distance between the window boundary frame and the motion area so as to evaluate and verify the detection result; the locking state judgment module is used for judging whether the door and window are not locked or not according to the detection result of the continuous frames; and the alarm and feedback module is used for triggering alarm and sending specific position and state information to management personnel. According to the invention, by combining the deep neural network and the background modeling technology, automatic detection of the unlocked state of the large-scale glass curtain wall building door and window is realized, the traditional manual detection is replaced, and the detection efficiency and accuracy are greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of building safety technology, and in particular to a method and system for identifying an unlocked open door based on a deep neural network and background modeling. Background Art

[0002] Existing methods for detecting unlocked doors and windows of glass curtain wall buildings mainly rely on manual inspection. Workers need to confirm whether each window is locked one by one. This practice is particularly inconvenient in large buildings because it takes a lot of time and labor to complete a comprehensive inspection. In addition, some existing methods focus on developing independent detection tools, but the application scenarios of these tools are usually limited to the detection of a single window, and real-time monitoring of the entire building cannot be achieved. In the face of extreme weather (such as typhoons or heavy rains), manual inspection is not only inefficient but also has certain risks.

[0003] Deficiencies of existing technology: 1. Low efficiency: Manual inspection efficiency is very low, especially in large buildings. It is difficult to complete accurate inspection of the status of all doors and windows in a short time.

[0004] 2. High cost: Since a lot of manual work is required, the cost of the entire testing process is relatively high.

[0005] 3. Obvious limitations: Some existing automatic detection equipment is usually only applicable to a single window, cannot achieve comprehensive detection of the entire building, and lacks a global perspective.

[0006] 4. Susceptible to environmental influences: Existing technologies are easily affected by external environmental factors such as weather and light, which may lead to missed detections or false detections, especially in poor lighting conditions.

[0007] 5. Poor adaptability: With the increase of high-rise buildings in cities, especially the widespread use of glass curtain wall buildings, the limitations of existing technologies have become more prominent and cannot effectively meet actual needs.

[0008] Therefore, the prior art has deficiencies and needs further improvement. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a method and system for identifying an unlocked open door based on deep neural network and background modeling.

[0010] To achieve the above object, the specific scheme of the present invention is as follows: The present invention provides an unlocked door recognition system based on deep neural network and background modeling, characterized in that the system comprises: Cameras, installed at the bottom of glass curtain wall buildings, are used to capture real-time video streams of door and window areas; A deep neural network module that automatically identifies the location and outline of windows and outputs the bounding box coordinates and classification labels of the window area; Background modeling and motion detection module, used to detect the motion status of doors and windows, and remove noise and small interference points through morphological operations to optimize the motion detection mask; A geometric analysis module that calculates the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion region to evaluate and verify the detection results; A locking state judgment module is used to judge whether the doors and windows are unlocked according to the detection results of consecutive frames; The alarm and feedback module is used to trigger an alarm when an unlocked state is detected and send specific location and status information to the management personnel.

[0011] Furthermore, the deep neural network module outputs the window area bounding box coordinates and classification labels by performing nonlinear transformation on the video frames.

[0012] Furthermore, the background modeling and motion detection module uses a K-nearest neighbor algorithm (KNN) to perform background subtraction and optimizes the motion detection mask through morphological processing.

[0013] Furthermore, the system also includes: a continuous state detection module, which is used to detect the window open state in n consecutive frames, and then determine that the window is not locked, wherein n is a preset continuous frame number threshold.

[0014] Furthermore, the alarm and feedback module includes a variety of alarm mechanisms, including SMS, email or notification of real-time monitoring system.

[0015] The present invention also provides a method for identifying an unlocked door based on a deep neural network and background modeling. Based on the above system, the method includes the following steps: S1, install fixed cameras at the bottom of the glass curtain wall building to monitor the status of each window in real time; S2, uses a deep neural network model to automatically identify the location and outline of the window and output the bounding box coordinates and classification labels of the window area; S3, performs background modeling and motion detection on the video frames, detects the motion status of doors and windows, removes noise and small interference points through morphological operations, and optimizes the motion detection mask; S4, calculates the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion area, evaluates whether the motion area overlaps with the window area, and verifies the detection results; S5, judging whether the doors and windows are unlocked according to the detection results of the continuous frames; S6, when an unlocked state is detected, an alarm is triggered and specific location and status information is sent to the management personnel.

[0016] Furthermore, the deep neural network model obtains a feature representation of the window area by performing a nonlinear transformation on the input video frame, and outputs the bounding box coordinates and classification label of the window area.

[0017] Furthermore, the background modeling uses a K-nearest neighbor algorithm (KNN) to detect motion areas in the video, removes noise and small interference points through morphological operations, and optimizes the motion detection mask.

[0018] Furthermore, the continuous state detection and locking state judgment includes: if the window opening state is detected in n consecutive frames, it is judged that the window is not locked, wherein n is a preset continuous frame number threshold.

[0019] Furthermore, the alarm and feedback mechanism includes multiple methods, including text messages, emails or notifications from real-time monitoring systems.

[0020] The technical solution of the present invention has the following beneficial effects: 1. Automated detection: By combining deep neural networks with background modeling technology, we have achieved automated detection of unlocked doors and windows in large-scale glass curtain wall buildings, replacing traditional manual inspections and greatly improving detection efficiency and accuracy.

[0021] 2. Real-time monitoring: The system can monitor the status of all windows in the building in real time, promptly detect and report unlocked doors and windows, especially before extreme weather arrives, and can provide early warning to ensure the safety of the building.

[0022] 3. Comprehensive coverage: Different from the existing single window detection tools, the present invention can comprehensively cover the entire building and monitor all windows, thus solving the problem of only partial detection in the past.

[0023] 4. Strong anti-interference ability: Through the feature extraction capability of the neural network and the motion detection algorithm of background modeling, the system can maintain good recognition performance under complex lighting conditions, reducing false detection or missed detection caused by environmental changes.

[0024] 5. High reliability: The continuous frame detection method is adopted to ensure the stability of the detection results, reduce the error caused by single frame detection, and improve the reliability and accuracy of the overall system.

[0025] 6. Timely feedback: The system has an instant alarm function. Once it detects that the doors and windows are not locked, it can send an alarm to the management personnel and provide specific location information to facilitate timely measures.

[0026] 7. Wide applicability: The present invention is applicable to any form of glass curtain wall buildings, is not limited by building height and window style, and has wide practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of unlocked windows in a building; Figure 2 is an example diagram of the neural network of the present invention identifying whether a window is locked; Figure 3 is a photo of an unlocked window fed back to the building management personnel by the alarm feedback mechanism of the present invention; Figure 4 is a system block diagram of the present invention; Figure 5 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments; it can be understood that the specific embodiments described herein are only used to explain the present invention rather than to limit the present invention; it should also be noted that, for ease of description, only the parts related to the present invention rather than all are shown in the accompanying drawings.

[0029] Combination Figure 1-Figure 5 As shown, the present invention provides an unlocked door recognition system based on deep neural network and background modeling, the system comprising: Cameras, installed at the bottom of glass curtain wall buildings, are used to capture real-time video streams of door and window areas; A deep neural network module that automatically identifies the location and outline of windows and outputs the bounding box coordinates and classification labels of the window area; Background modeling and motion detection module, used to detect the motion status of doors and windows, and remove noise and small interference points through morphological operations to optimize the motion detection mask; A geometric analysis module that calculates the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion region to evaluate and verify the detection results; A locking state judgment module is used to judge whether the doors and windows are unlocked according to the detection results of consecutive frames; The alarm and feedback module is used to trigger an alarm when an unlocked state is detected and send specific location and status information to the management personnel.

[0030] The deep neural network module outputs the window area bounding box coordinates and classification labels by performing nonlinear transformation on the video frames.

[0031] The background modeling and motion detection module uses the K-nearest neighbor algorithm (KNN) to perform background subtraction and optimizes the motion detection mask through morphological processing.

[0032] The system also includes: a continuous state detection module, which is used to detect the window opening state in n consecutive frames, and then determine that the window is not locked, wherein n is a preset continuous frame number threshold.

[0033] The alarm and feedback module includes multiple alarm mechanisms, including SMS, email or notification of real-time monitoring system.

[0034] The present invention also provides a method for identifying an unlocked door based on a deep neural network and background modeling. Based on the above system, the method includes the following steps: S1, install fixed cameras at the bottom of the glass curtain wall building to monitor the status of each window in real time; S2, uses a deep neural network model to automatically identify the location and outline of the window and output the bounding box coordinates and classification labels of the window area; S3, performs background modeling and motion detection on the video frames, detects the motion status of doors and windows, removes noise and small interference points through morphological operations, and optimizes the motion detection mask; S4, calculates the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion area, evaluates whether the motion area overlaps with the window area, and verifies the detection results; S5, judging whether the doors and windows are unlocked according to the detection results of the continuous frames; S6, when an unlocked state is detected, an alarm is triggered and specific location and status information is sent to the management personnel.

[0035] The deep neural network model obtains the feature representation of the window area by performing nonlinear transformation on the input video frame, and outputs the bounding box coordinates and classification label of the window area.

[0036] The background modeling uses a K-nearest neighbor algorithm (KNN) to detect motion areas in a video, removes noise and small interference points through morphological operations, and optimizes motion detection masks.

[0037] The continuous state detection and locking state judgment includes: if the window opening state is detected in n consecutive frames, it is judged that the window is not locked, wherein n is a preset continuous frame number threshold.

[0038] The alarm and feedback mechanism includes multiple methods, including text messages, emails or notifications from real-time monitoring systems.

[0039] Working principle overview: Camera installation and video acquisition Fixed cameras are installed at the bottom of glass curtain wall buildings to ensure that all windows are within the monitoring range. The cameras capture the video stream of the door and window areas in real time, providing a data source for subsequent analysis.

[0040] Neural network automatic recognition The pre-trained deep neural network model is used to process the video frames and automatically identify the position and outline of all windows. The neural network learns the feature representation of the window area through multiple layers of nonlinear transformation and outputs the bounding box coordinates and classification labels of the window area. During the training process, the model adjusts its weights and bias terms by minimizing the loss function to ensure that each window can be accurately identified and located even under complex lighting conditions.

[0041] Background modeling and motion detection Based on the K-nearest neighbor algorithm (KNN), the background modeling of the video frame is performed, and the difference between the current picture and the background is analyzed frame by frame to generate a binary mask map to identify possible motion areas (such as open doors and windows). The system optimizes the motion detection mask through morphological operations (such as closing and opening operations), removes noise and small interference points, and ensures that only the motion areas related to doors and windows are retained.

[0042] Window State Geometry Analysis Combining the window bounding box identified by the neural network and the motion area detected by background modeling, the intersection over union (IoU) is calculated to assess whether the motion area overlaps with the window area. In addition, the system calculates the Euclidean distance between the motion area and the center point of the window to further verify the accuracy of the detection results.

[0043] Continuous state detection and locking state judgment The window opening status is detected in each frame of the video, and the stability of the status is analyzed by combining the detection results of consecutive frames. If the window is detected to be open in several consecutive frames (usually 2 or 3 frames), it is judged that the window may not be locked.

[0044] Alarm and feedback mechanism When the system detects that a window is unlocked, it immediately triggers an alarm and sends feedback to the management staff, indicating the specific window location and status. The system can also save images and videos of the detection results for further verification and processing by the management staff.

[0045] Process Summary Install cameras and set up monitoring areas: Make sure the monitoring area covers all windows.

[0046] Neural network automatically recognizes windows: identifies the location and outline of windows.

[0047] Background modeling and motion detection: Detect the motion status of doors and windows.

[0048] Window state geometry analysis: Evaluate whether the motion region overlaps with the window region.

[0049] Continuous state detection and locking state judgment: judge whether the doors and windows are unlocked based on the continuous frames.

[0050] Alarm and feedback mechanism: When an unlocked state is detected, an alarm is triggered and the specific location and status information is sent to the management personnel.

[0051] These steps together form a complete solution that ensures efficient, real-time and accurate detection of the unlocked status of doors and windows in glass curtain wall buildings.

[0052] Embodiment 1: 1. Camera installation and monitoring area setting: Install fixed cameras at the bottom of the glass curtain wall building to ensure that the monitoring range covers all windows. The camera will capture the video stream of the door and window area in real time for subsequent status analysis and detection.

[0053] 2. Automatic window recognition using neural network: A neural network model is used to process video frames and automatically identify the positions and outlines of all windows in the glass curtain wall. The goal of this neural network model is to output the classification results and bounding boxes of the window area based on the input image information. Assuming the input image is 𝑋, the neural network learns the feature representation 𝐹(𝑋) of the window area through several layers of nonlinear transformation. The basic structure of the neural network can be expressed as:

[0054] is the input image or image feature, is the weight matrix of each layer, is the bias term of each layer, is the activation function. After being processed by the neural network, the output is the bounding box coordinates B of the window area and the classification result C, that is, ,in Denotes the bounding box coordinates of the window, and C denotes the classification label of the window area. During training, the neural network adjusts its weights and bias terms by minimizing the loss function, which usually includes the bounding box regression loss. and classification loss Right now , where 𝜆 is a hyperparameter that balances the classification loss and bounding box regression loss. The neural network model accurately segments the windows according to the characteristics of the building structure and outputs the bounding box of the window to the subsequent geometric analysis module, ensuring that each window can be accurately identified and located even under complex lighting conditions or different viewing angles.

[0055] 3. Background modeling and motion detection: Detect motion areas in the video based on the K-nearest neighbor algorithm (KNN) and perform background modeling on the video frame. This background subtraction method analyzes the difference between the current image and the background frame by frame, generates a binary mask image, and identifies possible motion areas (such as open doors and windows). The system optimizes the motion detection mask through morphological operations (such as closing and opening operations) to remove noise and small interference points, ensuring that only motion areas related to doors and windows are retained. Assume that the current frame is , the background frame is , binary mask map of motion area The formula can be:

[0056] Obtain, among which, It is an indicator function. When the pixel difference between the current frame and the background frame is greater than the preset threshold, it is judged as the foreground (moving area). In order to eliminate noise and small non-correlated areas, the system uses a motion mask Perform morphological processing, including closing and opening operations:

[0057] in, and They are structural elements used for morphological operations to ensure the smoothness and integrity of the motion area.

[0058] 4. Geometric analysis of window states: The system uses the window bounding boxes detected by the neural network , combined with the motion area detected by background modeling, the intersection over union (IoU) is calculated to evaluate whether the motion area overlaps with the window area. The calculation formula of IoU is:

[0059] in, is the window bounding box identified by the neural network, is the motion detection mask, is the intersection area of ​​the window area and the motion area, and are the areas of the window area and the motion area respectively. If , the system will consider the window to be in motion (i.e., possibly unlocked). In addition, the system also calculates the Euclidean distance between the motion area and the center point of the window , to further verify the accuracy of the detection. The formula for Euclidean distance is:

[0060] in, is the window bounding box The center point coordinates, is the coordinate of the center point of the motion area. If the Euclidean distance Less than a certain threshold , it is further confirmed that the motion area is related to the window.

[0061] 5. Continuous state detection and locking state judgment: The system detects the open state of the window in each frame of video. However, in order to improve the accuracy of detection and reduce false alarms, the system will further analyze the detection results of consecutive frames. Assuming that the system detects the open state of the window in the tth frame, the system will continue to detect the window in the next t+1 frame, t+2 frame, etc. to ensure the stability of the state. The state of each window is recorded as If the window is detected to be open in two or more consecutive frames, it is judged that the window may not be locked. The specific process is as follows: First, assume that the window is detected to be open in the tth frame, and record its state as 1 (indicating open). If the open state is still detected in the next frame t+1 , and the state duration difference between the two frames satisfies the condition:

[0062] It can be preliminarily determined that the window is likely to be unlocked. To further confirm the stability of the open state, the system needs to detect more frames. If the open state is satisfied in n consecutive frames, , the window is considered not locked, the specific conditions are:

[0063] Where n is the preset continuous frame number threshold, which is usually set to 2 or 3 frames to ensure the stability of detection and reduce false detection. During the detection process, if a frame state changes, such as the state of the t+kth frame is (meaning that the on state is not detected), the system will restart monitoring and ignore the current detection result until the on state is detected again continuously. The final judgment condition can be expressed by the formula:

[0064] 6. Alarm and feedback mechanism: When a window is detected to be unlocked, the system will immediately trigger an alarm and send feedback information to the building management personnel, indicating the specific window location and unlocked status. The system can save images and videos of the detection results for subsequent verification and processing by the management personnel.

[0065] By introducing a combination of deep neural networks and background modeling, it is able to automatically identify and detect the status of doors and windows, achieving real-time monitoring of large-scale glass curtain wall buildings, and solving the problem of low efficiency and easy missed inspections in traditional manual inspections. Its core advantage lies in the use of efficient neural networks to process window recognition in complex environments, combined with motion detection and geometric analysis, to ensure accurate detection of unlocked doors and windows in all weather conditions, timely warning, and avoid safety hazards.

[0066] The method combines deep neural network with background modeling to realize automatic recognition and unlocked state detection of doors and windows in glass curtain wall buildings. Its innovation lies in detecting and locating windows of glass curtain walls through neural network, and combining technical means such as intersection over union (IoU) and Euclidean distance to improve the accuracy and reliability of detection, especially in complex lighting conditions.

[0067] Example 2: Unlocked Door and Window Detection under Normal Lighting Conditions Scenario description: A high-rise glass curtain wall building needs to ensure that all windows are locked before a typhoon arrives. Fixed cameras have been installed at the bottom of the building to cover all window areas.

[0068] Steps: Camera installation and monitoring area settings The camera is installed at the bottom of the building to ensure that the monitoring range covers all windows.

[0069] The camera captures the video stream of the door and window area in real time.

[0070] Automatic window recognition using neural networks The video frames are processed using a trained deep neural network model to identify the locations and outlines of all windows.

[0071] Output the bounding box coordinates and classification label of the window area.

[0072] Background modeling and motion detection The K-nearest neighbor algorithm (KNN) is used to model the background of the video frames and detect the movement status of doors and windows.

[0073] The motion detection mask is optimized by removing noise and small interference points through morphological operations such as closing and opening.

[0074] Window State Geometry Analysis Calculate the intersection over union (IoU) and Euclidean distance between the window bounding box and the motion region to evaluate and verify the detection results.

[0075] If the IoU exceeds the set threshold and the Euclidean distance is less than the preset threshold, it is further confirmed that the motion area is related to the window.

[0076] Continuous state detection and locking state judgment If the window opening state is detected in n consecutive frames, it is determined that the window is not locked, where n is a preset consecutive frame number threshold.

[0077] If the window is detected to be open in several consecutive frames (for example, 2 or 3 frames), it is determined that the window is not locked.

[0078] Alarm and feedback mechanism When a window is detected to be unlocked, the system immediately triggers an alarm and sends the specific location and status information to the management personnel.

[0079] After receiving the alarm information, the management personnel can view the saved test result images and videos for subsequent verification and processing.

[0080] Effect Description This embodiment demonstrates that under normal lighting conditions, the system can efficiently and accurately identify and detect unlocked doors and windows, and promptly notify management personnel to handle the problem, thereby ensuring the safety of the building before extreme weather arrives.

[0081] Example 3: Unlocked Door and Window Detection under Complex Lighting Conditions Scenario Description A glass curtain wall building needs to check the locking status of all windows during dusk, when the lighting conditions are complex and the light gradually dims as the sun sets.

[0082] Steps: Camera installation and monitoring area settings Same as Example 1.

[0083] Automatic window recognition using neural networks Video frames are processed using a deep neural network model trained specifically to adapt to complex lighting conditions.

[0084] Ensure that each window can be accurately identified and located at different viewing angles.

[0085] Background modeling and motion detection Same as Example 1, but attention should be paid to the effect of background modeling under complex lighting conditions.

[0086] Window State Geometry Analysis Same as Example 1, but under complex lighting conditions, the detection accuracy and reliability are further improved by using technical means such as intersection over union (IoU) and Euclidean distance.

[0087] Continuous state detection and locking state judgment As in Example 1, when the window opening state is detected in several consecutive frames, the system can correctly determine that the window is not locked.

[0088] Alarm and feedback mechanism Same as in Example 1, ensuring that the management personnel can be notified in time for processing even under complex lighting conditions.

[0089] Effect Description This embodiment shows that even under complex lighting conditions, the technology of the present invention can ensure the accuracy of recognition, timely discover and report unlocked doors and windows, and ensure building safety.

[0090] These two embodiments illustrate that the present invention can operate effectively under different lighting conditions, ensuring that the unlocked state of doors and windows in glass curtain wall buildings is detected and processed in a timely manner.

[0091] The above description is only a preferred embodiment of the present invention, and does not limit the scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the protection scope of the present invention.

Claims

1. A system for identifying unlocked doors based on deep neural networks and background modeling, characterized in that: The system includes: Cameras, installed at the bottom of glass curtain wall buildings, are used to capture real-time video streams of door and window areas; A deep neural network module that automatically identifies the location and outline of windows and outputs the bounding box coordinates and classification labels of the window area; Background modeling and motion detection module, used to detect the motion status of doors and windows, and remove noise and small interference points through morphological operations to optimize the motion detection mask; A geometric analysis module that calculates the intersection-over-union ratio and Euclidean distance between the window bounding box and the motion region to evaluate and verify the detection results; A locking state judgment module is used to judge whether the doors and windows are unlocked according to the detection results of consecutive frames; The alarm and feedback module is used to trigger an alarm when an unlocked state is detected and send specific location and status information to the management personnel.

2. The system according to claim 1, characterized in that The deep neural network module outputs the window area bounding box coordinates and classification labels by performing nonlinear transformation on the video frames.

3. The system according to claim 1, characterized in that The background modeling and motion detection module uses a K-nearest neighbor algorithm to perform background subtraction and optimizes the motion detection mask through morphological processing.

4. The system according to claim 1, characterized in that The system also includes: a continuous state detection module, which is used to detect the window opening state in n consecutive frames, and then determine that the window is not locked, wherein n is a preset continuous frame number threshold.

5. The system according to claim 1, characterized in that The alarm and feedback module includes multiple alarm mechanisms, including SMS, email or notification of real-time monitoring system.

6. A method for identifying unlocked open doors based on deep neural network and background modeling, based on the system according to any one of claims 1 to 5, characterized in that: The method comprises the following steps: S1, install fixed cameras at the bottom of the glass curtain wall building to monitor the status of each window in real time; S2, uses a deep neural network model to automatically identify the location and outline of the window and output the bounding box coordinates and classification labels of the window area; S3, performs background modeling and motion detection on the video frames, detects the motion status of doors and windows, removes noise and small interference points through morphological operations, and optimizes the motion detection mask; S4, calculating the intersection-over-union ratio and the Euclidean distance between the window bounding box and the motion area, evaluating whether the motion area overlaps with the window area, and verifying the detection result; S5, judging whether the doors and windows are unlocked according to the detection results of the continuous frames; S6, when an unlocked state is detected, an alarm is triggered and specific location and status information is sent to the management personnel.

7. The method according to claim 6, characterized in that The deep neural network model obtains the feature representation of the window area by performing nonlinear transformation on the input video frame, and outputs the bounding box coordinates and classification label of the window area.

8. The method according to claim 6, characterized in that The background modeling adopts K nearest neighbor algorithm to detect the motion area in the video, removes noise and small interference points through morphological operation, and optimizes the motion detection mask.

9. The method according to claim 6, characterized in that The continuous state detection and locking state judgment includes: if the window opening state is detected in n consecutive frames, it is judged that the window is not locked, wherein n is a preset continuous frame number threshold.

10. The method according to claim 6, characterized in that The alarm and feedback mechanism includes multiple methods, including text messages, emails or notifications from real-time monitoring systems.

Citation Information

Patent Citations

  • Children high-altitude protection method based on deep learning model

    CN110826406A

  • Method and device for detecting open states of opening windows of glass curtain wall, equipment and medium

    CN113435508A

  • Window state detection method based on machine vision

    CN114677604A

  • Grain depot ventilation window state detection algorithm based on deep learning

    CN114821154A

  • Curtain wall opening and closing state detection method based on unmanned aerial vehicle, inspection system and medium

    CN117911907A