A machine vision-based elevator door abnormal state real-time monitoring system

Abstract

The application belongs to the technical field of elevator safety detection, and specifically relates to a real-time monitoring system for abnormal states of an elevator door based on machine vision, which comprises a visual acquisition module, an edge calculation module, a cloud analysis module and an early warning execution module; the visual acquisition module is used for acquiring multi-view visual data in the running process of the elevator door and performing pretreatment; a signal output end of the visual acquisition module is connected with a signal input end of the edge calculation module; the edge calculation module is internally provided with a multi-modal machine vision algorithm library, used for performing real-time analysis on the visual data and identifying abnormal states of the elevator door; the cloud analysis module is connected with a signal output end of the edge calculation module and used for performing deep analysis on historical monitoring data; and the early warning execution module is connected with a signal output end of the edge calculation module and an output end of the cloud analysis module, thereby solving the problems of incomplete, inaccurate and non-real-time detection of the elevator door in the prior art.

Description

Technical Field

[0001] This application belongs to the field of elevator safety inspection technology, specifically a real-time monitoring system for abnormal elevator door status based on machine vision. Background Technology

[0002] As an indispensable vertical transportation tool in modern urban high-rise buildings, the safe operation of elevators is directly related to people's lives and property safety. According to statistics, elevator door system failure is one of the main causes of elevator safety accidents, accounting for more than 60%. Common failures include door jamming, people or objects being trapped, excessively large door gaps, and opening / closing timeouts.

[0003] Traditional elevator door monitoring methods mainly rely on mechanical contact sensors and infrared sensing devices, which have many limitations: First, the monitoring dimensions are limited, only able to detect specific types of anomalies, and cannot comprehensively cover complex faults during elevator door operation; second, the response is delayed, often only being detected after a fault occurs or even causes a safety hazard, making it difficult to provide early warnings; third, they are greatly affected by environmental interference, and are prone to false alarms and missed alarms in complex scenarios such as strong light, dust, and electromagnetic interference; fourth, they lack data analysis and predictive capabilities, and cannot mine potential fault patterns from historical data, resulting in elevator maintenance being mostly reactive, costly, and inefficient.

[0004] In recent years, with the development of machine vision technology, some elevator door monitoring systems have begun to incorporate visual recognition technology. However, existing solutions still have shortcomings: most only detect single fault types, such as door jamming or object clamping; the algorithm models have poor robustness, and the recognition accuracy drops significantly in complex real-world scenarios such as changes in elevator door lighting or obstruction by dirt; data processing relies on cloud servers, which carries the risk of network latency and makes it difficult to meet the needs of real-time monitoring. Therefore, there is an urgent need for an intelligent system that can comprehensively, accurately, and in real-time monitor abnormal elevator door conditions. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of existing technologies by designing a machine vision-based real-time monitoring system for abnormal elevator door states. This system utilizes a visual acquisition module, an edge computing module, a cloud analysis module, and an early warning execution module to accurately identify, warn of, and predict faults in various abnormal elevator door states in real time. This improves elevator operation safety and the level of intelligent operation and maintenance, and solves the problems of incomplete, inaccurate, and unreal-time detection of elevator doors in existing technologies.

[0006] The above-mentioned technical objective of this application is achieved through the following technical solution: A machine vision-based real-time monitoring system for abnormal elevator door status includes a vision acquisition module, an edge computing module, a cloud analysis module, and an early warning execution module. The visual acquisition module is used to acquire multi-view visual data during the operation of the elevator door and perform preprocessing; the signal output terminal of the visual acquisition module is connected to the signal input terminal of the edge computing module. The edge computing module has a built-in multimodal machine vision algorithm library to perform real-time analysis of visual data and identify abnormal elevator door states. The cloud-based analysis module is connected to the signal output terminal of the edge computing module to perform in-depth analysis of historical monitoring data, thereby enabling fault prediction and operation and maintenance management. The early warning execution module is connected to the signal output terminal of the edge computing module and the output terminal of the cloud analysis module, and is used to execute early warning actions and elevator control commands.

[0007] Preferably, the visual acquisition module includes multiple sets of high-definition industrial cameras, which are evenly distributed on the top of the elevator car and the sides of the shaft. Each high-definition industrial camera has an adaptive lighting adjustment function, and the acquisition frame rate of each high-definition industrial camera is not less than 60fps.

[0008] Preferably, the multimodal machine vision algorithm library of the edge computing module includes a door contour detection algorithm, a dynamic feature analysis algorithm, an obstacle recognition algorithm, and a door gap anomaly detection algorithm, wherein: The door contour detection algorithm is based on an improved Hough transform and is used to locate the elevator door position and opening angle. The dynamic feature analysis algorithm is based on dynamic time warping technology and is used to compare the real-time running trajectory of the elevator door with a normal template to identify dynamic anomalies. The obstacle recognition algorithm integrates the YOLOv8 deep learning model and semantic segmentation algorithm to identify obstacles within the elevator door area; The door gap anomaly detection algorithm is based on image grayscale difference analysis technology and is used to detect door gap anomalies after the elevator door is closed.

[0009] Preferably, in the door contour detection algorithm, the position and opening / closing angle of the elevator door are located with an error controlled within ±0.5mm.

[0010] Preferably, the cloud analysis module has a built-in fault prediction model based on long short-term memory network, which can predict potential elevator door faults 7-15 days in advance by learning from historical monitoring data.

[0011] Preferably, the early warning execution module includes an audible and visual alarm, a signal transmission unit, and a control unit, which can realize local alarm, remote information push and elevator door control functions. The control unit is connected to the audible and visual alarm through the signal transmission unit, and the control unit's signal transmission unit is connected to the edge computing module and the cloud computing module.

[0012] Preferably, the warning response time of the audible and visual alarm is less than 0.5 seconds.

[0013] Preferably, the nonlinear correction term in the topological acoustic propagation model includes compensation terms for medium inhomogeneity gradient, cable bending stiffness coupling effect, and Rayleigh scattering phase perturbation.

[0014] The beneficial effects of this application are: 1. Multi-dimensional and comprehensive monitoring: Breaking through the limitations of traditional single fault monitoring, it can simultaneously identify more than 10 abnormal states such as elevator door jamming, people or objects being trapped, abnormal door gaps, and opening / closing timeouts, achieving full-process monitoring of elevator door operation.

[0015] 2. Real-time and accurate identification: The edge computing architecture combined with optimized machine vision algorithms ensures that the monitoring response time is less than 0.5 seconds and the anomaly identification accuracy rate is over 99.5%, which greatly reduces the false alarm and missed alarm rates.

[0016] 3. Early fault prediction: Based on the LSTM fault prediction model, potential fault patterns are mined from historical data, and early warnings are issued 7-15 days in advance, transforming elevator operation and maintenance from post-remediation to pre-event prevention, reducing operation and maintenance costs by more than 30%.

[0017] 4. Strong adaptability to complex scenes: The camera with adaptive lighting adjustment and robust algorithm can work stably in complex environments such as strong light, low light, and dirt obstruction, and is suitable for elevator scenarios in various residential, commercial and industrial buildings. Detailed Implementation

[0018] A machine vision-based real-time monitoring system for abnormal elevator door status includes a vision acquisition module, an edge computing module, a cloud analysis module, and an early warning execution module. The visual acquisition module is used to acquire multi-view visual data during the operation of the elevator door and perform preprocessing; the signal output terminal of the visual acquisition module is connected to the signal input terminal of the edge computing module, and then the image is transmitted to the edge computing module.

[0019] The edge computing module has a built-in multimodal machine vision algorithm library to perform real-time analysis of visual data and identify abnormal elevator door states.

[0020] The cloud-based analysis module is connected to the signal output of the edge computing module to perform in-depth analysis of historical monitoring data, enabling fault prediction and operation and maintenance management. The edge computing module periodically uploads historical monitoring data to the cloud-based analysis module for refinement. The LSTM fault prediction model learns from the data, analyzes the elevator door operation trend, and sends predictive maintenance suggestions to operation and maintenance personnel when potential fault signs are detected.

[0021] The early warning execution module is connected to the signal output terminals of the edge computing module and the cloud analysis module, and is used to execute early warning actions and elevator control commands. Upon receiving an abnormal signal, the early warning execution module activates the audible and visual alarm inside the elevator car and simultaneously sends a warning message containing details of the abnormality to maintenance personnel. For serious abnormalities, it sends a command to the elevator control system to stop or reopen the elevator doors.

[0022] Preferably, the visual acquisition module includes multiple sets of high-definition industrial cameras, which are evenly distributed on the top of the elevator car and the sides of the shaft. Each high-definition industrial camera has an adaptive lighting adjustment function, and the acquisition frame rate of each high-definition industrial camera is no less than 60fps. The camera's adaptive lighting adjustment function allows it to maintain clear imaging in complex lighting environments inside the elevator, such as strong light, weak light, and backlight, ensuring the capture of every detail of the elevator door's operation. Simultaneously, the visual acquisition module has a built-in image preprocessing unit that performs noise reduction, enhancement, and distortion correction on the acquired raw images, providing a high-quality data foundation for subsequent algorithm analysis.

[0023] Preferably, the multimodal machine vision algorithm library of the edge computing module includes a door contour detection algorithm, a dynamic feature analysis algorithm, an obstacle recognition algorithm, and a door gap anomaly detection algorithm, wherein: The door contour detection algorithm is based on an improved Hough transform and is used to locate the elevator door position and opening angle. The dynamic feature analysis algorithm is based on dynamic time warping technology and is used to compare the real-time running trajectory of the elevator door with a normal template to identify dynamic anomalies. The obstacle recognition algorithm integrates the YOLOv8 deep learning model and semantic segmentation algorithm to identify obstacles within the elevator door area; The door gap anomaly detection algorithm is based on image grayscale difference analysis technology. It scans the door gap area after the elevator door is closed and identifies abnormal situations such as excessively large door gaps or foreign objects stuck by detecting sudden changes in grayscale values ​​at the door gap.

[0024] Preferably, in the door contour detection algorithm, the position and opening / closing angle of the elevator door are located with an error controlled within ±0.5mm.

[0025] Preferably, the cloud analysis module has a built-in fault prediction model based on long short-term memory network, which can predict potential elevator door faults 7-15 days in advance by learning from historical monitoring data.

[0026] Preferably, the early warning execution module includes an audible and visual alarm, a signal transmission unit, and a control unit, enabling local alarm, remote information push, and elevator door control functions. The control unit is connected to the audible and visual alarm via the signal transmission unit, and the signal transmission unit is connected to the edge computing module and the cloud computing module. When the edge module detects a real-time anomaly, it immediately triggers a local audible and visual alarm and simultaneously sends early warning information to maintenance personnel and property management personnel via SMS, APP push, etc. The information includes detailed information such as the anomaly type, occurrence time, and elevator location. For serious anomalies, such as people being trapped or doors failing to close, the module can send a signal to the elevator control system to force the elevator doors to stop or reopen, preventing accidents.

[0027] Preferably, the warning response time of the audible and visual alarm is less than 0.5 seconds, thereby reducing the false alarm and missed alarm rates.

Claims

1. A real-time monitoring system for abnormal elevator door status based on machine vision, characterized in that, It includes a visual acquisition module, an edge computing module, a cloud analysis module, and an early warning execution module; The visual acquisition module is used to acquire multi-view visual data during the operation of the elevator door and perform preprocessing; the signal output terminal of the visual acquisition module is connected to the signal input terminal of the edge computing module. The edge computing module has a built-in multimodal machine vision algorithm library to perform real-time analysis of visual data and identify abnormal elevator door states. The cloud-based analysis module is connected to the signal output terminal of the edge computing module to perform in-depth analysis of historical monitoring data, thereby enabling fault prediction and operation and maintenance management. The early warning execution module is connected to the signal output terminal of the edge computing module and the output terminal of the cloud analysis module, and is used to execute early warning actions and elevator control commands.

2. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 1, characterized in that, The visual acquisition module includes multiple sets of high-definition industrial cameras. The high-definition industrial cameras are evenly distributed on the top of the elevator car and the side of the shaft. Each high-definition industrial camera has an adaptive lighting adjustment function, and the acquisition frame rate of each high-definition industrial camera is not less than 60fps.

3. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 1, characterized in that, The multimodal machine vision algorithm library of the edge computing module includes door contour detection algorithm, dynamic feature analysis algorithm, obstacle recognition algorithm, and door gap anomaly detection algorithm, wherein: The door contour detection algorithm is based on an improved Hough transform and is used to locate the elevator door position and opening angle. The dynamic feature analysis algorithm is based on dynamic time warping technology and is used to compare the real-time running trajectory of the elevator door with a normal template to identify dynamic anomalies. The obstacle recognition algorithm integrates the YOLOv8 deep learning model and semantic segmentation algorithm to identify obstacles within the elevator door area; The door gap anomaly detection algorithm is based on image grayscale difference analysis technology and is used to detect door gap anomalies after the elevator door is closed.

4. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 3, characterized in that, In the door contour detection algorithm, the position and opening angle of the elevator door are located with an error controlled within ±0.5mm.

5. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 1, characterized in that, The cloud-based analysis module has a built-in fault prediction model based on long short-term memory networks. By learning from historical monitoring data, it can predict potential elevator door faults 7-15 days in advance.

6. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 1, characterized in that, The early warning execution module includes an audible and visual alarm, a signal transmission unit, and a control unit, which can realize local alarm, remote information push and elevator door control functions. The control unit is connected to the audible and visual alarm through the signal transmission unit, and the control unit's signal transmission unit is connected to the edge computing module and the cloud computing module.

7. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 1, characterized in that, The warning response time of the audible and visual alarm is less than 0.5 seconds.

8. The real-time monitoring system for abnormal elevator door status based on machine vision according to claim 1, characterized in that: The nonlinear correction term in the topological acoustic propagation model includes compensation terms for medium inhomogeneity gradient, cable bending stiffness coupling effect, and Rayleigh scattering phase perturbation.