Elevator safety monitoring system fused with computer vision and method thereof

Abstract

The invention relates to the technical field of safety monitoring, in particular to an elevator safety monitoring system and method fused with computer vision, and the system comprises a data collection module which is used for synchronously collecting video data, sensor data and environment parameter data of an elevator; the data processing module is used for carrying out preprocessing and feature fusion on the collected data and outputting a fusion feature vector F; the risk assessment module is used for calculating a dynamic risk score according to the fusion feature vector; and the alarm module is used for triggering an alarm when the dynamic risk score exceeds a preset threshold value. According to the method, the feature weight is dynamically adjusted through the Bayesian optimization algorithm, so that important risk features obtain higher weight distribution; and meanwhile, the LSTM network is utilized to predict the dynamic threshold value based on the historical risk sequence, safety references of the elevator in different time periods and under different working conditions are adapted in real time, the problems of false report and missing report of a traditional fixed threshold value when the working conditions change are solved, and the dynamic adaptability and accuracy of risk assessment are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of security monitoring technology, specifically to a safety monitoring system and method for elevators that integrates computer vision. Background Technology

[0002] As an indispensable vertical transportation tool in modern buildings, the safety of elevator operation is directly related to the safety of people's lives and property. Traditional elevator safety monitoring systems rely heavily on various physical sensors, such as speed governors, safety brakes, and weighing devices, to monitor the operating status. These systems achieve protection by triggering mechanical or electrical safety devices under specific conditions such as overload and overspeed.

[0003] However, existing technologies have significant limitations: First, traditional systems can only respond to preset, single physical quantity thresholds and cannot make multi-dimensional, integrated judgments on the operating status; for example, the system cannot identify visually identifiable dangerous scenarios such as overcrowding, fighting, or pets getting their leashes caught in the elevator car; second, traditional sensors cannot effectively perceive environmental factors, such as the potential impact of wind speed and temperature changes caused by severe weather on the elevator's operating status, and lack the ability to assess environmental risks; finally, the alarm mechanisms of existing systems are mostly static and delayed, unable to perform dynamic risk assessment and early warning based on real-time data streams, and usually only trigger responses after a fault or danger occurs, lacking predictability.

[0004] Therefore, there is an urgent need in this field for an elevator safety monitoring system that can integrate multimodal information and has dynamic risk assessment and intelligent early warning capabilities to overcome the shortcomings of existing technologies. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a lift safety monitoring system and method that integrates computer vision, which can effectively solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention provides a lift safety monitoring system integrating computer vision, comprising:

[0008] The data acquisition module is used to synchronously collect video data, sensor data, and environmental parameter data of the elevator;

[0009] The data processing module, which is communicatively connected to the data acquisition module, is used to preprocess and fuse the acquired data, and output a fused feature vector F.

[0010] The risk assessment module is communicatively connected to the data processing module and is used to calculate a dynamic risk score based on the fused feature vector.

[0011] An alarm module, which is communicatively connected to the risk assessment module, is used to trigger an alarm when the dynamic risk score exceeds a preset threshold.

[0012] The sensor data includes load data W, vibration data V, and velocity data S.

[0013] The environmental parameter data includes light intensity data L, wind speed data Q, and temperature data T;

[0014] The data processing module uses a convolutional neural network (CNN) to extract spatial features from video data and a long short-term memory network (LSTM) to extract temporal series features from sensor data.

[0015] The risk assessment module is based on a dynamic risk scoring function. The dynamic risk scoring function is calculated as follows:

[0016] ;

[0017] in, The risk score calculated at time t, Let i be the i-th feature component of the fused feature vector. The weights of the i-th feature component are dynamically adjusted using the Bayesian optimization algorithm. The dynamic threshold predicted by the Long Short-Term Memory network;

[0018] The dynamic risk scoring function is trained and optimized based on historical operational data.

[0019] Furthermore, the specific steps for preprocessing and feature fusion of the collected data to output a fused feature vector include:

[0020] The collected sensor data is timestamped and cleaned to remove outliers caused by momentary sensor failure.

[0021] The cleaned sensor data and environmental parameter data are normalized and mapped to the [0,1] interval;

[0022] The preprocessed video data is input into a pre-trained convolutional neural network (CNN), and after feature maps are extracted, they are processed by pooling to obtain visual feature vectors.

[0023] The preprocessed sensor data and environmental parameter data are concatenated according to time windows, input into a Long Short-Term Memory (LSTM) network, and the hidden state output at the final time step is taken as the sensor feature vector.

[0024] The visual feature vector is concatenated with the sensor feature vector to obtain the fused feature vector F.

[0025] Furthermore, the convolutional neural network adopts the ResNet-18 architecture, which contains 18 deep residual layers for extracting multi-level spatial features of video data.

[0026] The Long Short-Term Memory (LSTM) network adopts a bidirectional, two-layer structure with 256 hidden units per layer. The final hidden states of the forward and backward LSTMs are concatenated to form the final output.

[0027] The visual feature vector is a 512-dimensional vector, obtained by global average pooling of the feature map of the last convolutional layer of ResNet-18.

[0028] The sensor feature vector is a 256-dimensional vector, which is obtained by dimensionality reduction of the final time step hidden state of the bidirectional LSTM network through a fully connected layer.

[0029] The connection operation is a vector concatenation operation, which concatenates the 512-dimensional visual feature vector with the 256-dimensional sensor feature vector in terms of feature dimensions to obtain a fused feature vector of 768 dimensions.

[0030] Furthermore, the method for constructing the dynamic risk scoring function includes:

[0031] Collect historical operating data of the elevator, including normal operating data and fault operating data, and label the historical operating data with corresponding risk tags. The risk tags adopt a three-level risk labeling system, where risk tag 0 indicates normal operating status, tag 1 indicates warning status, and tag 2 indicates dangerous status.

[0032] The weights are solved using the Bayesian optimization algorithm. The optimal initial value is obtained, where the objective function of Bayesian optimization is to minimize the mean square error between the predicted risk score and the true risk label. The Bayesian optimization uses a Gaussian process as a surrogate model and selects the next evaluation point through the expectation-enhancing acquisition function.

[0033] Construct a long short-term memory network model based on historical risk score time series. As input, predict the dynamic risk threshold at the current time point. ;

[0034] Integrating the weights With dynamic threshold This forms the dynamic risk scoring function.

[0035] Furthermore, the weights are solved using a Bayesian optimization algorithm. The optimal initial values ​​specifically include:

[0036] A Bayesian optimization problem model is established, with the objective function being to minimize the mean squared error between the predicted risk score and the actual risk label, as shown in the formula:

[0037] ;

[0038] in, Let be the feature weight vector to be optimized. Let be the objective function, representing the current weight vector. Below, the mean squared error between the predicted risk scores and the true risk labels of all training samples is... The number of training samples. Let j be the fused feature vector of the j-th training sample. Let i be the i-th component of the fused feature vector of the j-th training sample. This represents the true risk label for the j-th sample.

[0039] A Gaussian process is used as a surrogate model to model the objective function. The covariance function of the Gaussian process uses the squared exponential kernel function, and the calculation formula is as follows:

[0040] ;

[0041] in, The covariance function is used to measure the variance between any two weights. and Similarity between them For signal variance parameters, For length scale parameters, For weight and The Euclidean distance between them;

[0042] Next, the next evaluation point is selected using the expected improvement acquisition function, as shown in the formula:

[0043] ;

[0044] in, Let be the desired improvement function. This represents the current optimal objective function value. is the mathematical expectation operator, representing a probabilistic weighted average of possible outcomes.

[0045] Furthermore, the construction of the long short-term memory network model specifically includes:

[0046] A two-layer stacked long short-term memory network model is constructed, including a first long short-term memory layer and a second long short-term memory layer, and the output vector of the first long short-term memory layer is used as the input vector of the second long short-term memory layer.

[0047] The historical risk score sequence of the past k consecutive time steps The network model is input in chronological order, where k is the preset time window length;

[0048] Temporal features of the input sequence are extracted using the gating mechanism in the Long Short-Term Memory network model, which includes an input gate, a forget gate, and an output gate.

[0049] Using the fully connected output layer of the Long Short-Term Memory network model, future... Dynamic threshold prediction sequence at each time step ,in This is the preset prediction step size;

[0050] Extract the first predicted value from the output dynamic threshold prediction sequence and use it as the dynamic risk threshold for the current time step t. Used for the risk scoring function.

[0051] Furthermore, the alarm module executes a multi-level alarm strategy and sets risk scoring thresholds. , and ,and :

[0052] when Log entries are made at that time, but no warnings are issued.

[0053] when Duration exceeding When this occurs, a local alarm is triggered;

[0054] when When this happens, a remote alarm will be triggered immediately.

[0055] A method for safety monitoring of elevators that integrates computer vision includes the following steps:

[0056] S1. Synchronously collect video data, sensor data, and environmental parameter data of the elevator;

[0057] S2. Preprocess and fuse the collected data to output the fused feature vector F;

[0058] S3. Calculate the dynamic risk score based on the fused feature vector;

[0059] S4. Trigger an alarm when the dynamic risk score exceeds a preset threshold;

[0060] The sensor data includes load data W, vibration data V, and velocity data S.

[0061] The environmental parameter data includes light intensity data L, wind speed data Q, and temperature data T;

[0062] Convolutional Neural Network (CNN) is used to extract spatial features from video data, and Long Short-Term Memory Network (LSTM) is used to extract temporal series features from sensor data.

[0063] The risk assessment module is based on a dynamic risk scoring function. The dynamic risk scoring function is calculated as follows:

[0064] ;

[0065] in, The risk score calculated at time t, Let i be the i-th feature component of the fused feature vector. The weights of the i-th feature component are dynamically adjusted using the Bayesian optimization algorithm. The dynamic threshold is predicted by the Long Short-Term Memory network.

[0066] The technical solution provided by this invention has the following advantages compared with the known prior art:

[0067] This invention constructs a dynamic risk scoring function based on Bayesian optimization and LSTM. The Bayesian optimization algorithm dynamically adjusts the feature weights, giving higher weights to important risk features. At the same time, the LSTM network is used to predict dynamic thresholds based on historical risk sequences, which can be adapted to the safety benchmark of the elevator in different time periods and under different operating conditions in real time. This mechanism completely solves the problem of false alarms and false alarms when the operating conditions change with traditional fixed thresholds, and greatly improves the dynamic adaptability and accuracy of risk assessment.

[0068] This invention simultaneously collects video data, sensor data, and environmental parameter data. It extracts spatial features from the video using a convolutional neural network and combines this with a bidirectional, two-layer LSTM to extract temporal features from the sensor and environmental data, ultimately forming a fused feature vector. This multi-source fusion mechanism breaks through the limitations of traditional single-sensor systems, comprehensively covering various risk factors such as personnel status, machinery operation, and environmental impact, providing more complete feature support for risk assessment.

[0069] The alarm module of this invention is designed with a multi-level alarm strategy with three threshold levels. Low risk only logs the alarm, medium risk triggers a local alarm if the threshold is continuously exceeded, and high risk immediately triggers a remote alarm. The duration judgment is used to avoid false alarms due to instantaneous fluctuations. This hierarchical response mode ensures rapid handling of high-risk situations and reduces the interference of low-risk alarms on operation and management, thus ensuring safety while taking into account the operating efficiency of the elevator. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0071] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0072] Figure 2 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0074] The present invention will be further described below with reference to embodiments.

[0075] Example 1:

[0076] Reference Figure 1 An elevator safety monitoring system integrating computer vision includes:

[0077] The data acquisition module is used to synchronously collect video data, sensor data, and environmental parameter data of the elevator;

[0078] The data processing module, which communicates with the data acquisition module, is used to preprocess and fuse the acquired data, and output the fused feature vector F.

[0079] The risk assessment module communicates with the data processing module and is used to calculate dynamic risk scores based on fused feature vectors.

[0080] The alarm module communicates with the risk assessment module and is used to trigger an alarm when the dynamic risk score exceeds a preset threshold.

[0081] The sensor data includes load data W, vibration data V, and velocity data S.

[0082] Environmental parameter data includes light intensity data L, wind speed data Q, and temperature data T;

[0083] The data processing module uses a convolutional neural network (CNN) to extract spatial features from video data and a long short-term memory network (LSTM) to extract temporal series features from sensor data.

[0084] The risk assessment module is based on a dynamic risk scoring function. The dynamic risk scoring function is calculated as follows:

[0085] ;

[0086] in, The risk score calculated at time t, To fuse the i-th feature component of the feature vector, The weights of the i-th feature component are dynamically adjusted using the Bayesian optimization algorithm. The dynamic threshold predicted by the Long Short-Term Memory network;

[0087] The dynamic risk scoring function is trained and optimized based on historical operational data.

[0088] Specifically, the data acquisition module achieves millisecond-level timestamp synchronization via industrial Ethernet, ensuring strict alignment of video frames, sensor sampling values, and environmental parameters in the time dimension; the video data acquisition resolution is 1920×1080, with a frame rate of 25fps, which can capture details such as the state of people inside the car and component deformation; the sensor sampling rate is set to 1kHz (vibration) and 10Hz (load / speed), and the environmental parameter sampling rate is 1Hz, balancing real-time performance and data volume.

[0089] By simultaneously collecting multi-source data from video, sensors, and the environment, and combining CNN to extract visual spatial features with LSTM to extract sensor temporal features, a comprehensive perception of the elevator's operating status is achieved. The dynamic risk scoring function optimizes weights using Bayesian optimization and predicts thresholds using LSTM, enabling risk assessment to adapt to changes in operating conditions in real time and significantly improving the accuracy of risk identification.

[0090] Breaking through the limitations of traditional single-sensor monitoring, it integrates visual and multi-dimensional sensor data to cover various risk factors such as personnel status, machine operation, and environmental impact; the dynamic weight and threshold design avoids the false alarm / missed alarm problem of fixed thresholds under complex working conditions, enhancing the robustness of the system.

[0091] Furthermore, the specific steps for preprocessing and feature fusion of the collected data to output the fused feature vector include:

[0092] The collected sensor data is timestamped and cleaned to remove outliers caused by momentary sensor failure.

[0093] The cleaned sensor data and environmental parameter data are normalized and mapped to the [0,1] interval;

[0094] The preprocessed video data is input into a pre-trained convolutional neural network (CNN), and after feature maps are extracted, they are processed by pooling to obtain visual feature vectors.

[0095] The preprocessed sensor data and environmental parameter data are concatenated according to time windows, input into a Long Short-Term Memory (LSTM) network, and the hidden state output at the final time step is taken as the sensor feature vector.

[0096] The visual feature vector is concatenated with the sensor feature vector to obtain the fused feature vector F.

[0097] Specifically, additional technical details: Data cleaning employs... In principle, outliers deviating from the mean by three times the standard deviation are removed. A fifth-order Butterworth low-pass filter is applied to the high-frequency noise in the vibration data. Normalization is performed using... formula ,in and Extreme values ​​were taken from historical data over the past three months; the time window was set to a length of 10 seconds and a step size of 5 seconds to ensure the continuity and timeliness of the time series characteristics.

[0098] Data cleaning and normalization processes eliminate outlier interference and differences in data units, ensuring the reliability of input data. Effective fusion of visual and sensor features preserves spatial details and temporal variation information, providing comprehensive feature support for risk assessment. Timestamp alignment ensures temporal consistency across multi-source data, avoiding feature bias caused by temporal misalignment. Normalization allows different types of data to directly participate in fusion calculations, improving feature fusion efficiency. Step-by-step feature extraction and stitching takes into account the unique value of both static visual features and dynamic sensor features.

[0099] Furthermore, the convolutional neural network adopts the ResNet-18 architecture, which contains 18 deep residual layers to extract multi-level spatial features from video data.

[0100] The Long Short-Term Memory (LSTM) network adopts a bidirectional, two-layer structure with 256 hidden units per layer. The final hidden states of the forward and backward LSTMs are concatenated to form the final output.

[0101] The visual feature vector is a 512-dimensional vector, obtained by global average pooling of the feature map of the last convolutional layer of ResNet-18.

[0102] The sensor feature vector is a 256-dimensional vector, which is obtained by dimensionality reduction of the final time step hidden state of the bidirectional LSTM network through a fully connected layer;

[0103] The concatenation operation is a vector concatenation operation, which concatenates the 512-dimensional visual feature vector with the 256-dimensional sensor feature vector in terms of feature dimensions, resulting in a 768-dimensional fused feature vector.

[0104] Specifically, ResNet-18 contains 4 residual blocks, each containing 2-3 convolutional layers with a kernel size of 3×3. Skip connections are used to solve the gradient vanishing problem in deep networks. Global average pooling compresses the 7×7×512 feature map into a 1×1×512 vector, reducing parameter redundancy. The bidirectional LSTM outputs 256-dimensional hidden states both forward and backward, concatenating them to a 512-dimensional array. This is then reduced to 256 dimensions by a fully connected layer with 256 neurons, using ReLU as the activation function.

[0105] The deep residual layers of ResNet-18 effectively solve the gradient vanishing problem in deep networks, enabling the extraction of multi-level spatial features from low-level texture to high-level semantics in videos. The bidirectional two-layer LSTM fully captures the forward and backward temporal dependencies of sensor data, improving the completeness of temporal features. The 768-dimensional fusion vector integrates key visual and sensor information, providing high-dimensional, highly discriminative feature inputs for risk assessment. The ResNet-18 architecture is lightweight and stable, suitable for real-time video processing in elevator scenarios. Compared to unidirectional LSTM, the bidirectional LSTM can more comprehensively mine the contextual relationships of temporal data. The fixed-dimensional feature vector design facilitates subsequent risk score calculation and reduces model complexity.

[0106] Furthermore, methods for constructing dynamic risk scoring functions include:

[0107] Collect historical operating data of the elevator, including normal operating data and fault operating data, and label the historical operating data with corresponding risk labels. The risk label adopts a three-level risk labeling system, where risk label 0 indicates normal operating status, label 1 indicates warning status, and label 2 indicates dangerous status.

[0108] The weights are solved using the Bayesian optimization algorithm. The optimal initial value is obtained, where the objective function of Bayesian optimization is to minimize the mean square error between the predicted risk score and the true risk label. Bayesian optimization uses a Gaussian process as a surrogate model and selects the next evaluation point by improving the acquisition function.

[0109] Construct a long short-term memory network model based on historical risk score time series. As input, predict the dynamic risk threshold at the current time point. ;

[0110] Integrated weights With dynamic threshold This forms a dynamic risk scoring function.

[0111] Specifically, the historical operation data includes 100,000+ samples (70,000+ under normal conditions, 20,000+ under warning conditions, and 10,000+ under dangerous conditions), covering different seasons, time periods, and load conditions; risk labels are marked by engineers based on fault records, sensor threshold exceedances, and video anomaly judgment results; the number of Bayesian optimization iterations is set to 50, and 30 sets of weight vectors are randomly generated from the initial sampling points.

[0112] The three-level risk labeling system provides clear supervision signals for model training, improving the interpretability of risk assessment; the initial weight values ​​obtained by Bayesian optimization make the feature importance allocation more accurate, reducing risk prediction errors; the dynamic threshold predicted by LSTM enables the scoring function to adapt to the operating characteristics of the elevator at different times, enhancing the dynamic adaptability of risk assessment.

[0113] Historical data annotation combined with supervised learning ensures that the model learns the true risk patterns; Bayesian optimization is more efficient than methods such as grid search, and can find the optimal weights in fewer iterations; dynamic thresholds replace fixed thresholds, solving the threshold adaptation problem of traditional systems when operating conditions change.

[0114] Furthermore, a Bayesian optimization algorithm is used to solve for the weights. The optimal initial values ​​specifically include:

[0115] A Bayesian optimization problem model is established, with the objective function being to minimize the mean squared error between the predicted risk score and the actual risk label, as shown in the formula:

[0116] ;

[0117] in, Let be the feature weight vector to be optimized. Let be the objective function, representing the current weight vector. Below, the mean squared error between the predicted risk scores and the true risk labels of all training samples is... The number of training samples. Let j be the fused feature vector of the j-th training sample. Let i be the i-th component of the fused feature vector of the j-th training sample. This represents the true risk label for the j-th sample.

[0118] A Gaussian process is used as a surrogate model to model the objective function. The covariance function of a Gaussian process uses the squared exponential kernel function, and the calculation formula is as follows:

[0119] ;

[0120] in, The covariance function is used to measure the variance between any two weights. and Similarity between them For signal variance parameters, For length scale parameters, For weight and The Euclidean distance between them;

[0121] Next, the next evaluation point is selected using the expected improvement acquisition function, as shown in the formula:

[0122] ;

[0123] in, Let be the desired improvement function. This represents the current optimal objective function value. is the mathematical expectation operator, representing a probabilistic weighted average of possible outcomes.

[0124] Specifically, signal variance in Gaussian processes Initialized to 1.0, with the length scale l initialized to 0.5, dynamic optimization is achieved through maximum likelihood estimation. During the expectation boost function calculation, 1000 Monte Carlo sampling iterations are performed on each candidate weight vector to estimate the mathematical expectation. The objective function calculation employs a mini-batch strategy with a batch size of 256 to accelerate the iteration process. The mean squared error objective function directly quantifies the deviation between the predicted value and the true label, ensuring that weight optimization focuses on improving prediction accuracy. The Gaussian process surrogate model efficiently models the nonlinear relationship of the objective function, reducing the number of actual evaluations. The expectation boost sampling function guides the algorithm to prioritize exploring potentially better weight regions, accelerating the search for the optimal weights. The objective function design is intuitive and effective, ensuring the targeted nature of weight optimization. The squared exponential kernel function flexibly captures the similarity between weights, improving the fitting accuracy of the surrogate model. The expectation boost function balances exploration and utilization, making it easier to find the globally optimal weights compared to random search or greedy strategies.

[0125] Furthermore, constructing a long short-term memory network model specifically includes:

[0126] A two-layer stacked long short-term memory network model is constructed, including a first long short-term memory layer and a second long short-term memory layer. The output vector of the first long short-term memory layer is used as the input vector of the second long short-term memory layer.

[0127] Historical risk score sequences over the past k consecutive time steps Input the network model in chronological order, where k is the preset time window length;

[0128] Temporal features of the input sequence are extracted using the gating mechanism in the Long Short-Term Memory network model. The gating mechanism includes the input gate, the forget gate, and the output gate.

[0129] Generate future... Dynamic threshold prediction sequence at each time step ,in This is the preset prediction step size;

[0130] Extract the first predicted value from the output dynamic threshold prediction sequence and use it as the dynamic risk threshold for the current time step t. Used for risk scoring functions.

[0131] Specifically, the time window length k is set to 10, meaning it takes the risk scores from the past 10 time steps as input, and the prediction step size m is set to 3. Each LSTM layer has 256 hidden units, and the dropout rate is 0.2 to prevent overfitting. In the gating mechanism, the input gate, forget gate, and output gate all use the sigmoid activation function, while cell state updates use the tanh activation function. The fully connected output layer contains 3 neurons, corresponding to m=3 prediction steps, and the activation function is a linear function. The two-layer stacked LSTM enhances the extraction capability of temporal features, capturing deep dependencies in historical risk scores. The gating mechanism effectively filters noise information, retains key temporal patterns, and improves the stability of threshold prediction. Multi-step prediction ensures the current threshold... It can be adjusted to reflect future trends, making risk assessment more forward-looking. Compared with single-layer LSTM, the stacked structure has a stronger nonlinear fitting ability and is suitable for processing complex risk time series data; the gating mechanism solves the long-term dependency problem of traditional recurrent neural networks and avoids the loss of time series information; multi-step prediction provides redundant verification for the current threshold and reduces the impact of single-step prediction error.

[0132] Furthermore, the alarm module executes multi-level alarm strategies and sets risk scoring thresholds. , and ,and :

[0133] when Log entries are made at that time, but no warnings are issued.

[0134] when Duration exceeding When this occurs, a local alarm is triggered;

[0135] when When this happens, a remote alarm will be triggered immediately.

[0136] Specifically, the threshold value is based on the distribution of historical risk data: Corresponding to the upper limit of normal state, Corresponding warning threshold, Corresponding critical value; The alarm duration is set to 30 seconds to avoid false alarms due to momentary fluctuations. Local alarms are audible and visual alarms inside the car, with a buzzer frequency of 2kHz and a warning light flashing period of 1 second. Remote alarms are pushed to the administrator's APP and SMS platform via a 4G module, along with real-time risk characteristic data. Multi-level threshold division ensures that alarm responses are precisely matched with risk levels. Low-risk alarms are only recorded without interfering with operation, while medium- and high-risk alarms are triggered in stages, reducing the impact of invalid alarms on management efficiency. Duration judgment avoids false alarms caused by momentary fluctuations, improving alarm reliability. The tiered alarm strategy balances monitoring sensitivity and practicality, and safety redundancy with operational efficiency. The remote alarm mechanism ensures that high-risk situations can quickly reach management personnel, buying time for emergency response.

[0137] Example 2:

[0138] Reference Figure 2 A method for safety monitoring of elevators that integrates computer vision includes the following steps:

[0139] S1. Synchronously collect video data, sensor data, and environmental parameter data of the elevator;

[0140] S2. Preprocess and fuse the collected data to output the fused feature vector F;

[0141] S3. Calculate the dynamic risk score based on the fused feature vector;

[0142] S4. Trigger an alarm when the dynamic risk score exceeds a preset threshold;

[0143] The sensor data includes load data W, vibration data V, and velocity data S.

[0144] Environmental parameter data includes light intensity data L, wind speed data Q, and temperature data T;

[0145] Convolutional Neural Network (CNN) is used to extract spatial features from video data, and Long Short-Term Memory Network (LSTM) is used to extract temporal series features from sensor data.

[0146] The risk assessment module is based on a dynamic risk scoring function. The dynamic risk scoring function is calculated as follows:

[0147] ;

[0148] in, The risk score calculated at time t, To fuse the i-th feature component of the feature vector, The weights of the i-th feature component are dynamically adjusted using the Bayesian optimization algorithm. The dynamic threshold is predicted by the Long Short-Term Memory network.

[0149] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lift safety monitoring system integrating computer vision, characterized in that, include: The data acquisition module is used to synchronously collect video data, sensor data, and environmental parameter data of the elevator; The data processing module, which is communicatively connected to the data acquisition module, is used to preprocess and fuse the acquired data, and output a fused feature vector F. The risk assessment module is communicatively connected to the data processing module and is used to calculate a dynamic risk score based on the fused feature vector. An alarm module, which is communicatively connected to the risk assessment module, is used to trigger an alarm when the dynamic risk score exceeds a preset threshold. The sensor data includes load data W, vibration data V, and velocity data S. The environmental parameter data includes light intensity data L, wind speed data Q, and temperature data T; The data processing module uses a convolutional neural network (CNN) to extract spatial features from video data and a long short-term memory network (LSTM) to extract temporal series features from sensor data. The risk assessment module is based on a dynamic risk scoring function. The dynamic risk scoring function is calculated as follows: ； in, The risk score is calculated at time t. Let i be the i-th feature component of the fused feature vector. The weights of the i-th feature component are dynamically adjusted using the Bayesian optimization algorithm. The dynamic threshold predicted by the Long Short-Term Memory network; The dynamic risk scoring function is trained and optimized based on historical operational data.

2. The elevator safety monitoring system integrating computer vision according to claim 1, characterized in that, The specific steps for preprocessing and feature fusion of the collected data to output a fused feature vector include: The collected sensor data is timestamped and cleaned to remove outliers caused by momentary sensor failure. The cleaned sensor data and environmental parameter data are normalized and mapped to the [0,1] interval; The preprocessed video data is input into a pre-trained convolutional neural network (CNN), and after feature maps are extracted, they are processed by pooling to obtain visual feature vectors. The preprocessed sensor data and environmental parameter data are concatenated according to time windows, input into a Long Short-Term Memory (LSTM) network, and the hidden state output at the final time step is taken as the sensor feature vector. The visual feature vector is concatenated with the sensor feature vector to obtain the fused feature vector F.

3. The elevator safety monitoring system integrating computer vision according to claim 2, characterized in that, The convolutional neural network adopts the ResNet-18 architecture, which contains 18 deep residual layers, and is used to extract multi-level spatial features of video data. The Long Short-Term Memory (LSTM) network adopts a bidirectional, two-layer structure with 256 hidden units per layer. The final hidden states of the forward and backward LSTMs are concatenated to form the final output. The visual feature vector is a 512-dimensional vector, obtained by global average pooling of the feature map of the last convolutional layer of ResNet-18. The sensor feature vector is a 256-dimensional vector, which is obtained by dimensionality reduction of the final time step hidden state of the bidirectional LSTM network through a fully connected layer. The connection operation is a vector concatenation operation, which concatenates the 512-dimensional visual feature vector with the 256-dimensional sensor feature vector in terms of feature dimensions to obtain a fused feature vector of 768 dimensions.

4. The elevator safety monitoring system integrating computer vision according to claim 1, characterized in that, The method for constructing the dynamic risk scoring function includes: Collect historical operating data of the elevator, including normal operating data and fault operating data, and label the historical operating data with corresponding risk tags. The risk tags adopt a three-level risk labeling system, where risk tag 0 indicates normal operating status, tag 1 indicates warning status, and tag 2 indicates dangerous status. The weights are solved using the Bayesian optimization algorithm. The optimal initial value is obtained, where the objective function of Bayesian optimization is to minimize the mean square error between the predicted risk score and the true risk label. The Bayesian optimization uses a Gaussian process as a surrogate model and selects the next evaluation point through the expectation-boosting acquisition function. Construct a long short-term memory network model based on historical risk score time series. As input, predict the dynamic risk threshold at the current time point. ; Integrating the weights With dynamic threshold This forms the dynamic risk scoring function.

5. The elevator safety monitoring system integrating computer vision according to claim 4, characterized in that, The weights are solved using a Bayesian optimization algorithm. The optimal initial values ​​specifically include: A Bayesian optimization problem model is established, with the objective function being to minimize the mean squared error between the predicted risk score and the actual risk label, as shown in the formula: ； in, Let be the feature weight vector to be optimized. Let be the objective function, representing the current weight vector. Below, the mean squared error between the predicted risk scores and the true risk labels of all training samples is... The number of training samples. Let j be the fused feature vector of the j-th training sample. Let i be the i-th component of the fused feature vector of the j-th training sample. This represents the true risk label for the j-th sample. A Gaussian process is used as a surrogate model to model the objective function. The covariance function of the Gaussian process uses the squared exponential kernel function, and the calculation formula is as follows: ； in, The covariance function is used to measure the variance between any two weights. and Similarity between them For signal variance parameters, For length scale parameters, For weight and The Euclidean distance between them; Next, the next evaluation point is selected using the expected improvement acquisition function, as shown in the formula: ； in, Let be the desired improvement function. This represents the current optimal objective function value. is the mathematical expectation operator, representing a probabilistic weighted average of possible outcomes.

6. The elevator safety monitoring system integrating computer vision according to claim 4, characterized in that, The construction of the long short-term memory network model specifically includes: A two-layer stacked long short-term memory network model is constructed, including a first long short-term memory layer and a second long short-term memory layer, and the output vector of the first long short-term memory layer is used as the input vector of the second long short-term memory layer. Historical risk score sequences over the past k consecutive time steps The network model is input in chronological order, where k is the preset time window length; Temporal features of the input sequence are extracted through the gating mechanism in the Long Short-Term Memory network model, which includes an input gate, a forget gate, and an output gate. Using the fully connected output layer of the Long Short-Term Memory network model, future... Dynamic threshold prediction sequence at each time step ,in This is the preset prediction step size; Extract the first predicted value from the output dynamic threshold prediction sequence and use it as the dynamic risk threshold for the current time step t. Used for the risk scoring function.

7. The elevator safety monitoring system integrating computer vision according to claim 1, characterized in that, The alarm module executes a multi-level alarm strategy and sets risk scoring thresholds. , and ,and : when Log entries are made at that time, but no warnings are issued. when Duration exceeding When this occurs, a local alarm is triggered; when When this happens, a remote alarm will be triggered immediately.

8. A method for safety monitoring of elevators integrating computer vision, characterized in that, Includes the following steps: S1. Synchronously collect video data, sensor data, and environmental parameter data of the elevator; S2. Preprocess and fuse the collected data to output the fused feature vector F; S3. Calculate the dynamic risk score based on the fused feature vector; S4. Trigger an alarm when the dynamic risk score exceeds a preset threshold; The sensor data includes load data W, vibration data V, and velocity data S. The environmental parameter data includes light intensity data L, wind speed data Q, and temperature data T; Convolutional Neural Network (CNN) is used to extract spatial features from video data, and Long Short-Term Memory Network (LSTM) is used to extract temporal series features from sensor data. The risk assessment module is based on a dynamic risk scoring function. The dynamic risk scoring function is calculated as follows: ； in, The risk score is calculated at time t. Let i be the i-th feature component of the fused feature vector. The weights of the i-th feature component are dynamically adjusted using the Bayesian optimization algorithm. The dynamic threshold is predicted by the Long Short-Term Memory network.

Images