A warning system for preventing dangerous behaviors of falling in an elevator shaft

Abstract

The present invention relates to the technical field of behavior warning, and particularly to a dangerous behavior warning system for preventing falls in elevator shafts, which includes a multimodal data acquisition module, a data transmission module, a data processing module, a warning output module, and a system control module; the multimodal data acquisition module can collect real-time images, sounds, personnel positions, and elevator car weight data in the elevator shaft; the data transmission module uses technologies such as classification marking, compression encryption, and redundant transmission to ensure efficient and secure data transmission; the data processing module uses improved CNN, LSTM, and multimodal GNN algorithms based on the attention mechanism to perform fusion analysis on the data and calculate the danger coefficient R; the warning output module outputs warning information according to the R value classification and integrates a reinforcement learning agent to optimize the warning strategy. The present invention can accurately identify various dangerous behaviors, effectively improve the safety protection level of the elevator shaft, and has important application value.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior warning, and particularly to a dangerous behavior warning system for preventing falls in an elevator shaft. Background Technique

[0002] Traditional elevator safety protection measures mainly focus on the running safety of the elevator car itself, such as the braking system of the car, door machine protection, etc. The monitoring and warning capabilities for dangerous behaviors of personnel in the elevator shaft are relatively weak. Some early monitoring means may only rely on a single sensor or simple monitoring equipment, and it is difficult to comprehensively and accurately identify complex and diverse dangerous behaviors. For example, only through image monitoring by a camera, it may be affected by factors such as light and perspective, resulting in the inability to timely detect dangerous behaviors in the dark or monitoring blind spots; and only relying on a weight sensor, it is difficult to distinguish normal load changes from weight abnormalities caused by dangerous behaviors of personnel.

[0003] With the continuous development of technology, people's requirements for elevator safety are increasing day by day. There is an urgent need for a system that can integrate multiple types of information and use advanced algorithms for accurate identification and warning to effectively prevent dangerous accidents such as falls in the elevator shaft and ensure the safety of personnel in the elevator surrounding environment. Therefore, a dangerous behavior warning system for preventing falls in an elevator shaft is proposed to solve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to provide a dangerous behavior warning system for preventing falls in an elevator shaft to solve the problems raised in the above background technique.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A dangerous behavior warning system for preventing falls in an elevator shaft, comprising:

[0007] A multimodal data acquisition module: used for real-time acquisition of image data, sound data, personnel position data and elevator car weight data in the elevator shaft;

[0008] A data transmission module: used for transmitting the data collected by the multimodal data acquisition module to the data processing module;

[0009] A data processing module: used for performing fusion analysis on the received multimodal data, identifying dangerous behaviors through an improved dangerous behavior recognition algorithm, and generating hierarchical warning information. The improved dangerous behavior recognition algorithm includes:

[0010] An improved convolutional neural network CNN based on the attention mechanism, used for extracting spatial features of dangerous behaviors from image data;

[0011] Long Short-Term Memory Network (LSTM), which is used to extract dynamic temporal features from sound data, weight data, and temporal position data;

[0012] Multi-modal Graph Neural Network (GNN), which is used to fuse cross-modal spatio-temporal correlation features of image, sound, weight, and position data;

[0013] Feature fusion unit, which is used to fuse spatial features, temporal features, and cross-modal features, and calculate the risk coefficient through a fully connected layer , and its formula is: , where is the spatial feature vector output by the CNN, is the temporal feature vector output by the LSTM, is the cross-modal feature vector output by the GNN, and are the weight matrix and bias term, is the sigmoid activation function;

[0014] Warning output module: which is used to output warning information according to the grading result of the risk coefficient ;

[0015] System control module: which is used to dynamically adjust the data acquisition frequency, data processing flow, and warning output method.

[0016] As a preferred solution, the operation process of the improved Convolutional Neural Network (CNN) based on the attention mechanism includes:

[0017] Performing a convolution operation on the input image to generate a feature map ;

[0018] Calculating the weights of each feature map through the attention mechanism , and generating a weighted context vector , where is the correlation score between the feature map and , is the normalized attention weight, is the context vector of the th feature, and the dimension is the same as ;

[0019] Inputting the context vector into the fully connected layer and outputting the spatial feature vector of the dangerous behavior.

[0020] As a preferred solution, the operation process of the Long Short-Term Memory Network (LSTM) includes:

[0021] Performing an operation on the temporal data Perform gated calculation:

[0022] Forget gate: ;

[0023] Input gate: ;

[0024] Output gate: ;

[0025] Update cell state: ;

[0026] Output hidden state: , and extract the temporal feature vector ;

[0027] Among them, 、 、 are the weight matrices of the forget gate, input gate, and output gate respectively, 、 、 are the bias terms corresponding to the forget gate, input gate, and output gate respectively, is the hidden state at the previous moment, is the input data at the current moment, is the cell state at the current moment, 、 are the weight matrix and bias term for cell state update, is the hidden state at the current moment.

[0028] As a preferred solution, the operation process of the multi-modal graph neural network GNN includes:

[0029] Construct a multi-modal graph structure, and the nodes include image features 、voice features 、weight features 、location features ; the edge weights are calculated by spatio-temporal correlation;

[0030] Update the node features through the graph attention mechanism GAT: Among them, represents the attention weight between node and ;

[0031] Output the fused global feature vector

[0032] As a preferred solution, the data processing module further includes:

[0033] Adaptive learning unit: used to dynamically adjust the model parameters of CNN, LSTM, and GNN according to the environmental changes in the elevator shaft;

[0034] Multi-task learning unit: used to synchronously identify multiple dangerous behaviors such as people climbing, falling, and staying.

[0035] As a preferred solution, the multi-modal data acquisition module includes:

[0036] A wide-angle camera installed on the inner wall of the elevator shaft for collecting panoramic images;

[0037] A distributed microphone array for directionally collecting abnormal sounds and processing them through an improved noise reduction algorithm: , where and are parameters dynamically adjusted according to the elevator shaft noise spectrum, is the frequency domain representation of the original audio signal, is the noise spectrum estimate value, is the phase information of the original audio signal;

[0038] An infrared sensor for real-time tracking of the personnel position;

[0039] A high-precision weight sensor installed at the bottom of the elevator car for detecting sudden weight changes.

[0040] As a preferred solution, the warning output module integrates a reinforcement learning agent, and its decision-making process includes:

[0041] Define the state space: real-time danger coefficient Environmental parameters, historical warning records, and environmental parameters include light intensity and noise level;

[0042] Define the action space: dynamically adjust the warning threshold or the alarm method;

[0043] Update the policy through the Q-learning algorithm to maximize the reward function: , where , are the weight coefficients of the warning accuracy rate and the false alarm rate respectively, is the warning accuracy rate, is the false alarm rate.

[0044] As a preferred solution, the warning output module performs the following operations according to the classification result of the danger coefficient :

[0045] When , trigger an audible and visual alarm and send an emergency notice to the monitoring center;

[0046] When happens, send a warning message to the maintenance personnel terminal;

[0047] When happens, record the data into the database but do not trigger an alarm.

[0048] As can be seen from the technical solutions provided by the present invention above, a dangerous behavior warning system for preventing falls in an elevator shaft provided by the present invention has the following beneficial effects:

[0049] Improve accuracy and reliability: Through multi-modal data collection and fusion analysis, the system can more accurately identify dangerous behaviors, effectively prevent fall accidents in the elevator shaft, and improve the reliability and safety of the system;

[0050] Precisely identify and classify warnings: The system uses an improved dangerous behavior recognition algorithm, combined with a convolutional neural network (CNN), long short-term memory network (LSTM), and multi-modal graph neural network (GNN) based on the attention mechanism, to achieve precise identification and classification warnings of dangerous behaviors, which helps to take corresponding measures in a timely manner to prevent accidents;

[0051] Adaptive learning ability: The system has the ability of adaptive learning, can dynamically adjust the model parameters according to the environmental changes in the elevator shaft, improve the robustness and adaptability of the system, and ensure that the system can maintain stable performance in different environments. Brief Description of the Drawings

[0052] Figure 1 It is a schematic diagram of the overall structure of a dangerous behavior warning system for preventing falls in an elevator shaft according to the present invention. Detailed Embodiments

[0053] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0054] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and the specific embodiments.

[0055] As Figure 1 shown, an embodiment of the present invention provides a dangerous behavior warning system for preventing falls in an elevator shaft, including a multi-modal data collection module, a data transmission module, a data processing module, a warning output module, and a system control module.

[0056] In this embodiment, the multi-modal data collection module is used to collect image data, sound data, personnel position data, and elevator car weight data in the elevator shaft in real time;

[0057] Furthermore, the multi-modal data acquisition module consists of:

[0058] Hardware configuration:

[0059] Wide-angle camera: Installed on the inner wall of the elevator shaft, covering the car running area and the bottom of the shaft;

[0060] Distributed microphone array: Deployed at the top, middle and bottom of the shaft for directional sound source localization;

[0061] Infrared sensors: Evenly distributed along the vertical direction of the shaft to track the position of personnel in real time;

[0062] High-precision weight sensor: Integrated into the bottom support structure of the elevator car to detect weight changes;

[0063] Environmental sensors: Optionally equipped with light sensors, temperature and humidity sensors to compensate for environmental interference;

[0064] Communication interface:

[0065] Real-time synchronization bus: Achieve timestamp synchronization of sensor data through CAN or RS-485 protocol;

[0066] Edge computing unit: Built-in preprocessing algorithms (such as noise reduction, image compression) to reduce data transmission bandwidth;

[0067] Even further, the specific operation steps of the multi-modal data acquisition module are as follows:

[0068] Step 1: Sensor initialization and self-check

[0069] Power-on self-check:

[0070] When the camera starts, perform white balance calibration and detect lens occlusion;

[0071] The microphone array performs noise baseline sampling to establish the environmental noise spectrum ;

[0072] The weight sensor is zeroed and calibrated to eliminate the influence of the car's own weight;

[0073] The infrared sensor checks the detection distance to ensure full coverage of the shaft height;

[0074] Synchronization protocol startup:

[0075] Broadcast timestamp signals through the synchronization bus to ensure that the clock deviation of all sensors < 1ms;

[0076] Step 2: Multi-modal data parallel acquisition

[0077] Image data acquisition:

[0078] Trigger conditions: elevator door opening / closing, infrared sensor detecting people approaching the hoistway;

[0079] Acquisition mode:

[0080] Default low frame rate mode (5fps) to reduce power consumption;

[0081] When the infrared sensor detects abnormal movement, switch to high frame rate mode (30fps);

[0082] Adaptive exposure control: Dynamically adjust the exposure parameters according to the hoistway light intensity, the formula is: , where, is the real-time value of the light sensor, is the coefficient preset according to the light reflection characteristics of the hoistway material;

[0083] Sound data acquisition:

[0084] Directional beamforming: The microphone array calculates the azimuth angle of the sound source through the time delay difference , and focuses on the dangerous areas of the hoistway (such as the gap at the top of the car);

[0085] Dynamic noise reduction processing: Run the improved spectral subtraction algorithm in real time: , where, and are the parameters dynamically adjusted according to the noise spectrum of the elevator hoistway, , , is the current signal-to-noise ratio, is the frequency domain representation of the original audio signal, is the noise spectrum estimate value, is the phase information of the original audio signal;

[0086] Weight data acquisition:

[0087] Mutation detection: Sample the weight data at a frequency of 10 Hz. When the weight change rate is detected, trigger the emergency data cache;

[0088] Temperature compensation: Correct the thermal drift error of the weight sensor through the temperature sensor, the formula is: , where, is the sensor temperature coefficient, is the calibration temperature;

[0089] Position data acquisition:

[0090] Multi-sensor fusion positioning: Calculate the coordinates (x, y) of the person by fusing the infrared sensor and the camera vision data, with an error < 10cm;

[0091] Hazardous Area Determination: If a person enters a preset restricted area (such as within 0.5 meters of the wellbore edge), it is marked as a high-risk event;

[0092] Step 3: Data Preprocessing and Caching

[0093] Image Data Compression:

[0094] Use JPEG-XS encoding to perform lossless compression on high-frame-rate images, with a compression ratio ≥ 4:1;

[0095] Only retain the image frames containing moving targets (filtered by background difference method);

[0096] Sound Data Framing:

[0097] Frame the sound data with a frame length of 20 ms and an overlap of 10 ms, and extract MFCC (Mel Frequency Cepstral Coefficients) features;

[0098] Abnormal Data Marking:

[0099] Add an emergency label to weight mutations ( ) and abnormal sounds (such as decibel value > 90 dB);

[0100] Step 4: Synchronously Transmit to the Data Processing Module

[0101] Timestamp Alignment:

[0102] Attach a unified timestamp to all data packets ;

[0103] Precision error < 5 ms;

[0104] If the transmission delay exceeds the threshold (such as 200 ms), discard the old data and trigger retransmission;

[0105] Priority Queue Management:

[0106] Emergency data (such as weight mutations, calls for help) is given priority for transmission, with a delay < 50 ms;

[0107] Regular data is transmitted according to the FIFO (First In First Out) queue.

[0108] In this embodiment, the data transmission module is used to transmit the data collected by the multi-modal data acquisition module to the data processing module;

[0109] Furthermore, the specific operation steps of the data transmission module are as follows:

[0110] I. Data Preprocessing and Encapsulation

[0111] Data Classification and Priority Marking:

[0112] Emergency data (such as weight mutation, distress signal): Marked as priority 1, with a latency requirement ≤ 50 ms;

[0113] High-frame-rate image data (when abnormal movement is detected): Marked as priority 2, with a latency ≤ 100 ms;

[0114] Regular monitoring data (environmental parameters, low-frequency images): Marked as priority 3, with a latency ≤ 500 ms;

[0115] Data compression and encryption:

[0116] Image data: Adopt the JPEG-XS lossless compression algorithm, with a compression ratio ≥ 4:1;

[0117] Sound data: After extracting MFCC features, reduce the dimension, retain the first 20-dimensional coefficients to reduce the transmission bandwidth;

[0118] Encryption algorithm: Use AES-256 encryption, and the key is dynamically generated through a Hardware Security Module (HSM);

[0119] Data encapsulation format: |Packet header (4B)|Timestamp (8B)|Data type (1B)|Priority (1B)|Data length (2B)|Data payload (NB)|CRC check (2B)|;

[0120] II. Transmission protocol and channel management

[0121] Dual-channel redundant transmission:

[0122] Primary channel: Industrial Ethernet (EtherCAT), with a bandwidth of 100 Mbps, used to transmit priority 1-2 data;

[0123] Standby channel: LoRa wireless communication, with a bandwidth of 50 Kbps, used for emergency transmission of priority 1 data;

[0124] Anti-interference strategy:

[0125] Frequency Hopping Spread Spectrum (FHSS): Used for wireless channels, with a frequency switching interval of 10 ms, and the frequency hopping sequence is generated by hashing the shaft ID;

[0126] Differential signal transmission: Use LVDS (Low-Voltage Differential Signaling) for signals transmitted through cables, with a common-mode interference resistance ≥ 20 dB;

[0127] III. Transmission process control

[0128] Data sending:

[0129] Fragmented transmission: The length of a single data packet ≤ 512 B. For extremely long data (such as continuous image frames), it is fragmented and sent, and the fragment sequence number is embedded in the packet header;

[0130] Data reception and verification:

[0131] CRC check: The receiving end checks the CRC. If it fails, it sends a NAK (negative acknowledgment) request for retransmission.

[0132] Data recombination: Reorder according to the fragment sequence number and timestamp to ensure the temporal consistency of multimodal data.

[0133] Exception handling:

[0134] Packet loss retransmission: The data with priority 1 enables a three-time retransmission mechanism with a timeout of 20 ms.

[0135] Channel switching: When the continuous packet loss rate of the main channel > 5%, it automatically switches to the standby channel.

[0136] In this embodiment, the data processing module is used to perform fusion analysis on the received multimodal data, identify dangerous behaviors through an improved dangerous behavior recognition algorithm, and generate hierarchical warning information. The improved dangerous behavior recognition algorithm includes:

[0137] An improved convolutional neural network CNN based on the attention mechanism, which is used to extract the spatial features of dangerous behaviors from image data.

[0138] Long short-term memory network LSTM, which is used to extract dynamic temporal features from sound data, weight data, and temporal position data.

[0139] Multimodal graph neural network GNN, which is used to fuse the cross-modal spatio-temporal correlation features of images, sounds, weights, and position data.

[0140] Feature fusion unit, which is used to fuse spatial features, temporal features, and cross-modal features, and calculate the danger coefficient through a fully connected layer , and its formula is: , where is the spatial feature vector output by the CNN, is the temporal feature vector output by the LSTM, is the cross-modal feature vector output by the GNN, and are the weight matrix and bias term, is the sigmoid activation function; further, to achieve accurate recognition and hierarchical warning of dangerous behaviors in the elevator shaft scenario, the data processing module is designed based on multimodal data fusion and an improved algorithm. The specific operation steps are as follows:

[0141] I. Data preprocessing and standardization

[0142] Data cleaning:

[0143] Image data: Eliminate noise through Gaussian filtering. The formula is: , where is a Gaussian kernel, , and the standard deviation ;

[0144] Sound data: Noise reduction based on an improved spectral subtraction algorithm (see the steps of the multi-modal data acquisition module);

[0145] Weight data: Apply sliding window mean filtering with a window size T = 100 ms;

[0146] Data standardization:

[0147] Normalize each modal data to a unified dimension: , where is the mean of historical data, is the standard deviation;

[0148] Temporal alignment:

[0149] Align the multi-modal data according to the timestamp , with a maximum allowable deviation ;

[0150] II. Spatial feature extraction (improved CNN + attention mechanism)

[0151] Convolutional feature extraction:

[0152] Input the preprocessed image data into the improved CNN to generate an initial feature map ;

[0153] Attention weight calculation:

[0154] Dynamically focus on the key areas of dangerous behaviors through the attention mechanism: ; , where MLP is a multi-layer perceptron, represents vector concatenation;

[0155] Context vector generation:

[0156] Weightedly fuse the feature maps and output the spatial feature vector representation vector concatenation;

[0157] Context vector generation:

[0158] Weightedly fuse the feature maps and output the spatial feature vector : , ; where is the feature map and 's correlation score,

[0159] is the normalized attention weight, is the context vector of the th feature, with the same dimension as ;

[0160] III. Temporal Feature Extraction (LSTM Dynamic Modeling)

[0161] Input of temporal data:

[0162] Input the sound spectrum, weight change, and personnel position sequence into LSTM, with the time step T1 = 10.

[0163] Gating calculation and state update:

[0164] Forget gate: ;

[0165] Input gate: ;

[0166] Output gate: ;

[0167] Update the cell state: ;

[0168] Output the hidden state: , and extract the temporal feature vector ;

[0169] Among them, , , are the weight matrices of the forget gate, input gate, and output gate respectively, , , are the bias terms corresponding to the forget gate, input gate, and output gate respectively, is the hidden state of the previous moment, is the input data of the current moment, is the cell state of the current moment, , are the weight matrix and bias term for cell state update, is the hidden state of the current moment;

[0170] IV. Cross-modal Feature Fusion (Multi-modal Graph Neural Network GNN)

[0171] Graph structure construction:

[0172] Node definition: Image feature node ;

[0173] Sound feature node (MFCC coefficients);

[0174] Weight feature node ;

[0175] Location feature node ;

[0176] Edge weight: Calculated based on spatio-temporal correlation, the formula is: , where is the time decay coefficient;

[0177] Graph attention update:

[0178] Aggregate cross-modal features through the graph attention mechanism (GAT): , where represents the attention weight between node and , is the weight matrix of the attention mechanism, is the vector concatenation operation;

[0179] Global feature output:

[0180] Output the fused cross-modal feature vector ;

[0181] V. Danger coefficient calculation and grading warning

[0182] Feature fusion and danger coefficient calculation: Concatenate , , and input them into the fully connected layer: ; , is the sigmoid function;

[0183] Grading warning strategy: Emergency alarm ( );

[0184] Warning prompt ( );

[0185] Record for observation ( );

[0186] VI. Adaptive learning and model optimization

[0187] Online parameter adjustment:

[0188] Dynamically update the model parameters according to real-time data: , where is the cross-entropy loss function, is the learning rate;

[0189] Multi-task learning mechanism:

[0190] Synchronously optimize the recognition tasks of climbing, falling, and staying behaviors, and the loss function is: , where , , is the task weight, which is dynamically adjusted through gradient normalization.

[0191] In this embodiment, the warning output module is used to output warning information according to the classification result of the risk coefficient ;

[0192] Furthermore, the warning output module performs the following operations according to the classification result of the risk coefficient :

[0193] When , trigger an audible and visual alarm and send an emergency notice to the monitoring center;

[0194] When , send a warning message to the maintenance personnel terminal;

[0195] When , record the data in the database but do not trigger an alarm;

[0196] Even further, the specific operation steps of the warning output module are as follows:

[0197] I. Module initialization and self-check

[0198] Device startup:

[0199] Audible and visual alarm: Start the self-check program to test whether the buzzer and LED indicator are working properly; Display screen: Load the default interface and verify the resolution and communication interface (HDMI / RS-232);

[0200] Wireless communication module: Connect to the preset Wi-Fi / 4G / 5G network and test the handshake protocol with the monitoring center and maintenance terminal;

[0201] Self-check rules:

[0202] If the device status is abnormal (such as buzzer damage, network offline), trigger a fault alarm (priority 0) of the system control module and switch to the standby device (such as local storage of alarm records)

[0203] II. Receive and analyze the risk coefficient

[0204] Data reception:

[0205] Obtain the risk coefficient output by the data processing module and related information (location, time, risk type) through shared memory or message queue;

[0206] Analysis and caching:

[0207] Extract Value, verify data integrity (request retransmission when CRC check fails);

[0208] Cache the latest 10 warning records for historical backtracking and suppression of duplicate alarms;

[0209] III. Hierarchical warning trigger logic

[0210] Emergency alarm:

[0211] Audible and visual alarm:

[0212] The buzzer sounds in a high-frequency pulse mode (frequency 5 kHz, interval 0.2 s), and the red LED flashes (frequency 2 Hz);

[0213] Duration: until manual confirmation or the danger is lifted;

[0214] Information push:

[0215] Send an emergency notice to the monitoring center via the MQTT protocol, including location, image snapshot, and audio clip;

[0216] Linkage control:

[0217] Trigger the elevator emergency braking system (send a stop instruction via the CAN bus);

[0218] Activate the strong light warning for the hoistway lighting (brightness 1000 lumens, duration 30 seconds);

[0219] Warning prompt:

[0220] Display prompt:

[0221] Highlight the warning area on the monitoring interface and overlay a dynamic heat map;

[0222] Pop up a dialog box: "Potential risk detected: staying at the edge of the hoistway, please check in time!"

[0223] Maintenance terminal notification:

[0224] Push a work order to the maintenance management system via the HTTP API, with the priority marked as "medium"

[0225] Record observation:

[0226] Data storage:

[0227] Encrypt the data and write it to the local database (SQLite / MySQL), with fields including timestamp, value, and raw sensor data;

[0228] Synchronize to cloud storage (AWS S3 / Azure Blob), retention period ≥ 30 days

[0229] IV. Multi-channel Collaboration and Fault Tolerance Mechanism

[0230] Primary and Backup Channel Switching:

[0231] Primary Channel (Ethernet / Wi-Fi): Prioritize the transmission of emergency alarm data;

[0232] Backup Channel (LoRa / Bluetooth): Automatically enabled when the packet loss rate of the primary channel > 5% to ensure the delivery of critical information;

[0233] Duplicate Alarm Suppression:

[0234] If the same event is repeatedly triggered within 10 seconds, it is merged into a single alarm, and the "Duplicate Event ID" is marked in the log;

[0235] Feedback Confirmation Mechanism:

[0236] The monitoring center needs to click to confirm within 15 seconds after receiving the alarm, otherwise a secondary notification (phone call to the duty supervisor) is triggered;

[0237] Furthermore, the early warning output module integrates a reinforcement learning agent, and its decision-making process includes:

[0238] Define the state space: Real-time danger coefficient Environmental parameters, historical early warning records, and environmental parameters include light intensity and noise level;

[0239] Define the action space: Dynamically adjust the early warning threshold Or the alarm method;

[0240] Update the policy through the Q-learning algorithm to maximize the reward function:

[0241] , where 、 are the weight coefficients of the early warning accuracy rate and the false alarm rate respectively, is the early warning accuracy rate, is the false alarm rate.

[0242] In this embodiment, the system control module is used to dynamically adjust the data acquisition frequency, data processing flow, and early warning output method; the specific operations include:

[0243] I. Adjustment of Data Acquisition Frequency

[0244] Real-time Monitoring and Analysis:

[0245] Continuously monitor the personnel activities in the elevator hoistway, and judge whether personnel frequently enter and exit the area near the elevator hoistway or stay abnormally in the hoistway through infrared sensors and image data; for example, if the number of times personnel are detected near the hoistway exceeds the set threshold (such as 5 times / minute) within a certain period of time, it is determined that the personnel activities are frequent;

[0246] Analyze environmental data, including light intensity and noise level; the light intensity sensor and noise sensor provide real-time feedback data, and records are made when the light intensity suddenly drops to a level that may affect the image acquisition quality (such as below 50 lux) or the noise level increases significantly (such as exceeding the set noise decibel threshold of 80 dB);

[0247] Monitor the system resource occupancy, including CPU usage, memory occupancy, and data transmission bandwidth; if the CPU usage continuously exceeds 80% or the memory occupancy exceeds 70%, it is determined that the system is in a high-load state;

[0248] Frequency adjustment strategy formulation:

[0249] When personnel activities are frequent, the environment changes greatly, or the system is in a high-load state, increase the data acquisition frequency; for image data acquisition, increase the frame rate from the default low frame rate mode (5 fps) to the high frame rate mode (30 fps); the sound data acquisition frequency is increased from 10 Hz to 20 Hz; the personnel position data acquisition interval is shortened from 0.5 seconds to 0.2 seconds; the elevator car weight data acquisition frequency is increased from 10 Hz to 15 Hz;

[0250] When the elevator hoistway is in a relatively stable state for a long time, that is, personnel activities are scarce, environmental parameters are stable, and system resources are sufficient (CPU usage is below 40% and memory occupancy is below 50%), reduce the acquisition frequency; the image data acquisition frame rate is reduced to 3 fps, the sound data acquisition frequency is reduced to 5 Hz, the personnel position data acquisition interval is extended to 1 second, and the elevator car weight data acquisition frequency is reduced to 8 Hz;

[0251] II. Data processing flow adjustment

[0252] Data priority determination and resource allocation:

[0253] Quickly analyze the received data to identify emergency data; such as sudden weight change data (when the detected weight change rate ΔW / Δt > 50 kg / s), abnormal sound data (decibel value > 90 dB), and obvious dangerous behaviors identified through image recognition (such as personnel climbing or being in a dangerous area of the hoistway) are marked as high priority;

[0254] For high-priority data, allocate more computing resources immediately; in the data processing module, create dedicated threads or processes for it, increase the CPU core allocation ratio to more than 60%, and preferentially occupy memory resources to ensure that this data can be quickly processed by the improved dangerous behavior recognition algorithm;

[0255] Conventional monitoring data (such as environmental parameters, low-frequency images, etc.) are processed according to normal resource allocation and calculated in the idle resources of the system to avoid conflicts with emergency data processing;

[0256] Dynamic adjustment of model parameters:

[0257] Regularly evaluate the performance of each dangerous behavior recognition model (CNN, LSTM, GNN), and adjust the parameters according to the recognition accuracy rate and false alarm rate; for example, if the false alarm rate of CNN is relatively high when recognizing climbing behavior in images, the system control module, through the adaptive learning unit, appropriately reduces the weight of the features related to climbing behavior in CNN, and at the same time increases the attention weight to the key areas in the image (such as the well shaft edge, car top, etc.);

[0258] Adjust the model parameters according to the changes in the elevator shaft environment; when the ambient light intensity remains low, the CNN model increases the parameter settings for contrast enhancement and noise reduction in the image preprocessing stage; when the noise level is high, the LSTM model increases the filtering weight for noise when processing sound data to improve the adaptability and accuracy of the model in different environments; III. Adjustment of warning output methods

[0259] Adaptation of environment and danger level:

[0260] Select the warning method considering the current environmental conditions; in areas with strong light and many people (such as the elevator hall in a shopping mall), when the danger coefficient is reached, in addition to triggering an audible and visual alarm (the buzzer sounds in a high-frequency pulse mode, and the red LED flashes), detailed warning information is also displayed on the display screens inside and near the elevator car, including image snapshots of dangerous behaviors, location information, and possible danger types, and voice warnings are broadcast through the public address system;

[0261] When the danger coefficient is within the warning range of if the maintenance personnel's terminal is busy (judged by the network connection status and the CPU usage rate of the terminal) or the network connection is unstable (the packet loss rate exceeds 10%), the system automatically switches to the SMS notification method, sends the warning information to the maintenance personnel's mobile phone, and displays a brief warning content in a pop-up window on the display screen inside the elevator car;

[0262] Adjustment of historical records and duration:

[0263] Adjust the warning output in combination with historical warning records; if the same type of warning appears multiple times within a short period (such as within 10 minutes) but does not cause actual danger, when this type of warning appears again later, appropriately reduce the intensity of the audible and visual alarms (such as reducing the buzzer volume by 30% and the LED flashing frequency by 50%) and shorten the duration (from continuous alarm until manual confirmation or danger elimination, shorten it to alarm for 30 seconds and then pause for 10 seconds, and then cycle the alarm), to avoid excessive interference to personnel;

[0264] For newly emerging and relatively serious dangerous behaviors (such as when it is first detected that a person has an obvious sign of falling at the edge of the hoistway and the danger coefficient R≥0.9), extend the display time of the warning information on the monitoring center and the maintenance personnel terminal to ensure that relevant personnel can fully understand the dangerous situation, and increase the push frequency of the emergency notice, repeat the push every 2 minutes until the danger is eliminated or manually confirmed.

[0265] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A dangerous behavior warning system for preventing falling in an elevator shaft, characterized by: include: Multimodal data acquisition module: used to collect image data, sound data, personnel location data and elevator car weight data in the elevator shaft in real time; Data transmission module: used for transmitting the data collected by the multimodal data collection module to the data processing module; Data processing module: used to perform fusion analysis on the received multimodal data, identify dangerous behaviors through an improved dangerous behavior recognition algorithm and generate graded warning information. The improved dangerous behavior recognition algorithm includes: An improved convolutional neural network (CNN) based on the attention mechanism is used to extract spatial features of dangerous behaviors from image data; Long short-term memory network LSTM, used to extract dynamic time series features from sound data, weight data and time series position data; Multimodal graph neural network (GNN), used to fuse cross-modal spatiotemporal correlation features of image, sound, weight and location data; Feature fusion unit, used to fuse spatial features, temporal features and cross-modal features, and calculate the risk factor through the fully connected layer , the formula is: ,in, is the spatial feature vector output by CNN, is the time series feature vector output by LSTM, is the cross-modal feature vector output by GNN, and is the weight matrix and bias term, is the sigmoid activation function; Warning output module: used to Output warning information based on the classification results; System control module: used to dynamically adjust data collection frequency, data processing flow and warning output mode; The operation process of the improved convolutional neural network CNN based on the attention mechanism includes: Perform convolution operation on the input image to generate feature map ; Calculate the weight of each feature map through the attention mechanism , and generate a weighted context vector ,in, The feature map and The relevance score of is the normalized attention weight, For the The context vector of features has the same dimension as Consistency; The context vector Input the fully connected layer and output the spatial feature vector of dangerous behavior ; The operation process of the long short-term memory network LSTM includes: For time series data Perform gated calculations: Forget Gate: ; Input Gate: ; Output Gate: ; Update cell status: ; Output hidden state: , and extract the time series feature vector ; in, , , are the weight matrices of the forget gate, input gate, and output gate respectively. , , are the bias items corresponding to the forget gate, input gate, and output gate respectively. is the hidden state at the previous moment, is the input data at the current moment, is the cell state at the current moment, , is the weight matrix and bias term for cell state update, is the hidden state at the current moment; The operation process of the multimodal graph neural network GNN includes: Construct a multimodal graph structure, where nodes include image features Sound characteristics , weight characteristics , location features ,Edge weights are calculated by spatiotemporal correlations; Update node features through the graph attention mechanism GAT: ,in, , indicating a node and The attention weight, is the weight matrix of the attention mechanism, It is a vector concatenation operation; Output the fused global feature vector .

2. A dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1, characterized in that: The data processing module also includes: Adaptive learning unit: used to dynamically adjust the model parameters of CNN, LSTM and GNN according to the environmental changes of the elevator shaft; Multi-task learning unit: used to simultaneously identify multiple dangerous behaviors such as climbing, falling, and staying.

3. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1 is characterized in that: The multimodal data acquisition module comprises: A wide-angle camera installed on the inner wall of the elevator shaft is used to capture panoramic images; Distributed microphone array for directional collection of abnormal sounds and processing through improved noise reduction algorithms: ,in, and is a parameter dynamically adjusted according to the elevator shaft noise spectrum. is the frequency domain representation of the original audio signal, is the noise spectrum estimate, is the phase information of the original audio signal; Infrared sensors for real-time tracking of personnel locations; High-precision weight sensor, installed at the bottom of the elevator car, is used to detect sudden changes in weight.

4. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1, characterized in that: The warning output module integrates a reinforcement learning agent, and its decision-making process includes: Defining the state space: real-time risk factor Environmental parameters and historical warning records. Environmental parameters include light intensity and noise level; Defining the action space: Dynamically adjusting warning thresholds or alarm method; Update the strategy through the Q-learning algorithm to maximize the reward function: ,in, , are the weight coefficients of the early warning accuracy and false alarm rate, For the warning accuracy, is the false alarm rate.

5. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1 is characterized in that: The early warning output module is based on the risk factor The grading results are performed as follows: when When an alarm occurs, it triggers an audible and visual alarm and sends an emergency notification to the monitoring center; when When a warning message is sent, it will be sent to the terminal of the maintenance personnel; when When the alarm is triggered, the data is recorded to the database but no alarm is triggered.

Images