Dangerous behavior early warning system for preventing elevator shaft from falling

Abstract

The invention relates to the technical field of behavior early warning, in particular to a dangerous behavior early warning system for elevator shaft falling prevention, which comprises a multi-modal data acquisition module, a data transmission module, a data processing module, an early warning output module and a system control module, the multi-modal data acquisition module can acquire images, sound, personnel positions and elevator car weight data in an elevator shaft in real time; the data transmission module adopts technologies such as classification marking, compression encryption and redundancy transmission to ensure efficient and safe transmission of data; the data processing module performs fusion analysis on the data by using an improved CNN algorithm, an LSTM algorithm and a multi-modal GNN algorithm based on an attention mechanism, and calculates a danger coefficient R; and the early warning output module outputs early warning information in a grading manner according to the R value, and integrates a reinforcement learning agent to optimize an early warning strategy. According to the method, various dangerous behaviors can be accurately recognized, the safety protection level of the elevator shaft is effectively improved, and the method has important application value.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior warning, and in particular to a dangerous behavior warning system for preventing falling in an elevator shaft. Background Art

[0002] Traditional elevator safety protection measures mainly focus on the operational safety of the elevator car itself, such as the car's braking system and door machine protection, but the monitoring and early warning capabilities for dangerous behaviors of people in the elevator shaft are relatively weak. Some early monitoring methods may rely only on a single sensor or simple monitoring equipment, making it difficult to fully and accurately identify complex and diverse dangerous behaviors. For example, image monitoring through cameras alone may be affected by factors such as light and viewing angle, resulting in the inability to promptly detect dangerous behaviors in dark places or blind spots; and relying solely on weight sensors makes it difficult to distinguish between normal load changes and weight abnormalities caused by dangerous behaviors of people.

[0003] With the continuous development of technology, people's requirements for elevator safety are increasing day by day. There is an urgent need for a system that can integrate multiple information and use advanced algorithms for accurate identification and early warning, so as to effectively prevent dangerous accidents such as falling in the elevator shaft and ensure the safety of personnel in the surrounding environment of the elevator. Therefore, in response to the above problems, a dangerous behavior early warning system for preventing falling in the elevator shaft is proposed. Summary of the invention

[0004] The object of the present invention is to provide a dangerous behavior warning system for preventing falling in an elevator shaft, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A dangerous behavior early warning system for preventing falling in an elevator shaft, comprising: Multimodal data acquisition module: used to collect image data, sound data, personnel location data and elevator car weight data in the elevator shaft in real time; Data transmission module: used for transmitting the data collected by the multimodal data collection module to the data processing module; Data processing module: used to perform fusion analysis on the received multimodal data, identify dangerous behaviors through an improved dangerous behavior recognition algorithm and generate graded warning information. The improved dangerous behavior recognition algorithm includes: An improved convolutional neural network (CNN) based on the attention mechanism is used to extract spatial features of dangerous behaviors from image data; Long short-term memory network LSTM, used to extract dynamic time series features from sound data, weight data and time series position data; Multimodal graph neural network GNN, used to fuse cross-modal spatiotemporal correlation features of image, sound, weight and location data;. Feature fusion unit, used to fuse spatial features, temporal features and cross-modal features, and calculate the risk factor through the fully connected layer , the formula is: ,in, is the spatial feature vector output by CNN, is the time series feature vector output by LSTM, is the cross-modal feature vector output by GNN, and is the weight matrix and bias term, is the sigmoid activation function; Warning output module: used to Output warning information based on the classification results; System control module: used to dynamically adjust data collection frequency, data processing flow and warning output method.

[0006] As a preferred solution, the operation process of the improved convolutional neural network CNN based on the attention mechanism includes: Perform convolution operation on the input image to generate feature map ; Calculate the weight of each feature map through the attention mechanism , and generate a weighted context vector ,in, The feature map and The relevance score of is the normalized attention weight, For the The context vector of features has the same dimension as Consistency; The context vector Input the fully connected layer and output the spatial feature vector of dangerous behavior .

[0007] As a preferred solution, the operation process of the long short-term memory network LSTM includes: For time series data Perform gated calculations: Forget Gate: ; Input Gate: ; Output Gate: ; Update cell status: ; Output hidden state: , and extract the time series feature vector ; in, , , are the weight matrices of the forget gate, input gate, and output gate respectively. , , are the bias items corresponding to the forget gate, input gate, and output gate respectively. is the hidden state at the previous moment, is the input data at the current moment, is the cell state at the current moment, , is the weight matrix and bias term for cell state update, is the hidden state at the current moment.

[0008] As a preferred solution, the operation process of the multimodal graph neural network GNN includes: Construct a multimodal graph structure, where nodes include image features , Sound characteristics , weight characteristics , location features ,Edge weights are calculated by spatiotemporal correlations; Update node features through the graph attention mechanism GAT: ,in, , indicating a node and The attention weight, is the weight matrix of the attention mechanism, It is a vector concatenation operation; Output the fused global feature vector .

[0009] As a preferred solution, the data processing module also includes: Adaptive learning unit: used to dynamically adjust the model parameters of CNN, LSTM and GNN according to the environmental changes of the elevator shaft; Multi-task learning unit: used to simultaneously identify multiple dangerous behaviors such as climbing, falling, and staying.

[0010] As a preferred solution, the multimodal data acquisition module includes: A wide-angle camera installed on the inner wall of the elevator shaft is used to capture panoramic images; Distributed microphone array for directional collection of abnormal sounds and processing through improved noise reduction algorithms: ,in, and is a parameter dynamically adjusted according to the elevator shaft noise spectrum. is the frequency domain representation of the original audio signal, is the noise spectrum estimate, is the phase information of the original audio signal; Infrared sensors for real-time tracking of personnel locations; High-precision weight sensor, installed at the bottom of the elevator car, is used to detect sudden changes in weight.

[0011] As a preferred solution, the warning output module integrates a reinforcement learning agent, and its decision-making process includes: Defining the state space: real-time risk factor Environmental parameters and historical warning records. Environmental parameters include light intensity and noise level; Defining the action space: Dynamically adjusting warning thresholds or alarm method; Update the strategy through the Q-learning algorithm to maximize the reward function: ,in, , are the weight coefficients of the early warning accuracy and false alarm rate, For the warning accuracy, is the false alarm rate.

[0012] As a preferred solution, the early warning output module is based on the risk factor The grading results are performed as follows: when When an alarm occurs, it triggers an audible and visual alarm and sends an emergency notification to the monitoring center; when When a warning message is sent, it will be sent to the terminal of the maintenance personnel; when When the alarm is triggered, the data is recorded to the database but no alarm is triggered.

[0013] It can be seen from the technical solution provided by the present invention that the dangerous behavior warning system for preventing falling in an elevator shaft provided by the present invention has the following beneficial effects: Improve accuracy and reliability: Through multimodal data collection and fusion analysis, the system can more accurately identify dangerous behaviors, effectively prevent falling accidents in the elevator shaft, and improve the reliability and safety of the system; Accurate identification and graded warning: The system uses an improved dangerous behavior recognition algorithm, combined with a convolutional neural network (CNN), a long short-term memory network (LSTM), and a multimodal graph neural network (GNN) based on an attention mechanism, to achieve accurate identification of dangerous behaviors and graded warnings, which helps to take corresponding measures in a timely manner to prevent accidents; Adaptive learning capability: The system has adaptive learning capability and can dynamically adjust model parameters according to environmental changes in the elevator shaft, improve the robustness and adaptability of the system, and ensure that the system can maintain stable performance in different environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is a schematic diagram of the overall structure of a dangerous behavior early warning system for preventing falling in an elevator shaft. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a dangerous behavior warning system for preventing falling in an elevator shaft, including a multimodal data acquisition module, a data transmission module, a data processing module, a warning output module and a system control module.

[0018] In this embodiment, the multimodal data acquisition module is used to collect image data, sound data, personnel location data and elevator car weight data in the elevator shaft in real time; Furthermore, the multimodal data acquisition module comprises: Hardware configuration: Wide-angle camera: installed on the inner wall of the elevator shaft, covering the car operation area and the bottom of the shaft; Distributed microphone array: deployed at the top, middle and bottom of the shaft for directional sound source positioning; Infrared sensors: equidistantly distributed along the vertical direction of the shaft, real-time tracking of personnel positions; High-precision weight sensor: integrated into the bottom support structure of the elevator car to detect weight changes; Environmental sensors: optional light sensors, temperature and humidity sensors to compensate for environmental interference; Communication interface: Real-time synchronization bus: time stamp synchronization of sensor data via CAN or RS-485 protocol; Edge computing unit: built-in pre-processing algorithms (such as noise reduction and image compression) to reduce data transmission bandwidth; Furthermore, the specific operation steps of the multimodal data acquisition module are as follows: Step 1: Sensor initialization and self-test Power-on self-test: The camera performs white balance calibration when it starts and detects lens occlusion; The microphone array performs noise baseline sampling to establish the ambient noise spectrum ; The weight sensor is calibrated to zero to eliminate the influence of the car's own weight; The infrared sensor verifies the detection distance to ensure that the full height of the shaft is covered; Synchronous protocol startup: Broadcast timestamp signals via the synchronization bus to ensure that the clock deviation of all sensors is less than 1ms; Step 2: Parallel collection of multimodal data Image data acquisition: Trigger conditions: elevator door opening / closing, infrared sensor detecting people approaching the shaft; Acquisition Mode: The default low frame rate mode (5fps) reduces power consumption; When the infrared sensor detects abnormal movement, it switches to high frame rate mode (30fps); Adaptive exposure control: Dynamically adjust exposure parameters according to the shaft light intensity. The formula is: ,in, is the real-time value of the light sensor, It is a coefficient preset according to the reflective characteristics of the shaft material; Sound data collection: Directional beamforming: The microphone array calculates the azimuth of the sound source through the time delay difference , focusing on dangerous areas of the hoistway (such as the gap on the top of the car); Dynamic noise reduction processing: Real-time operation of the improved spectral subtraction algorithm: ,in, and is a parameter dynamically adjusted according to the elevator shaft noise spectrum. , , is the current signal-to-noise ratio, is the frequency domain representation of the original audio signal, is the noise spectrum estimate, is the phase information of the original audio signal; Weight data collection: Sudden change detection: The weight data is sampled at a frequency of 10 Hz. When , the emergency data cache is triggered; Temperature compensation: The thermal drift error of the weight sensor is corrected by the temperature sensor. The formula is: ,in, is the sensor temperature coefficient, To calibrate the temperature; Location data collection: Multi-sensor fusion positioning: through the fusion of infrared sensor and camera visual data, the coordinates (x, y) of the person are calculated with an error of <10cm; Dangerous area determination: If a person enters a preset restricted area (such as within 0.5 meters of the shaft edge), it is marked as a high-risk event; Step 3: Data preprocessing and caching Image Data Compression: Use JPEG-XS encoding to compress high frame rate images losslessly, with a compression ratio of ≥4:1; Only image frames containing moving objects are retained (filtered by background subtraction method); Sound data framing: Divide the frame into 20ms frames with 10ms overlap to extract MFCC (Mel-frequency cepstral coefficient) features; Abnormal data marking: For weight mutation ( ), add emergency tags to abnormal sounds (such as decibel value > 90dB); Step 4: Synchronous transmission to the data processing module Timestamp alignment: All packets are timestamped ; Accuracy error <5ms; If the transmission delay exceeds a threshold (e.g., 200ms), the old data is discarded and retransmission is triggered; Priority queue management: Emergency data (such as weight changes, calls for help) are transmitted first, with a delay of <50ms; Normal data is transferred in a FIFO (First In First Out) queue.

[0019] In this embodiment, the data transmission module is used to transmit the data collected by the multimodal data collection module to the data processing module; Further, the specific operation steps of the data transmission module are as follows: 1. Data preprocessing and packaging Data classification and priority marking: Urgent data (such as sudden weight changes, calls for help): marked as priority 1, with a delay requirement of ≤50ms; High frame rate image data (when abnormal movement is detected): marked as priority 2, delay ≤ 100ms; Conventional monitoring data (environmental parameters, low-frequency images): marked as priority 3, delay ≤ 500ms; Data compression and encryption: Image data: using JPEG-XS lossless compression algorithm, compression ratio ≥ 4:1; Sound data: Extract MFCC features and reduce the dimension, retain the first 20 dimensional coefficients, and reduce the transmission bandwidth; Encryption algorithm: AES-256 encryption is used, and the key is dynamically generated through the hardware security module (HSM); Data encapsulation format: | Header (4B) | Timestamp (8B) | Data type (1B) | Priority (1B) | Data length (2B) | Data payload (NB) | CRC check (2B) |; 2. Transmission Protocol and Channel Management Dual channel redundant transmission: Main channel: Industrial Ethernet (EtherCAT), bandwidth 100Mbps, used to transmit priority 1-2 data; Backup channel: LoRa wireless communication, bandwidth 50Kbps, used for emergency transmission of priority 1 data; Anti-interference strategy: Frequency Hopping Spread Spectrum (FHSS): used for wireless channels, the frequency switching interval is 10ms, and the frequency hopping sequence is generated by the well ID hash; Differential signal transmission: LVDS (low voltage differential signal) is used for cable transmission signals, and the ability to resist common mode interference is ≥20dB; 3. Transmission process control Data sending: Fragmented transmission: The length of a single data packet is ≤512B. Extra-long data (such as continuous image frames) is sent in fragments, and the fragment sequence number is embedded in the packet header. Data reception and verification: CRC check: The receiving end checks the CRC. If it fails, it sends a NAK (negative acknowledgment) to request retransmission; Data reorganization: reorder by shard sequence number and timestamp to ensure temporal consistency of multimodal data; Exception handling: Packet loss retransmission: Priority 1 data uses a three-time retransmission mechanism with a timeout of 20ms; Channel switching: When the continuous packet loss rate of the main channel is >5%, it automatically switches to the backup channel.

[0020] In this embodiment, the data processing module is used to perform fusion analysis on the received multimodal data, identify dangerous behaviors through an improved dangerous behavior recognition algorithm and generate graded warning information. The improved dangerous behavior recognition algorithm includes: An improved convolutional neural network (CNN) based on the attention mechanism is used to extract spatial features of dangerous behaviors from image data; Long short-term memory network LSTM, used to extract dynamic time series features from sound data, weight data and time series position data; Multimodal graph neural network (GNN), used to fuse cross-modal spatiotemporal correlation features of image, sound, weight and location data; Feature fusion unit, used to fuse spatial features, temporal features and cross-modal features, and calculate the risk factor through the fully connected layer , the formula is: ,in, is the spatial feature vector output by CNN, is the time series feature vector output by LSTM, is the cross-modal feature vector output by GNN, and is the weight matrix and bias term, is the sigmoid activation function; further, in order to achieve accurate identification and graded warning of dangerous behaviors in the elevator shaft scene, the data processing module is designed based on multimodal data fusion and improved algorithm. The specific operation steps are as follows: 1. Data preprocessing and standardization Data cleaning: Image data: Eliminate noise through Gaussian filtering, the formula is: ,in, is the Gaussian kernel, , standard deviation ; Sound data: noise reduction based on improved spectral subtraction algorithm (see the steps of multimodal data acquisition module); Weight data: Apply sliding window mean filtering with a window size of T = 100ms; Data Standardization: Normalize each modal data to a unified dimension: ,in, is the mean of historical data, is the standard deviation; Timing Alignment: Based on timestamp Aligning multimodal data, maximum allowed deviation ; 2. Spatial feature extraction (improved CNN + attention mechanism) Convolutional feature extraction: Input the preprocessed image data into the improved CNN to generate the initial feature map ; Attention weight calculation: Dynamically focus on key areas of dangerous behavior through attention mechanism: ; , MLP is a multi-layer perceptron, Represents vector concatenation; Context vector generation: Weighted fusion feature map, output spatial feature vector represents vector concatenation; Context vector generation: Weighted fusion feature map, output spatial feature vector : , ;in, The feature map and The relevance score of is the normalized attention weight, For the The context vector of features has the same dimension as Consistency; 3. Time series feature extraction (LSTM dynamic modeling) Time series data input: The sound spectrum, weight change, and personnel position sequence are input into LSTM with a time step of T1=10.

[0021] Gated calculation and state update: Forget Gate: ; Input Gate: ; Output Gate: ; Update cell status: ; Output hidden state: , and extract the time series feature vector ; in, , , are the weight matrices of the forget gate, input gate, and output gate respectively. , , are the bias items corresponding to the forget gate, input gate, and output gate respectively. is the hidden state at the previous moment, is the input data at the current moment, is the cell state at the current moment, , is the weight matrix and bias term for cell state update, is the hidden state at the current moment; 4. Cross-modal feature fusion (multimodal graph neural network GNN) Graph structure construction: Node definition: Image feature node ; Sound Feature Node (MFCC coefficients); Weight Feature Node ; Position Feature Node ; Edge weight: calculated based on spatiotemporal correlation, the formula is: ,in, , is the time attenuation coefficient; Graph attention update: Aggregating cross-modal features through graph attention mechanism (GAT): ,in, , indicating a node and The attention weight, is the weight matrix of the attention mechanism, It is a vector concatenation operation; Global feature output: Output the fused cross-modal feature vector ; V. Calculation of risk factor and graded warning Feature fusion and risk factor calculation: , , After concatenation, input the fully connected layer: ; , is the sigmoid function; Grading warning strategy: Emergency alarm ( ); Early warning reminder ( ); Record observations ( ); 6. Adaptive Learning and Model Optimization Online parameter adjustment: Dynamically update model parameters based on real-time data: ,in, is the cross entropy loss function, is the learning rate; Multi-task learning mechanism: The recognition tasks of climbing, falling, and stranding behaviors are optimized simultaneously, and the loss function is: ,in, , , is the task weight, which is dynamically adjusted through gradient normalization.

[0022] In this embodiment, the early warning output module is used to Output warning information based on the classification results; Furthermore, the early warning output module is based on the risk factor The grading results are performed as follows: when When an alarm occurs, it triggers an audible and visual alarm and sends an emergency notification to the monitoring center; when When a warning message is sent, it will be sent to the terminal of the maintenance personnel; when When the alarm is triggered, the data is recorded in the database but no alarm is triggered; Furthermore, the specific operation steps of the early warning output module are as follows: 1. Module initialization and self-test Device startup: Sound and light alarm: start the self-test program to test whether the buzzer and LED indicator are working properly; Display screen: load the default interface, verify the resolution and communication interface (HDMI / RS-232); Wireless communication module: connect to the preset Wi-Fi / 4G / 5G network and test the handshake protocol with the monitoring center and maintenance terminal; Self-check rules: If the device status is abnormal (such as buzzer damage, network offline), the fault alarm of the system control module is triggered (priority 0) and the backup device is switched (such as local storage alarm record) 2. Receiving and analyzing the risk factor

[0023] Data reception: Get the risk factor of the data processing module output through shared memory or message queue and related information (location, time, type of hazard); Parsing and caching: extract Value, check data integrity (request retransmission when CRC check fails); Cache the latest 10 warning records for historical backtracking and repeated alarm suppression; 3. Classified warning trigger logic Emergency Alarm: Sound and light alarm: The buzzer sounds in high-frequency pulse mode (frequency 5kHz, interval 0.2s), and the red LED flashes (frequency 2Hz); Duration: Until manual confirmation or danger is eliminated; Information push: Send emergency notifications to the monitoring center via the MQTT protocol, including location, image snapshots, and audio clips; Linkage control: Trigger the elevator emergency brake system (send a stop command via the CAN bus); Start the shaft lighting strong light warning (brightness 1000 lumens, lasting 30 seconds); Warning Tips: Display prompts: Highlight the warning area on the monitoring interface and overlay a dynamic heat map; A dialog box pops up: "Potential risk detected: Retention at the edge of the shaft, please check in time!" Maintenance terminal notification: Push the work order to the maintenance management system via HTTPAPI, with the priority marked as "medium" Record observations: Data Storage: The data is encrypted and written to the local database (SQLite / MySQL). The fields include timestamp, Value, sensor raw data; Synchronize to cloud storage (AWS S3 / Azure Blob), retention period ≥ 30 days 4. Multi-channel collaboration and fault-tolerance mechanism Active / standby channel switching: Main channel (Ethernet / Wi-Fi): prioritizes transmission of emergency alarm data; Backup channel (LoRa / Bluetooth): automatically enabled when the packet loss rate of the primary channel is >5% to ensure the delivery of critical information; Repeated alarm suppression: If the same event is triggered repeatedly within 10 seconds, it will be combined into a single alarm and marked as "Repeated Event ID" in the log; Feedback confirmation mechanism: After receiving the alarm, the monitoring center needs to click to confirm within 15 seconds, otherwise it will trigger the second-level notification (calling the on-duty supervisor); Furthermore, the warning output module integrates a reinforcement learning agent, whose decision-making process includes: Defining the state space: real-time risk factor Environmental parameters and historical warning records. Environmental parameters include light intensity and noise level; Defining the action space: Dynamically adjusting warning thresholds or alarm method; Update the strategy through the Q-learning algorithm to maximize the reward function: ,in, , are the weight coefficients of the early warning accuracy and false alarm rate, For the warning accuracy, is the false alarm rate.

[0024] In this embodiment, the system control module is used to dynamically adjust the data collection frequency, data processing flow and warning output mode; the specific operations include: 1. Data collection frequency adjustment Real-time monitoring and analysis: Continuously monitor the activities of people in the elevator shaft, and use infrared sensors and image data to determine whether people frequently enter and exit the area near the elevator shaft or stay abnormally in the shaft. For example, if the number of times a person appears near the shaft exceeds a set threshold (such as 5 times / minute) within a certain period of time, it is determined that the person is active frequently. Analyze environmental data, including light intensity and noise level. The light intensity sensor and noise sensor provide real-time feedback data. When the light intensity suddenly drops to a level that may affect the image acquisition quality (such as below 50 lux) or the noise level increases significantly (such as exceeding the set noise decibel threshold of 80 dB), the data is recorded. Monitor system resource usage, including CPU usage, memory usage, and data transmission bandwidth; if the CPU usage is continuously higher than 80% or the memory usage exceeds 70%, the system is considered to be in a high-load state; Frequency adjustment strategy formulation: When personnel activities are frequent, the environment changes greatly, or the system is under high load, the data collection frequency is increased; for image data collection, the frame rate is increased from the default low frame rate mode (5fps) to the high frame rate mode (30fps); the sound data collection frequency is increased from 10Hz to 20Hz; the personnel location data collection interval is shortened from 0.5 seconds to 0.2 seconds; the elevator car weight data collection frequency is increased from 10Hz to 15Hz; When the elevator shaft is in a relatively stable state for a long time, that is, when there are few people moving around, the environmental parameters are stable, and the system resources are sufficient (CPU usage is less than 40%, and memory usage is less than 50%), the acquisition frequency is reduced; the image data acquisition frame rate is reduced to 3fps, the sound data acquisition frequency is reduced to 5Hz, the personnel position data acquisition interval is extended to 1 second, and the elevator car weight data acquisition frequency is reduced to 8Hz; 2. Data Processing Flow Adjustment Data priority determination and resource allocation: The received data is quickly analyzed to identify emergency data; for example, weight mutation data (when the weight change rate ΔW / Δt>50kg / s is detected), abnormal sound data (decibel value>90dB), and obvious dangerous behaviors identified by images (such as people climbing or being in the dangerous area of ​​the shaft) are marked as high priority; For high-priority data, more computing resources are immediately allocated; in the data processing module, dedicated threads or processes are opened for it, the CPU core allocation ratio is increased to more than 60%, and memory resources are given priority to ensure that this data can be quickly processed by the improved dangerous behavior recognition algorithm; Conventional monitoring data (environmental parameters, low-frequency images, etc.) are processed according to normal resource allocation and calculated in the system's idle resources to avoid conflicts with emergency data processing; Dynamic adjustment of model parameters: Regularly evaluate the performance of each dangerous behavior recognition model (CNN, LSTM, GNN), and adjust parameters based on recognition accuracy and false alarm rate; for example, if the false alarm rate of CNN is high when recognizing climbing behavior in an image, the system control module will appropriately reduce the weight of the features related to climbing behavior in CNN through the adaptive learning unit, and increase the attention weight to key areas in the image (such as the edge of the shaft, the top of the car, etc.); Adjust model parameters according to changes in the elevator shaft environment; when the ambient light intensity remains low, the CNN model adds contrast enhancement and noise reduction parameter settings in the image preprocessing stage; when the noise level is high, the LSTM model increases the noise filtering weight when processing sound data to improve the adaptability and accuracy of the model in different environments; III. Adjustment of warning output mode Adaptation to environment and danger level: Consider the current environmental conditions to select the warning method; in areas with strong light and a large number of people (such as shopping mall elevator halls), when the risk factor is high, When an emergency occurs, in addition to triggering an audible and visual alarm (the buzzer sounds in a high-frequency pulse mode and the red LED flashes), detailed warning information is displayed on the display screens in and near the elevator car, including an image snapshot of the dangerous behavior, location information and possible types of danger, and a voice warning is played through the broadcasting system; When the risk factor is When the maintenance personnel's terminal is in a busy state (determined by the network connection status and terminal CPU usage) or the network connection is unstable (the packet loss rate exceeds 10%), the system automatically switches to the SMS notification mode, sends the warning information to the maintenance personnel's mobile phone, and displays a brief warning content in the form of a pop-up window on the display screen in the elevator car; History and duration adjustments: Adjust the warning output in combination with historical warning records; if the same type of warning appears multiple times within a short period of time (such as within 10 minutes) but does not cause actual danger, when the same type of warning appears again later, appropriately reduce the intensity of the sound and light alarm (such as reducing the buzzer volume by 30%, and the LED flashing frequency by 50%) and shorten the duration (from continuous alarm to manual confirmation or danger elimination, shorten to 30 seconds of alarm followed by a 10-second pause, and then re-circulate the alarm) to avoid excessive interference to personnel; For new and more serious dangerous behaviors (such as the first detection of obvious signs of a person falling at the edge of the shaft with a risk factor R ≥ 0.9), extend the display time of the warning information in the monitoring center and maintenance personnel terminals to ensure that relevant personnel can fully understand the dangerous situation, and increase the push frequency of emergency notifications, repeating the push every 2 minutes until the danger is eliminated or manually confirmed.

[0025] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A dangerous behavior warning system for preventing falling in an elevator shaft, characterized by: include: Multimodal data acquisition module: used to collect image data, sound data, personnel location data and elevator car weight data in the elevator shaft in real time; Data transmission module: used for transmitting the data collected by the multimodal data collection module to the data processing module; Data processing module: used to perform fusion analysis on the received multimodal data, identify dangerous behaviors through an improved dangerous behavior recognition algorithm and generate graded warning information. The improved dangerous behavior recognition algorithm includes: An improved convolutional neural network (CNN) based on the attention mechanism is used to extract spatial features of dangerous behaviors from image data; Long short-term memory network LSTM, used to extract dynamic time series features from sound data, weight data and time series position data; Multimodal graph neural network (GNN), used to fuse cross-modal spatiotemporal correlation features of image, sound, weight and location data; Feature fusion unit, used to fuse spatial features, temporal features and cross-modal features, and calculate the risk factor through the fully connected layer , the formula is: ,in, is the spatial feature vector output by CNN, is the time series feature vector output by LSTM, is the cross-modal feature vector output by GNN, and is the weight matrix and bias term, is the sigmoid activation function; Warning output module: used to Output warning information based on the classification results; System control module: used to dynamically adjust data collection frequency, data processing flow and warning output method.

2. A dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1, characterized in that: The operation process of the improved convolutional neural network CNN based on the attention mechanism includes: Perform convolution operation on the input image to generate feature map ; Calculate the weight of each feature map through the attention mechanism , and generate a weighted context vector ,in, The feature map and The relevance score of is the normalized attention weight, For the The context vector of features has the same dimension as Consistency; The context vector Input the fully connected layer and output the spatial feature vector of dangerous behavior .

3. A dangerous behavior warning system for preventing falling in an elevator shaft according to claim 2, characterized in that: The operation process of the long short-term memory network LSTM includes: For time series data Perform gated calculations: Forget Gate: ; Input Gate: ; Output Gate: ; Update cell status: ; Output hidden state: , and extract the time series feature vector ; in, , , are the weight matrices of the forget gate, input gate, and output gate respectively. , , are the bias items corresponding to the forget gate, input gate, and output gate respectively. is the hidden state at the previous moment, is the input data at the current moment, is the cell state at the current moment, , is the weight matrix and bias term for cell state update, is the hidden state at the current moment.

4. A dangerous behavior warning system for preventing falling in an elevator shaft according to claim 3, characterized in that: The operation process of the multimodal graph neural network GNN includes: Construct a multimodal graph structure, where nodes include image features , Sound characteristics , weight characteristics , location features ,Edge weights are calculated by spatiotemporal correlations; Update node features through the graph attention mechanism GAT: ,in, , indicating a node and The attention weight, is the weight matrix of the attention mechanism, It is a vector concatenation operation; Output the fused global feature vector .

5. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 4, characterized in that: The data processing module also includes: Adaptive learning unit: used to dynamically adjust the model parameters of CNN, LSTM and GNN according to the environmental changes of the elevator shaft; Multi-task learning unit: used to simultaneously identify multiple dangerous behaviors such as climbing, falling, and staying.

6. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1 is characterized in that: The multimodal data acquisition module comprises: A wide-angle camera installed on the inner wall of the elevator shaft is used to capture panoramic images; Distributed microphone array for directional collection of abnormal sounds and processing through improved noise reduction algorithms: ,in, and is a parameter dynamically adjusted according to the elevator shaft noise spectrum. is the frequency domain representation of the original audio signal, is the noise spectrum estimate, is the phase information of the original audio signal; Infrared sensors for real-time tracking of personnel locations; High-precision weight sensor, installed at the bottom of the elevator car, is used to detect sudden changes in weight.

7. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1 is characterized in that: The warning output module integrates a reinforcement learning agent, and its decision-making process includes: Defining the state space: real-time risk factor Environmental parameters and historical warning records. Environmental parameters include light intensity and noise level; Defining the action space: Dynamically adjusting warning thresholds or alarm method; Update the strategy through the Q-learning algorithm to maximize the reward function: ,in, , are the weight coefficients of the early warning accuracy and false alarm rate, For the warning accuracy, is the false alarm rate.

8. The dangerous behavior warning system for preventing falling in an elevator shaft according to claim 1 is characterized in that: The early warning output module is based on the risk factor The grading results are performed as follows: when When an alarm occurs, it triggers an audible and visual alarm and sends an emergency notification to the monitoring center; when When a warning message is sent, it will be sent to the terminal of the maintenance personnel; when When the alarm is triggered, the data is recorded to the database but no alarm is triggered.

Images