A high-risk behavior recognition and active intervention method and system in a car

Abstract

The application discloses a high-risk behavior recognition and active intervention method and system in an elevator car, and the method comprises the following steps: S100, collecting three data streams of vision, vibration and elevator running state; S200, filtering out abnormal high-frequency impact signals by means of an adaptive filter; S300, setting a vibration flow trigger threshold value, and revising the corresponding vibration flow trigger threshold value according to the several stages of elevator starting acceleration, stable running and braking deceleration; S400, extracting the spatial coordinates of the extremities of the passengers in the vision flow and the instantaneous speed vectors thereof, and calculating the physical shock source center coordinates of the impact signals on the car wall plate; S500, calibrating and unifying the vision coordinate system and the car physical coordinate system, calculating the Euclidean distance between the spatial coordinates of the extremities of the passengers and the physical shock source center coordinates, and then judging the dangerous behavior. The problems that single vision is easily affected by light and shielding and single vibration detection cannot distinguish between human damage and environmental interference are solved.

Description

Technical Field

[0001] This invention belongs to the field of hazard identification technology, and more specifically, relates to a system for identifying and actively intervening in high-risk behaviors inside an elevator car. Background Technology

[0002] As an indispensable vertical transportation tool in modern high-rise buildings, the elevator car, its core passenger-carrying component, is a relatively enclosed and limited space. With the increasing frequency of elevator use, the composition and behavioral patterns of passengers within the car are becoming increasingly complex. In actual operation, passengers frequently engage in abnormal behaviors such as violent jumping, banging on the car walls, forcibly prying open the doors, or blocking the doors from closing for extended periods. These behaviors not only easily cause violent shaking of the car, leading to door system malfunctions or accidental disconnection of the safety circuit, resulting in elevator entrapment and shutdown accidents, but also easily trigger panic among other passengers in the enclosed space, posing a serious potential threat to the personal safety of passengers and the integrity of the elevator equipment.

[0003] In existing technologies, elevator operation status control primarily relies on a microcomputer processing system within the elevator control cabinet and a surrounding sensor network. The elevator system monitors the traction machine's speed, car displacement, and the on / off status of various safety switches to ensure the elevator operates according to predetermined logic for lifting, lowering, and door opening / closing. Current control logic mainly focuses on monitoring the status of the elevator's mechanical components and electrical systems, emphasizing ensuring that the equipment's physical operating parameters meet safety standards, such as preventing mechanical failures like overspeeding, overshooting, or bottoming out.

[0004] However, in actual elevator operation, existing technologies are relatively inadequate and lagging in their methods of monitoring passenger behavior inside the elevator car. Traditional in-car monitoring mostly uses closed-circuit television video surveillance systems, primarily serving the purpose of post-event evidence collection and tracing, and is unable to achieve real-time semantic understanding of complex scenarios within the car. Due to the lack of perception methods capable of accurately capturing and analyzing human posture and behavioral intentions, the system cannot identify dangerous passenger behaviors in the first instance. This lack of monitoring prevents the control system from making adaptive interventions or providing voice warnings based on emergencies within the car, resulting in dangerous behaviors not being stopped in time, thus significantly increasing the probability of safety accidents during elevator operation. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for identifying and actively intervening in high-risk behaviors within an elevator car. By establishing a unified timestamp synchronization mechanism and spatial coordinate mapping system, this invention can cross-verify passenger behavior from both physical contact and behavioral intent dimensions. This multimodal redundancy verification mechanism effectively solves the problems of single-vision detection being susceptible to illumination and occlusion, and single vibration detection being unable to distinguish between human sabotage and environmental interference, greatly reducing the false alarm and missed detection rates for high-risk behaviors in elevators under complex operating conditions.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for identifying and actively intervening in high-risk behaviors within a car is provided, comprising the following steps: S100. Construct a multimodal sensing terminal inside the elevator car to collect three data streams: visual, vibration, and elevator operating status, and synchronize the three data streams with timestamps. S200. Construct a background noise baseline model, and filter out the inherent mechanical background noise of guide shoe friction and wire rope vibration through an adaptive filter to separate the abnormal high-frequency impact signal. S300. Set the vibration flow trigger threshold, extract the elevator operating status, and revise the corresponding vibration flow trigger threshold according to the elevator's starting acceleration, smooth operation, and braking deceleration stages. S400: Using a skeletal key point detection algorithm, the spatial coordinates and instantaneous velocity vectors of the passenger's limbs in the visual flow are extracted. Using a sensor array distributed on the car wall, the time difference of arrival algorithm is used to calculate the physical source center coordinates of the impact signal on the car wall panel. S500: The visual coordinate system and the physical coordinate system of the car are calibrated and unified. The Euclidean distance between the spatial coordinates of the passenger's limbs and the coordinates of the physical source center is calculated, and then dangerous behaviors are judged.

[0007] Furthermore, in step S100, when collecting visual stream data, an RGB-D camera is used to acquire two-dimensional images and three-dimensional depth point cloud data of passengers inside the car; when collecting vibration stream data, MEMS high-frequency vibration sensors are arranged in an array on the car wall panels and floor to collect mechanical wave signals of the car; the elevator operation status stream data is obtained by reading real-time operation data from the elevator main control board via the CAN bus, and the real-time operation data includes current speed, acceleration / deceleration status, current floor, and door opening / closing status.

[0008] Further, in step S100, the passenger's two-dimensional image data is Used to identify passenger contours and postures, the 3D depth point cloud data is The passenger's location is accurately determined through a spatial coordinate mapping function, using the three-dimensional depth point cloud data. The value is calibrated using a depth value correction function, specifically: ; in, The calibrated depth value. This is the original depth value. For camera pixel-level calibration function, This is the ambient light compensation function; The camera pixel-level calibration function Specifically: ; in, , , , , , All are calibration coefficients; The ambient light compensation function for: ; in, For adjustment coefficients, Real-time ambient light intensity. This is the standard ambient light intensity threshold.

[0009] Furthermore, in step S200, when constructing the background noise baseline model, it is necessary to collect data for a duration of [duration missing] when the elevator is empty or passengers are not behaving abnormally. Car vibration signal data , Number the sensor. The typical mechanical noise scenarios include the starting acceleration, smooth operation and braking deceleration stages, as well as guide shoe friction, wire rope vibration and car resonance. Then, based on the collected normal vibration data, a background noise benchmark model was constructed using a combination of statistical modeling and frequency domain analysis. ,in The signal frequency.

[0010] Furthermore, the background noise benchmark model First, a short-time Fourier transform is performed on the normal signal. Frequency domain features are obtained: ; in, For the Hanning window function, Sampling frequency, It is a time variable; Then, a time-frequency domain background baseline is constructed using statistical averaging: ; in, The number of samplings under the same working conditions. For the first Frequency domain signal of the second sample For frequency domain standard deviation, It is an adaptive weighting function; The adaptive weight function for: ; in, The attenuation coefficient is... This is the dominant frequency of mechanical noise.

[0011] Furthermore, an adaptive filter is then configured, and a filter coefficient update rule is designed based on the minimum mean square error algorithm to specifically filter out inherent mechanical background noise. At this point, let the filter input signal be the original vibration signal. The desired signal is the output of the background noise baseline model. The filter impulse response is The filtered output signal is: ; in, This represents the convolution operation; The formula for updating the filter coefficients is: ; in, This is the step size factor, which balances convergence speed and stability. For filtering error, It is the conjugate signal of the input signal.

[0012] Furthermore, after processing with an adaptive filter, the abnormal high-frequency impulse signal is separated. The method combines residual extraction with threshold filtering: ; in, This is the anomaly detection coefficient. The instantaneous standard deviation of the filter residual. Final output To characterize abnormal high-frequency impact signals that may pose high-risk behaviors, and to provide data support for subsequent vibration feature analysis; The instantaneous standard deviation of the filter residual for: ; in, The length of the sliding window. This represents the mean of the residuals within the window.

[0013] Furthermore, in step S300, the threshold is revised by combining the inherent vibration differences during the elevator operation phase to avoid false triggering or missed triggering, thus balancing the accuracy and real-time performance of the judgment. First, set the initial vibration flow trigger threshold. This threshold is determined based on the statistical characteristics of previously collected normal vibration signals and abnormal high-frequency impact signals, and the peak mean of the filtered vibration signals from each sensor under normal operating conditions is calculated. and the minimum peak value of abnormal signals The initial threshold is taken as a reasonable value between the two. Then, the feature vector of the elevator's operating state flow is extracted in real time. acceleration and deceleration states in Combination speed The current operating stage of the elevator is determined, which is divided into three scenarios: the start-up acceleration stage, the stable operation stage, and the braking deceleration stage, and corresponding threshold revision functions are constructed. Finally, the dynamic trigger thresholds for the corresponding operational phases are output in real time. The amplitude of the filtered abnormal high-frequency impulse signal and In contrast, when If the condition is met, the subsequent visual feature extraction and behavior judgment process is triggered; otherwise, it is determined to be inherent vibration or residual noise, and no subsequent operation is triggered.

[0014] Furthermore, for visual stream data, a skeletal keypoint detection algorithm is used to analyze the two-dimensional images captured by the camera. and 3D depth point cloud data Joint processing was performed, with a focus on extracting key skeletal points at the extremities of passengers; First, based on 3D depth point cloud data, the 3D spatial coordinates of each skeletal point in the visual coordinate system are calculated. Then, combined with continuous frame image data, the instantaneous velocity vector of each limb end skeletal point is calculated. By the ratio of the difference between the coordinates of consecutive frames to the frame interval, the direction and speed of the skeletal point's motion in 3D space are accurately characterized. The objective function is minimized by the gradient descent method, and the coordinates of the source center are obtained iteratively to determine the specific location of the abnormal vibration in the physical space of the car.

[0015] According to a second aspect of the present invention, a system for identifying and actively intervening in high-risk behaviors inside a car is provided, comprising: Data acquisition module: used to build a multimodal sensing terminal in the elevator car, collect three data streams: visual, vibration and elevator operation status, and synchronize the three data streams with timestamps; Anomaly Separation Module: Used to build a background noise baseline model, filter out inherent mechanical background noise such as guide shoe friction and wire rope vibration through an adaptive filter, and separate out abnormal high-frequency impact signals; Threshold revision module: Used to set the trigger threshold of vibration flow, extract the elevator operating status, and revise the corresponding vibration flow trigger threshold according to the elevator being in the stages of starting acceleration, smooth operation and braking deceleration. Coordinate positioning module: Used to extract the spatial coordinates and instantaneous velocity vectors of the passenger's limbs in the visual flow using a skeletal key point detection algorithm, and to calculate the physical source center coordinates of the impact signal on the car wall using a sensor array distributed on the car wall and a time difference of arrival algorithm. Hazard assessment module: It is used to calibrate and unify the visual coordinate system and the physical coordinate system of the car, calculate the Euclidean distance between the spatial coordinates of the passenger's limbs and the coordinates of the physical seismic source center, and then assess dangerous behaviors.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: 1. The method for identifying and actively intervening in high-risk behaviors within the elevator car of this invention establishes a unified timestamp synchronization mechanism and spatial coordinate mapping system. This invention enables cross-verification of passenger behavior from two dimensions: physical contact and behavioral intent. This multimodal redundancy verification mechanism effectively solves the problems of single visual detection being susceptible to illumination and occlusion, and single vibration detection being unable to distinguish between human sabotage and environmental interference. This significantly reduces the false alarm and missed detection rates for high-risk behaviors in elevators under complex operating conditions.

[0017] 2. The method for identifying and actively intervening in high-risk behaviors within the elevator car of this invention, by constructing a background noise benchmark model and combining it with an adaptive filter, can distinguish in real time between mechanical vibrations generated during normal elevator operation and impact vibrations generated by abnormal passenger behavior. The system dynamically adjusts the trigger threshold of the vibration flow according to different stages of the elevator, such as starting, stabilizing, and braking, ensuring high sensitivity when the elevator is stationary and high anti-interference capability when operating at high speed or experiencing significant vibrations during start-stop. This fundamentally solves the technical drawback of traditional mobile monitoring devices, which are prone to false alarms due to the movement of the carrier itself.

[0018] 3. This invention utilizes the coordinates of human limb extremities extracted by depth vision, combined with the physical earthquake source coordinates calculated by a vibration sensor array using a time-of-arrival algorithm, to perform Euclidean distance calculations within a unified car physical coordinate system. Only when the person's movement position and the actual impact position highly coincide in time and space is the behavior deemed dangerous. This accurately eliminates false dangerous scenarios such as movement without contact or vibration without movement, ensuring the objectivity and rigor of the alarm decision.

[0019] 4. When the system detects abnormally severe vibrations, but visual analysis rules out any intentional aggression from people inside the elevator car, the system can automatically classify the event as a potential mechanical failure and trigger a maintenance warning. This intelligent identification capability upgrades the elevator management system from simple post-event monitoring to an intelligent terminal with predictive maintenance functions, avoiding invalid complaint handling due to misjudgment of human sabotage, and ensuring that potential equipment hazards can be detected in a timely manner. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the steps of a method for identifying and actively intervening in high-risk behaviors inside an elevator car, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating a method for identifying and actively intervening in high-risk behaviors inside an elevator car, according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the operation interface of a high-risk behavior recognition and active intervention system in a car according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware layout of a high-risk behavior recognition and active intervention system in an elevator car according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the TDOA algorithm of a high-risk behavior recognition and active intervention system in a car according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating the active intervention process of a high-risk behavior identification and active intervention system in an elevator car, according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] Example 1 like Figure 1 , 2 As shown, this embodiment of the invention provides a method for identifying and actively intervening in high-risk behaviors inside an elevator car, comprising the following steps: S100. Construct a multimodal sensing terminal inside the elevator car to collect three data streams: visual, vibration, and elevator operating status, and synchronize the three data streams with timestamps. S200. Construct a background noise baseline model and filter out inherent mechanical background noise such as guide shoe friction and wire rope vibration through an adaptive filter to separate out abnormal high-frequency impact signals. S300. Set the vibration flow trigger threshold, extract the elevator operating status, and revise the corresponding vibration flow trigger threshold according to the elevator's starting acceleration, smooth operation, and braking deceleration stages. S400: Using a skeletal key point detection algorithm, the spatial coordinates and instantaneous velocity vectors of the passenger's limbs in the visual flow are extracted. Using a sensor array distributed on the car wall, the time difference of arrival algorithm is used to calculate the physical source center coordinates of the impact signal on the car wall panel. S500: The visual coordinate system and the physical coordinate system of the car are calibrated and unified. The Euclidean distance between the spatial coordinates of the passenger's limbs and the coordinates of the physical source center is calculated, and then dangerous behaviors are judged.

[0023] In step S100, when collecting visual stream data, an RGB-D camera is used to acquire two-dimensional images and three-dimensional depth point cloud data of passengers inside the car; when collecting vibration stream data, MEMS high-frequency vibration sensors are arranged in an array on the car wall and floor to collect mechanical wave signals of the car; the elevator operation status stream data is obtained by reading real-time operation data from the elevator main control board via the CAN bus, and the real-time operation data includes current speed, acceleration / deceleration status, current floor, and door opening / closing status.

[0024] In step S100, the passenger's two-dimensional image data is Used to identify passenger contours and postures, the 3D depth point cloud data is The passenger's location is accurately determined through a spatial coordinate mapping function, using the three-dimensional depth point cloud data. The value is calibrated using a depth value correction function, specifically: ; in, The calibrated depth value. This is the original depth value. For camera pixel-level calibration function, This is the ambient light compensation function; The camera pixel-level calibration function Specifically: ; in, , , , , , All are calibration coefficients; The ambient light compensation function for: ; in, For adjustment coefficients, Real-time ambient light intensity. This is the standard ambient light intensity threshold.

[0025] When collecting vibration flow data, let the coordinates of the MEMS high-frequency vibration sensor be... , The number of sensors is [number], and the acquired raw mechanical wave signal is [significant]. .

[0026] When collecting elevator operation status stream data, it is necessary to construct an operation status feature vector, specifically: ; in, At the current speed, In acceleration and deceleration state, The floor number is [number]. The door is in open / closed state; The acceleration / deceleration state for: ; The floors for: ; in, This represents the total number of floors. For the elevator to reach the At the moment of the layer, For unit step function, The door opening / closing state Represented by a 0-1 function, specifically: This indicates that the door is open. This indicates that the door is closed.

[0027] After data acquisition is complete, high-precision timestamp synchronization processing is performed, using a Kalman filter-based time synchronization algorithm to eliminate device latency and random errors. Specifically: let the original timestamps of the three data streams be... , , The unified reference time is Then, the time error state equation is established: ; in, For the first Error vectors between each data stream at any given time and the reference time. The noise is a process noise that follows a Gaussian distribution. , The process noise covariance matrix; The observation equation is: ; in, The observation error vector is obtained by comparing timestamps across devices. For the observation matrix, To observe the noise, it follows a Gaussian distribution. , To observe the noise covariance matrix.

[0028] The optimal time error estimate is then calculated recursively using Kalman filtering. The final synchronized unified timestamp is: ; in ; Simultaneously, a time consistency verification function is introduced: ; when Upon successful synchronization, the timing of the three data streams is determined to be consistent. This is the synchronization accuracy threshold.

[0029] In step S200, when constructing the background noise baseline model, it is necessary to collect data for a duration of [duration missing] when the elevator is empty or passengers are not behaving abnormally. Car vibration signal data , Number the sensor. The noise levels are measured during the acceleration, smooth operation, and braking deceleration phases, encompassing typical mechanical noise scenarios including guide shoe friction, wire rope vibration, and car resonance. Then, based on the collected normal vibration data, a background noise baseline model is constructed using a combination of statistical modeling and frequency domain analysis. ,in The signal frequency.

[0030] Background noise benchmark model First, a short-time Fourier transform is performed on the normal signal. Frequency domain features are obtained: ; in, For the Hanning window function, Sampling frequency, It is a time variable; Then, a time-frequency domain background baseline is constructed using statistical averaging: ; in, The number of samplings under the same working conditions. For the first Frequency domain signal of the second sample For frequency domain standard deviation, It is an adaptive weighting function; The adaptive weight function for: ; in, The attenuation coefficient is... To enhance the reliability of the reference frequency range for mechanical noise.

[0031] Then, an adaptive filter is configured, and a filter coefficient update rule is designed based on the minimum mean square error algorithm to specifically filter out inherent mechanical background noise. At this point, the filter input signal is assumed to be the original vibration signal. The desired signal is the output of the background noise baseline model. The filter impulse response is The filtered output signal is: ; in, This represents the convolution operation; The formula for updating the filter coefficients is: ; in, This is the step size factor, which balances convergence speed and stability. For filtering error, It is the conjugate signal of the input signal.

[0032] After processing by an adaptive filter, the abnormal high-frequency impulse signal is separated. The method combines residual extraction with threshold filtering: ; in, This is the anomaly detection coefficient. The instantaneous standard deviation of the filter residual. Final output To characterize abnormal high-frequency impact signals that may pose high-risk behaviors, and to provide data support for subsequent vibration feature analysis; The instantaneous standard deviation of the filter residual for: ; in, The length of the sliding window. This represents the mean of the residuals within the window.

[0033] In step S300, the threshold is revised by combining the inherent vibration differences during elevator operation to avoid false triggering or missed triggering, while balancing accuracy and real-time performance. First, an initial vibration flow trigger threshold is set. This threshold is determined based on the statistical characteristics of previously collected normal vibration signals and abnormal high-frequency impact signals, and the peak mean of the filtered vibration signals from each sensor under normal operating conditions is calculated. and the minimum peak value of abnormal signals The initial threshold is taken as a reasonable value between the two, specifically: ; in, The threshold adjustment coefficient is used to avoid false triggering by normal vibrations while ensuring that minor abnormal signals are not missed.

[0034] Then, the feature vector of the elevator's operating state flow is extracted in real time. acceleration and deceleration states in Combination speed The current operating stage of the elevator is determined and divided into three scenarios: the start-up acceleration stage, the stable operation stage, and the braking deceleration stage, and corresponding threshold revision functions are constructed.

[0035] The startup acceleration phase is as follows: when and At that time, it is determined to be in the startup acceleration phase, in which The elevator's rated operating speed is used as the reference point. During this phase, the car is affected by inertial forces, resulting in a large inherent vibration amplitude. Therefore, the trigger threshold needs to be increased, and the revised formula is as follows: ; in, The maximum acceleration of the elevator. The threshold correction coefficient is used during the acceleration phase. The threshold increases synchronously with the acceleration to accurately counteract inertial vibration interference.

[0036] The stable operation phase is as follows: when and At that time, it was determined to be in a stable operation phase. This is the acceleration threshold. The inherent vibrations in this stage are smooth and stable. Maintaining the initial threshold or moderately reducing it to improve sensitivity, the revised formula is: ; in The threshold reduction coefficient during the steady-state phase ensures that low-frequency abnormal signals generated by minor impacts or kicks can be captured.

[0037] The braking and deceleration phase is as follows: when and At this point, it is determined to be the braking deceleration phase. During this phase, the car is affected by braking inertia, and its inherent vibration amplitude is similar to that of the starting acceleration phase. The threshold revision formula is as follows: ; The same correction factor is used as in the acceleration phase. To maintain consistency in threshold adjustment, and at the same time adapt to the negative acceleration characteristics of the deceleration phase through absolute value conversion.

[0038] After the threshold is revised, boundary constraints need to be introduced to ensure that the threshold does not exceed a reasonable range. ,in , Simultaneously, a stage-switching buffer mechanism is implemented, employing an exponential smooth transition formula when the elevator switches from one stage to another. ,in For the duration of phase switching, This serves as a transition coefficient to avoid instantaneous misjudgments caused by sudden threshold changes.

[0039] Finally, the dynamic trigger thresholds for the corresponding operational phases are output in real time. The amplitude of the filtered abnormal high-frequency impulse signal and In contrast, when When the vibration signal is normal, it triggers the subsequent visual feature extraction and behavior judgment process; otherwise, it is determined to be inherent vibration or residual noise and does not trigger subsequent operations, thereby improving the accuracy of vibration signal judgment and providing a reliable triggering basis for high-risk behavior recognition.

[0040] In step S400, for the visual stream data, a skeletal keypoint detection algorithm is used to process the two-dimensional images captured by the camera. and 3D depth point cloud data Joint processing is performed, with a focus on extracting key skeletal points from the passenger's extremities. The set of skeletal points from the extremities is defined as follows: These correspond to eight core points: left fingertips, left palm, right fingertips, right palm, left toes, left heel, right toes, and right heel.

[0041] First, based on 3D depth point cloud data, calculate the 3D spatial coordinates of each skeleton point in the visual coordinate system. , Corresponding set Each point in the middle, superscript Define the visual coordinate system. Coordinate calculations need to incorporate depth correction results, i.e. , Let be the pixel coordinates of the skeleton points in the 2D image. Obtained by converting pixel coordinates to camera intrinsic parameters: , ,in The horizontal and vertical focal lengths of the camera. These are the coordinates of the principal point in the image, all of which are intrinsic parameters of the camera and were obtained through offline calibration.

[0042] Then, by combining the continuous frame image data, the instantaneous velocity vectors of each limb distal bone point are calculated. The central difference method is used to improve the calculation accuracy, specifically: ; In the formula, For image frame interval, For camera frame rate, The calculation method is the same. By using the ratio of the difference in coordinates between consecutive frames to the frame interval, the direction and speed of movement of skeletal points in three-dimensional space can be accurately characterized, providing a quantitative basis for determining whether a limb has experienced a violent impact.

[0043] For vibration flow data, an array of MEMS sensors distributed on the car walls is used. The physical source center coordinates of the impact signal were calculated using the time difference of arrival algorithm. superscript Identify the physical coordinate system of the elevator car.

[0044] Assume the mechanical wave generated by the abnormal impact signal propagates at a speed of v in the car wall panel. ,sensor and The times when the impact signal was received were respectively , , the time difference of arrival between the two sensors is , and the corresponding spatial distance difference satisfies: ; Considering the influence of sensor measurement noise, the least squares method is used to solve the optimal solution of the system of equations, and the objective function is defined as: ; Minimize the objective function through the gradient descent method , and the source center coordinates are obtained by iterative solution , and the specific position of the abnormal vibration in the physical space of the car is clarified.

[0045] In step S500, the visual coordinate system takes the optical center of the camera as the origin, and the physical coordinate system of the car takes the fixed geometric reference point in the car as the origin. The two are converted through rigid body transformation, and the conversion relationship includes a rotation matrix and a translation vector. When calibrating and unifying the visual coordinate system and the physical coordinate system of the car, it is necessary to arrange non-coplanar calibration target points in the car. The known coordinates of each target point in the physical coordinate system of the car are , and at the same time, the coordinates of each target point in the visual coordinate system are obtained through the camera . Based on the target point coordinate pairs, the least squares method is used to solve the rigid body transformation parameters, where the rotation matrix is , satisfying , is the identity matrix, and the translation vector is , and the conversion relationship between the two is: ; To reduce the calibration error, the objective function is set as: ; The optimal transformation parameters are obtained through iterative optimization to complete the calibration of the two coordinate systems. After calibration, it is necessary to verify the reprojection error , when it is determined that the calibration is qualified.

[0046] Based on the above transformation parameters, the visual coordinates of the skeletal points at the ends of the limbs of the passengers extracted in S400 are converted into the coordinates in the physical coordinate system of the car, and the conversion formula is unified as: . After conversion, the skeletal point coordinates and the physical source center coordinates of the vibration in S400 are in the same coordinate system, and the distance calculation can be directly performed.

[0047] Subsequently, the Euclidean distance between each skeletal point at the end of the limb and the source center is calculated to quantify the spatial correlation degree between the limb and the abnormal vibration position, and the distance is: ; To distinguish between physical contact / approach and distance states, a distance threshold is set. ,when At that time, the judgment of the first The skeletal points are close to the epicenter, which could potentially trigger abnormal vibrations.

[0048] When combining multi-dimensional information to determine high-risk behavior, it is necessary to link the dynamic threshold in step S300. The instantaneous velocity vector of the limb in step S400 During the elevator operation phase, a comprehensive judgment rule is established.

[0049] First, calculate the magnitude of the limb velocity vector: ; And set a speed threshold .

[0050] The final high-risk behavior determination logic employs multi-condition logical AND operation, defining a determination function. High-risk behavior is determined to exist when all of the following conditions are met: 1. Abnormal vibration signal trigger threshold: ; 2. Spatial distance condition: There exists at least one skeletal point that satisfies the condition. ; 3. Limb movement conditions: The velocity of the corresponding skeletal point must meet the requirements. ; 4. Operational Phase Verification: If the system is in the start-up / acceleration / braking / deceleration phase, an additional secondary verification of suspicious signals is required. The judgment function is formally expressed as follows: ; in, This is a runtime verification function. 0 indicates that the validation passed, and 0 indicates that it failed. For logical AND operation, Judgment result When this occurs, an active intervention process is triggered. If the vibration is determined to be normal or not caused by passengers, no intervention will be triggered.

[0051] Once high-risk behavior is identified, the system immediately activates a tiered intervention mechanism that balances safety, warning effect, and appropriateness to avoid secondary risks caused by inappropriate intervention. The specific process is as follows: 1. First Phase: Local Real-Time Alerts. The system plays a concise and clear voice alarm through the built-in speakers inside the elevator car, such as "Do not bump or kick the car. Such behavior may cause elevator malfunction. Please stop immediately." Simultaneously, the system controls the ceiling lights inside the car to flash in a fast-forward mode, effectively alerting violating passengers without causing panic among others. The system also records and stores data such as the current timestamp, elevator operating status, abnormal vibration signal fragments, and skeletal motion trajectory screenshots in a local cache, providing evidence for subsequent traceability.

[0052] 2. Second Phase: Mild Intervention and Backend Reporting. If the passenger does not stop the high-risk behavior within 1-3 seconds of triggering the intervention, the system sends a "mild intervention command" to the elevator main control board via the CAN bus. Upon receiving the command, the main control board maintains the current operating state but limits the subsequent acceleration / deceleration amplitude of the elevator to reduce the impact damage to the elevator's mechanical structure caused by the high-risk behavior. Simultaneously, the relevant data on the high-risk behavior is reported to the elevator monitoring center backend via Ethernet or a 4G module. The backend system automatically marks the elevator as "abnormal" and pushes alarm information to the maintenance personnel's terminal, clearly indicating the elevator number, floor, type of high-risk behavior, and real-time operating status.

[0053] 3. Third Stage: In-depth intervention and emergency response. If, more than 3 seconds after triggering, the passenger continues to engage in high-risk behavior or the vibration signal amplitude suddenly increases, the system determines it to be a serious risk. It immediately sends an emergency smooth stop command to the elevator main control board. The main control board prioritizes the nearest leveling floor, keeps the car door open after stopping, and simultaneously cuts off all subsequent elevator operation commands to prevent high-risk behavior from causing car displacement or damage to mechanical components. Local warnings are simultaneously enhanced with a looped voice alarm and continuous rapid flashing of the overhead lights. Furthermore, the system is linked to the property security personnel through the monitoring backend, pushing elevator location and high-risk behavior information to notify security personnel to respond on-site.

[0054] 4. Intervention Termination and System Reset: After the passenger stops the high-risk behavior, the system will continue to monitor for 10 seconds. If no new abnormal vibration signal is triggered during this period, the warning will be automatically terminated, the overhead light will return to normal, the speaker will stop broadcasting, the elevator main control board will release the operation restrictions, and the system will return to normal operation. If the incident is handled by security personnel, an intervention termination command can be sent through the terminal. The system will immediately reset and clear the abnormal marker, and the handling result will be entered into the background to complete the data recording closed loop.

[0055] 5. Special scenario adaptation: If the elevator is in the opening and closing stage when a high-risk behavior is triggered, the system will prioritize completing the opening and closing action before initiating the intervention process to avoid the risk of people being trapped if the door is not in place. If it is in the braking and deceleration stage, the system will prioritize ensuring that the elevator stops smoothly at the nearest floor before performing subsequent intervention operations, taking into account both the effectiveness of the intervention and the personal safety of passengers.

[0056] Throughout the entire intervention process, the system monitors the elevator's operating parameters and vibration signals in real time. If the vibration signal disappears and the skeletal points return to a stable state, the system automatically terminates the intervention and resets the elevator in advance without the need for manual intervention, thus maximizing the efficiency of elevator operation.

[0057] Example 2 like Figures 3-6 As shown, this embodiment of the invention provides a high-risk behavior identification and active intervention system in a car, comprising: Data acquisition module: used to build a multimodal sensing terminal in the elevator car, collect three data streams: visual, vibration and elevator operation status, and synchronize the three data streams with timestamps; Anomaly Separation Module: Used to build a background noise baseline model, filter out inherent mechanical background noise such as guide shoe friction and wire rope vibration through an adaptive filter, and separate out abnormal high-frequency impact signals; Threshold revision module: Used to set the trigger threshold of vibration flow, extract the elevator operating status, and revise the corresponding vibration flow trigger threshold according to the elevator being in the stages of starting acceleration, smooth operation and braking deceleration. Coordinate positioning module: Used to extract the spatial coordinates and instantaneous velocity vectors of the passenger's limbs in the visual flow using a skeletal key point detection algorithm, and to calculate the physical source center coordinates of the impact signal on the car wall using a sensor array distributed on the car wall and a time difference of arrival algorithm. Hazard assessment module: It is used to calibrate and unify the visual coordinate system and the physical coordinate system of the car, calculate the Euclidean distance between the spatial coordinates of the passenger's limbs and the coordinates of the physical seismic source center, and then assess dangerous behaviors.

[0058] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying and actively intervening in high-risk behaviors inside a car, characterized in that, Includes the following steps: S100. Construct a multimodal sensing terminal inside the elevator car to collect three data streams: visual, vibration, and elevator operating status, and synchronize the three data streams with timestamps. S200. Construct a background noise baseline model, and filter out the inherent mechanical background noise of guide shoe friction and wire rope vibration through an adaptive filter to separate out abnormal high-frequency impact signals. S300. Set the vibration flow trigger threshold, extract the elevator operating status, and revise the corresponding vibration flow trigger threshold according to the elevator's starting acceleration, smooth operation, and braking deceleration stages. S400: Using a skeletal key point detection algorithm, the spatial coordinates and instantaneous velocity vectors of the passenger's limbs in the visual flow are extracted. Using a sensor array distributed on the car wall, the time difference of arrival algorithm is used to calculate the physical source center coordinates of the impact signal on the car wall panel. S500: The visual coordinate system and the physical coordinate system of the car are calibrated and unified. The Euclidean distance between the spatial coordinates of the passenger's limbs and the coordinates of the physical source center is calculated, and then dangerous behaviors are judged.

2. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 1, characterized in that, In step S100, when collecting visual stream data, an RGB-D camera is used to acquire two-dimensional images and three-dimensional depth point cloud data of passengers inside the car; when collecting vibration stream data, MEMS high-frequency vibration sensors are arranged in an array on the car wall and floor to collect mechanical wave signals of the car; the elevator operation status stream data is obtained by reading real-time operation data from the elevator main control board via the CAN bus, and the real-time operation data includes current speed, acceleration / deceleration status, current floor, and door opening / closing status.

3. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 2, characterized in that, In step S100, the passenger's two-dimensional image data is Used to identify passenger contours and postures, the 3D depth point cloud data is The passenger's location is accurately determined through a spatial coordinate mapping function, using the three-dimensional depth point cloud data. The value is calibrated using a depth value correction function, specifically: ； in, The calibrated depth value. This is the original depth value. For camera pixel-level calibration function, This is the ambient light compensation function; The camera pixel-level calibration function Specifically: ； in, , , , , , All are calibration coefficients; The ambient light compensation function for: ； in, For adjustment coefficients, Real-time ambient light intensity. This is the standard ambient light intensity threshold.

4. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 3, characterized in that, In step S200, when constructing the background noise baseline model, it is necessary to collect data for a duration of [duration missing] when the elevator is empty or passengers are not behaving abnormally. Car vibration signal data , Number the sensor. The typical mechanical noise scenarios include the starting acceleration, smooth operation and braking deceleration stages, as well as guide shoe friction, wire rope vibration and car resonance. Then, based on the collected normal vibration data, a background noise benchmark model was constructed using a combination of statistical modeling and frequency domain analysis. ,in The signal frequency.

5. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 4, characterized in that, Background noise benchmark model First, a short-time Fourier transform is performed on the normal signal. Frequency domain features are obtained: ； in, For the Hanning window function, Sampling frequency, It is a time variable; Then, a time-frequency domain background baseline is constructed using statistical averaging: ； in, The number of samplings under the same working conditions. For the first Frequency domain signal of the second sample For frequency domain standard deviation, It is an adaptive weighting function; The adaptive weight function for: ； in, The attenuation coefficient is... This is the dominant frequency of mechanical noise.

6. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 5, characterized in that, Then, an adaptive filter is configured, and a filter coefficient update rule is designed based on the minimum mean square error algorithm to filter out inherent mechanical background noise. At this point, let the filter input signal be the original vibration signal. The desired signal is the output of the background noise baseline model. The filter impulse response is The filtered output signal is: ； in, This represents the convolution operation; The formula for updating the filter coefficients is: ； in, This is the step size factor, which balances convergence speed and stability. For filtering error, It is the conjugate signal of the input signal.

7. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 6, characterized in that, After processing by an adaptive filter, the abnormal high-frequency impulse signal is separated. The method combines residual extraction with threshold filtering: ； in, This is the anomaly detection coefficient. The instantaneous standard deviation of the filter residual. Final output To characterize abnormal high-frequency impact signals that may pose high-risk behaviors, and to provide data support for subsequent vibration feature analysis; The instantaneous standard deviation of the filter residual for: ； in, The length of the sliding window. This represents the mean of the residuals within the window.

8. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 7, characterized in that, In step S300, the threshold is revised by combining the inherent vibration differences during the elevator operation phase to avoid false triggering or missed triggering, while taking into account both the accuracy and real-time performance of the judgment. First, set the initial vibration flow trigger threshold. This threshold is determined based on the statistical characteristics of previously collected normal vibration signals and abnormal high-frequency impact signals, and the peak mean of the filtered vibration signals from each sensor under normal operating conditions is calculated. and the minimum peak value of abnormal signals The initial threshold is taken as a reasonable value between the two. Then, the feature vector of the elevator's operating state flow is extracted in real time. acceleration and deceleration states in Combination speed The current operating stage of the elevator is determined, which is divided into three scenarios: the start-up acceleration stage, the stable operation stage, and the braking deceleration stage, and corresponding threshold revision functions are constructed. Finally, the dynamic trigger thresholds for the corresponding operational phases are output in real time. The amplitude of the filtered abnormal high-frequency impulse signal and In contrast, when If the condition is met, the subsequent visual feature extraction and behavior judgment process is triggered; otherwise, it is determined to be inherent vibration or residual noise, and no subsequent operation is triggered.

9. The method for identifying and actively intervening in high-risk behaviors inside a car according to claim 8, characterized in that, For visual stream data, a skeletal keypoint detection algorithm is used to analyze the 2D images captured by the camera. and 3D depth point cloud data Joint processing was performed, with a focus on extracting key skeletal points at the extremities of passengers; First, based on 3D depth point cloud data, the 3D spatial coordinates of each skeletal point in the visual coordinate system are calculated. Then, combined with continuous frame image data, the instantaneous velocity vector of each limb end skeletal point is calculated. By the ratio of the difference between the coordinates of consecutive frames to the frame interval, the direction and speed of the skeletal point's motion in 3D space are accurately characterized. The objective function is minimized by the gradient descent method, and the coordinates of the source center are obtained iteratively to determine the specific location of the abnormal vibration in the physical space of the car.

10. A system for identifying and actively intervening in high-risk behaviors inside a car, used to implement the method for identifying and actively intervening in high-risk behaviors inside a car as described in any one of claims 1-9, characterized in that, include: Data acquisition module: used to build a multimodal sensing terminal in the elevator car, collect three data streams: visual, vibration and elevator operation status, and synchronize the three data streams with timestamps; Anomaly Separation Module: Used to build a background noise baseline model, filter out inherent mechanical background noise such as guide shoe friction and wire rope vibration through an adaptive filter, and separate out abnormal high-frequency impact signals; Threshold revision module: Used to set the trigger threshold of vibration flow, extract the elevator operating status, and revise the corresponding vibration flow trigger threshold according to the elevator's different stages of starting acceleration, smooth operation, and braking deceleration. Coordinate positioning module: Used to extract the spatial coordinates and instantaneous velocity vectors of the passenger's limbs in the visual flow using a skeletal key point detection algorithm, and to calculate the physical source center coordinates of the impact signal on the car wall using a sensor array distributed on the car wall and a time difference of arrival algorithm. Hazard assessment module: It is used to calibrate and unify the visual coordinate system and the physical coordinate system of the car, calculate the Euclidean distance between the spatial coordinates of the passenger's limbs and the coordinates of the physical seismic source center, and then assess dangerous behaviors.

Images