Escalator abnormal behavior recognition emergency system and method based on machine vision

Abstract

The invention belongs to the technical field of machine vision recognition, and discloses an escalator abnormal behavior recognition emergency system and method based on machine vision, after efficient joint extraction is carried out on escalator taking visual data to obtain joint features, key frames are extracted according to similarity, abnormal postures in the escalator taking process are recognized, and the escalator abnormal behavior recognition emergency system and method based on machine vision are obtained. And the contours of people and objects can be obtained through edge identification, and whether congestion occurs and large objects are detained or not can be identified. In addition, according to the abnormal posture and figure detection method, after a result is output, the result can be associated with an escalator motor, corresponding emergency processing is carried out, escalator control operation is automatically carried out, prevention measures are provided before potential safety hazards occur, and effective guarantee is provided for escalator taking safety of passengers.

Description

Technical Field

[0001] This invention belongs to the field of machine vision recognition technology, specifically relating to an emergency system and method for recognizing abnormal behavior of escalators based on machine vision. Background Technology

[0002] In modern urban transportation systems, escalators have become indispensable transportation facilities in public places such as subways, shopping malls, and office buildings, carrying the main transport capacity between different levels. With increasing passenger flow, escalator safety issues have become increasingly prominent. Inappropriate postures and improper behaviors of escalator riders are the main causes of falls, pinching injuries, and crushing injuries, which can even be life-threatening in severe cases. Therefore, how to monitor abnormal passenger behavior in real time and immediately take emergency measures after such behavior occurs to prevent further escalation of accidents has become an important issue for improving public safety.

[0003] Currently, safety monitoring of escalator passengers mainly relies on cameras around the escalator for surveillance and visual analysis to identify abnormal behavior. However, due to the complex deployment environment of escalators and the diverse forms of pedestrians, simple image processing methods struggle to accurately identify abnormal passenger postures. Furthermore, camera monitoring is significantly affected by factors such as ambient lighting and camera installation angles, and is prone to false alarms or missed alarms when there are obstructions or localized light noise. Therefore, there is an urgent need for a highly accurate visual analysis method to identify abnormal escalator behavior and control the escalator motor for emergency avoidance. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an emergency system and method for identifying abnormal behavior of escalators based on machine vision, which aims to solve the problems of difficulty and poor accuracy in identifying abnormal escalators in the prior art.

[0005] This invention provides an emergency system for recognizing abnormal behavior of escalators based on machine vision, including a video acquisition module, an abnormal behavior recognition module, an edge computing storage module, an edge communication module, an emergency processing module, and a cloud computing module;

[0006] Video capture module: used to acquire real-time video of the escalator in operation;

[0007] Edge computing storage module: used for video preprocessing, visualization feature extraction, real-time image storage, abnormal behavior case sample storage, and event storage;

[0008] Abnormal Behavior Recognition Module: Based on the pre-processed video and preliminary extracted visualization features provided by the edge computing storage module, it monitors abnormal postures of objects and passengers on escalators and evaluates the monitoring results.

[0009] Edge communication module: used to provide network interconnection with different high-level protocols, realize the conversion and forwarding of video data and models, connect the cloud wide area network and the edge local area network, provide network security firewall, and provide external updates for the entire system;

[0010] Emergency handling module: Used to receive emergency handling instructions sent by the abnormal behavior recognition module, change the operating status of the escalator, provide voice reminders to passengers, and communicate with the control system of the escalator.

[0011] Cloud computing module: Used to re-examine difficult samples, responsible for issuing system updates and model updates, realizing the aggregation of information from multiple escalators, displaying statistical information on the cloud platform, and providing channels to view the overall operation of the escalator on multiple devices.

[0012] Furthermore, the video acquisition module includes two cameras, which are respectively installed at the starting and ending points of the escalator. The cameras are capable of acquiring real-time images of the escalator in operation and of acquiring distance information from objects on the escalator to the camera lenses.

[0013] Furthermore, the abnormal behavior identification module includes:

[0014] Passenger boundary crossing monitoring submodule: Used to monitor abnormal passenger behavior that crosses boundaries on escalators;

[0015] Passenger fall detection submodule: Used to monitor abnormal passenger behavior such as falling on escalators;

[0016] Passenger Reverse Movement Monitoring Submodule: Used to monitor abnormal passenger behavior of moving in the wrong direction on escalators;

[0017] Passenger Sprint Monitoring Submodule: Used to monitor abnormal passenger sprinting behavior on escalators;

[0018] Passenger flow statistics and density calculation submodule: used to calculate passenger flow and passenger density on escalators;

[0019] Large Item Delay Alarm Submodule: Used to monitor large items left on escalators, issue alarms when escalators are congested or large items are left behind, and send corresponding emergency handling instructions to the emergency handling module based on the alarm.

[0020] Furthermore, the emergency response module includes:

[0021] Emergency control and processing unit: Based on emergency handling instructions, it links the motor of the escalator to control the escalator, thereby converting the escalator into deceleration, slow stop or emergency stop;

[0022] Voice broadcast unit: used to provide voice reminders to passengers;

[0023] Alarm unit: Used to alert the escalator control center and relevant personnel, notify them to take subsequent actions, and automatically save the alarm information into the system.

[0024] Furthermore, the edge computing storage module:

[0025] I. Built-in real-time body pose tracking and estimation model BlazePose, which can identify and locate the core joints of the human body in real-time video, thereby obtaining the joint angle, angular velocity and angular acceleration of each core joint.

[0026] The core joints include the shoulders, elbows, wrists, waist, knees, and ankles;

[0027] II. Capable of preprocessing video frames, including grayscale processing;

[0028] III. Built-in 3D Harris algorithm, which can extract edges from video frames to obtain the number of passengers on the escalator and the outline and corners of large luggage.

[0029] Furthermore, the abnormal behavior recognition module:

[0030] I. Built-in TGA-LSTM model, which can perform pose recognition based on the features of core joints in video frames.

[0031] II. Built-in CNN model: The CNN model takes the edge and corner information extracted by the 3D Harris algorithm as input and outputs the judgment results of the personnel density on the escalator and whether large items are stuck on the escalator.

[0032] Furthermore, the large items include strollers, wheelchairs, and large luggage.

[0033] This invention also provides a machine vision-based method for identifying abnormal escalator behavior, using the aforementioned escalator abnormal behavior identification emergency system, comprising the following steps:

[0034] S1. Acquire video data through the video acquisition module and filter noise;

[0035] S2. Preprocess the video data using the edge computing storage module;

[0036] Scenario 1: Monitoring abnormal postures of passengers on escalators;

[0037] The preprocessing for case 1 includes the following steps:

[0038] I. The BlazePose real-time body pose tracking estimation model is used to process video frames, and a bottom-up approach is adopted to detect human skeletal joints. First, based on the confidence of human skeletal joints, the position of the joints is detected to obtain information on each core joint of the body. Second, the affinity of human skeletal joints is used to connect each joint to obtain a human skeleton map. At the same time, the joint angle, angular velocity and angular acceleration of each core joint are obtained.

[0039] II. Based on the similarity of joint features, redundant frames are filtered out and key frames are extracted.

[0040] Scenario 2: Monitor changes in passenger density on escalators and whether there are large objects left on the escalators;

[0041] The preprocessing for case 2 includes the following steps:

[0042] First, perform grayscale processing on the video frames;

[0043] II. Edge extraction is performed on the grayscale video frames using the 3D Harris algorithm to obtain the number of passengers on the escalator and the outline and corner points of large luggage.

[0044] S3. The abnormal behavior recognition module monitors and identifies abnormal postures of objects and passengers on the escalator.

[0045] Monitoring abnormal postures of passengers on escalators:

[0046] The joint features of the keyframes are input into the abnormal behavior recognition module, and the pre-trained TGA-LSTM model in the abnormal behavior recognition module is used to perform pose recognition. If the edge pose recognition result is highly reliable, the pose recognition result is output. If the edge pose recognition result is not highly reliable, the sample is marked as a difficult sample and submitted to the cloud computing module for analysis.

[0047] Monitor changes in passenger density on escalators and whether large objects are left on the escalators:

[0048] Edge and corner information is input into a pre-trained CNN model, which then identifies the number of passengers and large items in video samples; passenger density on escalators. The calculation expression is as follows:

[0049]

[0050] in, The current number of passengers on the escalator; The maximum number of passengers allowed on the escalator; if the real-time passenger density on the escalator... If the density exceeds the set threshold, congestion is determined to have occurred;

[0051] S4. Based on the identification results of the abnormal behavior identification module, determine whether there is any abnormal behavior on the escalator. When abnormal behavior is detected, control the escalator motor through the emergency handling module to implement emergency handling measures to ensure passenger safety.

[0052] Beneficial effects:

[0053] 1. This invention proposes an emergency response system for escalator abnormal behavior recognition based on machine vision. Through efficient human posture estimation, pedestrian density estimation, and large object detection, it achieves rapid identification of abnormal escalator behavior and promptly implements effective emergency measures, automatically controlling escalator operation and issuing alarms to ensure pedestrian safety and greatly improve escalator safety. The application of this technology will provide strong technical support for the safety management of public transportation and commercial venues.

[0054] 2. This invention proposes an emergency system for recognizing abnormal escalator behavior based on machine vision. After efficiently extracting joint features from visualized escalator data, keyframes are extracted based on similarity to identify abnormal postures during escalation. Edge detection can also be used to obtain the contours of people and objects, identifying congestion and the presence of large objects. Furthermore, the abnormal posture and person detection methods described in this invention can be correlated with the escalator motor after outputting results for corresponding emergency handling and automatic escalator control. This provides preventative measures before potential safety hazards occur, effectively ensuring passenger safety.

[0055] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0056] Figure 1 This is a structural block diagram of the machine vision-based escalator abnormal behavior recognition emergency system of the present invention;

[0057] Figure 2 A diagram illustrating abnormal behavior monitored by a camera;

[0058] Figure 3 A flowchart for identifying abnormal postures of escalator passengers;

[0059] Figure 4 Joints of the human body extracted by BlazePose;

[0060] Figure 5This describes the process for estimating escalator passenger density and recognizing objects. Detailed Implementation

[0061] To make the technical solutions, advantages, and objectives of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the protection scope of this application.

[0062] like Figure 1 As shown, the present invention provides an emergency system for recognizing abnormal behavior of escalators based on machine vision, including a video acquisition module, an abnormal behavior recognition module, an edge computing storage module, an edge communication module, an emergency processing module, and a cloud computing module;

[0063] The video acquisition module is connected to the abnormal behavior recognition module and the edge computing storage module via video cables; the edge computing storage module is connected to the abnormal behavior recognition module, the edge communication module, and the emergency handling module via an internal local area network; the edge communication module is connected to the cloud computing module via a wide area network and a firewall is configured; the emergency handling module is connected to the escalator control system via a bus.

[0064] Video capture module

[0065] This is used to acquire real-time video of escalator operation. Specifically, depth cameras, designed to easily obtain distance information from objects to the camera, are typically set in two sets, installed at the beginning and end of the escalator, respectively. Specific positions and angles are as follows... Figure 2 As shown, the data captured by the camera can be transmitted via wired connection to the edge computing storage module.

[0066] Edge computing storage module

[0067] Used for video preprocessing, visualization feature extraction, real-time image storage, abnormal behavior case sample storage, and event storage.

[0068] Abnormal behavior recognition module

[0069] Based on machine learning algorithms, abnormal postures of passengers and objects on escalators are monitored through real-time image analysis, and the detection results are evaluated.

[0070] The abnormal behavior recognition module includes:

[0071] Passenger boundary crossing monitoring submodule: Used to monitor abnormal passenger behavior that crosses boundaries on escalators;

[0072] Passenger fall detection submodule: Used to monitor abnormal passenger behavior such as falling on escalators;

[0073] Passenger Reverse Movement Monitoring Submodule: Used to monitor abnormal passenger behavior of moving in the wrong direction on escalators;

[0074] Passenger Sprint Monitoring Submodule: Used to monitor abnormal passenger sprinting behavior on escalators;

[0075] Passenger flow statistics and density calculation submodule: used to calculate passenger flow and passenger density on escalators;

[0076] Large Item Delay Alarm Submodule: Used to monitor large items (including strollers, wheelchairs and large luggage) left on escalators. It is used to issue an alarm when escalators are congested or large items are left behind, and to send corresponding emergency handling instructions to the emergency handling module based on the alarm.

[0077] Edge communication module

[0078] Specifically, it is a smart gateway, whose function is to provide network interconnection with different high-level protocols, realize the conversion and forwarding of video data and models, connect the cloud wide area network and the edge local area network, provide network security firewall, and provide external updates for the entire system.

[0079] I. Built-in real-time body pose tracking and estimation model BlazePose, which can identify and locate the core joints of the human body in real-time video, thereby obtaining the joint angle, angular velocity and angular acceleration of each core joint.

[0080] The core joints include the shoulders, elbows, wrists, waist, knees, and ankles;

[0081] II. Capable of preprocessing video frames, including grayscale processing;

[0082] III. Built-in 3D Harris algorithm, capable of edge extraction from video frames, thereby obtaining the number of passengers on the escalator and the outline and corner points of large luggage.

[0083] Emergency Response Module

[0084] This module is used to receive emergency handling instructions sent by the abnormal behavior recognition module, change the operating status of the escalator, provide voice reminders to passengers when necessary, avoid safety accidents, and reduce harm to personnel in emergency situations. The emergency handling module is connected to the escalator control system.

[0085] The emergency response module includes:

[0086] Emergency control and processing unit: Based on emergency handling instructions, it links the motor of the escalator to control the escalator, thereby converting the escalator into deceleration, slow stop or emergency stop;

[0087] Voice broadcast unit: used to provide voice reminders to passengers;

[0088] Alarm unit: Used to alert the escalator control center and relevant personnel, notify them to take subsequent actions, and automatically save the alarm information into the system.

[0089] Cloud computing module

[0090] It is used to re-examine difficult samples, and is responsible for issuing system updates and model updates. It can realize the aggregation of information from multiple escalators, display statistical information on the cloud platform, and provide channels to view the overall operation of the escalator on multiple devices.

[0091] In one embodiment of this system, the system uses front and rear cameras of the video acquisition module to collect video data of the escalator's operation. This data is then transmitted via a video transmission cable to an edge computing storage module for video preprocessing and feature extraction. The original video is stored in a database. Subsequently, the processed video and video features are transmitted to an abnormal behavior recognition module for identification. If an abnormal behavior case cannot be accurately identified, the case is sent to a cloud computing module via an edge communication module for verification of the difficult sample. The cloud computing module returns the identification result to the edge computing storage module. If abnormal behavior is detected, the corresponding emergency command is sent to the emergency handling module to control the escalator motor and prevent safety accidents.

[0092] Reference Figure 3 This invention proposes a machine vision-based method for identifying and responding to abnormal postures of escalator passengers. Based on the BlazePose tool integrated into the MediaPipe framework, this method extracts human joints from the bottom up, extracts the motion features of joint vectors, filters out redundant frames, retains keyframe information, and inputs it into a TGA-LSTM abnormal posture analysis model to judge the behavior of escalator passengers. It accurately identifies abnormal behaviors and promptly implements emergency measures, controlling the escalator to switch between slow, decelerate, and emergency stop modes to reduce the occurrence of safety accidents.

[0093] The specific implementation steps are as follows:

[0094] S1. Capture escalator operation video using a depth camera and perform Gaussian filtering to smooth the video and filter out noise.

[0095] S2. For example Figure 4As shown, BlazePose is used to process depth image samples, employing a bottom-up approach to detect human skeletal joints. First, based on the confidence level of the human skeletal joints, their positions are detected, obtaining information on 23 joints of the body. Second, the affinity of the human skeletal joints is used to connect each joint, thus obtaining a human skeleton map. The angle θ between joint O and its two neighboring points A and B is calculated.

[0096]

[0097] in, Joint angle , , The vector of neighboring points is... , .

[0098] For multiple consecutive signal samples, calculate the angular velocity of the joint angle. and angular acceleration

[0099]

[0100]

[0101] in, This is the frame time interval.

[0102] The feature vector S is obtained by extracting the joint angle, joint angular velocity, and joint angular acceleration of each frame of signal.

[0103]

[0104] This is the set of all joint angles in the articulation diagram. , They are respectively A set of joint angular velocities and joint angular accelerations that correspond one-to-one.

[0105] Arrange the feature vectors in temporal order to obtain the global feature set of the entire video signal. ,have

[0106]

[0107] Where n is the total number of frames.

[0108] Furthermore, special attention is paid to the shoulder joints (2, 3), elbow joints (4, 5), lumbar region (14, 15), knee joints (16, 17), and ankles (18, 19), which are referred to as the core joints.

[0109] S3. Keyframe extraction is performed based on the similarity of the extracted feature vectors, and the global feature set is analyzed. Clustering of feature vectors within the frame removes similar redundant feature vectors, extracts keyframes, and reduces the computational load of the pose recognition algorithm.

[0110] S4. Construct a TGA-LSTM network as a pose classifier. LSTM is a temporal recurrent neural network commonly used for classifying, processing, and predicting time series data. TGA is a commonly used attention mechanism that allows the model to "focus" on the more important parts of the input sequence when processing different parts. It typically includes an encoder and a decoder. The encoder is responsible for transforming the input sequence into hidden state vectors at each time step, and the decoder generates the output sequence based on the encoder's hidden state vectors. The steps for constructing the TGA-LSTM network are as follows:

[0111] S4.1 Assume the input sequence is The output state of LSTM is

[0112] S4.2 For time frame t, input The obtained LSTM hidden state and memory state The attention weights are obtained by using them as input for attention calculation. :

[0113]

[0114] in The parameters to be learned are Softmax, which is a normalization exponential function. Its purpose is to represent the multi-class classification results in probabilistic form, mapping the outputs of multiple neurons to... Within the interval, this is transformed into a probability problem to be solved, which can be expressed as follows:

[0115]

[0116] in Input elements for nodes involved in Softmax.

[0117] For S4.3 The weighted sum is obtained by summing all time frames of the input sequence using weighted methods.

[0118]

[0119] S4.4 will and Connect the elements to form a fully connected layer, and obtain the final output:

[0120]

[0121] in for and splicing, and Parameters of fully connected layers

[0122] S4.5 introduces the cross-entropy loss function, which evaluates the accuracy of the TGA-LSTM model by calculating the cross-entropy between the model's prediction results and the true labels, and updates the weight parameters through backpropagation.

[0123] S5. Input the feature vectors of the keyframes into the TGA-LSTM network model for pose recognition. Evaluate the confidence of pose recognition based on the output probability of softmax. If the output probability of softmax decreases in a gradient manner, the top 1 of the softmax output is taken as the recognition result. If the output probability of softmax is almost equal, the sample is marked as a difficult sample and sent to the cloud computing module for verification.

[0124] S6. If any abnormal behavior (crossing the boundary, falling, going against the flow, running) occurs in the identified posture, the identification result will be sent to the emergency handling module to provide voice reminders and elevator control strategies for the corresponding abnormal behavior category, and to remind the inspection personnel to handle the abnormal problem.

[0125] In one embodiment of this system, the system uses front and rear cameras of the video acquisition module to collect video data of the escalator's operation. This data is then transmitted via a video transmission cable to an abnormal behavior recognition module for video preprocessing and edge extraction. The module identifies the number of passengers in the video and monitors the presence of large items. If the passenger density exceeds a threshold or large items are found to be lingering, a corresponding emergency command is sent to the emergency handling module to control the escalator motor and prevent accidents.

[0126] Reference Figure 5 This invention proposes a machine vision-based method for estimating escalator passenger density, recognizing objects, and responding to emergencies. This method monitors changes in escalator passenger density and detects the presence of large objects on the escalator. It uses the 3D Harris algorithm to extract the edges and corners of people or objects, which are then input into a CNN model for recognition. The specific implementation steps are as follows:

[0127] S1. Capture escalator operation video using a depth camera and perform median smoothing filtering to remove noise.

[0128] S2. Perform grayscale processing on each frame of the original video sample.

[0129] S3. Edge extraction is performed using the 3D Harris algorithm to obtain the number of passengers currently riding the escalator and the outline and corner points of large luggage items.

[0130] S3.1 Calculate the response value R for each pixel.

[0131] S3.2 normalizes it and sets a response threshold. The coarse screening yielded responses greater than The point is the edge point.

[0132] S3.3 Calculate the gradient and grayscale changes of the edge points in three dimensions, and calculate the energy function E(u,v,w).

[0133] S3.3 Determines whether a point is a corner point by judging the difference in energy function between each edge point and its surrounding edge points.

[0134] S4. Input the edge point and corner point information into the pre-trained CNN model to identify the number of occupants and large items (strollers, large luggage, wheelchairs) in the video samples.

[0135] S5. Count the current number of passengers on the escalator and calculate the real-time escalator passenger density Q.

[0136]

[0137] in, The current number of passengers on the escalator. This is the maximum number of passengers the escalator can carry.

[0138] S6. If the real-time escalator passenger density exceeds the density threshold (i.e., congestion is considered to have occurred), a signal is automatically sent to the emergency handling module to control the escalator to slow down or stop urgently, play the corresponding broadcast notification, and remind the inspection personnel to guide the crowd in time; if large items are found to be left behind, the escalator is controlled to stop urgently, and the inspection personnel are reminded to remove the left behind items in time.

[0139] It is hereby declared that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An emergency response system for recognizing abnormal escalator behavior based on machine vision, characterized in that: It includes a video acquisition module, an abnormal behavior recognition module, an edge computing storage module, an edge communication module, an emergency response module, and a cloud computing module; Video capture module: used to acquire real-time video of the escalator in operation; Edge computing storage module: used for video preprocessing, visualization feature extraction, real-time image storage, abnormal behavior case sample storage, and event storage; Abnormal Behavior Recognition Module: Based on the pre-processed video and preliminary extracted visualization features provided by the edge computing storage module, it monitors abnormal postures of objects and passengers on escalators and evaluates the monitoring results. Edge communication module: used to provide network interconnection with different high-level protocols, realize the conversion and forwarding of video data and models, connect the cloud wide area network and the edge local area network, provide network security firewall, and provide external updates for the entire system; Emergency handling module: Used to receive emergency handling instructions sent by the abnormal behavior recognition module, change the operating status of the escalator, provide voice reminders to passengers, and communicate with the control system of the escalator. Cloud computing module: Used to re-examine difficult samples, responsible for issuing system updates and model updates, realizing the aggregation of information from multiple escalators, displaying statistical information on the cloud platform, and providing channels to view the overall operation of the escalator on multiple devices.

2. The emergency response system for recognizing abnormal escalator behavior based on machine vision according to claim 1, characterized in that: The video acquisition module includes two cameras, which are respectively installed at the starting and ending points of the escalator. The cameras can acquire real-time images of the escalator in operation and can also acquire distance information from objects on the escalator to the camera lenses.

3. The emergency response system for recognizing abnormal escalator behavior based on machine vision according to claim 2, characterized in that: The abnormal behavior identification module includes: Passenger boundary crossing monitoring submodule: Used to monitor abnormal passenger behavior that crosses boundaries on escalators; Passenger fall detection submodule: Used to monitor abnormal passenger behavior such as falling on escalators; Passenger Reverse Movement Monitoring Submodule: Used to monitor abnormal passenger behavior of moving in the wrong direction on escalators; Passenger Sprint Monitoring Submodule: Used to monitor abnormal passenger sprinting behavior on escalators; Passenger flow statistics and density calculation submodule: used to calculate passenger flow and passenger density on escalators; Large Item Delay Alarm Submodule: Used to monitor large items left on escalators, issue alarms when escalators are congested or large items are left behind, and send corresponding emergency handling instructions to the emergency handling module based on the alarm.

4. The emergency response system for recognizing abnormal escalator behavior based on machine vision according to claim 3, characterized in that: The emergency response module includes: Emergency control and processing unit: Based on emergency handling instructions, it links the motor of the escalator to control the escalator, thereby converting the escalator into deceleration, slow stop or emergency stop; Voice broadcast unit: used to provide voice reminders to passengers; Alarm unit: Used to alert the escalator control center and relevant personnel, notify them to take subsequent actions, and automatically save the alarm information into the system.

5. The emergency system for recognizing abnormal escalator behavior based on machine vision according to claim 4, characterized in that: The edge computing storage module: I. Built-in real-time body pose tracking and estimation model BlazePose, which can identify and locate the core joints of the human body in real-time video, thereby obtaining the joint angle, angular velocity and angular acceleration of each core joint. The core joints include the shoulders, elbows, wrists, waist, knees, and ankles; II. Capable of preprocessing video frames, including grayscale processing; III. Built-in 3D Harris algorithm, which can extract edges from video frames to obtain the number of passengers on the escalator and the outline and corners of large luggage.

6. The emergency system for recognizing abnormal escalator behavior based on machine vision according to claim 5, characterized in that: The abnormal behavior identification module: I. Built-in TGA-LSTM model, which can perform pose recognition based on the features of core joints in video frames. II. Built-in CNN model: The CNN model takes the edge and corner information extracted by the 3D Harris algorithm as input and outputs the judgment results of the personnel density on the escalator and whether large items are stuck on the escalator.

7. The emergency system for recognizing abnormal escalator behavior based on machine vision according to claim 6, characterized in that: Large items include strollers, wheelchairs, and large luggage.

8. A method for identifying abnormal behavior of escalators based on machine vision, characterized in that, Using the escalator abnormal behavior identification and emergency system as described in any one of claims 1 to 7 includes the following steps: S1. Acquire video data through the video acquisition module and filter noise; S2. Preprocess the video data using the edge computing storage module; Scenario 1: Monitoring abnormal postures of passengers on escalators; The preprocessing for case 1 includes the following steps: I. The BlazePose real-time body pose tracking estimation model is used to process video frames, and a bottom-up approach is adopted to detect human skeletal joints. First, based on the confidence of human skeletal joints, the position of the joints is detected to obtain information on each core joint of the body. Second, the affinity of human skeletal joints is used to connect each joint to obtain a human skeleton map. At the same time, the joint angle, angular velocity and angular acceleration of each core joint are obtained. II. Based on the similarity of joint features, redundant frames are filtered out and key frames are extracted. Scenario 2: Monitor changes in passenger density on escalators and whether there are large objects left on the escalators; The preprocessing for case 2 includes the following steps: First, perform grayscale processing on the video frames; II. Edge extraction is performed on the grayscale video frames using the 3D Harris algorithm to obtain the number of passengers on the escalator and the outline and corner points of large luggage. S3. The abnormal behavior recognition module monitors and identifies abnormal postures of objects and passengers on the escalator. Monitoring abnormal postures of passengers on escalators: The joint features of the keyframes are input into the abnormal behavior recognition module, and the pre-trained TGA-LSTM model in the abnormal behavior recognition module is used to perform pose recognition. If the edge pose recognition result is highly reliable, the pose recognition result is output. If the edge pose recognition result is not highly reliable, the sample is marked as a difficult sample and submitted to the cloud computing module for analysis. Monitor changes in passenger density on escalators and whether large objects are left on the escalators: Edge and corner information is input into a pre-trained CNN model, which then identifies the number of passengers and large items in video samples; passenger density on escalators. The calculation expression is as follows: in, The current number of passengers on the escalator; The maximum number of passengers allowed on the escalator; if the real-time passenger density on the escalator... If the density exceeds the set threshold, congestion is determined to have occurred; S4. Based on the identification results of the abnormal behavior identification module, determine whether there is any abnormal behavior on the escalator. When abnormal behavior is detected, control the escalator motor through the emergency handling module to implement emergency handling measures to ensure passenger safety.

Images