A step type vertical traffic facility danger monitoring and safety protection system and method

Abstract

A step type vertical traffic facility danger monitoring and safety protection system, comprising a danger detection device and a wearable falling detection component, a falling protection device, a voice prompting device, a comprehensive management system and a communication device, the danger detection device comprises a visible light image acquisition device and an infrared image acquisition device, is used for acquiring visible light images through the visible light image acquisition device, acquiring infrared images through the infrared image acquisition device of a near infrared wave band, and analyzing and processing the infrared images and the visible light images by using an image processing algorithm, realizing fire source and smoke detection, determining which level of steps a human body stands on, and detecting falling danger through a posture detection algorithm.The present application can simultaneously detect three kinds of dangers of falling of pedestrians and humanoid robots, fire and smoke, and give protection measures according to the dangers, especially can reduce or avoid the harm or damage caused by falling of pedestrians and robots.

Description

Technical Field

[0001] This invention relates to the fields of safety protection and intelligent building technology, and in particular to a hazard monitoring and safety protection system for stepped vertical transportation facilities, as well as a protection method for the aforementioned hazard monitoring and safety protection system for stepped vertical transportation facilities. Background Technology

[0002] For vertical transportation facilities in buildings, such as stairs and steps, pedestrians or expensive humanoid robots are more likely to fall and suffer greater injuries or losses than on roads. For public safety, there is an urgent need to develop a protective system to improve the safety of passage through vertical transportation facilities like stairs and steps, especially in residential areas for the elderly, young children, and patients, or in training and workplaces for humanoid robots. Existing solutions mostly focus on single technologies such as pedestrian fall detection, fire detection, and smoke detection, lacking proactive physical protection mechanisms that are linked in real-time to dangerous situations. In particular, there is a gap in measures and facilities for proactive protection of pedestrians or humanoid robots who fall on stairs or steps.

[0003] Although wearable fall protection products and some intellectual property exist, they are not used by everyone, and their protection scope is limited. In summary, while individual hazard monitoring technologies or personal protective equipment have some applications, a systematic solution that deeply couples multi-source hazard detection (falls, fires, smoke, etc.) with active physical protection actuators is still lacking in the field of building vertical transportation facilities. Existing technologies, especially in public facilities, have failed to address the key challenge of "how to intervene immediately after monitoring."

[0004] Therefore, developing a system that integrates hazard monitoring and proactive protection is of significant practical necessity and application value. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a hazard monitoring and safety protection system for vertical transportation facilities such as steps that can simultaneously detect three hazards: pedestrian and humanoid robot falls, fire, and smoke, and provide protective measures according to the hazards. In particular, it can reduce or avoid injuries or damages caused by pedestrians and robots falling.

[0006] Another technical problem to be solved by the present invention is to provide a protection method for the above-mentioned hazard monitoring and safety protection system for stepped vertical transportation facilities.

[0007] The technical problem to be solved by this invention is achieved through the following technical solution. This invention is a hazard monitoring and safety protection system for stepped vertical transportation facilities, including a hazard detection device, a wearable fall detection component, a fall protection device, a voice prompt device, a comprehensive management system, and a communication device.

[0008] The hazard detection device includes a visible light image acquisition device and an infrared image acquisition device. It is used to acquire visible light images through the visible light image acquisition device and infrared images through the near-infrared band infrared image acquisition device. The infrared images and visible light images are analyzed and processed using image processing algorithms to detect fire sources and smoke, determine which step a person is standing on, and detect the risk of falling through posture detection algorithms.

[0009] The wearable fall detection component includes a fall detection electronic device worn by or embedded in the body of the person or robot being detected.

[0010] The fall protection device includes a flexible energy-absorbing device concealed under the tread and behind the kick surface of the step. When a fall signal is received, the flexible energy-absorbing device pops out and covers the tread of the step to form protection, reduce injury and damage, and protect pedestrians or humanoid robots.

[0011] For an upward-moving pedestrian or humanoid robot, when a signal is received that the pedestrian or humanoid robot has fallen, the flexible energy-absorbing devices on all levels of steps above and below the tread where the fallen pedestrian or humanoid robot is located will deploy. If there is an unfallen person or humanoid robot above the fallen pedestrian or humanoid robot, the flexible energy-absorbing devices on all levels of steps above the unfallen person or humanoid robot will not deploy. If there is an unfallen person or humanoid robot below the fallen pedestrian or humanoid robot, the flexible energy-absorbing devices on all levels of steps below the unfallen person or humanoid robot will not deploy.

[0012] For pedestrians or humanoid robots going downhill, when a signal is received that a pedestrian or humanoid robot has fallen, the flexible energy-absorbing devices will only pop up on each step below the tread where the fallen pedestrian or humanoid robot is located. If there are pedestrians or humanoid robots below the fallen pedestrian or humanoid robot that have not fallen, the flexible energy-absorbing devices below the tread where the unfallen pedestrian or humanoid robot is located will not pop up.

[0013] When a fire signal or an emergency evacuation signal is received, the fall protection device will not activate and the flexible energy absorption device will not deploy.

[0014] The voice prompt device is used to play pre-recorded audio signals or for managers to communicate with on-site personnel.

[0015] The integrated management system is used to monitor and manage video information, perform system maintenance and fault diagnosis, collect data statistics, interact with other management systems, and communicate with the site via a voice prompt device.

[0016] The communication device is used for information transmission between the hazard detection device, the fall protection device, the voice prompt device, the integrated management system, the wearable fall detection component, and other management systems. The hazard detection device and the wearable fall detection component transmit hazard detection signals to the integrated management system through the communication device. After comprehensive analysis, the integrated management system sends control signals to the control box of the fall protection device through the communication device. The control box sends a trigger signal through the control line to activate the trigger of the fall protection device, triggering the flexible energy-absorbing device to pop out and cover the tread of the step, forming protection for pedestrians or humanoid robots, and sending the corresponding voice signal to the voice prompt device to play the pre-recorded audio signal.

[0017] The technical problem to be solved by the present invention can also be further achieved through the following technical solutions: For the above-mentioned step-type vertical transportation facility hazard monitoring and safety protection system, after the visible light image acquisition device and the infrared image acquisition device capture video or images, the video or images are transmitted to the integrated management system, which uses image processing algorithms to detect fire sources and smoke hazards, determine which step a person or humanoid robot is standing on, and detect fall hazards through posture detection algorithms.

[0018] Alternatively, the visible light image acquisition device and the infrared image acquisition device can directly detect fire sources and smoke hazards through image processing algorithms, determine which step a person or humanoid robot is standing on, detect fall hazards through posture detection algorithms, and transmit the results to the integrated management system.

[0019] The technical problem to be solved by the present invention can also be further achieved through the following technical solution: For the above-mentioned step-type vertical transportation facility hazard monitoring and safety protection system, the image processing algorithm is a convolutional neural network algorithm. The input of the convolutional neural network algorithm is a fusion feature map of infrared image and visible light image. The fusion feature map is formed by preprocessing the image to extract the thermal radiation contour of the infrared image and preprocessing the visible light image to extract the color and texture features of the visible light image; the thermal radiation contour features and color and texture features are fused at multiple scales to form a fusion feature map.

[0020] The technical problem to be solved by this invention can also be further achieved through the following technical solution: For the above-mentioned hazard monitoring and safety protection system for stepped vertical transportation facilities, the system detects fall hazards using a posture detection algorithm. The system outputs key human body points through the YOLOv8-pose model, including: left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joint coordinates. It then calculates the distance between the left and right shoulder and ankle joints, and calculates the left and right distances between the left and right hip joints and the left and right shoulder joints, respectively. The trunk vector is calculated by: calculating the left and right upper arm vectors between the left and right shoulder joints and the left and right elbow joints, respectively; calculating the left and right forearm vectors between the left and right wrist joints and the left and right elbow joints, respectively; calculating the left and right thigh vectors between the left and right knee joints and the left and right hip joints, respectively; and calculating the left and right lower leg vectors between the left and right knee joints and the left and right ankle joints, respectively. Then, the trunk tilt angle is calculated by: calculating the angle between the left and right upper arms and the trunk, calculating the bending angles of the left and right elbow joints, calculating the bending angles of the left and right torso, and calculating the bending angles of the left and right knee joints.

[0021] The step on which a person is standing is determined by the step on which the ankle joint is located, as output by the YOLOv8-pose model.

[0022] The technical problem to be solved by the present invention can also be further achieved through the following technical solutions. For the above-mentioned step-type vertical traffic facility hazard monitoring and safety protection system, the torso tilt angle is defined as the angle between the torso vector and the vertical direction, the left and right shoulder-ankle distances are defined as the distances between the left and right shoulder joints and the left and right ankle joints, respectively, the left and right upper arm torso angles are defined as the angles between the left and right upper arm vectors and the torso vectors, respectively, the left and right elbow joint bending angles are defined as the angles between the left and right reverse upper arm vectors and the forearm vectors, respectively, the left and right torso bending angles are defined as the angles between the corresponding reverse torso vectors and the thigh vectors, respectively, and the left and right knee joint bending angles are defined as the angles between the left and right reverse thigh vectors and the calf vectors, respectively.

[0023] The technical problem to be solved by the present invention can also be further achieved through the following technical solutions. For the above-mentioned step-type vertical transportation facility hazard monitoring and safety protection system, the posture detection algorithm detects the risk of falling by determining whether a fall has occurred based on a dynamic threshold set according to the current height. The dynamic thresholds include: dynamic threshold for trunk tilt angle, dynamic threshold for trunk tilt angle change rate, dynamic threshold for shoulder-ankle distance, dynamic threshold for shoulder-ankle distance change rate, dynamic threshold for knee flexion angle, dynamic threshold for knee flexion angle change rate, dynamic threshold for elbow flexion angle, dynamic threshold for elbow flexion angle change rate, dynamic threshold for upper arm-torso angle, dynamic threshold for upper arm-torso angle change rate, dynamic threshold for body flexion angle, and dynamic threshold for body flexion angle change rate.

[0024] The technical problem to be solved by this invention can also be further achieved through the following technical solution: For the above-mentioned hazard monitoring and safety protection system for stepped vertical transportation facilities, the determination of whether a fall has occurred is made when at least one of the following criteria is met:

[0025] In n consecutive frames (n≥3), the torso tilt angle exceeds the dynamic threshold of the torso tilt angle;

[0026] In n consecutive frames, the rate of change of the torso tilt angle exceeds the dynamic threshold of the rate of change of the torso tilt angle;

[0027] In n consecutive frames, the distance between the left or right shoulder and ankle exceeds the dynamic threshold for shoulder and ankle distance;

[0028] In n consecutive frames, the rate of change of the left or right shoulder-ankle distance exceeds the dynamic threshold of the rate of change of shoulder-ankle distance;

[0029] In n consecutive frames, the left or right knee flexion angle exceeds the dynamic threshold of the knee flexion angle.

[0030] In n consecutive frames, the rate of change of the left or right knee flexion angle exceeds the dynamic threshold of the rate of change of the knee flexion angle.

[0031] In n consecutive frames, the left or right elbow flexion angle exceeds the dynamic threshold of the elbow flexion angle.

[0032] In n consecutive frames, the rate of change of the left or right elbow flexion angle exceeds the dynamic threshold of the rate of change of elbow flexion angle.

[0033] In n consecutive frames, the angle between the left or right upper arm and torso exceeds the dynamic threshold of the upper arm-torso angle.

[0034] In n consecutive frames, the rate of change of the angle between the left or right upper arm and the torso exceeds the dynamic threshold of the rate of change of the angle between the upper arm and the torso.

[0035] In n consecutive frames, the left or right torso bending angle exceeds the dynamic threshold of torso bending angle;

[0036] In n consecutive frames, the rate of change of the left or right body bending angle exceeds the dynamic threshold of the rate of change of body bending angle.

[0037] The technical problem to be solved by the present invention can also be further achieved through the following technical solutions: For the above-mentioned step-type vertical transportation facility hazard monitoring and safety protection system, the integrated management system is implemented by application software running on a computer, mobile phone or tablet computer. The application software has the following functions: fall detection, recording and saving recent videos, recording the time and location of the hazard, recording videos and images when the hazard occurs, diagnosing equipment faults, managing the database, monitoring the site, having voice communication with the site, and exchanging information with other management systems.

[0038] The technical problem to be solved by the present invention can also be further achieved through the following technical solution: For the above-mentioned hazard monitoring and safety protection system for stepped vertical transportation facilities, two sets of visible light image acquisition devices and infrared image acquisition devices are provided, which are respectively installed at the upper and lower ends of the stepped vertical transportation facilities, so that the front of a person or humanoid robot can be captured regardless of whether it is going up or down.

[0039] The technical problem to be solved by the present invention can also be further achieved through the following technical solution: For the above-mentioned hazard monitoring and safety protection system for stepped vertical transportation facilities, a method for hazard monitoring and safety protection of stepped vertical transportation facilities is provided, the steps of which are as follows:

[0040] (1) Data acquisition steps:

[0041] Visible light image acquisition devices and infrared image acquisition devices are deployed at the upper and lower ends of the stepped vertical transportation facilities to acquire visible light images and near-infrared infrared images respectively. At the same time, wearable fall detection components collect motion state data of the detected person or robot in real time.

[0042] (2) Image analysis and hazard identification steps:

[0043] Visible light and infrared images are input into image processing algorithms for analysis and processing. Thermal radiation contour features of infrared images and color texture features of visible light images are extracted. After multi-scale feature fusion, a convolutional neural network algorithm is used to detect fire sources and smoke. At the same time, the coordinates of human body key points are output through the YOLOv8-pose model. The ankle joint coordinates determine which step the human body is standing on. Based on the key point coordinates, the torso tilt angle, shoulder-torso distance, upper arm-torso angle, elbow joint flexion angle, torso flexion angle, knee joint flexion angle and their rate of change are calculated.

[0044] (3) Fall detection steps:

[0045] Based on the current step height, a dynamic threshold is set. The angle and distance parameters calculated in step (2) are compared with the corresponding dynamic threshold and the dynamic threshold of change rate. When at least one of the judgment criteria is met, it is judged as a fall and a fall signal is generated. At the same time, the wearable fall detection component sends the detected fall signal to the integrated management system through the communication device.

[0046] (4) Comprehensive decision-making and protection triggering steps:

[0047] The integrated management system receives signals from the hazard detection device and the wearable fall detection component, performs comprehensive analysis, and determines the type of hazard. If it is determined to be a fall hazard, it executes corresponding protective strategies based on the direction of travel of the fallen pedestrian or humanoid robot.

[0048] If the fall is an upward fall, the flexible energy-absorbing devices on the steps above and below the person who fell will be triggered to pop out. However, if there is a person who has not fallen above, the flexible energy-absorbing devices on the steps above the person who has not fallen will not pop out. If there is a person who has not fallen below, the flexible energy-absorbing devices on the steps below the person who has not fallen will not pop out.

[0049] If the fall is downward, the flexible energy-absorbing devices of the steps below the tread where the faller is located will be triggered to pop out. However, if there is a person below the faller who has not fallen, the flexible energy-absorbing devices of the steps below the person who has not fallen will not pop out.

[0050] The fall protection device will not be triggered if there is a fire or emergency evacuation signal.

[0051] The integrated management system sends control signals to the control box of the fall protection device through the communication device. The control box sends a trigger signal through the control line to activate the trigger, causing the flexible energy-absorbing device to pop out and cover the step surface. At the same time, it sends the corresponding voice signal to the voice prompt device to play the pre-recorded audio signal.

[0052] (5) Information management and interaction steps:

[0053] The integrated management system records the time and location of the emergency, along with corresponding videos and images, saves recent video data, performs fault diagnosis on equipment, and interacts with other management systems. Managers can use the integrated management system to communicate with the site remotely for monitoring and command.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] 1. This invention can comprehensively monitor pedestrian falls, fires, and smoke in vertical transportation facilities such as stairs or steps, and actively protect fallen pedestrians and humanoid robots, making passage safer;

[0056] 2. Compared with wearable fall protection devices, this invention protects a wider range of people and has greater universality;

[0057] 3. This invention solves the inconvenience caused by wearable fall protection devices and achieves seamless protection;

[0058] 4. This invention solves the problem of inadequate protection when wearable fall protection devices are used when users "resist wearing" or "forget to wear" them. Attached Figure Description

[0059] Figure 1 This is a general structural diagram of the present invention;

[0060] Figure 2 This is a structural diagram of a fall protection device;

[0061] Figure 3 This is a diagram illustrating key points on the front of the human body.

[0062] The attached diagram lists the components represented by each number as follows:

[0063] 1. Hazard detection device; 1-1. Visible light image acquisition device; 1-2. Infrared image acquisition device; 2. Fall protection device; 2-1. Flexible energy absorption device; 2-2. Trigger; 2-3. Control box; 2-4. Control cable; 2-5. Step tread; 2-6. Step riser; 3. Voice prompt device; 4. Integrated management system; 5. Communication device; 5-1. Communication cable; 6. Wearable fall detector; 7. Nose; 8. Right eye; 9. Left eye; 10. Right ear; 11. Left ear; 12. Right shoulder; 13. Left shoulder; 14. Right elbow; 15. Left elbow; 16. Right wrist; 17. Left wrist; 18. Right hip; 19. Left hip; 20. Right knee; 21. Left knee; 22. Right ankle; 23. Left ankle 24. Right upper arm vector; 25. Right torso vector; 26. Left upper arm vector; 27. Left torso vector; 28. Right upper arm torso angle; 29. ​​Left upper arm torso angle; 30. Reverse right upper arm vector; 31. Right forearm vector; 32. Reverse left upper arm vector; 33. Left forearm vector; 34. Right elbow flexion angle; 35. Left elbow flexion angle; 36. Reverse right torso vector; 37. Right thigh vector; 38. Reverse left torso vector; 39. Left thigh vector; 40. Right torso flexion angle; 41. Left torso flexion angle; 42. Reverse right thigh vector; 43. Right calf vector; 44. Reverse left thigh vector; 45. Left calf vector; 46. Right knee flexion angle; 47. Left knee flexion angle. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Example 1, such as Figure 1 and Figure 2As shown, this embodiment provides a hazard monitoring and safety protection system for stepped vertical transportation facilities, including a hazard detection device 1, a wearable fall detector 6, a fall protection device 2, a voice prompt device 3, a comprehensive management system 4, and a communication device 5. The hazards include falls of people or humanoid robots, fires, and smoke. While fall detection can be performed using one or two of the visible light image acquisition device 1-1, infrared image acquisition device 1-2 (near-infrared band 700-1400nm), and wearable fall detector 6, fire and smoke detection can also be performed using the aforementioned devices alone. One of the visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 is used. However, for more reliable detection, this embodiment preferably uses three detection devices simultaneously: the visible light image acquisition device 1-1, the infrared image acquisition device 1-2, and the wearable fall detector 6 to achieve fall detection. The detection of fire and smoke also uses both the visible light image acquisition device 1-1 and the infrared image acquisition device 1-2. The determination of which step a person or humanoid robot is standing on also uses both the visible light image acquisition device 1-1 and the infrared image acquisition device 1-2.

[0066] The visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 can be common cameras, network cameras (IPCs), industrial cameras, or modules with integrated image sensors. They can also be two independent image or video acquisition devices, or a dual-spectrum image acquisition device that can simultaneously capture visible light or video and infrared images or videos. Preferably, this embodiment uses a dual-spectrum image acquisition device that can simultaneously capture visible light or video and infrared images or videos. Further, a dual-light camera that can simultaneously capture visible light and infrared images is used, that is, the visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 are integrated into the same camera.

[0067] The visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 are single-function devices that can only capture images. After capturing videos or images, they are transmitted to the integrated management system. The integrated management system uses image processing algorithms to detect fire sources and smoke hazards, determine which step a person or humanoid robot is standing on, and detect fall hazards using posture detection algorithms.

[0068] The image processing algorithm is a convolutional neural network algorithm. The input of the convolutional neural network algorithm is a fusion feature map of infrared image and visible light image. The fusion feature map is formed by preprocessing the image to extract the thermal radiation contour of infrared image, preprocessing the visible light image to reduce the influence of uneven lighting and shadows, and extracting the color and texture features of visible light image; the thermal radiation contour features and color and texture features are fused at multiple scales to form a fusion feature map.

[0069] The method for detecting fall hazards using a posture detection algorithm is implemented through the YOLOv8-pose model, as shown in the attached figure. Figure 3 As shown, although the YOLOv8-pose model can output the coordinates of 17 key points of the human body, preferably, to reduce the false detection rate and make the algorithm more robust, this embodiment only uses 12 key points, including: left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. Using the coordinates of these 12 key points, the left and right shoulder-ankle distances between the left and right shoulder joints and the left and right ankle joints are calculated, the left trunk vector 27 and right trunk vector 25 between the left and right hip joints and the left and right shoulder joints are calculated, and the left upper arm vector 26 and right upper arm vector 25 between the left and right shoulder joints and the left and right elbow joints are calculated. Calculate the upper arm vector 24, the left forearm vector 33 and the right forearm vector 31 between the left and right wrist joints and the left and right elbow joints respectively, the left thigh vector 39 and the right thigh vector 37 between the left and right knee joints and the left and right hip joints respectively, and the left lower leg vector 45 and the right lower leg vector 43 between the left and right knee joints and the left and right ankle joints respectively; then calculate the left and right torso tilt angles respectively, the left upper arm torso angle 29 and the right upper arm torso angle 28 respectively, the left elbow joint flexion angle 35 and the right elbow joint flexion angle 34 respectively, the left torso flexion angle 41 and the right torso flexion angle 40 respectively, and the left knee joint flexion angle 47 and the right knee joint flexion angle 46 respectively.

[0070] The confidence threshold of the YOLOv8-pose model is selected as 0.4 to 0.7, preferably 0.5 in this embodiment; the IOU threshold is selected as 0.3 to 0.6, preferably 0.45 in this embodiment.

[0071] The left and right torso tilt angles are defined as the angles between the left and right torso vectors and the vertical direction, respectively; the left shoulder-ankle distance and right shoulder-ankle distance are defined as the distances between the left shoulder joint 13 and the right shoulder joint 12 and the left ankle joint 23 and the right ankle joint 22, respectively; the left upper arm-torso angle 29 and the right upper arm-torso angle 28 are defined as the angles between the left upper arm vector 26 and the right upper arm vector 24 and the left torso vector 27 and the right torso vector 25, respectively; the left elbow joint flexion angle 35 and the right elbow joint flexion angle 34 are respectively... The angles 33 and 31 are defined as the angles between the left reverse upper arm vector and the right reverse upper arm vector, respectively; the left torso bending angle 41 and the right torso bending angle 40 are defined as the angles between the left reverse torso vector 38 and the right reverse torso vector 36, respectively, and the left thigh vector 39 and the right thigh vector 37, respectively; the left knee joint bending angle 47 and the right knee joint bending angle 46 are defined as the angles between the left reverse thigh vector 44 and the right reverse thigh vector 42, respectively, and the left calf vector 45 and the right calf vector 43, respectively.

[0072] The method detects fall risks using a posture detection algorithm and determines whether a fall has occurred based on dynamic thresholds set according to the current height. These dynamic thresholds include: dynamic thresholds for trunk tilt angle, dynamic thresholds for the rate of change of trunk tilt angle, dynamic thresholds for shoulder-to-ankle distance, dynamic thresholds for the rate of change of shoulder-to-ankle distance, dynamic thresholds for knee flexion angle, dynamic thresholds for the rate of change of knee flexion angle, dynamic thresholds for elbow flexion angle, dynamic thresholds for the rate of change of elbow flexion angle, dynamic thresholds for the upper arm-to-thigh angle, dynamic thresholds for the rate of change of upper arm-to-thigh angle, dynamic thresholds for body flexion angle, and dynamic thresholds for the rate of change of body flexion angle.

[0073] The determination of which step a person or humanoid robot is standing on is based on the left ankle joint 23 or right ankle joint 22 of the human body output by the YOLOv8-pose model. When going up, the higher ankle joint is used to determine the step, and when going down, the lower ankle joint is used to determine the step.

[0074] The visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 are located at the upper and lower ends of the stepped vertical transportation facility, respectively. Two sets are deployed so that they can capture the front view of a person or humanoid robot whether going up or down.

[0075] The method of detecting fall hazards through posture detection algorithm, for determining the fall of pedestrians and humanoid robots going uphill, the visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 are installed at the upper end of the stepped vertical transportation facility and close to the highest step, and ensure that the field of view can cover all steps of the stepped vertical transportation facility and the pedestrians and humanoid robots on the steps.

[0076] The method for detecting fall hazards using a posture detection algorithm, and the dynamic threshold settings for determining falls by pedestrians and humanoid robots moving upwards, are as follows (step height and distance are in meters, and the number of steps is the total number of steps from the lowest step to the step where the person or humanoid robot is standing):

[0077] (1) Dynamic threshold of the upward torso tilt angle: set as A1 degrees + a1 × step height × step number, where 20 degrees ≤ A1 ≤ 80 degrees, 0.1 ≤ a1 ≤ 2.5; preferably, in this embodiment A1 = 45 degrees, a1 = 1.5.

[0078] (2) Threshold for dynamic change rate of torso tilt angle: set as ∇A1 degrees + α1 × step height × step number, where ∇A1 is the rate of change of torso tilt angle, 1 degree ≤ ∇A1 ≤ 10 degrees, 0.1 ≤ α1 ≤ 0.75; preferably, in this embodiment, ∇A1 = 5 degrees, α1 = 0.25.

[0079] (3) Dynamic threshold for the rate of change of the upward shoulder-ankle distance: set as ∇B1 + β1 × step height × number of steps, where ∇B1 is the rate of change of the upward shoulder-ankle distance, 0.05 ≤ ∇B1 ≤ 0.5, 0.01 ≤ β1 ≤ 0.1; preferably, in this embodiment ∇B1 = 0.1, β1 = 0.05.

[0080] (4) Dynamic threshold of upward knee flexion angle: set as C1 degree + c1 × step height × step number, where 20 degrees ≤ C1 ≤ 70 degrees, 0.25 ≤ c1 ≤ 2.5; preferably, in this embodiment, C1 = 45 degrees, c1 = 1.5.

[0081] (5) Dynamic threshold of the rate of change of the upward knee flexion angle: set as ∇C1 degrees + γ1 × step height × number of steps, where ∇C1 is the rate of change of the upward knee flexion angle, 0.5 degrees ≤ ∇C1 ≤ 30 degrees, 0.1 ≤ γ1 ≤ 3.0; preferably, in this embodiment, ∇C1 = 3 degrees, γ1 = 0.5.

[0082] (6) Dynamic threshold of the rate of change of the upward elbow joint flexion angle: set as ∇D1 degrees + δ1 × step height × step number, where ∇D1 is the rate of change of the upward elbow joint flexion angle, 0.1 degrees ≤ ∇D1 ≤ 30 degrees, 0.05 ≤ δ1 ≤ 5.0; preferably, in this embodiment, ∇D1 = 3 degrees, δ1 = 0.75.

[0083] (7) Dynamic threshold of the angle between the upper arm and torso: set as E1 degree + e1 × step height × number of steps, where 3 degrees ≤ E1 ≤ 90 degrees, 0.1 ≤ e1 ≤ 2.5; preferably, in this embodiment, E1 = 20 degrees, e1 = 0.5.

[0084] (8) Dynamic threshold for the rate of change of the angle between the upper arm and torso: set as ∇E1 degrees + ξ1 × step height × number of steps, where ∇E1 is the rate of change of the angle between the upper arm and torso, 0.1 degrees ≤ ∇E1 ≤ 30 degrees, 0.01 ≤ ξ1 ≤ 1.0; preferably, in this embodiment, ∇E1 = 2.5 degrees, ξ1 = 0.25.

[0085] (9) Dynamic threshold of the upward body bending angle: set as F1 degree + f1 × step height × step number, where 50 degrees ≤ F1 ≤ 90 degrees, 0.1 ≤ f1 ≤ 2.5; preferably, in this embodiment F1 = 60 degrees, f1 = 2.5.

[0086] (10) Dynamic threshold of the rate of change of the upward body bending angle: set as ∇F1 degrees + λ1 × step height × number of steps, where ∇F1 is the rate of change of the upward body bending angle, 0.1 degrees ≤ ∇F1 ≤ 30 degrees, 0.01 ≤ λ1 ≤ 0.25; preferably, in this embodiment, ∇F1 = 1 degree, λ1 = 0.15.

[0087] The criteria for determining falls by using a posture detection algorithm to detect fall hazards, and for determining falls by pedestrians and humanoid robots moving upwards (hereinafter referred to as the upward fall determination criteria), are as follows:

[0088] (1) Criterion 1 for determining an upward fall: In n consecutive frames (n=3 in this embodiment), the torso tilt angle is greater than the dynamic threshold of the upward torso tilt angle;

[0089] (2) Criterion 2 for determining an upward fall: In n consecutive frames, the rate of change of the dynamic torso tilt angle is greater than the dynamic threshold of the rate of change of the upward torso tilt angle.

[0090] (3) Criterion for determining an upward fall: In n consecutive frames, the rate of change of the distance between the left or right shoulder and ankle is greater than the dynamic threshold of the rate of change of the upward shoulder and ankle distance;

[0091] (4) Criterion for determining an upward fall: In n consecutive frames, the bending angle of the left or right knee joint is less than the dynamic threshold of the upward knee joint bending angle;

[0092] (5) Criterion for determining an upward fall: In n consecutive frames, the rate of change of the left or right knee flexion angle is greater than the dynamic threshold of the rate of change of the upward knee flexion angle.

[0093] (6) Criterion for determining an upward fall: In n consecutive frames, the rate of change of the left or right elbow joint flexion angle is greater than the dynamic threshold of the rate of change of the upward elbow joint flexion angle.

[0094] (7) Criterion for determining an upward fall: In n consecutive frames, the angle between the left or right upper arm and torso is greater than the dynamic threshold for the angle between the upper arm and torso in the upward movement;

[0095] (8) Criterion for determining an upward fall: In n consecutive frames, the rate of change of the angle between the left or right upper arm and the torso is greater than the dynamic threshold of the rate of change of the angle between the upper arm and the torso.

[0096] (9) Criterion for determining an upward fall: In n consecutive frames, the left or right upward body bending angle is less than the dynamic threshold of the upward body bending angle;

[0097] (10) Criterion for determining an upward fall: In n consecutive frames, the rate of change of the left or right upward body bending angle is greater than the dynamic threshold of the rate of change of the upward body bending angle.

[0098] The method of detecting fall hazards using posture detection algorithm determines whether an upward-moving pedestrian or humanoid robot has fallen, based on the condition that at least one of the upward-moving fall determination criteria is met. Preferably, in this embodiment, if any three of the upward-moving fall determination criteria 2, 3, 4, 5, 6, 8, and 10 are met, the upward-moving pedestrian or humanoid robot is determined to have fallen.

[0099] The rate of change is the absolute value of the difference between the corresponding values ​​calculated within two frames divided by the time elapsed between the two frames.

[0100] The method of detecting fall hazards through posture detection algorithm, for determining the fall of pedestrians and humanoid robots going downhill, the visible light image acquisition device and the infrared image acquisition device are installed at the lower end of the stepped vertical transportation facility and close to the lowest step, and ensure that the field of view can cover all steps of the stepped vertical transportation facility and the pedestrians and humanoid robots on the steps.

[0101] The method for detecting fall hazards using a posture detection algorithm, and the dynamic threshold settings for determining falls by pedestrians and humanoid robots are as follows (step height and distance are in meters, and the number of steps is the total number of steps from the lowest step to the step where the person or humanoid robot is standing):

[0102] (1) Dynamic threshold of downward trunk tilt angle: set as A2 degrees - a2 × step height × step number, where 20 degrees ≤ A2 ≤ 80 degrees, 0.01 ≤ a2 ≤ 2.5; preferably, in this embodiment A2 = 40 degrees, a2 = 1.0.

[0103] (2) Threshold for dynamic change rate of downward trunk tilt angle: set as: ∇A2 degrees - α2 × step height × number of steps, where ∇A2 is the rate of change of trunk tilt angle, 1 degree ≤ ∇A2 ≤ 20 degrees, 0.01 ≤ α2 ≤ 0.3; preferably, in this embodiment, ∇A2 = 2.5 degrees, α2 = 0.15.

[0104] (3) Dynamic threshold for the rate of change of the descending shoulder-ankle distance: set as ∇B2-β2×step height×step number, where ∇B2 is the rate of change of the descending shoulder-ankle distance, 0.01≤∇B2≤0.5, 0.001≤β2≤0.05; preferably, in this embodiment, ∇B2=0.12, β2=0.005.

[0105] (4) Dynamic threshold of the downward knee flexion angle: set as C2 degrees - c2 × step height × number of steps, where 45 degrees ≤ C2 ≤ 90 degrees, 0.025 ≤ c2 ≤ 5.0; preferably, in this embodiment, C2 = 60 degrees, c2 = 2.5.

[0106] (5) Dynamic threshold for the rate of change of the downward knee flexion angle: set as ∇C2 degrees - γ2 × step height × number of steps, where ∇C2 is the rate of change of the downward knee flexion angle, 0.1 degrees ≤ ∇C2 ≤ 30 degrees, 0.01 ≤ γ2 ≤ 0.25; preferably, in this embodiment, ∇C2 = 3.5 degrees, γ2 = 0.15.

[0107] (6) Dynamic threshold of the rate of change of the downward elbow flexion angle: set as ∇D2 degrees - δ2 × step height × number of steps, where ∇D2 is the rate of change of the downward elbow flexion angle, 0.1 degrees ≤ ∇D2 ≤ 30 degrees, 0.01 ≤ δ2 ≤ 0.5; preferably, in this embodiment, ∇D2 = 6 degrees, δ2 = 0.25.

[0108] (7) Dynamic threshold of the downward upper arm-to-torso angle: set as E2 degrees - e2 × step height × step number, where 1 degree ≤ E2 ≤ 90 degrees, 0.05 ≤ e2 ≤ 2.5; preferably, in this embodiment E2 = 4 degrees, e2 = 0.45.

[0109] (8) Dynamic threshold for the rate of change of the downward upper arm-to-torso angle: set as ∇E2 degrees - ξ2 × step height × number of steps, where ∇E2 is the rate of change of the downward upper arm-to-torso angle, 0.05 degrees ≤ ∇E2 ≤ 30 degrees, 0.01 ≤ ξ2 ≤ 0.5; preferably, in this embodiment, ∇E2 = 5 degrees, ξ2 = 0.25.

[0110] (9) Dynamic threshold of downward body bending angle: set as F2 degrees - f2 × step height × number of steps, where 30 degrees ≤ F2 ≤ 120 degrees, 0.05 ≤ f2 ≤ 2.0; preferably, in this embodiment, F2 = 100 degrees, f2 = 2.5.

[0111] (10) Dynamic threshold for the rate of change of the downward body bending angle: set as ∇F2 degrees - λ2 × step height × number of steps, where ∇F2 is the rate of change of the downward body bending angle, 0.05 degrees ≤ ∇F2 ≤ 30 degrees, 0.01 ≤ λ2 ≤ 0.5; preferably, in this embodiment, ∇F2 = 10 degrees, λ2 = 0.3.

[0112] The criteria for determining falls by using a posture detection algorithm to detect fall hazards, and for determining falls by pedestrians and humanoid robots (hereinafter referred to as the "fall determination criteria"), are as follows:

[0113] (1) Downward fall judgment criterion 1: In n consecutive frames (n=3 in this embodiment), the torso tilt angle is greater than the downward torso tilt angle dynamic threshold;

[0114] (2) Downward fall judgment criterion 2: In n consecutive frames, the rate of change of the dynamic change of the torso tilt angle is greater than the threshold of the dynamic change of the downward torso tilt angle;

[0115] (3) Criterion 3 for determining a fall: In n consecutive frames, the rate of change of the left or right shoulder-ankle distance is greater than the dynamic threshold of the rate of change of the downhill shoulder-ankle distance;

[0116] (4) Criterion 4 for determining a fall: In n consecutive frames, the bending angle of the left or right knee joint is less than the dynamic threshold of the bending angle of the knee joint during the fall;

[0117] (5) Criterion 5 for determining a fall: In n consecutive frames, the rate of change of the left or right knee flexion angle is greater than the dynamic threshold of the rate of change of the knee flexion angle during the fall.

[0118] (6) Downward fall judgment criterion 6: In n consecutive frames, the rate of change of the left or right elbow joint flexion angle is greater than the dynamic threshold of the rate of change of the downward elbow joint flexion angle.

[0119] (7) Downward fall judgment criterion 7: In n consecutive frames, the angle between the left or right upper arm and torso is greater than the dynamic threshold of the downward upper arm and torso angle;

[0120] (8) Downward fall judgment criterion 8: In n consecutive frames, the rate of change of the angle between the left or right upper arm and the torso is greater than the dynamic threshold of the rate of change of the downward upper arm and torso angle;

[0121] (9) Downward fall judgment criterion 9: In n consecutive frames, the left or right downward body bending angle is less than the downward body bending angle dynamic threshold;

[0122] (10) Downward fall judgment criterion 10: In n consecutive frames, the rate of change of the left or right downward body bending angle is greater than the dynamic threshold of the rate of change of the downward body bending angle.

[0123] The method of detecting fall hazards using posture detection algorithm determines whether a pedestrian or humanoid robot has fallen by satisfying at least one of the following fall detection criteria: Preferably, in this embodiment, if any four of the following fall detection criteria 1, 2, 3, 4, 5, 6, 7, 9, and 10 are satisfied, the pedestrian or humanoid robot is determined to have fallen.

[0124] The wearable fall detector 6 is a fall detection device worn or embedded in the body of a pedestrian or robot. It includes, but is not limited to, devices that generate attitude data using sensors such as electronic gyroscopes, accelerometers, electronic tiltmeters, fiber optic gyroscopes, laser gyroscopes, image acquisition devices, and lidar, and detect fall hazards through algorithm calculation. Preferably, in this embodiment, it is an electronic watch worn by a pedestrian. The wearable fall detector 6 can upload the detected fall hazards to the integrated management system through a communication device.

[0125] When the fall protection device 2 receives a signal that a pedestrian or humanoid robot has fallen, the flexible energy-absorbing devices 2-1 on all levels of steps above and below the tread surface 2-5 where the fallen pedestrian or humanoid robot is located will deploy. If there is an unfallen person or humanoid robot above the fallen pedestrian or humanoid robot, the flexible energy-absorbing devices 2-1 on all levels of steps above the tread surface 2-5 where the unfallen person or humanoid robot is located will not deploy. If there is an unfallen person or humanoid robot below the fallen pedestrian or humanoid robot, the energy-absorbing devices 2-1 on all levels of steps below the tread surface 2-5 where the unfallen person or humanoid robot is located will not deploy.

[0126] When the fall protection device 2 receives a signal that a pedestrian or humanoid robot has fallen, the flexible energy-absorbing devices 2-1 will only pop out on each step below the tread surface 2-5 where the fallen pedestrian or humanoid robot is located. If there are pedestrians or humanoid robots below the fallen pedestrian or humanoid robot, the flexible energy-absorbing devices 2-1 below the tread surface 2-5 where the fallen pedestrian or humanoid robot is located will not pop out.

[0127] The fall protection device 2 consists of the flexible energy-absorbing device 2-1, the trigger 2-2, and the control box 2-3, which are installed under the step tread 2-5 and behind the step riser 2-6.

[0128] The flexible energy-absorbing device 2-1 is installed below the step tread and behind the step riser. It can be an inflatable airbag or a flexible buffer pad. Preferably, in this embodiment, an inflatable airbag is used. The trigger 2-2 is a gas generator. The control box 2-3 is equipped with an air circuit breaker, controller, power module, relay, wiring terminals, etc. The controller is a programmable logic controller (PLC) with communication function.

[0129] When a fire signal is detected, the fall protection device 2 will not activate, and the flexible energy-absorbing device 2-1 will not pop out, thus preventing crowding and trampling.

[0130] The trigger 2-2 may be, but is not limited to, a gas generator for an inflatable airbag, an electric actuator for ejecting a flexible buffer pad, or a pneumatic actuator.

[0131] The control box contains, but is not limited to, circuit breakers, controllers, communication modules, power modules, relays, and terminals. When it receives a fall hazard signal from the integrated management system 4 via the communication device 5, it sends a trigger signal to the trigger, causing the flexible energy-absorbing device 2-1 to pop out, thus achieving the protective function.

[0132] The fire signal originates from the fire source and smoke signal identified by the hazard detection device 1, or is received from other management systems via the communication device 5, or from other fire detection devices.

[0133] The voice prompt device 3 is a regular speaker capable of playing audio signals, which come from the integrated management system 4. The audio signals include: fire alarm sounds, fall protection device usage instructions, safe passage instructions, distress signals, orderly evacuation instructions, and reassuring voices. The fall protection device usage instructions are played when a fall occurs and the flexible energy-absorbing device 2-1 pops up, such as, "Please don't panic, we have triggered the protective device to protect you. If you are not injured, please stand up and carefully pass through. If you are injured, please wait patiently for rescue." The safe passage instructions are played when there is fire or smoke, such as, "There is a fire. Please don't panic, do not crowd, bend over and lower your head to pass through." The distress signal is played when a fall occurs, such as, "Hello, is anyone there? Someone has fallen, please help." The reassuring voice is played when a fall occurs and the person remains motionless for an extended period, such as, "A distress signal has been sent, please wait patiently for rescue."

[0134] The integrated management system 4 is implemented by application software running on a computer. The functions of the application software include: recording and saving recent videos, recording the time and location of the incident, recording videos and images when the incident occurs, diagnosing equipment faults, managing the database, monitoring the scene, storing and sending audio signals, communicating with the scene via voice, and exchanging information with other management systems.

[0135] Other management systems include, but are not limited to, intelligent building systems, security monitoring systems, fire control systems, and fire surveillance systems.

[0136] The communication device 5 can achieve communication via wired or wireless means. Wired communication cables include, but are not limited to, twisted-pair cables, network cables, and fiber optic cables. Wireless communication includes, but is not limited to, Wi-Fi, Bluetooth, LoRa, and Li-Fi. If high reliability is required or there is electromagnetic interference, this embodiment preferably uses wired communication. If wiring is difficult or costly, this embodiment preferably uses wireless communication. This embodiment uses wired communication, specifically implemented using a network cable.

[0137] Working principle of the invention:

[0138] After the hazard detection device 1 and the wearable fall detector 6 detect a hazard, they upload the hazard signal to the integrated management system 4 via the communication device 5. The integrated management system 4 analyzes the hazard and transmits control commands to the control box 2-3 via the communication device 5. If a fall is detected, the control box 2-3 sends a trigger signal via the control cable 2-4, causing the trigger 2-2 of the fall protection device 2 to activate. This triggers the flexible energy-absorbing device 2-1 to pop out and cover the tread surface of the step, especially the edges, providing protection for pedestrians or robots and reducing the severity of injury. Simultaneously, a voice signal is sent to the voice prompt device. 3. The voice prompt device 3 plays the corresponding voice prompt; if the hazard detection device 1 and the wearable fall detector 6 simultaneously detect a fall hazard and a fire source or smoke, or if a fall hazard is detected and a fire signal is received from other management systems, the trigger 2-2 of the fall protection device 2 will not activate and the flexible energy absorption device 2-1 will not pop up in order to ensure smooth passage; if no fall hazard is detected, but only a fire or smoke is detected, the hazard signal will be uploaded to the integrated management system 4 through the communication device 5. After analyzing the hazard, the integrated management system 4 will send the corresponding voice prompt signal to the voice prompt device 3, and the voice prompt device 3 will play the corresponding alarm or safety evacuation prompt.

[0139] Example 2, as shown in the attached document Figure 1 and attached Figure 2 As shown, this embodiment provides a hazard monitoring and safety protection system for vertical transportation facilities with stairs, including a hazard detection device 1, a wearable fall detector 6, a fall protection device 2, a voice prompt device 3, a comprehensive management system 4, and a communication device 5. The difference between this embodiment and Embodiment 1 is that the fall hazard detection uses one or two of the visible light image acquisition device 1-1, the infrared image acquisition device 1-2, and the wearable fall detector 6. The fire signal comes from the detection result obtained from one of the visible light image acquisition device 1-1 and the infrared image acquisition device 1-2, or from a fire signal sent by other management systems or devices. The visible light image acquisition device 1-1 is used to determine which step a person or humanoid robot is standing on.

[0140] The visible light image acquisition device 1-1 and the infrared image acquisition device 1-2 adopt intelligent camera devices, which directly realize the detection of fire source and smoke hazard through image processing algorithm, detect the fall hazard through posture detection algorithm, determine which step the person or humanoid robot is standing on, and transmit the detection results to the integrated management system 4 through communication device 5.

[0141] The remaining structure, principles and methods are the same as in Example 1, and will not be repeated here.

[0142] Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A hazard monitoring and safety protection system for stepped vertical transportation facilities, characterized in that: This includes hazard detection devices and wearable fall detection components, fall protection devices, voice prompt devices, integrated management systems, and communication devices. The hazard detection device includes a visible light image acquisition device and an infrared image acquisition device. It is used to acquire visible light images through the visible light image acquisition device and infrared images through the near-infrared band infrared image acquisition device. The infrared images and visible light images are analyzed and processed using image processing algorithms to detect fire sources and smoke, determine which step a person is standing on, and detect the risk of falling through posture detection algorithms. The wearable fall detection component includes a fall detection electronic device worn by or embedded in the body of the person or robot being detected. The fall protection device includes a flexible energy-absorbing device concealed under the tread and behind the kick surface of the step. When a fall signal is received, the flexible energy-absorbing device pops out and covers the tread of the step to form protection, reduce injury and damage, and protect pedestrians or humanoid robots. For an upward-moving pedestrian or humanoid robot, when a signal is received that the pedestrian or humanoid robot has fallen, the flexible energy-absorbing devices on all levels of steps above and below the tread where the fallen pedestrian or humanoid robot is located will deploy. If there is an unfallen person or humanoid robot above the fallen pedestrian or humanoid robot, the flexible energy-absorbing devices on all levels of steps above the tread where the unfallen person or humanoid robot is located will not deploy. If there is an unfallen person or humanoid robot below the fallen pedestrian or humanoid robot, the flexible energy-absorbing devices on all levels of steps below the tread where the unfallen person or humanoid robot is located will not deploy. For pedestrians or humanoid robots going downhill, when a signal is received that a pedestrian or humanoid robot has fallen, the flexible energy-absorbing devices will only pop up on each step below the tread where the fallen pedestrian or humanoid robot is located. If there are pedestrians or humanoid robots below the fallen pedestrian or humanoid robot that have not fallen, the flexible energy-absorbing devices below the tread where the unfallen pedestrian or humanoid robot is located will not pop up. When a fire signal or an emergency evacuation signal is received, the fall protection device will not activate and the flexible energy absorption device will not deploy. The voice prompt device is used to play pre-recorded audio signals or for managers to communicate with on-site personnel. The integrated management system is used to monitor and manage video information, perform system maintenance and fault diagnosis, collect data statistics, interact with other management systems, and communicate with the site via a voice prompt device. The communication device is used for information transmission between the hazard detection device, the fall protection device, the voice prompt device, the integrated management system, the wearable fall detection component, and other management systems. The hazard detection device and the wearable fall detection component transmit hazard detection signals to the integrated management system through the communication device. After comprehensive analysis, the integrated management system sends control signals to the control box of the fall protection device through the communication device. The control box sends a trigger signal through the control line to activate the trigger of the fall protection device, triggering the flexible energy-absorbing device to pop out and cover the tread of the step, forming protection for pedestrians or humanoid robots, and sending the corresponding voice signal to the voice prompt device to play the pre-recorded audio signal.

2. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 1, characterized in that: After the visible light image acquisition device and the infrared image acquisition device capture videos or images, they are transmitted to the integrated management system. The integrated management system uses image processing algorithms to detect fire sources and smoke hazards, determine which step a person or humanoid robot is standing on, and detect fall hazards using posture detection algorithms. Alternatively, the visible light image acquisition device and the infrared image acquisition device can directly detect fire sources and smoke hazards through image processing algorithms, determine which step a person or humanoid robot is standing on, detect fall hazards through posture detection algorithms, and transmit the results to the integrated management system.

3. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 1 or 2, characterized in that: The image processing algorithm is a convolutional neural network algorithm. The input of the convolutional neural network algorithm is a fusion feature map of infrared image and visible light image. The fusion feature map is formed by preprocessing the image to extract the thermal radiation contour of infrared image and preprocessing the visible light image to extract the color and texture features of visible light image. The thermal radiation contour features and color and texture features are fused at multiple scales to form a fusion feature map.

4. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 1 or 2, characterized in that: The method detects fall hazards using a posture detection algorithm. Key human body points output by the YOLOv8-pose model include: left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joint coordinates. It calculates the distance between the left and right shoulder and ankle, the left and right trunk vectors between the left and right hip joints and the left and right shoulder joints, the left and right upper arm vectors between the left and right shoulder joints and the left and right elbow joints, the left and right forearm vectors between the left and right wrist joints and the left and right elbow joints, the left and right thigh vectors between the left and right knee joints and the left and right hip joints, and the left and right lower leg vectors between the left and right knee joints and the left and right ankle joints. Furthermore, it calculates the trunk tilt angle, the angle between the left and right upper arm and trunk, the bending angle of the left and right elbow joints, the bending angle of the left and right torso, and the bending angle of the left and right knee joints. The step on which a person is standing is determined by the step on which the ankle joint is located, as output by the YOLOv8-pose model.

5. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 4, characterized in that: The trunk tilt angle is defined as the angle between the trunk vector and the vertical direction. The left and right shoulder-ankle distances are defined as the distances between the left and right shoulder joints and the left and right ankle joints, respectively. The left and right upper arm trunk angles are defined as the angles between the left and right upper arm vectors and the trunk vector, respectively. The left and right elbow joint flexion angles are defined as the angles between the left and right reverse upper arm vectors and the forearm vector, respectively. The left and right torso flexion angles are defined as the angles between the corresponding reverse torso vectors and the thigh vector, respectively. The left and right knee joint flexion angles are defined as the angles between the left and right reverse thigh vectors and the calf vector, respectively.

6. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 4, characterized in that: The posture detection algorithm detects the risk of falling by determining whether a fall has occurred based on dynamic thresholds set according to the current height. The dynamic thresholds include: dynamic thresholds for trunk tilt angle, dynamic thresholds for the rate of change of trunk tilt angle, dynamic thresholds for shoulder-ankle distance, dynamic thresholds for the rate of change of shoulder-ankle distance, dynamic thresholds for knee flexion angle, dynamic thresholds for the rate of change of knee flexion angle, dynamic thresholds for elbow flexion angle, dynamic thresholds for the rate of change of elbow flexion angle, dynamic thresholds for the upper arm-to-thigh angle, dynamic thresholds for the rate of change of upper arm-to-thigh angle, dynamic thresholds for body flexion angle, and dynamic thresholds for the rate of change of body flexion angle.

7. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 6, characterized in that: The determination of whether a fall has occurred is made when at least one of the following criteria is met: In n consecutive frames, the torso tilt angle exceeds the dynamic threshold of the torso tilt angle; In n consecutive frames, the rate of change of the torso tilt angle exceeds the dynamic threshold of the rate of change of the torso tilt angle; In n consecutive frames, the distance between the left or right shoulder and ankle exceeds the dynamic threshold for shoulder and ankle distance; In n consecutive frames, the rate of change of the left or right shoulder-ankle distance exceeds the dynamic threshold of the rate of change of shoulder-ankle distance; In n consecutive frames, the left or right knee flexion angle exceeds the dynamic threshold of the knee flexion angle. In n consecutive frames, the rate of change of the left or right knee flexion angle exceeds the dynamic threshold of the rate of change of the knee flexion angle. In n consecutive frames, the left or right elbow flexion angle exceeds the dynamic threshold of the elbow flexion angle. In n consecutive frames, the rate of change of the left or right elbow flexion angle exceeds the dynamic threshold of the rate of change of elbow flexion angle. In n consecutive frames, the angle between the left or right upper arm and torso exceeds the dynamic threshold of the upper arm-torso angle. In n consecutive frames, the rate of change of the angle between the left or right upper arm and the torso exceeds the dynamic threshold of the rate of change of the angle between the upper arm and the torso. In n consecutive frames, the left or right torso bending angle exceeds the dynamic threshold of torso bending angle; In n consecutive frames, the rate of change of the left or right torso bending angle exceeds the dynamic threshold of the rate of change of torso bending angle. Where n≥3.

8. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 1, characterized in that: The integrated management system is implemented by application software running on a computer, mobile phone or tablet computer. The application software has the following functions: fall detection, recording and saving recent videos, recording the time and location of the incident, recording videos and images of the incident, diagnosing equipment faults, managing the database, monitoring the site, making voice calls with the site, and exchanging information with other management systems.

9. The hazard monitoring and safety protection system for stepped vertical transportation facilities according to claim 1, characterized in that: Two sets of visible light image acquisition devices and infrared image acquisition devices are installed at the upper and lower ends of the stepped vertical transportation facility, respectively, so that they can capture the front view of a person or humanoid robot whether going up or down.

10. A method for monitoring hazards and protecting the safety of stepped vertical transportation facilities, characterized in that: This method uses the hazard monitoring and safety protection system for stepped vertical transportation facilities as described in any one of claims 1-9, and its steps are as follows: (1) Data acquisition steps: Visible light image acquisition devices and infrared image acquisition devices are deployed at the upper and lower ends of the stepped vertical transportation facilities to acquire visible light images and near-infrared infrared images respectively. At the same time, wearable fall detection components collect motion state data of the detected person or robot in real time. (2) Image analysis and hazard identification steps: Visible light and infrared images are input into image processing algorithms for analysis and processing. Thermal radiation contour features of infrared images and color texture features of visible light images are extracted. After multi-scale feature fusion, a convolutional neural network algorithm is used to detect fire sources and smoke. At the same time, the coordinates of human body key points are output through the YOLOv8-pose model. The ankle joint coordinates determine which step the human body is standing on. Based on the key point coordinates, the torso tilt angle, shoulder-torso distance, upper arm-torso angle, elbow joint flexion angle, torso flexion angle, knee joint flexion angle and their rate of change are calculated. (3) Fall detection steps: Based on the current step height, a dynamic threshold is set. The angle and distance parameters calculated in step (2) are compared with the corresponding dynamic threshold and the dynamic threshold of change rate. When at least one of the judgment criteria is met, it is judged as a fall and a fall signal is generated. At the same time, the wearable fall detection component sends the detected fall signal to the integrated management system through the communication device. (4) Comprehensive decision-making and protection triggering steps: The integrated management system receives signals from the hazard detection device and the wearable fall detection component, performs comprehensive analysis, and determines the type of hazard. If it is determined to be a fall hazard, it executes corresponding protective strategies based on the direction of travel of the fallen pedestrian or humanoid robot. If the fall is an upward fall, the flexible energy-absorbing devices on the steps above and below the person who fell will be triggered to pop out. However, if there is a person who has not fallen above, the flexible energy-absorbing devices on the steps above the person who has not fallen will not pop out. If there is a person who has not fallen below, the flexible energy-absorbing devices on the steps below the person who has not fallen will not pop out. If the fall is downward, the flexible energy-absorbing devices of the steps below the fallen person will be triggered to pop out. However, if there is a person below who has not fallen, the flexible energy-absorbing devices of the steps below the person who has not fallen will not pop out. If it is a fire or emergency evacuation signal, the fall protection device will not be triggered; The integrated management system sends control signals to the control box of the fall protection device through the communication device. The control box sends a trigger signal through the control line to activate the trigger, causing the flexible energy-absorbing device to pop out and cover the step surface. At the same time, it sends the corresponding voice signal to the voice prompt device to play the pre-recorded audio signal. (5) Information management and interaction steps: The integrated management system records the time and location of the emergency, along with the corresponding video and images, saves recent video data, performs fault diagnosis on the equipment, and interacts with other management systems. Managers can use the integrated management system to conduct voice calls with the site, enabling remote monitoring and command.

Images