Multi-feature decision fusion escalator fall detection method

Abstract

The invention discloses a multi-feature decision fusion escalator fall detection method. Belongs to the technical field of computer vision and behavior recognition. The technical problem that high-precision and high-robustness fall detection is difficult to realize in a real and diversified environment in the prior art is solved. The method comprises the following steps: acquiring an escalator monitoring video stream, and splitting the escalator monitoring video stream into an image sequence; detecting the head area of a person on the escalator, calculating the head motion acceleration, and setting an acceleration threshold to judge whether the human body falls down; detecting a human body bounding box, calculating the length-width ratio change of the bounding box, and setting a length-width ratio change threshold to judge whether the human body falls down or not; a human body key point tracker is detected to be associated with human body targets in different frames, a key point sequence is analyzed through a space-time diagram convolutional network, and whether the human body falls down or not is judged; the three detection routes are processed in parallel, and three output results are fused to obtain a more accurate detection result; and setting a joint loss function, and updating the parameters of the XGBoost network model.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and behavior recognition technology, specifically relating to a multi-feature decision fusion method for escalator fall detection. Background Technology

[0002] Escalators, as efficient transportation equipment, pose a risk of secondary injuries due to passenger imbalance during operation, necessitating intelligent safety protection measures. Existing protection methods—manual monitoring suffers from response delays and high costs, while electromagnetic wave sensing only provides basic voice prompts—are ineffective in responding to emergencies. With the development of deep learning and object detection technologies, vision-based intelligent monitoring solutions can achieve real-time detection and millisecond-level response to escalator safety hazards. Improved object detection algorithms, deployed through embedded platforms, can efficiently analyze video streams, offering high efficiency, low cost, and adaptability to multiple scenarios. Compared to traditional methods, object detection-based intelligent early warning systems significantly reduce the accident rate, becoming the preferred solution for addressing escalator safety issues.

[0003] Traditional methods for detecting human falls involve analyzing changes in the geometric features of the human body's outline, rotational momentum, motion vectors, and motion curves to detect falls.

[0004] Traditional machine learning requires manual feature design and extraction. For complex data like images and videos, designing comprehensive and effective features is extremely difficult. In fall detection, for example, changes in human posture and movement involve multi-dimensional information such as joint angles, contours, and speed, making manual extraction prone to missing key features. Deep learning models (such as Convolutional Neural Networks (CNNs)) can automatically learn features from large amounts of image and video data. Through layers of convolutional and pooling layers, they abstract features from low-level edges and textures to high-level semantics from raw pixel data. Therefore, most researchers currently use deep learning-based methods for fall detection. Existing technologies use AlphaPose's joint coordinate information as input and a spatiotemporal graph convolutional network to classify actions and determine if a fall has occurred. Accuracy on the UR and Le2i datasets reaches 97.28% and 96.86% respectively, but accuracy drops in crowded scenes.

[0005] In summary, traditional methods lack sufficient feature representation capabilities, while deep learning methods have limited adaptability to different scenarios. Specifically, these limitations manifest as: weak ability to distinguish similar behaviors, poor resistance to occlusion and environmental interference, reliance on prior knowledge or specific scenario data, and performance degradation in complex dynamic scenarios (such as crowding and occlusion). These issues make it difficult for existing methods to achieve high-precision and robust fall detection in real-world, diverse environments. Summary of the Invention

[0006] To address the technical problem that existing technologies struggle to achieve high-precision and robust fall detection in diverse real-world environments, this application provides a multi-feature decision fusion method for escalator fall detection, comprising the following steps: S1. Acquire the escalator monitoring video stream via camera and split the video stream into image sequences. ; S2, First detection route: Detect the head area of ​​people on the escalator, calculate the head movement acceleration, and set an acceleration threshold to determine whether a person has fallen; S3, Second detection route: Detect human body bounding box, calculate the aspect ratio change of the bounding box, and set an aspect ratio change threshold to determine whether the human body has fallen; S4, Third Detection Route: Detect key points of the human body, associate human targets in different frames through the ByteTrack tracker, and analyze the key point sequence using a spatiotemporal graph convolutional network to determine whether the human body has fallen; S5. Three detection routes are processed in parallel. An XGBoost fusion strategy is designed to fuse the three output results to obtain more accurate fall detection results. S6. Set the joint loss function and update the parameters of the XGBoost network model using the training sample set.

[0007] Furthermore, in step S1, after acquiring the escalator monitoring video stream, smoothing and video resolution normalization are performed, and the video stream color space is converted to YUV format to obtain a preprocessed image sequence. .

[0008] Furthermore, step S2 specifically includes: Step S21: Use the Ultralytics model to detect the head region of the person, and output the coordinates of the head bounding box. ,in, , The coordinates of the head bounding box center are: The width and height of the header bounding box. The confidence level after Sigmoid activation. The target score represents the head bounding box, and head tracking yields the head position sequence. ; Step S22: Calculate the displacement of the head center point between consecutive frames. : ,in, For the first Frame header bounding box center coordinates for Frame header bounding box center coordinates; Step S23: According to the frame interval Calculate the first Instantaneous speed during frame output : ; Step S24: Calculate the velocity change , obtained the Instantaneous acceleration during frame output : ; Step S25: Set acceleration threshold ,when When identified as a potential fall event, continuous If the instantaneous acceleration exceeds the acceleration threshold during frame output, it is determined as a fall.

[0009] Furthermore, step S3 specifically includes: Step S31: Use the CenterNet model to detect the complete human body bounding box and output the coordinates of the human body bounding box. ,in , The coordinates of the center of the human body bounding box. The width and height of the human body bounding box. The confidence level after Sigmoid activation; Step S32, calculate the first... Aspect Ratio of Human Body Bounding Box during Frame Output : ,in, Indicates the first Frame human body bounding box width, Indicates the first The height of the human body bounding box in frame; human body tracking yields the aspect ratio sequence of the human body bounding box. ; Step S33: According to the frame interval Calculate and track the first Frame output aspect ratio change rate : ,in, Indicates the first Aspect ratio of the human body bounding box during frame output; Step S34: Set the aspect ratio change threshold ,when When identified as a potential fall event, continuous If the aspect ratio change rate exceeds the aspect ratio change threshold during frame output, it is determined to be a fall.

[0010] Furthermore, step S4 specifically includes: Step S41, Select Key points of the human body were extracted from image sequences using a 4-level high-resolution HRNet network. The spatial characteristics are obtained from the first Level feature map : Generating heatmaps of key points in the human body using deconvolution layers. : ,in, This represents the 4th level feature map. Represents the feature mapping function. Represents the learnable weight matrix. Indicates the bias term. This represents the Sigmoid activation function; non-maximum suppression is used to determine the coordinates of key points on the human body. : ,in, Indicates the first Heatmap of key points This indicates a non-maximum suppression operation, resulting in a set of keypoints. : , Represents the pixel coordinates of key points. Indicates the confidence level of key points; Step S42: Associate key points in different frames using the ByteTrack tracker, maintain ID consistency, and obtain complete temporal key point data. ,in, This indicates the total number of tracked targets in the current frame. Indicates for the first The tracking target, from the first Frame to the A set of key points in the historical data of a frame; Step S43: Construct a spatiotemporal graph convolutional network model, with complete temporal key point data as input. The output is the first... The probability of a target falling is tracked, and a threshold is set to determine whether a fall has occurred.

[0011] Furthermore, step S5 specifically includes: Head acceleration characteristics Length-to-width ratio variation characteristics and attitude analysis features The features are concatenated to obtain a feature vector. An XGBoost fusion model is then constructed, with the concatenated feature vector as the input and the fusion-based fall probability as the output. A threshold is set to determine whether a fall has occurred.

[0012] Furthermore, head acceleration characteristics pass get; in, This indicates the maximum instantaneous acceleration value within the monitoring window. ; This represents the average acceleration within the monitoring window. ; The variance of acceleration is represented by... .

[0013] Furthermore, the characteristics of aspect ratio variation pass get, in, This indicates the maximum instantaneous aspect ratio within the monitoring window. ; This represents the average aspect ratio within the monitoring window. ; The variance representing the aspect ratio, .

[0014] Furthermore, attitude analysis features pass Obtain, among which, , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates all key points are at The average instantaneous velocity along the axial direction, Indicates all key points are at Standard deviation of instantaneous velocity in the axial direction; , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates all key points are at The average instantaneous velocity along the axial direction, Indicates all key points are at Standard deviation of instantaneous velocity in the axial direction; , Indicates the first The trend of confidence changes at key points This represents the mean trend of confidence levels across all key points. This represents the standard deviation of the confidence level change trend for all key points.

[0015] Furthermore, the joint loss function is ,in Indicates the weighting coefficient; The loss function represents the acceleration branch. ,in, This represents the binary determination result of step S2. ( = Did not fall, =Fall down), , Represents the Sigmoid function; The loss function represents the aspect ratio branch. ,in, This represents the binary determination result of step S3. ( = Did not fall, =Fall down), ; The loss function represents the pose branch. ,in, This represents the binary determination result of step S4. ( = Did not fall, =Fall down), This represents the fall probability output by the spatiotemporal graph convolutional network. This represents the XGBoost fusion probability loss function. , This represents the binary determination result of step S5. ( = Did not fall, =Fall down), This represents the fall probability output by the XGBoost fusion model.

[0016] The beneficial effects of the method described in this invention are as follows: (1) The multi-feature decision fusion architecture of the present invention enables each detection route to form complementary advantages at the feature representation level through the collaborative analysis of head motion dynamics features, human body geometric features and spatiotemporal key point features: when a certain model fails to detect due to occlusion or viewpoint change, other models can still maintain effective judgment, thereby achieving a better balance between recall and precision.

[0017] (2) To improve the reliability of decision results and further enhance the accuracy of fall detection by using the XGBoost network; This invention does not simply set fusion weight coefficients and then simply fuse the prediction results of the three prediction methods through the XGBoost network. Instead, this invention fuses the feature vectors obtained from the three prediction methods respectively, merges the three feature vectors into a new feature vector, and then outputs the fall probability after fusion through the XGBoost network. This feature vector fusion method can more deeply explore the complementary information between different prediction methods, avoid the information loss of simple weighted fusion, thereby enhancing the model's discrimination ability and improving the accuracy of the judgment. Attached Figure Description

[0018] Figure 1 This is a flowchart of the escalator fall detection method based on multi-feature decision fusion in an embodiment of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0020] Example 1 This embodiment provides a multi-feature decision fusion method for escalator fall detection, such as... Figure 1 As shown, it includes the following steps: S1. Acquire the escalator monitoring video stream via camera and split the video stream into image sequences. ; S2, First detection route: Detect the head area of ​​people on the escalator, calculate the head movement acceleration, and set an acceleration threshold to determine whether a person has fallen; S3, Second detection route: Detect human body bounding box, calculate the aspect ratio change of the bounding box, and set an aspect ratio change threshold to determine whether the human body has fallen; S4, Third Detection Route: Detect key points of the human body, associate human targets in different frames through the ByteTrack tracker, and analyze the key point sequence using a spatiotemporal graph convolutional network to determine whether the human body has fallen; S5. Three detection routes are processed in parallel. An XGBoost fusion strategy is designed to fuse the three output results to obtain more accurate fall detection results. S6. Set the joint loss function and update the parameters of the XGBoost network model using the training sample set.

[0021] Example 2 This embodiment further defines embodiment 1 and provides further explanation of step S1.

[0022] Step S1 is as follows: Assume that the video stream is acquired through an escalator monitoring camera, with a resolution of 1920×1080, a frame rate of 25fps, and RGB format.

[0023] The video is smoothed using Gaussian filtering, its resolution is normalized, and the Y channel in the YUV color space is subjected to CLAHE processing to significantly enhance image contrast while suppressing noise amplification. Then, the CLAHE-enhanced YUV image data is bilinearly interpolated and scaled to a uniform size, and the color space is converted to YUV format, resulting in a preprocessed image sequence. .

[0024] Example 3 This embodiment further defines embodiment 1 and provides further explanation of step S2.

[0025] Step S2 is as follows: Step S21: Use the Ultralytics model to detect the head region of the person, and output the coordinates of the head bounding box. ,in, , The coordinates of the head bounding box center are: The width and height of the header bounding box. The confidence level after Sigmoid activation. The target score of the head bounding box represents the probability estimate of the presence of a target object within the bounding box. Head tracking yields the head position sequence. ; Step S22: Calculate the displacement of the head center point between consecutive frames. : ,in, For the first Frame header bounding box center coordinates for Frame header bounding box center coordinates; Step S23: According to the frame interval Calculate the first Instantaneous speed during frame output : ; Step S24: Calculate the velocity change , obtained the Instantaneous acceleration during frame output : ; Step S25: Set acceleration threshold ,when When identified as a potential fall event, continuous If the instantaneous acceleration exceeds the acceleration threshold during frame output, it is determined as a fall. It can take the value 5.

[0026] Example 4 This embodiment further defines embodiment 1 and provides further explanation of step S3.

[0027] Step S3 is as follows: Step S31: Use the CenterNet model to detect the complete human body bounding box and output the coordinates of the human body bounding box. ,in , The coordinates of the center of the human body bounding box. The width and height of the human body bounding box. The confidence level after Sigmoid activation; Step S32, calculate the first... Aspect Ratio of Human Body Bounding Box during Frame Output : ,in, Indicates the first Frame human body bounding box width, Indicates the first The height of the human body bounding box in frame; human body tracking yields the aspect ratio sequence of the human body bounding box. ; Step S33: According to the frame interval , here A value of 0.04 can be used to calculate the tracking number. Frame output aspect ratio change rate : ,in, Indicates the first Aspect ratio of the human body bounding box during frame output; Step S34: Set the aspect ratio change threshold ,when When identified as a potential fall event, continuous If the aspect ratio change rate exceeds the aspect ratio change threshold during frame output, it is determined to be a fall. It can take the value 5.

[0028] Example 5 This embodiment further defines embodiment 1 and provides further explanation of step S4.

[0029] Step S4 is as follows: Step S41, Select Key points of the human body were extracted from the image sequence using a 4-level high-resolution HRNet network. Spatial features, preprocessed image sequences After processing through convolutional layers, Batch Normalization (BN) layers, and ReLU layers, the th... Level feature map : The resolutions of the output feature maps at each level are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively.

[0030] In this embodiment, 17 keypoints were selected for human keypoint detection based on standards set manually according to human anatomy and motion analysis requirements. These 17 keypoints originate from the human pose annotation standards of the COCO dataset. Specifically, the 17 points include: nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles. Human annotators annotated human joints in images according to the COCO specifications, forming a training set. Each keypoint contains coordinates and a visibility label. During training, the keypoint coordinates are converted into a Gaussian distributed heatmap as a supervision signal. HRNet learns localization by minimizing the error between the predicted heatmap and the real heatmap. During inference, it extracts the positions of extreme points from the heatmap and combines them with offset correction to obtain the final keypoint coordinates.

[0031] Generating heatmaps of key human body points using deconvolution layers. : ,in, This represents the 4th level feature map. Represents the feature mapping function. Represents the learnable weight matrix. Indicates the bias term. This represents the Sigmoid activation function; non-maximum suppression is used to determine the coordinates of key points on the human body. : ,in, Indicates the first Heatmap of key points This indicates a non-maximum suppression operation, resulting in a set of keypoints. : , Represents the pixel coordinates of key points. Indicates the confidence level of key points; Step S42: Associate key points in different frames using the ByteTrack tracker, maintain ID consistency, and obtain complete temporal key point data. ,in, This indicates the total number of tracked targets in the current frame. Indicates for the first The tracking target, from the first Frame to the A set of key points in the historical data of a frame; Step S43: Construct a Spatiotemporal Graph Convolutional Network (STGCN) model, with complete temporal keypoint data as input. The output is the first... The probability of a target falling is tracked, and a threshold is set to determine whether a fall has occurred.

[0032] The specific workflow of the spatiotemporal graph convolutional network model is as follows: the input is complete temporal key point data. ; Constructing the adjacency matrix of the spatial graph : , Indicates the first , The Euclidean distance between the coordinates of the key points. This represents the width of the Gaussian kernel.

[0033] Constructing the adjacency matrix of the time graph : .

[0034] The adjacency matrix of the spatial graph is processed by spatial domain convolution to obtain : Degree matrix Calculation and normalization preparation: for subsequent symmetric normalization

[0035] , Output spatial feature matrix : , Represents the key point feature matrix. Represents spatial convolution weights, This indicates the activation of the modified linear unit.

[0036] After time-domain convolution, the spatiotemporal feature matrix is ​​obtained. : , This represents a one-dimensional temporal convolution.

[0037] Fully connected layer prediction: , The probability of falling is , Represents the normalized exponential function, Represents the classifier weights. This indicates the classifier bias.

[0038] STGCN outputs the probability of falling. Set threshold =0.7, when When identified as a potential fall event, continuous A frame exceeding a threshold is considered a fall. It can take the value 5.

[0039] After completing the spatiotemporal graph convolution of STGCN, the system performs difference analysis on keypoints with the same ID in two consecutive frames, and then extracts motion features from the keypoint sequence: , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates the first Key points Always Velocity in the axial direction, Indicates the first Key points Always Velocity in the axial direction; , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates the first Key points Always Velocity in the axial direction, Indicates the first Key points Always Velocity in the axial direction; , Indicates the first The trend of confidence changes at key points Indicates the first Key points The confidence level of the detection at any given moment. Indicates the first Key points The confidence level of the detection at any given time.

[0040] Example 5 This embodiment further defines embodiment 1 and provides further explanation of step S5.

[0041] Step S5 is as follows: Head acceleration characteristics Length-to-width ratio variation characteristics and attitude analysis features The features are concatenated to obtain a feature vector. An XGBoost fusion model is then constructed, with the concatenated feature vector as the input and the fusion-based fall probability as the output. A threshold is set to determine whether a fall has occurred.

[0042] Head acceleration characteristics pass get; in, This indicates the maximum instantaneous acceleration value within the monitoring window. ; This represents the average acceleration within the monitoring window. ; The variance of acceleration is represented by... .

[0043] Aspect Ratio Variation Characteristics pass get, in, This indicates the maximum instantaneous aspect ratio within the monitoring window. ; This represents the average aspect ratio within the monitoring window. ; The variance representing the aspect ratio, , Indicates the first The aspect ratio change rate during frame output.

[0044] Attitude analysis features pass Obtain, among which, , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates all key points are at The average instantaneous velocity along the axial direction, Indicates all key points are at Standard deviation of instantaneous velocity in the axial direction; , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates all key points are at The average instantaneous velocity along the axial direction, Indicates all key points are at Standard deviation of instantaneous velocity in the axial direction; , Indicates the first The trend of confidence changes at key points This represents the mean trend of confidence levels across all key points. This represents the standard deviation of the confidence level change trend for all key points.

[0045] Here, take , Head acceleration characteristics ; Aspect Ratio Variation Characteristics ; Attitude analysis features ; The features are concatenated to obtain: .

[0046] Construct an XGBoost fusion model, taking the concatenated feature vector as input, and perform single-tree prediction. Indicates the first Tree samples The predicted value; , Indicates the learning rate. Indicates the first Prediction function for trees.

[0047] Leaf node weight calculation Represents leaf nodes j The optimal weight, Belongs to leaf node j The sample set.

[0048] , Indicates sample The first gradient, Indicates sample The second gradient, This represents the L2 regularization coefficient.

[0049] Split gain Calculate the following for candidate split points: , L / R represents the left and right subsets after splitting. This represents the complexity control parameter.

[0050] (S5.3) XGBoost outputs the final fall probability: , M represents the total number of trees.

[0051] Set fusion threshold ,when It was determined to be a fall event.

[0052] Example 5 This embodiment further defines embodiment 1 and provides further explanation of step S6.

[0053] Step S6 is as follows: Collect video clips from cameras monitoring escalators to construct a training dataset containing normal behavior and fall behavior; manually annotate the dataset, including head bounding boxes, body bounding box aspect ratios, coordinates of 17 key points, and fall labels (0 / 1); preprocess the features through modules S2-S4.

[0054] The joint loss function is ,in Indicates the weighting coefficient; The loss function represents the acceleration branch. ,in, This represents the binary determination result of step S2. ( = Did not fall, =Fall down), , Represents the Sigmoid function; The loss function represents the aspect ratio branch. ,in, This represents the binary determination result of step S3. ( = Did not fall, =Fall down), ; The loss function represents the pose branch. ,in, This represents the binary determination result of step S4. ( = Did not fall, =Fall down), This represents the fall probability output by the spatiotemporal graph convolutional network. This represents the XGBoost fusion probability loss function. , This represents the binary determination result of step S5. ( = Did not fall, =Fall down), This represents the fall probability output by the XGBoost fusion model.

[0055] The outputs of the three routes are computed in parallel. The loss term is calculated for each route, and the total loss is obtained by weighted summation. The model parameters are then updated using gradient descent.

Claims

1. A multi-feature decision fusion method for escalator fall detection, characterized in that, The method includes the following steps: S1. Acquire the escalator monitoring video stream via camera and split the video stream into image sequences. ; S2, First detection route: Detect the head area of ​​people on the escalator, calculate the head movement acceleration, and set an acceleration threshold to determine whether a person has fallen; S3, Second detection route: Detect human body bounding box, calculate the aspect ratio change of the bounding box, and set an aspect ratio change threshold to determine whether the human body has fallen; S4, Third Detection Route: Detect key points of the human body, associate human targets in different frames through the ByteTrack tracker, and analyze the key point sequence using a spatiotemporal graph convolutional network to determine whether the human body has fallen; S5. Three detection routes are processed in parallel. An XGBoost fusion strategy is designed to fuse the three output results to obtain more accurate fall detection results. S6. Set the joint loss function and update the parameters of the XGBoost network model using the training sample set.

2. The escalator fall detection method based on multi-feature decision fusion according to claim 1, characterized in that, In step S1, after acquiring the escalator monitoring video stream, smoothing and video resolution normalization are performed, and the video stream color space is converted to YUV format to obtain a preprocessed image sequence. .

3. The escalator fall detection method based on multi-feature decision fusion according to claim 2, characterized in that, Step S2 is as follows: Step S21: Use the Ultralytics model to detect the head region of the person, and output the coordinates of the head bounding box. ,in, , The coordinates of the head bounding box center are: The width and height of the header bounding box. The confidence level after Sigmoid activation. The target score represents the head bounding box, and head tracking yields the head position sequence. ; Step S22: Calculate the displacement of the head center point between consecutive frames. : ,in, For the first Frame header bounding box center coordinates for Frame header bounding box center coordinates; Step S23: According to the frame interval Calculate the first Instantaneous speed during frame output : ； Step S24: Calculate the velocity change , obtained the Instantaneous acceleration during frame output : ； Step S25: Set acceleration threshold ,when When identified as a potential fall event, continuous If the instantaneous acceleration exceeds the acceleration threshold during frame output, it is determined as a fall.

4. The escalator fall detection method based on multi-feature decision fusion according to claim 3, characterized in that, Step S3 is as follows: Step S31: Use the CenterNet model to detect the complete human body bounding box and output the coordinates of the human body bounding box. ,in , The coordinates of the center of the human body bounding box The width and height of the human body bounding box. The confidence level after Sigmoid activation; Step S32, calculate the first... Aspect Ratio of Human Body Bounding Box during Frame Output : ,in, Indicates the first Frame human body bounding box width, Indicates the first The height of the human body bounding box in frame; human body tracking yields the aspect ratio sequence of the human body bounding box. ; Step S33: According to the frame interval Calculate and track the first Frame output aspect ratio change rate : ,in, Indicates the first Aspect ratio of the human body bounding box during frame output; Step S34: Set the aspect ratio change threshold ,when When identified as a potential fall event, continuous If the aspect ratio change rate exceeds the aspect ratio change threshold during frame output, it is determined to be a fall.

5. The escalator fall detection method based on multi-feature decision fusion according to claim 4, characterized in that, Step S4 is as follows: Step S41, Select Key points of the human body were extracted from image sequences using a 4-level high-resolution HRNet network. The spatial characteristics are obtained from the first Level feature map : ; Generating heatmaps of key human body points using deconvolution layers. : ,in, This represents the 4th level feature map. Represents the feature mapping function. Represents the learnable weight matrix. Indicates the bias term. This represents the Sigmoid activation function; non-maximum suppression is used to determine the coordinates of key points on the human body. : ,in, Indicates the first Heatmap of key points This indicates a non-maximum suppression operation, resulting in a set of keypoints. : , Represents the pixel coordinates of key points. Indicates the confidence level of key points; Step S42: Associate key points in different frames using the ByteTrack tracker, maintain ID consistency, and obtain complete temporal key point data. ,in, This indicates the total number of tracked targets in the current frame. Indicates for the first The tracking target, from the first Frame to the A set of key points in the historical data of a frame; Step S43: Construct a spatiotemporal graph convolutional network model, with complete temporal key point data as input. The output is the first... The probability of a target falling is tracked, and a threshold is set to determine whether a fall has occurred.

6. The escalator fall detection method based on multi-feature decision fusion according to claim 5, characterized in that, Step S5 is as follows: Head acceleration characteristics Length-to-width ratio variation characteristics and attitude analysis features The features are concatenated to obtain a feature vector. An XGBoost fusion model is then constructed, with the concatenated feature vector as the input and the fusion-based fall probability as the output. A threshold is set to determine whether a fall has occurred.

7. The escalator fall detection method based on multi-feature decision fusion according to claim 6, characterized in that, Head acceleration characteristics pass get; in, This indicates the maximum instantaneous acceleration value within the monitoring window. ; This represents the average acceleration within the monitoring window. ; The variance of acceleration is represented by... .

8. The escalator fall detection method based on multi-feature decision fusion according to claim 7, characterized in that, Aspect Ratio Variation Characteristics pass get, in, This indicates the maximum instantaneous aspect ratio within the monitoring window. ; This represents the average aspect ratio within the monitoring window. ; The variance representing the aspect ratio, .

9. The escalator fall detection method based on multi-feature decision fusion according to claim 8, characterized in that, Attitude analysis features pass Obtain, among which, , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates all key points are at The average instantaneous velocity along the axial direction, Indicates all key points are at Standard deviation of instantaneous velocity in the axial direction; , Indicates the first The key point is Instantaneous velocity in the axial direction, Indicates all key points are at The average instantaneous velocity along the axial direction, Indicates all key points are at Standard deviation of instantaneous velocity in the axial direction; , Indicates the first The trend of confidence changes at key points This represents the average trend of confidence levels across all key points. It represents the standard deviation of the confidence level change trend for all key points.

10. The escalator fall detection method based on multi-feature decision fusion according to claim 9, characterized in that, The joint loss function is ,in Indicates the weighting coefficient; The loss function represents the acceleration branch. ,in, This represents the binary determination result of step S2. ( = Did not fall, =Fall down), , Represents the Sigmoid function; The loss function represents the aspect ratio branch. ,in, This represents the binary determination result of step S3. ( = Did not fall, =Fall down), ; The loss function represents the pose branch. ,in, This represents the binary determination result of step S4. ( = Did not fall, =Fall down), This represents the fall probability output by the spatiotemporal graph convolutional network. This represents the XGBoost fusion probability loss function. , This represents the binary determination result of step S5. ( = Did not fall, =Fall down), This represents the fall probability output by the XGBoost fusion model.

Images