Escalator passenger falling behavior real-time detection and active warning method and device based on human posture estimation

Abstract

The embodiment of the application provides a kind of based on human posture estimation's escalator passenger fall behavior real-time detection and active early warning method and device, constructs each passenger image area by background difference and multi-scale human detection, establishes passenger motion trajectory sequence in combination with skeleton key point posture feature extraction and time sequence difference tracking, and through multistage confidence comparison and time length constraint drive graded warning signal linkage escalator deceleration protection, effectively solve the deficiency of traditional technology in motion target extraction, posture behavior modeling and graded warning linkage etc., for the intelligent real-time detection and active safety early warning of escalator passenger fall behavior provides technical support.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a method and device for real-time detection and proactive early warning of passenger fall behavior on escalators based on human posture estimation. Background Technology

[0002] Existing methods for detecting passenger falls on escalators have significant shortcomings. Traditional systems perform poorly in video acquisition and moving target extraction, typically relying on manual monitoring or simple frame difference detection for abnormal behavior assessment. They lack the ability to perform background subtraction and foreground separation on the original image sequence and combine it with a multi-scale human detection model to accurately locate the bounding boxes of each passenger. This results in insufficient accuracy in extracting moving target regions and reliability in locating individual passengers in the complex dynamic background of escalators, making it difficult to support subsequent refined posture analysis.

[0003] Furthermore, existing technologies face bottlenecks in passenger posture feature extraction and fall behavior modeling. Most systems lack the ability to simultaneously calculate torso tilt angle, centroid height change rate, and limb folding degree using a set of skeletal keypoint coordinates to construct posture feature vectors. They also fail to combine historical frame posture features for temporal difference extraction of posture abrupt change features and struggle to leverage target tracking models to construct continuous motion trajectory sequences through cross-frame identity association. This results in insufficient temporal dynamic feature modeling capabilities for fall behavior, impacting detection accuracy and recall.

[0004] Existing systems have technical shortcomings in fall confidence assessment and tiered early warning linkage. They lack a complete closed-loop early warning mechanism that performs multi-level comparisons of posture change characteristics with preset fall judgment thresholds to obtain confidence scores, combines this with constraints on the duration of lying on the ground to generate tiered early warning signals, and simultaneously pushes these signals to monitoring clients and mobile terminals, while also linking with the escalator control system to implement deceleration protection. This affects the ability to respond quickly and proactively intervene in fall incidents. Solving these problems is of great significance for improving the level of intelligent monitoring of passenger safety on escalators in public places. Summary of the Invention

[0005] To address the problems in existing technologies, this application provides a method and device for real-time detection and proactive early warning of passenger fall behavior in escalators based on human posture estimation. This method effectively solves the shortcomings of traditional technologies in terms of moving target extraction, posture behavior modeling, and hierarchical early warning linkage, providing technical support for intelligent real-time detection and proactive safety early warning of passenger fall behavior in escalators.

[0006] To solve at least one of the above problems, this application provides the following technical solution:

[0007] In a first aspect, this application provides a method for real-time detection and proactive early warning of passenger fall behavior on escalators based on human posture estimation, including:

[0008] The original image sequence is obtained by collecting video streams of passenger activities through camera equipment deployed above the escalator area. Background subtraction and foreground separation are performed on the original image sequence to obtain moving target area images. The moving target area images are input into a multi-scale human detection model to perform target localization to obtain a set of bounding boxes for each passenger. Based on the set of bounding boxes for each passenger, the moving target area images are cropped to obtain the image regions for each passenger.

[0009] The image regions of each passenger are input into the human skeleton detection model to extract the set of skeleton key point coordinates. Based on the set of skeleton key point coordinates, the torso tilt angle, the rate of change of the center of mass height, and the degree of limb folding are calculated to obtain the posture feature vector. The posture feature vector is temporally differiated with the posture features of historical frames to obtain the posture change feature. The posture change feature is input into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger.

[0010] The posture change features in each passenger movement trajectory sequence are compared with the preset fall judgment threshold to obtain a fall confidence score. When the fall confidence score exceeds the preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. The graded warning signal is pushed to the monitoring client and mobile terminal and the escalator control system is linked to perform deceleration protection.

[0011] Furthermore, it also includes: the camera device collects video of the escalator area at a preset acquisition frequency to obtain a video stream of passenger activity; the camera device extracts key frames from the video stream of passenger activity to obtain an original image sequence; the camera device sorts the original image sequence according to the acquisition timestamp and writes it into a frame buffer queue; and the camera device performs adaptive brightness adjustment and color equalization on the images in the frame buffer queue to obtain a preprocessed image sequence.

[0012] The preprocessed image sequence is read from the frame buffer queue and a pixel-by-pixel difference operation is performed with the pre-established background model to obtain a difference image. Binarization segmentation and morphological filtering are performed on the difference image to obtain a foreground mask image. The foreground mask image is then superimposed with the corresponding frame in the preprocessed image sequence to obtain a moving target region image.

[0013] Furthermore, it also includes: inputting the image of the moving target region into the feature pyramid network of the multi-scale human detection model to perform multi-level convolution operations to obtain a multi-scale feature map set; performing candidate box generation and non-maximum suppression filtering based on the multi-scale feature map set to obtain a set of bounding boxes for each passenger; and associating and storing the coordinates of each bounding box in the set of bounding boxes with the confidence score to obtain a list of bounding box attributes.

[0014] Read the bounding box coordinates that meet the preset filtering threshold conditions from the bounding box attribute list, perform region cropping on the moving target region image based on the bounding box coordinates to obtain the initial image region of each passenger, and perform size normalization processing on the initial image region of each passenger to obtain the image region of each passenger.

[0015] Furthermore, it also includes: inputting each passenger image region into a human skeleton detection model constructed based on an hourglass neural network to perform multi-scale feature fusion and key point heatmap regression to obtain a set of joint heatmap responses; performing peak localization extraction on the set of joint heatmap responses to extract coordinate points covering the main joints of the human body to obtain a set of skeleton key point coordinates; and connecting the set of skeleton key point coordinates according to the human body topology to obtain a skeleton topology map.

[0016] The torso tilt angle is obtained by calculating the angle between the line connecting the shoulder key point and the hip key point in the skeleton topology map and the vertical direction. The weighted centroid position of each key point is calculated based on the coordinate set of the skeleton key points and the centroid position of the previous frame is subtracted to obtain the centroid height change rate. The limb contraction ratio is calculated based on the relative distance between the elbow key point and the knee key point in the skeleton topology map to obtain the limb folding degree. The torso tilt angle, the centroid height change rate and the limb folding degree are combined and encapsulated to obtain the posture feature vector.

[0017] Furthermore, it also includes: reading the historical frame pose features corresponding to the current frame from the pose feature cache queue, performing a difference operation on the pose feature vector and the historical frame pose features item by item according to the feature dimension to obtain a pose difference vector, performing absolute value calculation and weighted summation on the differences in each dimension of the pose difference vector to obtain a pose change feature, and writing the pose feature vector into the pose feature cache queue as the historical frame pose features of subsequent frames;

[0018] The attitude change features and the set of bounding boxes of each passenger in the current frame are input into the target tracking model to perform appearance feature encoding to obtain the identity feature vector of each passenger. Based on the identity feature vector of each passenger, similarity matching is performed with the trajectory identity features stored in the historical trajectory database to obtain the inter-frame association pairing result. The inter-frame association pairing result is used to stitch the trajectory according to the passenger identity identifier and update the historical trajectory database to obtain the motion trajectory sequence of each passenger.

[0019] Furthermore, it also includes: extracting the posture change features corresponding to each frame from the sequence of each passenger's movement trajectory and organizing them in chronological order to obtain posture change time series data; reading the preset fall judgment threshold conditions from the configuration parameter library and parsing them into a trunk tilt threshold, a center of mass descent threshold, and a limb folding threshold to obtain a multi-level judgment threshold set; and comparing the features of each dimension in the posture change time series data with the corresponding thresholds in the multi-level judgment threshold set to obtain the trigger identifiers of each dimension.

[0020] The initial confidence score is obtained by weighting and accumulating the trigger identifiers of each dimension according to the preset weight coefficients. The temporal accumulation factor is calculated based on the number of consecutive trigger frames of the attitude change time series data within the sliding window. The fall confidence score is obtained by multiplying the initial confidence score and the temporal accumulation factor.

[0021] Furthermore, it also includes: comparing the fall confidence score with a preset trigger threshold to obtain a fall trigger determination result; when the fall trigger determination result is in a triggered state, starting a lying timer and continuously accumulating the lying duration; comparing the lying duration with a preset time constraint to obtain a duration satisfaction flag; when the fall trigger determination result is in a triggered state and the duration satisfaction flag is valid, matching the warning level based on the interval where the fall confidence score is located to obtain a graded warning signal;

[0022] The graded early warning signals are encapsulated into early warning data packets according to the communication protocol format. The early warning data packets are then synchronously pushed to the monitoring client and mobile terminal through the network interface to trigger pop-up alarms and message reminders. Based on the early warning level in the graded early warning signals, escalator control commands are generated. The escalator control commands are sent to the escalator control system to trigger deceleration protection actions and link the camera equipment to perform image capture and record and store the preceding and following time periods.

[0023] Secondly, this application provides a real-time detection and active early warning device for escalator passenger fall behavior based on human posture estimation, comprising:

[0024] The image cropping module is used to acquire a video stream of passenger activities through a camera device deployed above the escalator area to obtain an original image sequence, perform background subtraction and foreground separation on the original image sequence to obtain a moving target region image, input the moving target region image into a multi-scale human detection model to perform target localization to obtain a set of bounding boxes for each passenger, and crop the moving target region image based on the set of bounding boxes for each passenger to obtain a passenger image region.

[0025] The posture analysis module is used to input the image regions of each passenger into the human skeleton detection model to extract the set of skeleton key point coordinates, calculate the torso tilt angle, the rate of change of the center of mass height and the degree of limb folding based on the set of skeleton key point coordinates to obtain a posture feature vector, perform temporal difference between the posture feature vector and the posture features of historical frames to obtain posture change features, and input the posture change features into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger.

[0026] The fall protection module is used to perform multi-level comparisons between the posture change features in the movement trajectory sequence of each passenger and the preset fall judgment threshold to obtain a fall confidence score. When the fall confidence score exceeds the preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. The graded warning signal is pushed to the monitoring client and mobile terminal and the escalator control system is linked to perform deceleration protection.

[0027] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for real-time detection and active warning of escalator passenger fall behavior based on human posture estimation.

[0028] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation.

[0029] Fifthly, this application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation.

[0030] As can be seen from the above technical solution, this application provides a method and device for real-time detection and active early warning of passenger fall behavior in escalators based on human posture estimation. It constructs image regions for each passenger through background subtraction and multi-scale human detection, establishes passenger motion trajectory sequences by combining skeleton key point posture feature extraction and temporal difference tracking, and drives graded early warning signals to link escalator deceleration protection through multi-level confidence comparison and time constraints. It effectively solves the shortcomings of traditional technologies in motion target extraction, posture behavior modeling and graded early warning linkage, and provides technical support for intelligent real-time detection and active safety early warning of passenger fall behavior in escalators. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in an embodiment of this application.

[0033] Figure 2 This is a structural diagram of the escalator passenger fall detection and active early warning device based on human posture estimation in the embodiments of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0036] In view of the problems existing in the prior art, this application provides a method and device for real-time detection and active early warning of passenger fall behavior in escalators based on human posture estimation. The method constructs image regions of each passenger through background subtraction and multi-scale human detection, establishes passenger motion trajectory sequences by combining skeleton key point posture feature extraction and temporal difference tracking, and drives graded early warning signals to link escalator deceleration protection through multi-level confidence comparison and time constraints. This effectively solves the shortcomings of traditional technologies in terms of moving target extraction, posture behavior modeling and graded early warning linkage, and provides technical support for intelligent real-time detection and active safety early warning of passenger fall behavior in escalators.

[0037] To effectively address the shortcomings of traditional technologies in areas such as moving target extraction, posture behavior modeling, and hierarchical early warning linkage, and to provide technical support for intelligent real-time detection and proactive safety warning of passenger fall behavior on escalators, this application provides an embodiment of a method for real-time detection and proactive early warning of passenger fall behavior on escalators based on human posture estimation. See [link to embodiment]. Figure 1 The method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation specifically includes the following:

[0038] Step S101: Obtain the original image sequence by acquiring the video stream of passenger activities through the camera equipment deployed above the escalator area; perform background subtraction and foreground separation on the original image sequence to obtain the moving target area image; input the moving target area image into the multi-scale human detection model to perform target localization to obtain the bounding box set of each passenger; and perform region cropping on the moving target area image based on the bounding box set of each passenger to obtain the image region of each passenger.

[0039] This embodiment uses cameras deployed above the escalator area to continuously capture video of passenger activity. The cameras capture real-time footage of the escalator's operating area at a preset capture frequency, configured based on the escalator's speed and passenger density to ensure continuous changes in passenger posture are captured between adjacent frames. The captured passenger activity video stream is transmitted to the frame buffer queue of the edge computing node, awaiting subsequent keyframe extraction and processing.

[0040] Based on the passenger activity video stream, this embodiment performs keyframe extraction to obtain the original image sequence. The extraction process involves extracting image frames from the video stream at fixed frame intervals, with each frame appended with a timestamp to indicate its position on the timeline. The original image sequence is then sorted in ascending order by timestamp and written to a frame buffer queue. Subsequently, each frame in the queue undergoes adaptive brightness adjustment and color equalization processing to eliminate interference from lighting changes in the escalator area on subsequent detection, resulting in a preprocessed image sequence.

[0041] Accordingly, this embodiment reads the preprocessed image sequence from the frame buffer queue and performs pixel-by-pixel differencing with a pre-established background model. The background model is obtained during system initialization by statistically modeling images of escalator areas during periods without passengers; its pixel value distribution characterizes the static appearance features of the escalator steps and handrails. The differencing operation subtracts the pixel values ​​of each pixel in the current frame from the corresponding pixel values ​​in the background model. Pixels whose absolute difference exceeds the dynamic threshold are marked as foreground candidate points, resulting in a differencing image.

[0042] Specifically, considering that escalator steps move periodically and uniformly during normal operation, the background model modeling process uses a Gaussian mixture model to statistically fit the time series of grayscale values ​​at each pixel location. This incorporates the pixel grayscale fluctuations caused by the periodic movement of the steps into the variance modeling range of the background distribution, enabling the background model to represent the dynamic background statistical distribution characteristics of the escalator area, including the periodic movement pattern of the steps. The relatively stable texture of the handrail area is used as the spatial reference benchmark for the background model. The difference operation subtracts the statistical expectation value of each pixel value in the current frame from the corresponding position in the background model. Pixels whose absolute difference exceeds the background variance confidence interval at that position are marked as foreground candidate points, resulting in the difference image.

[0043] Based on the difference image, this embodiment performs binarization segmentation and morphological filtering to obtain a foreground mask image. Binarization segmentation sets foreground candidate points in the difference image to valid values ​​and other pixels to invalid values. Morphological filtering sequentially performs erosion and dilation operations; erosion eliminates isolated noise points, and dilation fills in empty areas within the human body contour. The foreground mask image is then superimposed on the corresponding frames in the preprocessed image sequence, retaining only the pixel information within the valid mask area to obtain the moving target region image.

[0044] After inputting the image of the moving target region into the multi-scale human detection model, this embodiment performs target localization processing. The multi-scale human detection model adopts a feature pyramid network structure, which performs multi-level convolution operations on the input image to generate feature maps of different resolutions. This network structure performs candidate box generation on each level of feature map, enabling the model to detect passenger targets with different sizes due to varying distances from the camera device. After the candidate boxes are generated, non-maximum suppression filtering is performed to remove redundant boxes with excessive overlap, resulting in a set of bounding boxes for each passenger.

[0045] Based on the aforementioned set of passenger bounding boxes, this embodiment associates and stores the coordinates of each bounding box with its corresponding confidence score to obtain a bounding box attribute list. Bounding box records whose confidence scores meet a preset threshold are selected from this list. Based on their coordinate information, region cropping is performed on the moving target region image to extract the rectangular image blocks containing each passenger, thus obtaining the initial image region for each passenger. Size normalization processing is performed on each initial image region of the passenger, scaling image blocks of different sizes to a uniform size to obtain the image region for each passenger. Each passenger image region is written into an image region cache queue for subsequent steps, such as step S102, where the human skeleton detection model reads and performs skeleton key point extraction.

[0046] Step S102: Input the image regions of each passenger into the human skeleton detection model to extract the set of skeleton key point coordinates. Based on the set of skeleton key point coordinates, calculate the torso tilt angle, the rate of change of the center of mass height, and the degree of limb folding to obtain the posture feature vector. Perform temporal difference between the posture feature vector and the posture features of historical frames to obtain the posture change feature. Input the posture change feature into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger.

[0047] From the passenger image regions written into the image region cache queue in step S101, this embodiment reads each image region one by one and inputs it into the human skeleton detection model. The human skeleton detection model adopts an hourglass-shaped neural network structure, which extracts global semantic features through stepwise downsampling and then restores spatial resolution through stepwise upsampling, enabling the network to fuse feature representations at different scales. The model performs multi-scale feature fusion and keypoint heatmap regression on the input image, generating Gaussian response peaks at the spatial locations corresponding to each joint, thus obtaining the heatmap response set of each joint.

[0048] Based on the heatmap response sets of each joint, this embodiment performs peak localization to extract key point coordinates for each heatmap. The peak localization process searches for the pixel position with the largest response value in each heatmap; this position is the estimated two-dimensional coordinate of the corresponding joint. The human skeleton detection model outputs seventeen coordinate points covering the major joints of the human body, including head, shoulder, elbow, wrist, hip, knee, and ankle key points. All key point coordinates are organized according to the human body topology, establishing connections between adjacent joints to obtain a skeleton topology map and a set of skeleton key point coordinates.

[0049] Accordingly, this embodiment calculates the torso tilt angle based on the aforementioned skeleton topology map. The calculation process first extracts the coordinates of the left shoulder keypoint, right shoulder keypoint, left hip keypoint, and right hip keypoint from the skeleton keypoint coordinate set. The midpoint of the left and right shoulder keypoints is taken as the shoulder center, and the midpoint of the left and right hip keypoints is taken as the hip center. The line connecting the shoulder center and the hip center represents the orientation of the human torso, and the angle between this line and the perpendicular direction of the image is the torso tilt angle. When the passenger maintains a standing posture, the torso tilt angle is close to zero; when the passenger falls, the torso tilt angle increases sharply.

[0050] Based on the set of coordinates of the key points of the skeleton, this embodiment calculates the rate of change of the center of gravity height. The calculation process involves performing a weighted summation of the ordinates of all key points to obtain the center of gravity height value of the current frame. The weighting coefficients are configured according to the contribution ratio of each joint to the human body mass distribution. The center of gravity height values ​​of previous frames are read from the posture feature cache queue. The center of gravity height value of the current frame is then differiated from the center of gravity height value of the previous frame and divided by the inter-frame time interval to obtain the rate of change of the center of gravity height. This rate of change characterizes the vertical velocity of the passenger's center of gravity; during a fall, the rate of change of the center of gravity height exhibits a negative abrupt increase.

[0051] Based on the relative positions of the limb joints in the skeletal topology diagram, this embodiment calculates the degree of limb folding. The calculation process extracts the coordinates of the elbow and knee key points, and calculates the bending angles of the upper and lower limbs respectively. When a passenger falls, the limbs are in a curled-up state, the distance between the elbow key point and the torso shortens, and the angle between the knee and hip key points changes significantly. This embodiment calculates the degree of limb folding by comparing the relative distances between the joints with the baseline distances in the initial standing posture. The torso tilt angle, the rate of change of the center of mass height, and the degree of limb folding are combined and encapsulated in a fixed-dimensional order to obtain a posture feature vector.

[0052] Based on the attitude feature vector, this embodiment performs temporal differencing to obtain attitude abrupt change features. The attitude features of historical frames adjacent to the current frame's timestamp are read from the attitude feature cache queue. The attitude feature vector of the current frame and the attitude features of the historical frames are then subtracted along their corresponding dimensions to obtain an attitude difference vector. The absolute values ​​of the differences in each dimension of the attitude difference vector are calculated, and then weighted summation is performed according to preset weighting coefficients to obtain the attitude abrupt change features. These attitude abrupt change features represent the drastic change in passenger attitude between adjacent frames using scalar values; this value shows a peak response at the moment of the fall. In this embodiment, the attitude feature vector of the current frame is written into the attitude feature cache queue as the historical frame attitude features for calculating temporal differencing in subsequent frames.

[0053] After inputting the posture change features and the set of bounding boxes of each passenger in the current frame into the target tracking model, this embodiment performs cross-frame identity association. The target tracking model performs appearance feature encoding on the image regions within each passenger bounding box to generate fixed-dimensional identity feature vectors. The identity feature vectors of each passenger are matched with the trajectory identity features stored in the historical trajectory database. The matching process calculates the distance metric between each identity feature vector in the current frame and the end features of the historical trajectory. The pairing with the smallest distance and meeting the association threshold condition is confirmed as the cross-frame correspondence of the same passenger. In this embodiment, the inter-frame association pairing results are stitched together according to the passenger identity identifier, and the position sequence and posture sequence of each passenger in the historical trajectory database are updated to obtain the motion trajectory sequence of each passenger. The motion trajectory sequence of each passenger is written into the trajectory storage area for subsequent steps S103 to read and perform fall detection and warning generation.

[0054] Step S103: The posture change features in each passenger movement trajectory sequence are compared with the preset fall judgment threshold to obtain a fall confidence score. When the fall confidence score exceeds the preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. The graded warning signal is pushed to the monitoring client and mobile terminal and the escalator control system is linked to perform deceleration protection.

[0055] From the passenger motion trajectory sequences written to the trajectory storage area in step S102, this embodiment reads the temporal data of posture change features corresponding to each passenger one by one. The temporal data of posture change features is organized into a one-dimensional sequence according to the frame number, recording the degree of drastic posture change of the passenger in each frame within a continuous observation period. This embodiment performs sliding window segmentation processing on the sequence, and the window length is configured according to the duration of a typical fall process on an escalator. The posture change features within each window are extracted for subsequent multi-level comparison operations.

[0056] Based on the aforementioned sliding window segmentation, this embodiment performs multi-level comparisons between the posture change features within each window and preset fall detection thresholds. The first level of comparison targets the peak amplitude of the posture change features, comparing the maximum value of the posture change features within the window with the peak threshold. If the peak amplitude exceeds this threshold, a suspected fall is identified. The second level of comparison targets the cumulative change in torso tilt angle, extracting the torso tilt angle of each frame within the window period from the motion trajectory sequence and calculating the absolute value of the difference in torso tilt angle between the first and last frames of the window. If this difference exceeds the angle change threshold, a significant torso tilt is identified. The third level of comparison targets the persistently negative feature of the centroid height change rate, statistically analyzing the percentage of frames with a continuously negative centroid height change rate within the window. If this percentage exceeds the persistence threshold, a continuous downward trend in the centroid is identified.

[0057] Accordingly, this embodiment calculates the fall confidence score based on multi-level comparison results. The calculation process maps the judgment results of the three levels of comparison to single-level confidence contribution values, and assigns weight coefficients to each level's contribution value based on the distinguishing ability of that level's characteristics in fall judgment. To facilitate subsequent trigger judgment and warning level classification, this embodiment provides a numerical calculation formula for confidence score:

[0058] Q = a * u + b * v + c * e - d * z.

[0059] In the formula, Q represents the fall confidence score; u represents the peak response contribution value, derived from the mapping result of the ratio of the peak value of the posture change feature to the threshold condition in the first-level comparison; v represents the angle change contribution value, derived from the normalized result of the cumulative change in the torso tilt angle in the second-level comparison; e represents the persistence contribution value, derived from the quantification result of the proportion of negative frames in the centroid height change rate in the third-level comparison; z represents the posture stability compensation value, derived from the inverse mapping of the variance of the posture feature vector within the window, used to suppress jitter interference during normal walking; a, b, c, and d are non-negative weights, given by the system configuration parameters and limited to a bounded interval, where the sum of a, b, and c is a unit value. The calculated Q is written into the confidence evaluation buffer, waiting for subsequent trigger judgment reading.

[0060] Specifically, to ensure that the fall confidence score Q does not become invalid or negative due to excessive suppression of the posture stability compensation term in any scenario, the value of d must simultaneously satisfy the following constraints. First, let Zw be the statistical upper bound of the posture stability compensation value z in the normal walking scenario, Uf, Vf, and Ef be the minimum expected values ​​of u, v, and e respectively when a real fall event occurs, and let κ∈(0,1) be the preset safety margin coefficient, then d must satisfy:

[0061] d⋅Zw≤κ⋅(a⋅Uf+b⋅Vf+c⋅Ef)

[0062] This constraint ensures that Q does not fall below the preset trigger threshold in real-world fall scenarios. Secondly, the upper bound D of d is determined during the system calibration phase based on statistical analysis of normal walking samples from the field, specifically by setting...

[0063] D=κ⋅(a+b+c) / zˉ

[0064] Where zˉ is the statistical mean of z under normal walking conditions. Thirdly, d is limited to a bounded closed interval [0, D], where D is given by system configuration parameters and calibrated based on actual scenario sample data during system deployment. The above constraints ensure that the d⋅z term only plays a noise suppression role and will not cause structural damage to the positive judgment interval of the fall confidence score Q.

[0065] Based on the fall confidence score, this embodiment performs trigger condition determination. The current passenger's fall confidence score is read from the confidence assessment buffer and compared with a preset trigger threshold. When the fall confidence score exceeds the preset trigger threshold, this embodiment further extracts the number of consecutive frames in the passenger's motion trajectory sequence where the torso tilt angle exceeds the lying-down angle threshold. This number of consecutive frames is multiplied by the inter-frame time interval to obtain the lying-down duration. The lying-down duration is compared with a time constraint; when the lying-down duration meets the time constraint, the fall event is confirmed, and the warning signal generation stage begins.

[0066] Specifically, the lying angle threshold condition is a torso tilt angle determination threshold pre-stored in the system configuration parameter library, used as the critical determination criterion for distinguishing between a passenger's standing / walking posture and a lying posture. The value of the lying angle threshold condition is determined based on the difference in typical distribution of torso tilt angles in standing and lying postures according to ergonomics: when a passenger maintains a normal standing or walking posture, the angle between the line connecting the center of the shoulder and the center of the hip in the skeletal topology diagram and the perpendicular direction of the image is usually within the range of 0° to 30°; after a passenger falls and lands, this angle usually continues to exceed 60°. Accordingly, this embodiment sets the default value of the lying angle threshold condition to 60°, that is, when the torso tilt angle continuously exceeds this threshold in consecutive frames, it is determined that the passenger is in a lying state and the lying timer is triggered. The lying angle threshold condition can be modified and configured according to the tilt angle of the escalator body and the installation pitch angle of the camera equipment to compensate for geometric projection deviations under different installation scenarios.

[0067] Based on the numerical range of the fall confidence score, this embodiment performs a warning level classification to generate tiered warning signals. The warning level classification is determined according to the range of the fall confidence score. A Level 1 warning signal is generated when the confidence score is in a low range, indicating a suspected fall; a Level 2 warning signal is generated when the confidence score is in a medium range, indicating a confirmed fall; and a Level 3 warning signal is generated when the confidence score is in a high range and the duration of lying on the ground exceeds the extended time constraint, indicating an emergency rescue situation. The tiered warning signal carries the passenger's identification, the coordinates of the fall location, the fall confidence score, and a warning level label.

[0068] Specifically, the division of warning levels into intervals is determined based on three preset boundary values ​​Qt, Qa, and Qb, which satisfy the following conditions:

[0069] 0 <Qt<Qa<Qb≤Qn

[0070] Where Qt is the trigger threshold and Qn is the theoretical upper limit of the fall confidence score. The first interval is defined as [Qt, Qa). When the confidence score is in this interval, it indicates that the current frame's posture features have triggered some judgment dimensions but have not yet fully met the multi-dimensional joint conditions of the fall features. The system determines it as a suspected fall state and generates a level one warning signal. The escalator control system simultaneously executes a deceleration preparatory action. The second interval is defined as [Qa, Qb). When the confidence score is in this interval, it indicates that the multi-dimensional posture features have all exceeded the corresponding judgment thresholds and the time-series accumulation factor is high. The system determines it as a confirmed fall state and generates a level two warning signal. The escalator control system executes emergency deceleration until it stops. The third interval is defined as [Qb, Qn]. When the confidence score is in this interval and the duration of lying on the ground exceeds the extended time constraint, the system determines it as an emergency rescue state and generates a level three warning signal, simultaneously triggering a manual rescue call process. The specific values ​​of Qt, Qa, and Qb are determined during the system calibration phase based on the statistical distribution of confidence scores of real fall samples and normal walking samples, so that the false alarm rate and false negative rate of each interval meet the preset performance indicators. In this embodiment, the default reference values ​​of the three are 0.40, 0.60, and 0.80 respectively in the range of normalized confidence scores [0, 1], and can be adjusted according to the actual deployment scenario.

[0071] After the graded warning signals are encapsulated, this embodiment performs multi-channel push and device linkage. During the push process, the graded warning signals are sent to the monitoring client via the local area network. Upon receiving the signal, the monitoring client displays a warning prompt window on the screen and marks the location of the fallen passenger. Simultaneously, this embodiment pushes the graded warning signals to the mobile terminals of maintenance personnel via the mobile communication network. Upon receiving the signal, the mobile terminals trigger vibration alerts and voice broadcasts. When the warning level reaches level two or above, this embodiment sends a deceleration protection command to the escalator control system. Upon receiving the command, the escalator control system gradually reduces the speed of each step until it stops, providing safe conditions for on-site rescue. The push records of the graded warning signals and the linkage execution results are written to the event log storage area for subsequent event retrospective and statistical analysis.

[0072] As can be seen from the above description, the real-time detection and proactive early warning method for escalator passenger fall behavior based on human posture estimation provided in this application can construct image regions for each passenger through background subtraction and multi-scale human detection, establish passenger motion trajectory sequences by combining skeleton key point posture feature extraction and temporal difference tracking, and drive graded early warning signals to link escalator deceleration protection through multi-level confidence comparison and time constraints. This effectively solves the shortcomings of traditional technologies in terms of moving target extraction, posture behavior modeling, and graded early warning linkage, and provides technical support for intelligent real-time detection and proactive safety early warning of escalator passenger fall behavior.

[0073] In one embodiment of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in this application, it may further include the following:

[0074] Step S201: The camera device acquires video of the escalator area at a preset acquisition frequency to obtain a video stream of passenger activity. Keyframe extraction is performed on the video stream of passenger activity to obtain an original image sequence. The original image sequence is sorted according to the acquisition timestamp and written into a frame buffer queue. Brightness adaptive adjustment and color equalization are performed on the images in the frame buffer queue to obtain a preprocessed image sequence.

[0075] Step S202: Read the preprocessed image sequence from the frame buffer queue and perform pixel-by-pixel difference operation with the pre-established background model to obtain a difference image. Perform binarization segmentation and morphological filtering on the difference image to obtain a foreground mask image. Perform mask superposition operation on the foreground mask image and the corresponding frame in the preprocessed image sequence to obtain a moving target region image.

[0076] This embodiment utilizes cameras deployed above the escalator area to perform continuous video capture. The cameras capture real-time footage of the escalator's operating area at a preset capture frequency, configured based on the escalator's speed and passenger density to ensure continuous changes in passenger posture are captured between adjacent frames. The captured passenger activity video stream is transmitted to an edge computing node, awaiting subsequent keyframe extraction and processing.

[0077] Based on the passenger activity video stream, this embodiment performs keyframe extraction to obtain the original image sequence. The extraction process extracts image frames from the video stream at fixed frame intervals. The frame interval is configured according to the dynamic characteristics of the scene; a smaller interval is used during periods of high passenger traffic to capture more dense motion information. Each image frame is appended with a timestamp to indicate its position on the timeline. The original image sequence is then sorted in ascending order of its timestamps and written into a frame buffer queue. The queue capacity is limited based on system processing latency and memory resources.

[0078] Based on the frame buffer queue, this embodiment performs adaptive brightness adjustment on each frame image in the queue. The adjustment process first calculates the global average brightness of the current frame image, and compares this average value with a preset target brightness value to obtain a brightness offset. According to the brightness offset, a linear mapping is performed on the grayscale values ​​of each pixel in the image, adjusting the pixel values ​​of overly dark areas upward and compressing the pixel values ​​of overly bright areas downward, thereby eliminating the brightness fluctuations in the escalator area caused by changes in skylight lighting.

[0079] Accordingly, this embodiment performs chroma equalization processing on the image after brightness adjustment. The processing converts the image from its original color space to a color space that separates brightness and chroma, and performs histogram equalization on the chroma channel to expand the dynamic range of the chroma distribution. After equalization, the image is converted back to its original color space, resulting in a preprocessed image sequence. Each frame in the preprocessed image sequence retains the same timestamp as the original image sequence and is written back to the corresponding position in the frame buffer queue for subsequent step S202 to read and perform background difference operations.

[0080] After reading the preprocessed image sequence from the frame buffer queue, this embodiment performs pixel-by-pixel differencing on each frame with a pre-established background model. The background model is obtained during system initialization by statistically modeling images of escalator areas during periods without passengers. Each pixel in the model stores the expected grayscale value and grayscale fluctuation range for that location. The differencing operation subtracts the expected grayscale value of the corresponding location in the background model from the pixel value of the current frame. Pixels whose absolute difference exceeds the grayscale fluctuation range of that location are marked as foreground candidate points, resulting in a differencing image.

[0081] Based on the difference image, this embodiment performs binarization segmentation to generate an initial foreground mask. During the segmentation process, the pixel values ​​of foreground candidate points in the difference image are set as valid marker values, while the remaining pixels are set as invalid marker values, forming a binary mask image. Subsequently, morphological filtering is performed on the binary mask image, followed by erosion and dilation operations. The erosion operation uses a structuring element of a preset size to shrink the mask image, eliminating isolated noise points with areas smaller than the structuring element. The dilation operation uses a structuring element of the same size to expand the erosion result, filling in the voids within the human body contour caused by texture differences, thus obtaining the foreground mask image.

[0082] After performing a mask overlay operation on the foreground mask image and the corresponding frame in the preprocessed image sequence, this embodiment obtains the moving target region image. The overlay operation uses the foreground mask image as the selection condition, retaining only the pixel information within the validly marked area of ​​the mask; the pixel values ​​of the invalidally marked area are set as background fill values. The moving target region image retains the color and texture details of the original image while removing interference information from the static background area. The moving target region image is written to the target region cache queue for subsequent steps S301 to read and perform multi-scale human detection and target localization processing.

[0083] In one embodiment of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in this application, it may further include the following:

[0084] Step S301: Input the image of the moving target region into the feature pyramid network of the multi-scale human detection model to perform multi-level convolution operations to obtain a multi-scale feature map set. Based on the multi-scale feature map set, perform candidate box generation and non-maximum suppression filtering to obtain a set of bounding boxes for each passenger. Associate and store the coordinates of each bounding box and the confidence score in each passenger bounding box set to obtain a list of bounding box attributes.

[0085] Step S302: Read the coordinates of the bounding boxes whose confidence scores meet the preset screening threshold from the bounding box attribute list, perform region cropping on the moving target region image based on the bounding box coordinates to obtain the initial image region of each passenger, and perform size normalization processing on the initial image region of each passenger to obtain the image region of each passenger.

[0086] From the moving target region image written to the target region cache queue in step S202 above, this embodiment reads image data frame by frame and inputs it into a multi-scale human detection model. The multi-scale human detection model adopts a feature pyramid network structure, which consists of a bottom-up feature extraction path and a top-down feature fusion path. The bottom-up path performs layer-by-layer convolution operations on the input image, and after each convolution operation, a downsampling operation is performed to generate a feature map sequence with progressively decreasing spatial resolution but progressively richer semantic information.

[0087] Based on the aforementioned feature map sequence, this embodiment performs a top-down feature fusion operation. The fusion process begins with the feature map at the highest semantic level, performing an upsampling operation to restore spatial resolution, and then performing element-wise addition with the adjacent lower-level feature maps. This fusion process propagates downwards along the feature pyramid, ensuring that each level of feature map simultaneously possesses high-level semantic information and low-level spatial detail information, resulting in a multi-scale feature map set. Each level of feature map in the multi-scale feature map set corresponds to a different receptive field; higher-level feature maps are suitable for detecting small passenger targets farther from the camera, while lower-level feature maps are suitable for detecting large passenger targets closer to the camera.

[0088] Accordingly, in this embodiment, candidate bounding box generation is performed at each level of the multi-scale feature map set. The generation process pre-defines multiple sets of anchor boxes with different aspect ratios at each spatial location in each feature map, and performs boundary regression and category prediction operations on each anchor box. The boundary regression operation outputs the anchor box center point coordinate offset and aspect ratio scaling factor, while the category prediction operation outputs the confidence score that the anchor box contains a human target. The coordinate offset and scaling factor of each anchor box are applied to the original anchor box parameters to obtain the adjusted candidate bounding box coordinates. Candidate bounding boxes from all levels are then converged to form a candidate bounding box set.

[0089] Based on the candidate box set, this embodiment performs non-maximum suppression filtering to obtain the passenger bounding box set. The filtering process first sorts the candidate box set in descending order of confidence score, selecting the candidate box with the highest confidence score as the retained box. Then, the intersection-union ratio (IUR) between the retained box and the remaining candidate boxes is calculated. Candidate boxes with IUR values ​​exceeding the overlap threshold are considered redundant and are removed. This embodiment repeats the above selection and removal process for the remaining candidate boxes until all candidate boxes have been processed, resulting in the passenger bounding box set.

[0090] After associating the coordinate information of each bounding box in the passenger bounding box set with its corresponding confidence score, this embodiment generates a bounding box attribute list. Each record in the list contains five fields: the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the bounding box, the height of the bounding box, and the confidence score. Records are stored in descending order of confidence score. The bounding box attribute list is written to an attribute list cache for subsequent step S302 to read and perform region clipping processing.

[0091] After reading the bounding box attribute list from the attribute list cache, this embodiment performs a filtering threshold condition determination. The determination process iterates through each record in the list, comparing the confidence score of each record with the preset filtering threshold condition. Only records whose confidence scores meet the filtering threshold condition are retained, resulting in a valid bounding box record set. The coordinate information of each record in the valid bounding box record set is extracted for subsequent region clipping operations.

[0092] Based on the coordinates of each bounding box in the effective bounding box record set, this embodiment performs region cropping on the moving target region image to obtain initial image regions for each passenger. The cropping process extracts pixel data of the corresponding rectangular region from the moving target region image based on the coordinates of the upper left corner and the width and height parameters of each bounding box. Subsequently, size normalization processing is performed on each passenger's initial image region, scaling image regions of different sizes to a uniform target size. The scaling process uses a bilinear interpolation algorithm to maintain the continuity of image texture, resulting in each passenger image region. Each passenger image region is written into an image region cache queue for subsequent steps, such as step S401, where the human skeleton detection model reads and performs skeleton key point extraction processing.

[0093] In one embodiment of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in this application, it may further include the following:

[0094] Step S401: Input each passenger image region into the human skeleton detection model constructed based on the hourglass neural network to perform multi-scale feature fusion and key point heatmap regression to obtain the heatmap response set of each joint. Perform peak localization on the heatmap response set of each joint to extract the coordinate points covering the main joints of the human body to obtain the skeleton key point coordinate set. Connect the skeleton key point coordinate set according to the human body topology to obtain the skeleton topology map.

[0095] Step S402: Calculate the angle between the line connecting the shoulder key point and the hip key point in the skeleton topology map and the vertical direction to obtain the torso tilt angle; calculate the weighted centroid position of each key point based on the coordinate set of the skeleton key points and subtract it from the centroid position of the previous frame to obtain the centroid height change rate; calculate the limb contraction ratio based on the relative distance between the elbow key point and the knee key point in the skeleton topology map to obtain the limb folding degree; combine and encapsulate the torso tilt angle, the centroid height change rate, and the limb folding degree to obtain the posture feature vector.

[0096] From each passenger image region written to the image region cache queue in step S302, this embodiment reads image data one by one and inputs it into the human skeleton detection model. The human skeleton detection model employs an hourglass-shaped neural network structure, with symmetrical encoder and decoder paths at its core. The encoder path performs progressive downsampling and convolution operations on the input image, extracting global semantic features of human pose as the spatial resolution gradually decreases. The decoder path performs progressive upsampling and convolution operations on the encoder output, reconstructing fine-grained joint position information as the spatial resolution gradually recovers.

[0097] In this embodiment, a skip connection is established across layers between the encoder and decoder paths of the hourglass-shaped neural network. The skip connection performs element-wise addition of the feature maps from each layer of the encoder with the feature maps from the corresponding layers of the decoder, enabling the decoder to directly acquire the edge and texture details preserved by the encoder during the spatial resolution restoration process. After multi-scale feature fusion, the decoder outputs a feature map with the same spatial resolution as the input image. This feature map is then fed into the keypoint prediction head to perform heatmap regression calculations.

[0098] Accordingly, this embodiment performs keypoint heatmap regression on the fused features to obtain heatmap response sets for each joint. The regression operation performs pixel-by-pixel response value prediction on the feature map, generating an independent heatmap channel for each predefined human joint. The response value of each pixel in the heatmap represents the probability that the location is the corresponding joint, and pixels near the actual joint location exhibit a Gaussian distribution of response peaks. The heatmap response sets for each joint include heatmap data for seventeen joint channels: head, shoulder, elbow, wrist, hip, knee, and ankle.

[0099] Based on the heatmap response sets of each joint, this embodiment performs peak localization to extract the coordinates of each joint. The localization process searches for the pixel position with the largest response value in the heatmap of each joint channel, and uses the horizontal and vertical coordinates of this position as the estimated two-dimensional coordinates of the corresponding joint. To improve localization accuracy, this embodiment performs sub-pixel-level interpolation on the response value distribution around the peak pixel, fine-tuning the peak coordinates based on the gradient direction of the response values ​​of adjacent pixels. After the coordinates of all joints are extracted, they are organized according to a predefined joint index order to obtain the set of skeleton keypoint coordinates.

[0100] After establishing connections between the joints in the set of key skeletal coordinates based on human anatomy, this embodiment obtains a skeletal topology map. In the topology map, the head key point is connected to the shoulder center to represent the neck; the left and right shoulder key points are connected to the corresponding elbow key points to represent the upper arm; the left and right elbow key points are connected to the corresponding wrist key points to represent the forearm; the shoulder center is connected to the hip center to represent the torso; the left and right hip key points are connected to the corresponding knee key points to represent the thigh; and the left and right knee key points are connected to the corresponding ankle key points to represent the lower leg. The skeletal topology map is written to the skeletal data cache for subsequent step S402 to read and perform posture feature calculations.

[0101] After reading the skeleton topology map from the skeleton data cache, this embodiment calculates the torso tilt angle. The calculation process first extracts the coordinates of the left and right shoulder keypoints from the skeleton keypoint coordinate set, and then calculates the average of the horizontal and vertical coordinates of the two points to obtain the shoulder center coordinates. The same method is used to calculate the average of the left and right hip keypoints to obtain the hip center coordinates. The line connecting the shoulder center and the hip center forms the torso direction vector, and the angle between this vector and the perpendicular direction of the image is the torso tilt angle. When the passenger maintains a normal standing posture, the torso tilt angle is close to zero; when the passenger leans forward or falls sideways, the torso tilt angle increases sharply.

[0102] Based on the set of coordinates of the key points of the skeleton, this embodiment calculates the rate of change of the center of gravity height. The calculation process performs a weighted summation operation on the ordinates of all key points. The weighting coefficients are configured according to the contribution ratio of each joint to the human body mass distribution, with the weighting coefficients for joints in the torso region being higher than those for the distal limbs. The weighted summation result is the center of gravity height value of the current frame. In this embodiment, the center of gravity height values ​​of previous frames are read from the posture feature cache queue. The center of gravity height value of the current frame is then differentially calculated from the center of gravity height value of the previous frame and divided by the inter-frame time interval to obtain the rate of change of the center of gravity height. When a passenger falls, the rate of change of the center of gravity height exhibits a continuously negative value.

[0103] Based on the coordinate information of the elbow and knee keypoints in the skeleton topology map, this embodiment calculates the degree of limb folding. The calculation process extracts the Euclidean distances between the left and right elbow keypoints and their corresponding shoulder keypoints, as well as the Euclidean distances between the left and right knee keypoints and their corresponding hip keypoints. These distance values ​​are then compared with the baseline distance values ​​collected during the system initialization phase in a standard standing posture. A ratio less than a unit value indicates that the limb is in a contracted state. This embodiment calculates the limb folding degree by averaging the contraction ratios of each limb. The trunk tilt angle, the rate of change of the center of mass height, and the degree of limb folding are combined and encapsulated in a fixed-dimensional order to obtain a posture feature vector. This posture feature vector is written into a posture feature cache queue for subsequent steps S501 to read and perform temporal difference and cross-frame identity association processing.

[0104] In one embodiment of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in this application, it may further include the following:

[0105] Step S501: Read the historical frame pose features corresponding to the current frame from the pose feature cache queue, perform difference operation on the pose feature vector and the historical frame pose features item by item according to the feature dimension to obtain the pose difference vector, perform absolute value calculation and weighted summation on the difference of each dimension in the pose difference vector to obtain the pose change feature, and write the pose feature vector into the pose feature cache queue as the historical frame pose features of subsequent frames.

[0106] Step S502: Input the attitude change feature and the set of bounding boxes of each passenger in the current frame into the target tracking model to perform appearance feature encoding to obtain the identity feature vector of each passenger. Based on the identity feature vector of each passenger and the trajectory identity features stored in the historical trajectory library, perform similarity matching to obtain the inter-frame association pairing result. Perform trajectory stitching of the inter-frame association pairing result according to the passenger identity identifier and update the historical trajectory library to obtain the motion trajectory sequence of each passenger.

[0107] From the attitude feature vector written to the attitude feature cache queue in step S402, this embodiment reads the current frame attitude feature vector and retrieves the attitude features of historical frames adjacent to its timestamp. The attitude feature cache queue maintains a fixed-length sliding window in ascending order of timestamps, storing the attitude feature vectors of the most recent frames within the window. In this embodiment, based on the current frame timestamp, the previous frame position is located in the queue, and the historical frame attitude features stored at that position are read. These features have the same dimensional structure as the current frame attitude feature vector.

[0108] Based on the current frame pose feature vector and the pose features of historical frames, this embodiment performs a dimension-by-dimensional interpolation operation to obtain a pose difference vector. The interpolation operation subtracts the value of the corresponding dimension in the pose features of historical frames from the value of the torso tilt angle dimension in the current frame pose feature vector, obtaining the torso tilt angle difference value. The same interpolation operation is performed on the centroid height change rate dimension and the limb folding degree dimension, respectively, to obtain the centroid height change rate difference value and the limb folding degree difference value. These three difference values ​​are then organized in their original dimensional order to obtain the pose difference vector.

[0109] Accordingly, this embodiment performs absolute value calculation and weighted summation on the differences in each dimension of the attitude difference vector. The calculation process first takes the absolute value of each dimension value in the attitude difference vector to eliminate the influence of positive and negative signs on the characterization of the degree of abrupt change. To facilitate subsequent fall detection and warning triggering, this embodiment provides a formula for calculating attitude abrupt change features:

[0110] M = p * g + q * h + r * j.

[0111] In the formula, M represents the posture change feature; g represents the absolute value of the trunk tilt angle difference, which characterizes the change in trunk orientation between adjacent frames; h represents the absolute value of the centroid height change rate difference, which characterizes the change in the vertical motion velocity of the centroid between adjacent frames; j represents the absolute value of the limb folding degree difference, which characterizes the change in limb contraction state between adjacent frames; p, q, and r are non-negative weight coefficients, given by the system configuration parameters, and the sum of the three is a unit value.

[0112] After the attitude change feature calculation is completed, this embodiment writes the current frame attitude feature vector into the attitude feature cache queue. During the writing process, a timestamp is appended to the current frame attitude feature vector and added to the tail of the queue. When the queue length exceeds a preset capacity limit, the earliest written historical frame attitude feature at the head of the queue is removed. The newly written current frame attitude feature vector in the attitude feature cache queue will serve as the historical frame attitude feature for subsequent frames when performing temporal difference operations.

[0113] After inputting the pose change features and the set of bounding boxes for each passenger in the current frame into the target tracking model, this embodiment performs appearance feature encoding. The target tracking model comprises two substructures: an appearance encoding network and an association matching module. The appearance encoding network performs convolution and global pooling operations on the image regions within each passenger bounding box, mapping image regions of different sizes into fixed-dimensional identity feature vectors. Each passenger identity feature vector encodes passenger clothing color, texture pattern, and body contour information, used to distinguish the visual appearance differences between different passengers.

[0114] Based on the passenger identity feature vectors, this embodiment performs similarity matching with the historical trajectory database. The historical trajectory database stores the trajectory records of each tracked passenger. Each trajectory record includes a passenger identity identifier, the identity feature vector of the trajectory's end frame, and a sequence of location coordinates. The matching process calculates the cosine distance between the passenger identity feature vectors of the current frame and the trajectory's end identity feature vectors in the historical trajectory database. Pairings with the smallest distance values ​​that meet the association threshold condition are identified as cross-frame correspondences of the same passenger, resulting in inter-frame association pairing results.

[0115] After stitching the inter-frame association pairing results together according to passenger identity identifiers, this embodiment updates the historical trajectory database to obtain the motion trajectory sequences of each passenger. During the stitching process, the position coordinates and attitude change features of passengers who have completed pairing in the current frame are appended to the end of the corresponding trajectory record sequence, and the end identity feature of the trajectory record is updated with the current frame's identity feature vector. For passengers in the current frame who cannot be paired with historical trajectories, this embodiment creates new trajectory records for them and assigns a unique identity identifier. The motion trajectory sequences of each passenger are written to the trajectory storage area for subsequent steps S601 to read and perform fall confidence calculation and graded early warning generation processing.

[0116] In one embodiment of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in this application, it may further include the following:

[0117] Step S601: Extract the posture change features corresponding to each frame from the motion trajectory sequence of each passenger and organize them in chronological order to obtain posture change time series data. Read the preset fall judgment threshold conditions from the configuration parameter library and parse them into trunk tilt threshold, center of mass descent threshold and limb folding threshold to obtain a multi-level judgment threshold set. Compare the features of each dimension in the posture change time series data with the corresponding thresholds in the multi-level judgment threshold set to obtain the trigger identifiers of each dimension.

[0118] Step S602: The trigger identifiers of each dimension are weighted and accumulated according to the preset weight coefficient to obtain the initial confidence score. The temporal accumulation factor is calculated based on the number of consecutive trigger frames of the attitude change time series data within the sliding window. The initial confidence score and the temporal accumulation factor are multiplied to obtain the fall confidence score.

[0119] From the passenger motion trajectory sequences written to the trajectory storage area in step S502, this embodiment reads the trajectory records corresponding to each passenger one by one and extracts the attitude change features of each frame. The extraction process traverses the frame data stored in the trajectory records in chronological order, and reads the attitude change feature values ​​from each frame. The attitude change features of all frames are arranged in ascending order of timestamps and organized into a one-dimensional sequence to obtain the attitude change time series data. The attitude change time series data records the evolution of the passenger's attitude change intensity in each frame within the observation period.

[0120] Based on the aforementioned posture change time-series data, this embodiment reads preset fall detection threshold conditions from the configuration parameter library. The configuration parameter library stores various parameters required for system operation in key-value pairs. This embodiment retrieves the corresponding parameter value set based on the identifier key of the fall detection parameter group. The parsing process splits the parameter value set into three independent values: trunk tilt threshold, center of mass descent threshold, and limb folding threshold. The trunk tilt threshold represents the critical amplitude at which the trunk tilt angle difference triggers an alarm; the center of mass descent threshold represents the critical amplitude at which the center of mass height change rate difference triggers an alarm; and the limb folding threshold represents the critical amplitude at which the limb folding degree difference triggers an alarm. The combination of these three values ​​yields a multi-level judgment threshold set.

[0121] Accordingly, this embodiment performs a step-by-step comparison between the features of each dimension in the posture change time series data and the corresponding thresholds in the multi-level judgment threshold set. The first level of comparison targets the torso tilt dimension, comparing the absolute value of the torso tilt angle difference value in the posture difference vector with the torso tilt threshold. If the difference value exceeds the threshold, the trigger flag for this dimension is set to an active state. The second level of comparison targets the centroid descent dimension, comparing the absolute value of the centroid height change rate difference value with the centroid descent threshold. If the difference value exceeds the threshold, the trigger flag for this dimension is set to an active state. The third level of comparison targets the limb folding dimension, comparing the absolute value of the limb folding degree difference value with the limb folding threshold. If the difference value exceeds the threshold, the trigger flag for this dimension is set to an active state, thus obtaining the trigger flags for each dimension.

[0122] Based on the trigger identifiers of each dimension, this embodiment performs a weighted summation to obtain an initial confidence score. The summation process maps each dimension's trigger identifier from a valid state to a unit contribution value and from an invalid state to zero. Then, the contribution values ​​of the three dimensions are multiplied by their corresponding preset weight coefficients and summed. The weight coefficients are configured according to the distinguishing ability of each dimension's features in determining a fall, with the weight coefficient for the torso tilt dimension being higher than that of the other two dimensions. The initial confidence score represents the immediacy of the pose anomaly in the current frame.

[0123] Based on the distribution characteristics of the attitude change time-series data within the sliding window, this embodiment calculates the temporal accumulation factor. The calculation process involves establishing a fixed-length sliding window on the attitude change time-series data, with the window's end aligned with the current frame position. This embodiment counts the number of consecutive frames within the window where all trigger markers are valid, and then calculates the ratio of this number of consecutive frames to the window length to obtain the continuous trigger ratio. The temporal accumulation factor is obtained from the continuous trigger ratio through a non-linear mapping; a higher continuous trigger ratio results in a larger temporal accumulation factor, which enhances the confidence response to persistent attitude anomalies.

[0124] After multiplying the initial confidence score with the temporal accumulation factor, this embodiment obtains the fall confidence score. The product operation allows the fall confidence score to simultaneously reflect the immediate anomaly level of the current frame and the persistent anomaly level within the recent temporal window. Occasional attitude fluctuations in a single frame are suppressed due to the smaller temporal accumulation factor, while attitude anomalies across multiple consecutive frames are amplified due to the larger temporal accumulation factor. The fall confidence score is written to the confidence evaluation buffer for subsequent steps S701 to read and execute trigger condition determination and graded warning signal generation processing.

[0125] In one embodiment of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation in this application, it may further include the following:

[0126] Step S701: Compare the fall confidence score with a preset trigger threshold to obtain a fall trigger determination result. When the fall trigger determination result is in a triggered state, start a lying timer and continuously accumulate the lying duration. Compare the lying duration with a preset time constraint to obtain a duration satisfaction flag. When the fall trigger determination result is in a triggered state and the duration satisfaction flag is valid, match the warning level based on the interval where the fall confidence score is located to obtain a graded warning signal.

[0127] Step S702: Encapsulate the graded early warning signal according to the communication protocol format to obtain an early warning data packet, and push the early warning data packet synchronously to the monitoring client and mobile terminal through the network interface to trigger pop-up alarms and message reminders. Generate escalator control commands based on the early warning level in the graded early warning signal, and send the escalator control commands to the escalator control system to trigger deceleration protection actions and link the camera equipment to perform image capture and record and store the preceding and following time periods.

[0128] From the fall confidence score written to the confidence assessment buffer in step S602 above, this embodiment reads the confidence value corresponding to the current passenger and compares it with a preset trigger threshold. The comparison process determines the magnitude of the fall confidence score against the preset trigger threshold. When the fall confidence score exceeds the preset trigger threshold, the fall trigger determination result is set to a triggered state; when the fall confidence score does not exceed the preset trigger threshold, the fall trigger determination result is set to a non-triggered state. The fall trigger determination result is written to the determination status register for subsequent lying-down timing and warning level matching.

[0129] When the fall trigger determination result is in a triggered state, this embodiment starts a prone timer and continuously accumulates the prone duration. The prone timer initializes its timing start point when the triggered state is first detected, and then checks whether the fall trigger determination result maintains the triggered state within each frame processing cycle. When the triggered state persists, this embodiment performs a difference operation between the current frame timestamp and the timing start timestamp to obtain the prone duration. When the triggered state is interrupted, this embodiment resets the prone timer to its initial state and clears the accumulated prone duration.

[0130] Accordingly, this embodiment compares the duration of the lying-down position with a preset time constraint to obtain a duration fulfillment flag. The preset time constraint represents the minimum lying-down duration required to confirm a fall event, used to filter out false triggers caused by non-fall actions such as brief squatting or bending over. When the duration of the lying-down position reaches or exceeds the preset time constraint, this embodiment sets the duration fulfillment flag to an active state. When the duration of the lying-down position does not reach the preset time constraint, this embodiment sets the duration fulfillment flag to an inactive state, and the system continues to accumulate the lying-down position duration but does not generate a warning signal temporarily.

[0131] When the fall trigger determination result is in a triggered state and the duration meets the requirement to be marked as valid, this embodiment matches the warning level based on the interval of the fall confidence score. The interval matching process compares the fall confidence score with preset multi-level interval boundary values. When the confidence score is in the first interval, a level one warning level is matched to represent a suspected fall; when the confidence score is in the second interval, a level two warning level is matched to represent a confirmed fall; and when the confidence score is in the third interval and the duration of lying on the ground exceeds the extended time constraint, a level three warning level is matched to represent an emergency rescue state. This embodiment combines and encapsulates the matched warning level with the passenger's identity identifier, the coordinates of the fall location, the fall confidence score, and the event timestamp to obtain a graded warning signal.

[0132] After encapsulating the tiered early warning signals according to the communication protocol format, this embodiment obtains an early warning data packet. The encapsulation process follows the system communication protocol specifications, adding a header field to the front of the tiered early warning signal. This header field includes the data packet length, message type identifier, sender address, and sequence number. A verification field is added to the back of the tiered early warning signal. This verification field is calculated from the data packet body using a cyclic redundancy check algorithm and is used by the receiver to verify the integrity of the data packet. The early warning data packet is written to a transmission buffer, awaiting network interface scheduling for transmission.

[0133] After the warning data packet is pushed to the monitoring client and mobile terminal via the network interface, this embodiment triggers a multi-channel alarm response. Upon receiving the warning data packet, the monitoring client parses the tiered warning signals, pops up a warning prompt window on the monitoring interface, and highlights the location of the fallen passenger on the escalator. Upon receiving the warning data packet, the mobile terminal triggers a vibration alert and voice broadcast. Maintenance personnel can view details of the fall event and on-site image snapshots through the mobile terminal.

[0134] Based on the warning levels in the tiered warning signals, this embodiment generates escalator control commands and sends them to the escalator control system. When the warning level is Level 1, a deceleration preparation command is generated. Upon receiving the command, the escalator control system reduces the step speed to a preset slow speed. When the warning level is Level 2 or 3, an emergency deceleration command is generated. Upon receiving the command, the escalator control system executes a step-by-step deceleration until the step speed stops completely. This embodiment simultaneously sends image capture commands and video recording and storage commands to the camera device. The camera device captures the current frame image and writes the video stream for a preset period before and after the fall event into the event recording storage area. The push records of the tiered warning signals, the execution results of the escalator control commands, and the video recording storage path are written into the event log storage area for subsequent event review and statistical analysis.

[0135] To effectively address the shortcomings of traditional technologies in areas such as moving target extraction, posture behavior modeling, and hierarchical early warning linkage, and to provide technical support for intelligent real-time detection and proactive safety warning of escalator passenger fall behavior, this application provides an embodiment of a human posture estimation-based escalator passenger fall behavior real-time detection and proactive warning device for implementing all or part of the aforementioned human posture estimation-based escalator passenger fall behavior real-time detection and proactive warning method. See [link to embodiment]. Figure 2 The real-time detection and active early warning device for escalator passenger fall behavior based on human posture estimation specifically includes the following components:

[0136] The image cropping module 10 is used to acquire a video stream of passenger activities through a camera device deployed above the escalator area to obtain an original image sequence, perform background subtraction and foreground separation on the original image sequence to obtain a moving target region image, input the moving target region image into a multi-scale human detection model to perform target localization to obtain a set of bounding boxes for each passenger, and crop the moving target region image based on the set of bounding boxes for each passenger to obtain a passenger image region.

[0137] The posture analysis module 20 is used to input the image regions of each passenger into the human skeleton detection model to extract the set of skeleton key point coordinates, calculate the torso tilt angle, the rate of change of the center of mass height and the degree of limb folding based on the set of skeleton key point coordinates to obtain a posture feature vector, perform temporal difference between the posture feature vector and the posture features of historical frames to obtain posture change features, and input the posture change features into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger.

[0138] The fall protection module 30 is used to perform multi-level comparisons between the posture change features in the movement trajectory sequence of each passenger and the preset fall judgment threshold conditions to obtain a fall confidence score. When the fall confidence score exceeds the preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. The graded warning signal is pushed to the monitoring client and mobile terminal and the escalator control system is linked to perform deceleration protection.

[0139] As can be seen from the above description, the real-time detection and active early warning device for escalator passenger fall behavior based on human posture estimation provided in this application embodiment can construct image regions of each passenger through background subtraction and multi-scale human detection, establish passenger motion trajectory sequences by combining skeleton key point posture feature extraction and temporal difference tracking, and drive graded early warning signals to link escalator deceleration protection through multi-level confidence comparison and time constraints. It effectively solves the shortcomings of traditional technologies in motion target extraction, posture behavior modeling and graded early warning linkage, and provides technical support for intelligent real-time detection and active safety early warning of escalator passenger fall behavior.

[0140] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation.

[0141] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation.

[0142] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-mentioned method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation.

[0143] In this embodiment of the invention, image regions for each passenger are constructed by background subtraction and multi-scale human detection. Passenger motion trajectory sequences are established by combining skeleton key point pose feature extraction and temporal difference tracking. Furthermore, the tiered warning signals are driven by multi-level confidence comparison and time constraints to link escalator deceleration protection. This effectively solves the shortcomings of traditional technologies in motion target extraction, posture behavior modeling, and tiered warning linkage, providing technical support for intelligent real-time detection and proactive safety warning of escalator passenger fall behavior.

[0144] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time detection and proactive early warning of passenger fall behavior on escalators based on human posture estimation, characterized in that, The method includes: The original image sequence is obtained by collecting video streams of passenger activities through camera equipment deployed above the escalator area. Background subtraction and foreground separation are performed on the original image sequence to obtain moving target area images. The moving target area images are input into a multi-scale human detection model to perform target localization to obtain a set of bounding boxes for each passenger. Based on the set of bounding boxes for each passenger, the moving target area images are cropped to obtain the image regions for each passenger. The image regions of each passenger are input into the human skeleton detection model to extract the set of skeleton key point coordinates. Based on the set of skeleton key point coordinates, the torso tilt angle, the rate of change of the center of mass height, and the degree of limb folding are calculated to obtain the posture feature vector. The posture feature vector is temporally differiated with the posture features of historical frames to obtain the posture change feature. The posture change feature is input into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger. The posture change features in each passenger movement trajectory sequence are compared with the preset fall judgment threshold to obtain a fall confidence score. When the fall confidence score exceeds the preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. The graded warning signal is pushed to the monitoring client and mobile terminal and the escalator control system is linked to perform deceleration protection.

2. The method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation according to claim 1, characterized in that, The process of acquiring a raw image sequence by collecting passenger activity video streams using camera equipment deployed above the escalator area, and performing background subtraction and foreground separation on the raw image sequence to obtain an image of the moving target region includes: The camera device captures video of the escalator area at a preset acquisition frequency to obtain a video stream of passenger activity. Keyframe extraction is performed on the video stream to obtain an original image sequence. The original image sequence is sorted according to the acquisition timestamp and written into a frame buffer queue. The images in the frame buffer queue are subjected to adaptive brightness adjustment and color equalization to obtain a preprocessed image sequence. The preprocessed image sequence is read from the frame buffer queue and a pixel-by-pixel difference operation is performed with the pre-established background model to obtain a difference image. Binarization segmentation and morphological filtering are performed on the difference image to obtain a foreground mask image. The foreground mask image is then superimposed with the corresponding frame in the preprocessed image sequence to obtain a moving target region image.

3. The method for real-time detection and proactive early warning of passenger fall behavior on escalators based on human posture estimation according to claim 1, characterized in that, The step of inputting the moving target region image into a multi-scale human detection model to perform target localization and obtain a set of bounding boxes for each passenger, and then cropping the moving target region image based on the set of bounding boxes for each passenger to obtain the image regions for each passenger, includes: The moving target region image is input into the feature pyramid network of the multi-scale human detection model to perform multi-level convolution operations to obtain a multi-scale feature map set. Based on the multi-scale feature map set, candidate box generation and non-maximum suppression filtering are performed to obtain a set of passenger bounding boxes. The coordinates of each bounding box in the set of passenger bounding boxes are associated with the confidence score and stored to obtain a list of bounding box attributes. Read the bounding box coordinates that meet the preset filtering threshold conditions from the bounding box attribute list, perform region cropping on the moving target region image based on the bounding box coordinates to obtain the initial image region of each passenger, and perform size normalization processing on the initial image region of each passenger to obtain the image region of each passenger.

4. The method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation according to claim 1, characterized in that, The step of inputting each passenger image region into a human skeleton detection model to extract a set of skeleton key point coordinates, and calculating the torso tilt angle, the rate of change of the center of gravity height, and the degree of limb folding based on the set of skeleton key point coordinates to obtain a posture feature vector, includes: Each passenger image region is input into a human skeleton detection model built on an hourglass neural network to perform multi-scale feature fusion and key point heatmap regression to obtain a set of joint heatmap responses. Peak localization is performed on the set of joint heatmap responses to extract coordinate points covering the main joints of the human body to obtain a set of skeleton key point coordinates. The set of skeleton key point coordinates is connected according to the human body topology to obtain a skeleton topology map. The torso tilt angle is obtained by calculating the angle between the line connecting the shoulder key point and the hip key point in the skeleton topology map and the vertical direction. The weighted centroid position of each key point is calculated based on the coordinate set of the skeleton key points and the centroid position of the previous frame is subtracted to obtain the centroid height change rate. The limb contraction ratio is calculated based on the relative distance between the elbow key point and the knee key point in the skeleton topology map to obtain the limb folding degree. The torso tilt angle, the centroid height change rate and the limb folding degree are combined and encapsulated to obtain the posture feature vector.

5. The method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation according to claim 1, characterized in that, The step of performing temporal difference analysis between the attitude feature vector and the attitude features of historical frames to obtain attitude change features, and then inputting the attitude change features into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger, includes: Read the pose features of the historical frame corresponding to the current frame from the pose feature cache queue, perform the difference operation between the pose feature vector and the pose features of the historical frame item by item according to the feature dimension to obtain the pose difference vector, perform absolute value calculation and weighted summation on the difference of each dimension in the pose difference vector to obtain the pose change feature, and write the pose feature vector into the pose feature cache queue as the pose features of the historical frame of the subsequent frame. The attitude change features and the set of bounding boxes of each passenger in the current frame are input into the target tracking model to perform appearance feature encoding to obtain the identity feature vector of each passenger. Based on the identity feature vector of each passenger, similarity matching is performed with the trajectory identity features stored in the historical trajectory database to obtain the inter-frame association pairing result. The inter-frame association pairing result is used to stitch the trajectory according to the passenger identity identifier and update the historical trajectory database to obtain the motion trajectory sequence of each passenger.

6. The method for real-time detection and proactive early warning of escalator passenger fall behavior based on human posture estimation according to claim 1, characterized in that, The step of performing a multi-level comparison between the posture change features in the movement trajectory sequences of each passenger and a preset fall determination threshold to obtain a fall confidence score includes: The posture change features corresponding to each frame are extracted from the motion trajectory sequence of each passenger and organized in chronological order to obtain posture change time series data. The preset fall judgment threshold conditions are read from the configuration parameter library and parsed into trunk tilt threshold, center of mass descent threshold and limb folding threshold to obtain a multi-level judgment threshold set. The features of each dimension in the posture change time series data are compared with the corresponding thresholds in the multi-level judgment threshold set to obtain the trigger identifiers of each dimension. The initial confidence score is obtained by weighting and accumulating the trigger identifiers of each dimension according to the preset weight coefficients. The temporal accumulation factor is calculated based on the number of consecutive trigger frames of the attitude change time series data within the sliding window. The fall confidence score is obtained by multiplying the initial confidence score and the temporal accumulation factor.

7. The method for real-time detection and proactive early warning of passenger fall behavior on escalators based on human posture estimation according to claim 1, characterized in that, When the fall confidence score exceeds a preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. This graded warning signal is then pushed to the monitoring client and mobile terminal, and the escalator control system is activated to perform deceleration protection. This includes: The fall confidence score is compared with a preset trigger threshold to obtain a fall trigger determination result. When the fall trigger determination result is in a triggered state, a lying timer is started and the lying duration is continuously accumulated. The lying duration is compared with a preset time constraint to obtain a duration satisfaction indicator. When the fall trigger determination result is in a triggered state and the duration satisfaction indicator is valid, a graded warning signal is obtained by matching the warning level based on the interval in which the fall confidence score is located. The graded early warning signals are encapsulated into early warning data packets according to the communication protocol format. The early warning data packets are then synchronously pushed to the monitoring client and mobile terminal through the network interface to trigger pop-up alarms and message reminders. Based on the early warning level in the graded early warning signals, escalator control commands are generated. The escalator control commands are sent to the escalator control system to trigger deceleration protection actions and link the camera equipment to perform image capture and record and store the preceding and following time periods.

8. A real-time detection and active early warning device for escalator passenger fall behavior based on human posture estimation, characterized in that, The device includes: The image cropping module is used to acquire a video stream of passenger activities through a camera device deployed above the escalator area to obtain an original image sequence, perform background subtraction and foreground separation on the original image sequence to obtain a moving target region image, input the moving target region image into a multi-scale human detection model to perform target localization to obtain a set of bounding boxes for each passenger, and crop the moving target region image based on the set of bounding boxes for each passenger to obtain a passenger image region. The posture analysis module is used to input the image regions of each passenger into the human skeleton detection model to extract the set of skeleton key point coordinates, calculate the torso tilt angle, the rate of change of the center of mass height and the degree of limb folding based on the set of skeleton key point coordinates to obtain a posture feature vector, perform temporal difference between the posture feature vector and the posture features of historical frames to obtain posture change features, and input the posture change features into the target tracking model to perform cross-frame identity association to obtain the motion trajectory sequence of each passenger. The fall protection module is used to perform multi-level comparisons between the posture change features in the movement trajectory sequence of each passenger and the preset fall judgment threshold to obtain a fall confidence score. When the fall confidence score exceeds the preset trigger threshold and the duration of lying on the ground meets the time constraint, a graded warning signal is generated. The graded warning signal is pushed to the monitoring client and mobile terminal and the escalator control system is linked to perform deceleration protection.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for real-time detection and active early warning of escalator passenger fall behavior based on human posture estimation as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the real-time detection and proactive warning method for escalator passenger fall behavior based on human posture estimation as described in any one of claims 1 to 7.

Images