Cableway platform tourist falling detection method, system and equipment based on visual identification and medium

Abstract

The invention relates to the technical field of cableway platform intelligent management, in particular to a cableway platform tourist falling detection method, system and device based on visual identification and a medium, and the method comprises the steps: collecting a video stream of a platform region, carrying out the pedestrian detection, obtaining the position of a bounding box of a tourist, and storing the position of the bounding box; a unique track ID and continuous track data are generated for each tourist by using a multi-target tracking algorithm; for each tourist, extracting a human body posture key point coordinate in each frame of the video stream and calculating a posture feature, and when the posture feature meets a preset posture collapse judgment rule, judging that the posture collapse is suspected; carrying out time sequence fusion on the continuous tracks of each tourist, and carrying out real falling event judgment based on the judgment condition of suspected posture collapse; and matching a preset platform area according to the position of the real falling event, outputting a risk level and triggering a corresponding control instruction. According to the invention, the falling behavior can be accurately and stably identified, and the safety protection capability of the cableway platform is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of ropeway station intelligent management, in particular to a ropeway station visitor fall detection method, system, device and medium based on visual recognition. BACKGROUND

[0002] As an important passenger transport device in scenic spots and ski resorts, the ropeway station area is a key node for passenger gathering and boarding / alighting, with dense flow and complex dynamics. Ensuring the safe operation of this area and preventing secondary accidents caused by visitor falls is one of the core requirements for improving the safety management level of public passenger transport facilities.

[0003] Currently, for safety monitoring in the ropeway station scenario, the existing technology mainly adopts the following methods: 1) relying on manual duty, and discovering abnormalities by workers through visual observation or monitoring video; 2) regional intrusion detection technology based on video analysis, identifying and alarming the intrusion behavior by setting a virtual warning area; 3) using pressure pads, infrared beams and other physical sensors to detect the out-of-bound or stay behavior of visitors at specific points.

[0004] However, the existing safety monitoring method for the ropeway station has obvious limitations: the existing safety monitoring method essentially focuses on the position or movement trajectory of personnel, and cannot effectively capture and analyze the specific body posture change behavior of “falling”; due to the lack of analysis capability for continuous posture sequences, the stability and reliability of the existing technology significantly decrease when dealing with common light changes, personnel occlusion and dense flow in the station environment; the existing scheme can only output simple trigger alarms, and cannot associate the detected events with the actual danger level in different areas of the station, so as to provide accurate and hierarchical linkage decision basis for the ropeway control system. SUMMARY

[0005] The safety monitoring method for the ropeway station mainly relies on position or trajectory monitoring, cannot accurately identify the specific body posture change behavior of “falling”, and has poor stability in complex environments, so it is difficult to realize accurate linkage associated with the regional danger level. The application provides a ropeway station visitor fall detection method, system, device and medium based on visual recognition, which realizes accurate and stable identification and automatic hierarchical linkage of the falling behavior through visual human posture analysis, time sequence fusion and regional association, and significantly improves the safety protection capability of the ropeway station.

[0006] In a first aspect, the application provides a ropeway station visitor fall detection method based on visual recognition, including the following steps: collecting a video stream of a station area and performing pedestrian detection on the video stream, obtaining a bounding box position of a tourist, and generating a unique track ID and continuous track data for each tourist using a multi-target tracking algorithm; For each tourist, the human pose key point coordinates of the tourist are extracted in each frame of the video stream, and a pose feature is calculated. When the pose feature meets a preset pose collapse judgment rule, it is determined that there is a suspected pose collapse; For each tourist, the human pose key point coordinates of the tourist are extracted in each frame of the video stream, and a pose feature is calculated. When the pose feature meets a preset pose collapse judgment rule, it is determined that there is a suspected pose collapse; According to the position of the real fall event, a preset station area is matched, a risk level is output, and a corresponding control instruction is triggered.

[0007] Further, a plurality of cameras deployed at the cableway station are used to collect the video stream of the station area, and the shooting range of the cameras covers the waiting area, the upper hatch, the lower hatch, the car deceleration section and the car off-station passage of the cableway station.

[0008] Further, after collecting the video stream of the station area, the video stream is preprocessed, and the preprocessing includes at least one of image decoding, frame format standardization, illumination compensation, noise filtering and region cropping.

[0009] Further, the pedestrian detection on the video stream specifically includes: The image of each frame in the video stream is input into the YOLOv8 model, and a plurality of candidate detection boxes are output, each candidate detection box containing a class label, a bounding box coordinate and a confidence score A first threshold is set, and candidate detection boxes with a confidence score lower than the first threshold are filtered; Non-maximum suppression is performed on the remaining candidate detection boxes after filtering. When the intersection over union of two candidate detection boxes is greater than a preset second threshold, only the candidate detection box with a higher confidence score is retained, and a bounding box set of all pedestrians in the current frame is obtained.

[0010] Further, the DeepSORT algorithm is used as the multi-target tracking algorithm to generate a unique track ID and continuous track data for each tourist. The execution steps of the DeepSORT algorithm in each frame include: An appearance feature extraction network is used to extract an appearance feature vector from a pedestrian image cropped from each pedestrian detection box; If the current frame is an initial frame, the position of each pedestrian in the current frame is used as the starting point of the track of the pedestrian; If the current frame is not an initial frame, a Kalman filter is used to predict the state of the track existing in the previous frame to obtain the predicted position of each track in the current frame and generate a track prediction box; calculate the motion Mahalanobis distance and the appearance feature cosine distance between the detection box of the current frame and the track prediction box; Based on the motion Mahalanobis distance and the appearance feature cosine distance, the association between the detection box of the current frame and the existing track is completed through cascading matching and the Hungarian algorithm.

[0011] Further need to explain is, appearance feature extraction network is a deep convolutional neural network pre-trained on pedestrian re-identification dataset, its input is the scaled pedestrian image, the output is a fixed dimension feature vector.

[0012] Further need to explain is, appearance feature extraction network is obtained by training in the following way: Use a large-scale pedestrian re-identification dataset for training, the training target is to make the cosine distance between the feature vectors extracted from different images of the same pedestrian as small as possible, and the cosine distance between the feature vectors of different pedestrians as large as possible.

[0013] Further need to explain is, the track management mechanism of DeepSORT algorithm includes: For the track successfully matched with the detection box, update its bounding box position, motion speed and appearance feature, and maintain the original track ID; For the detection box that is not matched with any track, initialize a new track and assign a new track ID; For the track that is not matched to the detection box for a continuous preset number of frames, determine that the target has left the monitoring area and delete it.

[0014] Further need to explain is, the extraction of human posture key point coordinates specifically includes: For the visitor who has been assigned a unique track ID, the corresponding visitor sub-image is cropped from the original image of the current frame according to its bounding box coordinates; Scale and normalize the visitor sub-image to the size of 256x256 pixels to obtain the standard visitor sub-image; Input the standard visitor sub-image into the human posture estimation model; The human posture estimation model outputs the pixel coordinates of the human key points and the confidence of each key point, including the tip of the nose, eyes, ears, shoulders, elbows, wrists, hips, knees and ankles; Map the pixel coordinates of each key point back to the original frame coordinate system and bind them with the corresponding visitor's track ID.

[0015] Further need to explain is, the human posture estimation model is HRNet model, HRNet model maintains high-resolution feature representation through the following network structure: A high-resolution convolutional subnetwork is used as the first stage; The subnetworks from high resolution to low resolution are added step by step to form multiple stages, in which the subnetworks of different resolutions are connected in parallel; The cross-resolution information fusion is repeatedly performed between the multiple resolution subnetworks in parallel; The feature maps output by all resolution subnetworks are finally aggregated to predict the key point heat map.

[0016] Further, after obtaining the key point coordinates, the key point coordinates are smoothed by using a sliding window time filtering algorithm, and the key point coordinates with a confidence lower than a preset threshold are compensated or interpolated.

[0017] Further, the posture features include: body tilt angle , which is calculated by the angle between the line connecting the center point of the shoulder and the center point of the hip and the vertical direction, wherein the center point of the shoulder is the midpoint of the line connecting the two shoulder key points, and the center point of the hip is the midpoint of the line connecting the two hip key points; key point height change amount , which is calculated by the height difference of the key points between consecutive frames key point falling speed , which is calculated by the key point height change amount per unit time.

[0018] Further, the posture collapse judgment rule is that at least two of the following conditions are met: body tilt angle is greater than a preset angle threshold ; key point height change amount is greater than a preset height change threshold ; and the key point falling speed of the head is greater than a preset speed threshold .

[0019] Further, the angle threshold has a value range of 40°-50°.

[0020] Further, the continuous trajectories of each visitor are time fused, and real fall events are determined based on the determination of suspected posture collapse, specifically including: A fixed-length time window is constructed for the continuous trajectory of each visitor, and the posture features and suspected collapse markers of each frame in the window are saved; In each time window, the number of frames marked as suspected posture collapse by the visitor is counted ; When the number of frames marked as suspected posture collapse is greater than a preset threshold , it is determined that a real fall event occurs.If the height of the tourist's key points has not returned to the normal range, it is determined that a real fall has occurred; in, This is the preset frame rate threshold.

[0021] It should be further noted that the length of the time window is 15-30 frames, and the frame number threshold... The value is 30%-50% of the number of frames in the time window.

[0022] It should be further noted that the criterion for determining whether the height of the key points has not returned to the normal range is: the height of the key points of the head continuously decreases by more than a preset height change threshold. .

[0023] It should be further noted that it also includes: A comprehensive judgment is made on the detection results of multiple consecutive time windows; When frames corresponding to suspected posture collapse are continuously distributed across multiple consecutive windows, it is determined that a real fall event has occurred.

[0024] It should also be noted that this includes a movement speed filtering mechanism: If a tourist is detected to be moving horizontally continuously at a speed greater than a preset speed threshold, it will not be considered a real fall event.

[0025] It should be further noted that the preset platform areas matched based on the location of the actual fall incident include: The cableway platform is zoned and divided into multiple predefined areas, including the upper hatch area, lower hatch area, carriage travel path area, hazardous equipment area, waiting area entrance and general platform area. Each predefined area is represented by a polygon boundary. The center of the bounding box corresponding to the tourist who actually fell is used as the location coordinate of the actual fall. The location coordinates are compared with the boundaries of each predefined area to determine whether the point is inside the polygon, thus determining the predefined area where the actual fall occurred.

[0026] It should be further explained that the output risk level specifically classifies fall incidents according to the degree of danger in the area, where: Level 1 high risk corresponds to actual falls occurring in the upper hatch area, lower hatch area, carriage travel path area, or dangerous equipment area. Level 2 medium risk corresponds to actual fall events occurring at the entrance of the waiting area or in densely populated areas. Densely populated areas are predefined regions where the number of tourists identified in the current frame exceeds a preset threshold. Level 3 low risk corresponds to actual falls occurring in general platform areas.

[0027] It should be further noted that the corresponding control commands that trigger the commands include: For a Level 1 high-risk event, a deceleration or stop command is sent to the cableway control system; For Level 2 medium-risk events, trigger audible and visual alarms and platform voice prompts; For Level 3 low-risk events, record the event and notify the station staff on duty.

[0028] It should be further noted that the deceleration or stopping command is sent to the cableway control system via the TCP protocol.

[0029] It should also be noted that a security verification mechanism is included: The system requires double confirmation or locking logic to prevent accidental triggering; All linked commands and actual fall events are synchronously recorded in the log system.

[0030] It should be further noted that the information recorded by the log system includes: Event time, location, video frame number, and risk level; The result of the control command execution.

[0031] It should also be noted that a closed-loop confirmation mechanism is included: If a deceleration or stop command is issued and the cableway control system returns the execution result, the execution result is recorded and confirmed in a closed loop with the issued deceleration or stop command. If no response is received after issuing a deceleration or stop command, a secondary trigger will be executed or manual intervention will be notified.

[0032] Secondly, this application provides a visual recognition-based cable car platform passenger fall detection system to implement the aforementioned cable car platform passenger fall detection method, including: The tourist trajectory generation module is used to collect video streams from the platform area and perform pedestrian detection on them, obtain the bounding box positions of tourists, and use a multi-object tracking algorithm to generate a unique trajectory ID and continuous trajectory data for each tourist. The suspected posture collapse determination module is used to extract the coordinates of key points of human posture in each frame of the video stream for each tourist and calculate posture features. When the posture features meet the preset posture collapse judgment rules, it is determined as a suspected posture collapse. The real fall event determination module is used to perform temporal fusion of the continuous trajectory of each tourist and to determine real fall events based on the judgment of suspected posture collapse. The location matching and command triggering module is used to match the preset platform area based on the location of the actual fall event, output the risk level, and trigger the corresponding control command.

[0033] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for detecting passenger falls at cableway platforms.

[0034] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for detecting passenger falls at cableway platforms.

[0035] As can be seen from the above technical solutions, this application has the following advantages: 1. This application achieves fully automatic and accurate identification and graded response to passenger falls on cable car platforms by acquiring platform video streams, performing pedestrian detection and tracking, extracting key points of human posture and calculating posture features to identify suspected posture collapses, then determining actual fall events through temporal fusion, and finally matching preset areas based on event locations and outputting risk levels and linkage control commands. This fundamentally solves the core problem that existing technologies cannot effectively identify posture anomalies and are difficult to achieve precise linkage control, enabling the system to replace traditional monitoring methods that rely on manual labor or simple sensors, significantly improving the automation level and real-time intervention capability of platform safety management.

[0036] 2. This application can accurately identify the rapid posture collapse characteristics from standing to touching the ground by extracting and analyzing the key point coordinates of tourists' human postures, effectively distinguishing falls from normal movements such as bending over and squatting, thus solving the problem that existing technologies cannot accurately identify the specific abnormal behavior of "falling".

[0037] 3. By constructing a continuous trajectory for each tourist and performing temporal fusion judgment, this application can effectively filter out single-frame misjudgments caused by momentary occlusion or light and shadow interference, thereby improving the stability and robustness of fall detection in complex and ever-changing actual platform environments.

[0038] 4. This application automatically outputs different levels of risk categories by matching the location of the fall incident with the platform area, realizing a leap from simple alarm to risk assessment, and providing a key decision-making basis for subsequent precise safety linkage. Attached Figure Description

[0039] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1This is a flowchart of a visual recognition-based method for detecting tourist falls at cable car platforms, as described in one embodiment of this application.

[0041] Figure 2 This is a schematic block diagram of a cable car platform passenger fall detection system based on visual recognition, as described in one embodiment of this application.

[0042] Figure 3 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation

[0043] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The following describes in detail the method for detecting passenger falls at cable car platforms according to this application. Specific details, such as particular system structures and technologies, are presented for illustrative purposes and not for limitation, to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0045] In the cableway platform fall detection method disclosed in this application, the term "comprising" indicates the presence of the described feature, whole, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0046] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0047] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0048] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0049] The cableway platform fall detection method provided in this application embodiment is executed by computer equipment. Correspondingly, the cableway platform fall detection system based on vision recognition runs in the computer equipment.

[0050] Figure 1 This is a flowchart of a visual recognition-based method for detecting passenger falls at cable car platforms, according to one embodiment of this application. Figure 1 The implementing entity could be a cable car platform visitor fall detection system. Depending on different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

[0051] like Figure 1 As shown, the visual recognition-based method for detecting passenger falls on cable car platforms includes: Step S1: Collect video streams of the platform area and perform pedestrian detection on them to obtain the bounding box positions of tourists. Then, use a multi-object tracking algorithm to generate a unique trajectory ID and continuous trajectory data for each tourist.

[0052] By acquiring video streams from the platform area and performing pedestrian detection, the bounding box positions of tourists are obtained. A multi-object tracking algorithm is then used to generate a unique trajectory ID and continuous trajectory data for each tourist, enabling continuous and accurate tracking. This avoids identity switching or loss due to occlusion or overlapping movements in crowded environments, ensuring consistent monitoring of each tourist's movement trajectory and behavior. It provides a reliable, individualized data source for fall detection and improves the system's tracking stability and data integrity in complex scenarios. In some specific embodiments, multiple cameras deployed at the cableway platform are used to capture video streams of the platform area, with the cameras covering the waiting area, upper hatch, lower hatch, deceleration section, and departure passage of the cableway platform.

[0053] By using multiple cameras deployed at the cableway station to collect video streams of the station area, the cameras' shooting range covers the waiting area, upper cabin, lower cabin, deceleration section of the car, and departure passage of the car, ensuring no blind spots in monitoring and fully covering the key areas of the station. It can capture all high-risk locations where falls may occur, providing a complete visual data foundation for comprehensive and accurate fall detection, and improving the overall coverage and reliability of the system.

[0054] In some specific embodiments, after acquiring the video stream of the platform area, the video stream is preprocessed. The preprocessing includes at least one of image decoding, frame format normalization, illumination compensation, noise filtering, and region cropping.

[0055] By preprocessing the video stream, including at least one of image decoding, frame format normalization, illumination compensation, noise filtering, and region cropping, the quality and consistency of the video images are improved, and the interference of environmental factors such as illumination changes and noise on subsequent pedestrian detection and pose estimation is reduced. This improves the accuracy and robustness of the entire system and ensures the stability of the algorithm under varying conditions.

[0056] In some specific embodiments, pedestrian detection of the video stream specifically includes: Each frame of the video stream is input into the YOLOv8 model, which outputs several candidate bounding boxes. Each candidate bounding box contains a class label, bounding box coordinates, and a confidence score. Set a first threshold to filter candidate detection boxes whose confidence scores are lower than the first threshold; Non-maximum suppression is performed on the remaining candidate detection boxes after filtering. When the intersection-union ratio of two candidate detection boxes is greater than the preset second threshold, only the candidate detection boxes with higher confidence scores are retained, thus obtaining the set of bounding boxes of all pedestrians in the current frame.

[0057] By inputting the image of each frame in the video stream into the YOLOv8 model to output several candidate detection boxes, setting a first threshold to filter low-confidence detection boxes, and performing non-maximum suppression operation, high-confidence boxes are retained when the intersection-union ratio is greater than the second threshold. This achieves efficient and accurate pedestrian detection, reduces false detections and missed detections, and ensures that only reliable detection results enter the subsequent tracking stage through a dual filtering mechanism, thereby improving the basic detection performance and overall efficiency of the system.

[0058] In some specific embodiments, the DeepSORT algorithm is used as a multi-target tracking algorithm to generate a unique trajectory ID and continuous trajectory data for each tourist. The execution steps of the DeepSORT algorithm in each frame include: Using a pre-trained appearance feature extraction network, appearance feature vectors are extracted from pedestrian images cropped from each pedestrian detection box; If the current frame is the initial frame, then the position of each pedestrian in the current frame is taken as the starting point of the pedestrian's trajectory; If the current frame is not the initial frame, the Kalman filter is used to predict the state of the existing trajectories in the previous frame, and the predicted position of each trajectory in the current frame is obtained to generate the trajectory prediction box. Calculate the motion Mahalanobis distance and appearance feature cosine distance between the detection box and the trajectory prediction box in the current frame; Based on the motion Mahalanobis distance and the appearance feature cosine distance, the association between the detection box of the current frame and the existing trajectory is completed through cascaded matching and Hungarian algorithm.

[0059] By utilizing the DeepSORT algorithm as a multi-target tracking algorithm, appearance feature vectors are extracted in each frame, state prediction is performed using a Kalman filter, motion Mahalanobis distance and appearance feature cosine distance are calculated, and the association between the detection box and the trajectory is completed through cascaded matching and Hungarian algorithm, achieving stable and continuous multi-target tracking. Even in complex scenes, it can effectively handle identity switching and occlusion problems, maintain trajectory consistency, provide reliable trajectory IDs for subsequent pose analysis, and enhance the tracking robustness of the system.

[0060] In some specific embodiments, the appearance feature extraction network is a deep convolutional neural network pre-trained on a pedestrian re-identification dataset, whose input is a scaled pedestrian image and whose output is a fixed-dimensional feature vector.

[0061] By using a deep convolutional neural network pre-trained on a pedestrian re-identification dataset as the appearance feature extraction network, the input scaled pedestrian image outputs a fixed-dimensional feature vector, providing a highly discriminative and robust appearance representation. This can accurately identify the differences between different pedestrians, reduce identity confusion during the tracking process, enhance the accuracy and stability of multi-target tracking, and ensure the consistency and comparability of feature extraction.

[0062] In some specific embodiments, the appearance feature extraction network is trained in the following manner: The training was conducted using a large-scale pedestrian re-identification dataset, with the goal of minimizing the cosine distance between feature vectors extracted from different images of the same pedestrian and maximizing the cosine distance between feature vectors of different pedestrians.

[0063] By training an appearance feature extraction network using a large-scale pedestrian re-identification dataset, the training objective is to minimize the cosine distance between feature vectors extracted from different images of the same pedestrian and maximize the cosine distance between feature vectors of different pedestrians. This optimizes the discriminative ability of the feature extraction network, enabling it to generate highly discriminative feature representations. This effectively reduces identity switching errors and improves tracking accuracy and system reliability in multi-target tracking.

[0064] In some specific embodiments, the trajectory management mechanism of the DeepSORT algorithm includes: For trajectories that successfully match the detection box, update their bounding box position, motion speed, and appearance features, while maintaining the original trajectory ID; For a detection bounding box that does not match any trajectory, initialize a new trajectory and assign a new trajectory ID; For a trajectory that fails to match a detection box for a consecutive preset number of frames, the target is determined to have left the monitoring area and is deleted.

[0065] By implementing the trajectory management mechanism of the DeepSORT algorithm, including updating the bounding box position, movement speed, and appearance features of the matched trajectory, initializing new detection boxes as new trajectories, and deleting consecutive unmatched trajectories, the lifecycle of the trajectory is dynamically maintained. This allows for timely handling of newly entering and leaving tourists, avoiding trajectory accumulation and resource waste, ensuring the efficiency and real-time performance of the tracking system, and improving the accuracy of long-term tracking and resource management efficiency.

[0066] Step S2: For each tourist, extract the coordinates of their human posture key points in each frame of the video stream and calculate the posture features. When the posture features meet the preset posture collapse judgment rules, it is judged as a suspected posture collapse.

[0067] By extracting the coordinates of key points of each tourist's human posture in each frame of the video stream and calculating posture features, when the posture features meet the preset posture collapse judgment rules, it is judged as a suspected posture collapse. It can capture changes in tourists' body posture in real time and preliminarily identify possible fall events based on objective rules, reducing the error of subjective judgment, improving the timeliness and accuracy of fall detection, providing key clues for subsequent confirmation of fall events, and enhancing the system's sensitivity and response capability to abnormal postures.

[0068] In some specific embodiments, extracting the coordinates of key points in human posture includes: For tourists who have been assigned a unique trajectory ID, the corresponding tourist sub-image is cropped from the original image of the current frame based on their bounding box coordinates; The tourist sub-image is scaled and normalized to a size of 256×256 pixels to obtain the standard tourist sub-image; Input standard tourist sub-images into the human pose estimation model; The human pose estimation model outputs the pixel coordinates of human key points and the confidence score of each key point. Key points include the tip of the nose, both eyes, both ears, both shoulders, both elbows, both wrists, both hips, both knees and both ankles. Map the pixel coordinates of each key point back to the original frame coordinate system and bind them to the corresponding tourist's trajectory ID.

[0069] By cropping the corresponding tourist sub-image from the original image of the current frame based on the bounding box coordinates of each tourist with a unique trajectory ID, scaling and normalizing it to a size of 256×256 pixels, inputting it into the human pose estimation model, outputting key point coordinates, and mapping it back to the original coordinate system, accurate human pose key point detection is achieved. This provides detailed body part location information for each tourist, thereby supporting accurate pose feature calculation and fall judgment, and ensuring the accuracy and usability of key point data.

[0070] In some specific embodiments, the human pose estimation model is the HRNet model, which maintains high-resolution feature representation through the following network structure: Use a high-resolution convolutional subnetwork as the first stage; Subnetworks of varying resolutions are added progressively, from high resolution to low resolution, forming multiple stages in which subnetworks of different resolutions are connected in parallel. Cross-resolution information fusion is repeatedly performed between multiple parallel resolution sub-networks; Finally, the feature maps output by all resolution sub-networks are aggregated to predict keypoint heatmaps.

[0071] By using the HRNet model as the human pose estimation model, its network structure maintains high-resolution feature representation, including high-resolution convolutional sub-networks, parallel multi-resolution sub-networks, and cross-resolution information fusion. Finally, the feature map is aggregated to predict the key point heatmap, which improves the accuracy and robustness of key point detection. Especially in complex poses or occlusion situations, it can preserve detailed information, reduce key point localization errors, and enhance the reliability of pose estimation.

[0072] In some specific embodiments, after obtaining the key point coordinates, the key point coordinates are smoothed. The smoothing process uses a sliding window time filtering algorithm to compensate or interpolate the key point coordinates with a confidence level lower than a preset threshold.

[0073] By smoothing the keypoint coordinates after acquisition and using a sliding window time filtering algorithm to compensate or interpolate the keypoint coordinates with confidence levels below a preset threshold, the jitter and noise of keypoint detection are reduced, and the stability and reliability of coordinate data are improved. This makes the pose feature calculation based on keypoints more accurate, reduces the risk of misjudgment, and ensures the continuity and consistency of pose analysis.

[0074] In some specific embodiments, the pose features include: Body tilt angle The angle between the line connecting the center point of the shoulder and the center point of the hip and the vertical direction is calculated, where the center point of the shoulder is the midpoint of the line connecting the two key points of the shoulder, and the center point of the hip is the midpoint of the line connecting the two key points of the hip. Change in height of key points Calculated by the height difference of key points between consecutive frames Key point descent speed It is calculated by the change in height of key points per unit time.

[0075] By defining posture features including body tilt angle, keypoint height change, and keypoint descent speed, and calculating the body tilt angle by the angle between the line connecting the shoulder center point and the hip center point and the vertical direction, calculating the height change by the height difference between consecutive frames, and calculating the descent speed by the height change of keypoints per unit time, the changes in tourists' body posture are quantified. This provides objective indicators for fall detection, reduces subjectivity, improves the scientific nature and repeatability of the detection, and ensures the accuracy and practicality of posture feature calculation.

[0076] In some specific embodiments, the attitude collapse judgment rule is to simultaneously satisfy at least two of the following conditions: Body tilt angle greater than the preset angle threshold ; Change in height of key points Greater than the preset height change threshold ; and the speed of descent of the key points of the head Speed ​​greater than the preset threshold .

[0077] By setting the posture collapse judgment rule to simultaneously meet at least two of the following conditions: body tilt angle greater than the angle threshold, key point height change greater than the height change threshold, and head key point descent speed greater than the speed threshold, the sensitivity and specificity of detection are balanced, false alarms caused by fluctuations in a single indicator are avoided, the accuracy and reliability of fall recognition are improved, and it is ensured that only genuine posture collapses are marked as suspected falls.

[0078] In some specific embodiments, the angle threshold The value range is 40°-50°.

[0079] By setting the angle threshold to a range of 40°-50°, a scientific and verified critical value for body tilt angle is provided, which can effectively distinguish between normal posture and severe tilt related to falls, optimize the accuracy of posture collapse judgment, avoid missed or false detections caused by thresholds that are set too high or too low, and improve the system's ability to capture real fall events.

[0080] Step S3: Perform temporal fusion of the continuous trajectory of each tourist and determine the actual fall event based on the judgment of suspected posture collapse.

[0081] By performing temporal fusion on the continuous trajectories of each tourist and determining actual fall events based on suspected posture collapse, the system combines multi-frame data from the time series to verify suspected events, reducing false alarms caused by instantaneous actions or noise. By comprehensively analyzing trajectory and posture information, the system improves the reliability and stability of fall event determination, ensuring that only continuous and real falls are confirmed, thereby reducing the false alarm rate and improving the system's judgment accuracy.

[0082] In some specific embodiments, the continuous trajectory of each tourist is fused temporally, and the actual fall event is determined based on the judgment of suspected posture collapse, specifically including: A fixed-length time window is constructed for each tourist's continuous trajectory, and the posture features and suspected collapse markers of each frame are saved. Within each time window, count the number of frames marked by the tourist as potentially exhibiting posture collapse. ; when If the height of the tourist's key points has not returned to the normal range, it is determined that a real fall has occurred; in, This is the preset frame rate threshold.

[0083] By constructing a fixed-length time window for each tourist's continuous trajectory, saving the posture features and suspected collapse markers of each frame, and counting the number of frames suspected of posture collapse within the time window, a real fall event is determined when the number of frames exceeds a threshold and the height of the key point is not restored. This introduces time persistence verification to ensure that the fall event is continuous rather than instantaneous, reducing false judgments of brief actions, improving the reliability of the judgment, and enhancing the system's ability to confirm real events.

[0084] In some specific embodiments, the length of the time window is 15-30 frames, and the frame number threshold is... The value is 30%-50% of the number of frames in the time window.

[0085] By setting the time window length to 15-30 frames and the frame number threshold to 30%-50% of the time window frame number, a reasonable time range and data ratio are provided for event determination, balancing response speed and accuracy, avoiding delays caused by excessively long windows or misjudgments caused by excessively short windows, optimizing the timing analysis performance of fall detection, and ensuring the best trade-off between real-time performance and reliability of the system.

[0086] In some specific embodiments, the criterion for determining whether the height of the key points has not returned to the normal range is: the height of the key points of the head continuously decreases by more than a preset height change threshold. .

[0087] By setting the criterion for determining whether the height of the key points has not returned to the normal range as a continuous drop in the height of the head's key points exceeding a preset height change threshold, a clear height recovery verification condition is provided. This ensures that only a continuous descent rather than a recovery posture is recognized as a real fall, further reducing false alarms, improving detection accuracy, and enhancing the system's ability to capture the fall process.

[0088] In some specific embodiments, it also includes: A comprehensive judgment is made on the detection results of multiple consecutive time windows; When frames corresponding to suspected posture collapse are continuously distributed across multiple consecutive windows, it is determined that a real fall event has occurred.

[0089] By comprehensively judging the detection results of multiple consecutive time windows, when the frames corresponding to suspected posture collapse are continuously distributed in multiple consecutive windows, it is determined that a real fall event has occurred. This enhances the robustness of event judgment, can handle continuous falls across windows, avoids missed detections caused by window segmentation, ensures accurate identification of long-term fall events, and improves the system's adaptability to complex scenarios.

[0090] In some specific embodiments, a motion speed filtering mechanism is also included: If a tourist is detected to be moving horizontally continuously at a speed greater than a preset speed threshold, it will not be considered a real fall event.

[0091] By introducing a movement speed filtering mechanism, when a tourist is detected to be moving horizontally continuously at a speed greater than a preset movement speed threshold, it is not considered a real fall event. This distinguishes between falls and normal rapid movement, reduces false alarms caused by running or fast walking, improves the specificity of fall detection, and ensures that the system focuses on real dangerous situations.

[0092] Step S4: Match the preset platform area according to the location of the actual fall event, output the risk level and trigger the corresponding control command.

[0093] By matching the location of a real fall event with a preset platform area, outputting the risk level and triggering corresponding control commands, the system can automatically assess the degree of risk based on the specific location of the fall and execute corresponding emergency measures, such as slowing down, stopping, or issuing an alarm. This enables differentiated handling of fall events with different levels of danger, optimizes resource allocation, improves the efficiency and targeting of platform safety management, and ensures a rapid and appropriate response to minimize the impact of accidents.

[0094] In some specific embodiments, matching a preset platform area based on the location of the actual fall event includes: The cableway platform is zoned and divided into multiple predefined areas, including the upper hatch area, lower hatch area, carriage travel path area, hazardous equipment area, waiting area entrance and general platform area. Each predefined area is represented by a polygon boundary. The center of the bounding box corresponding to the tourist who actually fell is used as the location coordinate of the actual fall. The location coordinates are compared with the boundaries of each predefined area to determine whether the point is inside the polygon, thus determining the predefined area where the actual fall occurred.

[0095] By zoning the cableway platform and dividing it into multiple predefined areas using polygonal boundaries, the location of a fall event is determined by comparing it with the boundaries of each area to see if the point is inside the polygon. This enables precise location and classification of fall events, providing spatial context for risk assessment and response measures, and enhancing the pertinence and efficiency of safety management.

[0096] In some specific embodiments, the output risk level is specifically to automatically classify fall events according to the degree of danger in the area, wherein: Level 1 high risk corresponds to actual falls occurring in the upper hatch area, lower hatch area, carriage travel path area, or dangerous equipment area. Level 2 medium risk corresponds to actual fall events occurring at the entrance of the waiting area or in densely populated areas. Densely populated areas are predefined regions where the number of tourists identified in the current frame exceeds a preset threshold. Level 3 low risk corresponds to actual falls occurring in general platform areas.

[0097] By automatically classifying fall incidents according to the degree of danger in each area, Level 1 (high risk) corresponds to the upper and lower hatches, the carriage travel path, or areas with dangerous equipment; Level 2 (medium risk) corresponds to the entrance of the waiting area or densely populated areas; and Level 3 (low risk) corresponds to general platform areas. This achieves scientific risk assessment, ensures that resources are prioritized for high-risk incidents, optimizes emergency response strategies, and improves the efficiency and effectiveness of platform safety management.

[0098] In some specific embodiments, the corresponding control commands that trigger the commands include: For a Level 1 high-risk event, a deceleration or stop command is sent to the cableway control system; For Level 2 medium-risk events, trigger audible and visual alarms and platform voice prompts; For Level 3 low-risk events, record the event and notify the station staff on duty.

[0099] By sending deceleration or stopping commands to the cableway control system for Level 1 high-risk events, triggering audible and visual alarms and platform voice prompts for Level 2 medium-risk events, and recording the event and notifying platform staff for Level 3 low-risk events, a tiered response is achieved. This ensures that fall events of different risk levels are handled appropriately, avoiding overreaction while guaranteeing timely intervention, and improving the accuracy and practicality of the safety response.

[0100] In some specific embodiments, deceleration or stopping commands are sent to the cableway control system via the TCP protocol.

[0101] By using the TCP protocol to send deceleration or stopping commands to the cableway control system, the reliability and real-time performance of command transmission are ensured. Stable communication between systems is achieved using standard network protocols, reducing the risk of command loss or delay, thereby improving the accuracy of control response and the overall reliability of the system.

[0102] In some specific embodiments, a security verification mechanism is also included: The system requires double confirmation or locking logic to prevent accidental triggering; All linked commands and actual fall events are synchronously recorded in the log system.

[0103] By introducing a safety confirmation mechanism, requiring double confirmation or locking logic to avoid accidental triggering, and synchronously recording all linkage commands and actual fall events in the log system, the system's security and reliability are improved, preventing accidental shutdowns or alarms caused by misoperation. At the same time, a complete audit trail is provided for post-event analysis, enhancing the system's credibility and maintainability.

[0104] In some specific embodiments, the information recorded by the log system includes: Event time, location, video frame number, and risk level; The result of the control command execution.

[0105] By recording event time, location, video frame number, risk level, and control command execution results through the log system, comprehensive historical data records are provided, which facilitates post-event analysis, system evaluation, and troubleshooting, supports continuous improvement and compliance checks, and provides a solid data foundation for system optimization and operation management.

[0106] In some specific embodiments, a closed-loop confirmation mechanism is also included: If a deceleration or stop command is issued and the cableway control system returns the execution result, the execution result is recorded and confirmed in a closed loop with the issued deceleration or stop command. If no response is received after issuing a deceleration or stop command, a secondary trigger will be executed or manual intervention will be notified.

[0107] Through a closed-loop confirmation mechanism, if the cableway control system returns an execution result after issuing a deceleration or stop command, it records and confirms the result in a closed loop. If no response is received, a secondary trigger is executed or manual intervention is notified. This ensures the reliability of the execution of control commands, avoids security vulnerabilities caused by communication failures or system malfunctions, and improves the integrity of emergency response and system robustness.

[0108] The following are embodiments of a cable car platform fall detection system based on visual recognition provided in this application. This cable car platform fall detection system based on visual recognition belongs to the same inventive concept as the cable car platform fall detection methods in the above embodiments. For details not described in detail in the embodiments of the cable car platform fall detection system, please refer to the embodiments of the above-mentioned cable car platform fall detection methods based on visual recognition.

[0109] like Figure 2 As shown, the visual recognition-based cable car platform fall detection system includes: The tourist trajectory generation module is used to collect video streams from the platform area and perform pedestrian detection on them, obtain the bounding box positions of tourists, and use a multi-object tracking algorithm to generate a unique trajectory ID and continuous trajectory data for each tourist. The suspected posture collapse determination module is used to extract the coordinates of key points of human posture in each frame of the video stream for each tourist and calculate posture features. When the posture features meet the preset posture collapse judgment rules, it is determined as a suspected posture collapse. The real fall event determination module is used to perform temporal fusion of the continuous trajectory of each tourist and to determine real fall events based on the judgment of suspected posture collapse. The location matching and command triggering module is used to match the preset platform area based on the location of the actual fall event, output the risk level, and trigger the corresponding control command.

[0110] The cableway platform passenger fall detection system in this embodiment is used to implement a visual recognition-based method for detecting passenger falls on cableway platforms.

[0111] This application also provides an electronic device for implementing the various embodiments of this application. Figure 3 To illustrate the hardware structure of an electronic device according to various embodiments of this application, as shown in the following diagram... Figure 3 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor.

[0112] Those skilled in the art will understand that the electronic device structure involved in the embodiments of this application does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0113] In embodiments of this application, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0114] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.

[0115] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.

[0116] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0117] This application also provides a storage medium storing a program product capable of implementing a vision-based method for detecting passenger falls at cable car platforms. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the foregoing "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.

[0118] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visual recognition-based ropeway station passenger fall detection method, characterized in that, The method comprises the following steps: Collecting a video stream of a platform area and performing pedestrian detection thereon to obtain a bounding box position of a tourist, and generating a unique track ID and continuous track data for each tourist by using a multi-target tracking algorithm; For each tourist, extracting the human pose key point coordinates in each frame of the video stream and calculating the pose feature, and determining a suspected pose collapse when the pose feature meets a preset pose collapse judgment rule; Performing time series fusion on the continuous track of each tourist, and determining a real fall event based on the determination of suspected pose collapse; Matching the position of the real fall event with a preset platform area, outputting a risk level, and triggering a corresponding control instruction.

2. The ropeway station visitor fall detection method according to claim 1, characterized in that, The pedestrian detection on the video stream specifically comprises: Inputting the image of each frame in the video stream into a YOLOv8 model to output a plurality of candidate detection boxes, each candidate detection box containing a class label, a bounding box coordinate and a confidence score Setting a first threshold, filtering the candidate detection boxes with a confidence score lower than the first threshold; Performing a non-maximum suppression operation on the remaining candidate detection boxes after filtering, and when the intersection over union of two candidate detection boxes is greater than a preset second threshold, only retaining the candidate detection box with a higher confidence score to obtain a bounding box set of all pedestrians in the current frame.

3. The ropeway station visitor fall detection method according to claim 1, characterized in that, The DeepSORT algorithm is used as the multi-target tracking algorithm to generate a unique track ID and continuous track data for each tourist. The execution steps of the DeepSORT algorithm in each frame include: Using a pre-trained appearance feature extraction network to extract an appearance feature vector from the pedestrian image cropped from each pedestrian detection box; If the current frame is an initial frame, the position of each pedestrian in the current frame is taken as the starting point of the pedestrian track; If the current frame is not an initial frame, a Kalman filter is used to predict the state of the existing track in the previous frame to obtain the predicted position of each track in the current frame, and a track prediction box is generated; Calculate the motion Mahalanobis distance and appearance feature cosine distance between the detection box of the current frame and the track prediction box; Based on the motion Mahalanobis distance and appearance feature cosine distance, the association between the detection box of the current frame and the existing track is completed through cascading matching and the Hungarian algorithm.

4. The ropeway station visitor fall detection method according to claim 1, characterized in that, The pose feature includes: Body inclination angle is calculated by the angle between the line connecting the center point of the shoulder and the center point of the hip and the vertical direction, wherein the center point of the shoulder is the midpoint of the line connecting the two shoulder key points, and the center point of the hip is the midpoint of the line connecting the two hip key points. Key point height change amount By calculating the key point height difference between consecutive frames Key point descent speed is calculated by the amount of change in the height of the key point per unit time.

5. The ropeway station visitor fall detection method according to claim 1, characterized in that, Performing time series fusion on the continuous track of each tourist, and determining a real fall event based on the determination of suspected pose collapse, specifically comprising: Constructing a fixed-length time window for the continuous track of each tourist, and saving the pose feature and suspected collapse label of each frame in the time window; counting the number of frames within each time window that are labeled by the tourist as suspected pose collapse ; When , and the key point height of the visitor is not restored to the normal range, it is determined that a real fall event occurs; wherein, is a preset frame number threshold.

6. The ropeway station visitor fall detection method according to claim 1, characterized in that, Matching the position of the real fall event with a preset platform area includes: Region calibration is performed on the cableway platform, and the platform is divided into a plurality of predefined regions, including an upper hatch area, a lower hatch area, a car travel path area, a dangerous equipment area, a waiting area entrance and a general platform area. Each predefined region is represented by a polygon boundary; The center of the bounding box corresponding to the tourist who has a real fall event is taken as the position coordinate of the real fall event, and the position coordinate is used to determine whether the point is inside the polygon by comparing the boundary of each predefined region, to determine the predefined region where the real fall event occurs.

7. The ropeway station visitor fall detection method according to claim 6, characterized in that, The output risk level is specifically an automatic classification of the fall event according to the area danger level, wherein: The first high risk corresponds to a real fall event occurring in the upper hatch area, the lower hatch area, the vehicle compartment travel path area, or the dangerous equipment area; The second medium risk corresponds to a real fall event occurring in the waiting area entrance or the crowded area, and the crowded area is a predefined area in the current frame where the number of identified tourists exceeds a preset threshold; The third low risk corresponds to a real fall event occurring in the general platform area. 8.A visual recognition-based ropeway station passenger fall detection system, characterized in that, The method for detecting tourist falls at a cableway platform as claimed in any one of claims 1-7, comprising: a tourist trajectory generation module for collecting video streams of the platform area and performing pedestrian detection thereon, obtaining the bounding box position of the tourists, and generating a unique trajectory ID and continuous trajectory data for each tourist using a multi-target tracking algorithm; a suspected posture collapse determination module for extracting the human body posture key point coordinates of each tourist in each frame of the video stream and calculating the posture feature, and determining suspected posture collapse when the posture feature meets the preset posture collapse determination rule; a real fall event determination module for performing time series fusion on the continuous trajectory of each tourist, and determining a real fall event based on the determination of suspected posture collapse; a position matching and instruction triggering module for matching the position of the real fall event with the preset platform area, outputting the risk level, and triggering the corresponding control instruction.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor is used to execute the computer program to realize the steps of the cableway platform tourist fall detection method as claimed in any one of claims 1-7.

10. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the steps of the cableway platform tourist fall detection method as claimed in any one of claims 1-7.

Images