Escalator passenger abnormal behavior detection method

Abstract

The invention discloses an escalator passenger abnormal behavior detection method, and the method comprises the steps: obtaining escalator scene passenger image data which at least comprises squatting, falling, head stretching and hand stretching pictures; the escalator scene passenger image data is labeled, labeling at least comprises posture types, human body frames and human body key point positions, a human body posture detection algorithm and the labeled image data are used for training a model, the positions of escalator floor plates and stairs are obtained through an image sensor, and the first target list is traversed; logic judgment is conducted on the relative positions or angles of the key points, multi-target tracking is conducted on the second target list, the tracking list is traversed, and when the occurrence time of a target event exceeds a set value, it is judged that the event occurs; when the event is judged to be squatting and hand extension according to the occurrence of the event, abnormal behaviors such as squatting, falling, head extension and hand extension of the passenger in the elevator can be accurately detected.

Description

Technical Field

[0001] The invention relates to the technical field of image recognition, in particular to a method for detecting abnormal behavior of escalator passengers. Background Art

[0002] With the development of social economy, escalators are becoming more and more popular. Shopping malls, subway stations, airports and other public places generally have escalators. Due to the dense population and high mobility, escalator safety accidents occasionally occur. Passenger abnormal behavior detection equipment based on video recognition has begun to be used in the field of escalators. Usually, passenger abnormal behavior detection includes squatting, falling, sticking out the head and hands, walking in the opposite direction, congestion, detention, carrying large items, etc.

[0003] CN117557949A An escalator safety device based on image recognition edge computing provides pre-protection against escalator safety incidents in three stages: the first stage is to identify whether the shoes worn by passengers are hole shoes, high heels and other shoes that are more likely to cause foot pinching accidents; the second stage is to define the warning area of ​​the safety yellow line area by analyzing the video code stream, and identify whether the passenger's feet have crossed the safety yellow line; the third stage is to perform AI deep learning on the leg posture of the passengers on the escalator through the analysis of the video code stream, and identify whether the passenger has fallen. The problem is that, first of all, many real fall incidents do not have obvious characteristics before they occur. The type of shoes worn and the feet exceeding the safety yellow line may be one of the reasons for the fall, but not all reasons. The effect of its early warning needs to be tested in practice. Thirdly, the characteristics of the fall identified in the third stage are judged by the legs, which is different from this patent.

[0004] CN117746311 A A passenger escalator dangerous behavior recognition method based on image tracking and deep learning adopts two posture detection models, optical flow algorithm and target tracking. It needs to detect each frame and fuse the results of the two posture detection models. It consumes a lot of computing power resources and is not suitable for use in small edge AI devices. Summary of the invention

[0005] The purpose of the present invention is to solve the above-mentioned problem and to design a method for detecting abnormal behavior of escalator passengers.

[0006] To achieve the above-mentioned purpose, the technical solution of the present invention further comprises the following steps in the above-mentioned method for detecting abnormal behavior of passengers on an escalator:

[0007] Acquire elevator passenger image data, wherein the elevator passenger image data at least includes pictures of squatting, falling, head sticking out, and hands sticking out;

[0008] Annotating the elevator passenger image data, wherein the annotation includes at least a posture category, a human body frame, and a human body key point position, to obtain annotated image data;

[0009] Using a human body posture detection algorithm and the labeled image data to train a model, to obtain a posture detection model;

[0010] Acquire the escalator floor plate and step positions through an image sensor to obtain a first target list, wherein the first target list at least includes a human frame position, a human posture category, and a human key point position;

[0011] Traversing the first target list, performing logical judgment on the relative positions or angles of key points, and obtaining a second target list;

[0012] Perform multi-target tracking on the second target list, record the time, ID and posture category of the target appearance, and obtain a tracking list;

[0013] The tracking list is traversed, and when the time for the target event to occur exceeds a set value, it is determined that the event has occurred.

[0014] Further, in the above-mentioned method for detecting abnormal behavior of escalator passengers, the escalator floor plate and step positions are obtained by an image sensor to obtain a first target list, and the first target list includes at least a human frame position, a human posture category and a human key point position, including:

[0015] The escalator floor plate and step positions are obtained through the image sensor, and the detection device reads the video stream of the camera to obtain the video stream image data;

[0016] Read the video stream image data at a set time interval, and decode the video stream image data into a picture to obtain picture data;

[0017] Performing reasoning on the image data using a posture detection model, wherein the output of the reasoning includes confidence, posture category, human frame position, and human key point position;

[0018] The key points of the human body at least include the center of the head, the left eye, the right eye, the left ear, the right ear, the left shoulder, the right shoulder, the left elbow, the right elbow, the left hand, the right hand, the left hip, the right hip, the left knee, the right knee, the left foot, and the right foot;

[0019] The targets whose confidence is greater than the threshold and are within the set floor plate and step area are valid targets, and a target list is obtained.

[0020] Furthermore, in the above-mentioned method for detecting abnormal behavior of escalator passengers, traversing the first target list, performing logical judgment on the relative positions or angles of key points, and obtaining the second target list includes:

[0021] Traversing the first target list, and performing logical judgment on the relative positions or angles of key points in the first target list;

[0022] When the posture category is squatting, the knee key point position of the human body must be higher than the hip key point position;

[0023] When the posture category is falling, the foot key point must be higher than the hip key point or the head key point must be lower than the shoulder key point;

[0024] When the posture is head extended, the angle between the key point of the head and the middle line of the human frame must be greater than the set angle;

[0025] When the gesture is that the hand is extended, the distance between the hand and the center line of the human body frame must be greater than the set value, and the target is retained; otherwise, the target is deleted to obtain the second target list.

[0026] Further, in the above-mentioned method for detecting abnormal behavior of escalator passengers, the second target list is subjected to multi-target tracking, and the time, ID and posture category of the target appearance are recorded to obtain a tracking list, including:

[0027] Perform multi-target tracking on the second target list, and record the time, ID, and posture category of the target appearance;

[0028] Targets that are not detected within the set frame interval n are deleted from the tracking list;

[0029] The purpose of setting the frame interval for deleting the tracking target here is to make the sensitivity of event detection adjustable;

[0030] When it is set that if one frame is not detected, the target in the list will be deleted. If the target frame is not detected continuously, it will be deleted. In this case, the sensitivity of event detection is low;

[0031] On the contrary, when the set frame interval number is larger, the system is allowed to consider that the target continues to exist when the target is not detected in the frame interval -1 frame. The sensitivity of event detection is higher and a tracking list is obtained.

[0032] Furthermore, in the above-mentioned method for detecting abnormal behavior of escalator passengers, traversing the tracking list and determining that an event occurs when the time for the target event to occur exceeds a set value includes:

[0033] Traversing the tracking list, when the time when the target event occurs exceeds a set value, it is determined that the event occurs;

[0034] The target events at least include squatting, falling, extending the head, and extending the hands; the set value is used to measure the duration of the event;

[0035] When the tracking list is empty, it indicates that no event has occurred. If the remote speed reduction flag is 1, a speed recovery command is sent to the escalator;

[0036] When the event is determined to be squatting or extending the hand according to the occurrence of the event, the warning level is set to general; when the event is falling or extending the head, the warning level is set to severe.

[0037] Further, in the above-mentioned method for detecting abnormal behavior of escalator passengers, when the event is determined to be squatting or hand extension according to the occurrence of the event, the warning level is set to general, and when the event is falling or head extension, the warning level is set to severe, including:

[0038] If the event is determined to be squatting or hand extension, the warning level is set to general; if the event is falling or head extension, the warning level is set to severe;

[0039] The warning level is sent to the sound and light warning module as serious, and a voice warning is broadcast to remind passengers. At the same time, an alarm record is generated and uploaded to the platform;

[0040] The management personnel can view the event pictures and videos in real time on the platform, and after manual inspection and confirmation, remotely operate the escalator to emergency stop, slow stop, speed reduction, and speed recovery on the platform interface;

[0041] When the event warning is serious, a speed reduction command is sent to the escalator to reduce the escalator running speed and set the remote speed reduction flag to 1.

[0042] Furthermore, a system for detecting abnormal behavior of escalator passengers is implemented, characterized in that the system comprises the following modules:

[0043] A data acquisition module, used to acquire elevator passenger image data, wherein the elevator passenger image data at least includes pictures of squatting, falling, head sticking out, and hand sticking out;

[0044] An image annotation module is used to annotate the elevator passenger image data, wherein the annotation includes at least a posture category, a human body frame and a human body key point position, to obtain annotated image data;

[0045] A model training module, used to train a model using a human posture detection algorithm and the labeled image data to obtain a posture detection model;

[0046] A target acquisition module, used to acquire the escalator floor plate and step positions through an image sensor to obtain a first target list, wherein the first target list at least includes a human frame position, a human posture category, and a human key point position;

[0047] A logic judgment module, used for traversing the first target list, performing logic judgment on the relative positions or angles of key points, and obtaining a second target list;

[0048] A target tracking module, used to track multiple targets in the second target list, record the time, ID and posture category of the target, and obtain a tracking list;

[0049] An event determination module is used to traverse the tracking list and determine that an event occurs when the time when the target event occurs exceeds a set value;

[0050] The event warning module is used to set the warning level to general when the event is determined to be squatting or hand extension according to the occurrence of the event, and to set the warning level to severe when the event is falling or head extension.

[0051] Furthermore, a system for detecting abnormal behavior of escalator passengers is implemented, characterized in that the system includes the following submodules:

[0052] The acquisition submodule is used to obtain the escalator floor plate and step position through the image sensor, and the detection device reads the video stream of the camera to obtain the video stream image data;

[0053] A decoding submodule, used for reading the video stream image data at a set time interval, and decoding the video stream image data into a picture to obtain picture data;

[0054] An inference submodule, used to perform inference on the image data using a posture detection model, wherein the inference output includes confidence, posture category, human frame position, and human key point position;

[0055] A key point submodule, used to determine the positions of the key points of the human body, including at least the center of the head, the left eye, the right eye, the left ear, the right ear, the left shoulder, the right shoulder, the left elbow, the right elbow, the left hand, the right hand, the left hip, the right hip, the left knee, the right knee, the left foot, and the right foot;

[0056] The judgment submodule is used to determine the targets whose confidence is greater than the threshold and are within the set floor plate and step area as valid targets, and obtain a target list.

[0057] Furthermore, a system for detecting abnormal behavior of escalator passengers is implemented, characterized in that the system includes the following submodules:

[0058] A traversal submodule, used for traversing the first target list and performing logical judgment on the relative positions or angles of key points in the first target list;

[0059] The first judgment submodule is used to judge that when the posture category is squatting, the position of the knee key point of the human body must be higher than the position of the hip key point;

[0060] The second judgment submodule is used to judge that when the posture category is falling, it must meet the requirements that the foot key point is higher than the hip key point or the head key point is lower than the shoulder key point;

[0061] The third judgment submodule is used to judge that when the posture is the head extending, the angle between the key point of the head and the middle line of the human frame must be greater than the set angle;

[0062] The fourth judgment submodule is used to judge that when the posture is the hand extending, the distance between the hand and the center line of the human body frame must be greater than a set value, and the target is retained; otherwise, the target is deleted to obtain a second target list.

[0063] The beneficial effect is that, by acquiring the image data of elevator passengers, the image data of elevator passengers at least includes pictures of squatting, falling, head extension, and hand extension; annotating the image data of elevator passengers, the annotation at least includes posture category, human frame, and human key point position, to obtain annotated image data; using the human posture detection algorithm and the annotated image data to train the model, to obtain the posture detection model; acquiring the escalator floor plate and step position through the image sensor, to obtain the first target list, the first target list at least includes the human frame position, human posture category, and human key point position; traversing the first target list, performing logical judgment on the relative position or angle of the key point, to obtain the second target list; performing multi-target tracking on the second target list, recording the time, id, and posture category of the target occurrence, to obtain the tracking list; traversing the tracking list, when the time of the target event exceeds the set value, it is determined that the event occurs; when the event is determined to be squatting or hand extension according to the occurrence of the event, the warning level is set to general, and when the event is falling or head extension, the warning level is set to severe. It can accurately detect abnormal behaviors of passengers in the elevator, such as squatting, falling, head extension, and hand extension. For serious situations such as falls, timely detection and measures can be taken to prevent passengers from suffering more serious injuries due to long-term lack of assistance. Reasonable warning levels are divided according to the degree of danger of different behaviors. By acquiring and analyzing elevator passenger image data, all-round real-time monitoring of passenger behavior in the elevator is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0065] Figure 1 This is a schematic diagram of a first embodiment of a method for detecting abnormal behavior of escalator passengers in an embodiment of the present invention;

[0066] Figure 2It is a schematic diagram of a second embodiment of a method for detecting abnormal behavior of escalator passengers in an embodiment of the present invention;

[0067] Figure 3 Schematic diagram of a third embodiment of a method for detecting abnormal behavior of escalator passengers in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0069] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0070] The present invention will be described in detail below in conjunction with the accompanying drawings. Figure 1 As shown, a method for detecting abnormal behavior of escalator passengers comprises the following steps:

[0071] Step 101, obtaining elevator passenger image data, the elevator passenger image data at least including squatting, falling, head extended, and hand extended pictures;

[0072] Step 102: label the elevator passenger image data, where the labeling includes at least posture category, human body frame and human body key point positions, to obtain labeled image data;

[0073] Step 103: training a model using a human posture detection algorithm and annotated image data to obtain a posture detection model;

[0074] Specifically, the human posture detection algorithm is the YOLO algorithm.

[0075] Step 104: Acquire the escalator floor plate and step positions through an image sensor to obtain a first target list, the first target list at least including a human frame position, a human posture category, and a human key point position;

[0076] Specifically, in this embodiment, the escalator floor plate and step position are obtained by an image sensor, and the detection device reads the video stream of the camera to obtain video stream image data; the video stream image data is read at a set time interval, and the video stream image data is decoded into a picture to obtain picture data; the picture data is inferred by a posture detection model, and the output of the inference includes confidence, posture category, human body frame position and human body key point position; the human body key point position includes at least the center point of the head, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, right hip, left knee, right knee, left foot and right foot; the targets with confidence greater than the threshold and within the set floor plate and step area are valid targets, and a target list is obtained.

[0077] Step 105, traverse the first target list, perform logical judgment on the relative positions or angles of the key points, and obtain a second target list;

[0078] Specifically, in this embodiment, the first target list is traversed, and a logical judgment is made on the relative positions or angles of the key points in the first target list; when the posture category is squatting, the position of the human body's knee key point must be higher than the position of the hip key point; when the posture category is falling, the foot key point must be higher than the hip key point or the head key point must be lower than the shoulder key point; when the posture is head extension, the angle between the head key point and the center line of the human body frame must be greater than the set angle; when the posture is hand extension, the distance between the hand and the center line of the human body frame must be greater than the set value, and the target is retained, otherwise the target is deleted to obtain the second target list.

[0079] Step 106: perform multi-target tracking on the second target list, record the time, ID and posture category of the target appearance, and obtain a tracking list;

[0080] Specifically, in this embodiment, multiple targets are tracked on the second target list, and the time, ID, and posture category of the target appearance are recorded; the target that is not detected in the set frame interval n is deleted from the tracking list; the frame interval for deleting the tracked target is set here to make the sensitivity of event detection adjustable; when it is set that 1 frame is not detected, the target in the list is deleted, if the target frame is not detected continuously, it is deleted, and the sensitivity of event detection is low at this time; on the contrary, when the set frame interval number is large, the system is allowed to consider that the target continues to exist when the target is not detected in the frame interval -1 frame, the sensitivity of event detection is high, and a tracking list is obtained.

[0081] Step 107, traverse the tracking list, and when the time when the target event occurs exceeds the set value, it is determined that the event occurs;

[0082] Specifically, in this embodiment, the tracking list is traversed, and when the time of the target event exceeds the set value, it is determined that the event has occurred; the target event includes at least squatting, falling, head extension, and hand extension; the set value is used to measure the duration of the event; when the tracking list is empty, it indicates that no event has occurred at present. If the remote speed reduction flag is 1, a speed recovery command is sent to the escalator. When the event is determined to be squatting or hand extension according to the event occurrence, the warning level is set to general; when the event is falling or head extension, the warning level is set to severe.

[0083] Specifically, in this embodiment, if the event is determined to be squatting or extending the hand, the warning level is set to general; when the event is a fall or the head is extended, the warning level is set to severe; the warning level is sent to the sound and light warning module as severe, and a voice warning is broadcast to remind passengers. At the same time, an alarm record is generated and uploaded to the platform; the management personnel view the event pictures and videos in real time on the platform, and after manual inspection and confirmation, remotely operate the escalator in emergency stop, slow stop, speed reduction, and speed recovery on the platform interface; when the event warning is severe, a speed reduction command is sent to the escalator to reduce the escalator running speed, and the remote speed reduction flag is set to 1.

[0084] The beneficial effect is that by acquiring the image data of elevator passengers, the image data of elevator passengers at least includes pictures of squatting, falling, head extension, and hand extension; marking the image data of elevator passengers, the marking at least includes posture category, human frame and human key point position, and obtains the marked image data; using the human posture detection algorithm and the labeled image data training model to obtain the posture detection model; acquiring the escalator floor plate and step position through the image sensor, obtaining the first target list, the first target list at least includes the human frame position, human posture category and human key point position; traversing the first target list, performing logical judgment on the relative position or angle of the key point, and obtaining the second target list; performing multi-target tracking on the second target list, recording the time, id and posture category of the target appearance, and obtaining the tracking list; traversing the tracking list, when the time of the target event appears exceeds the set value, it is determined that the event occurs; when the event is determined to be squatting or hand extension according to the event occurrence, the warning level is set to general, and when the event is falling or head extension, the warning level is set to severe. It can accurately detect abnormal behaviors of passengers in the elevator, such as squatting, falling, head extension, and hand extension. For serious situations such as falls, timely detection and measures can be taken to prevent passengers from suffering more serious injuries due to long-term lack of assistance. Reasonable warning levels are divided according to the degree of danger of different behaviors. By acquiring and analyzing elevator passenger image data, all-round real-time monitoring of passenger behavior in the elevator is achieved.

[0085] Its beneficial effect is that the target detection model can be used to output the category of human posture, combined with the key point location information for secondary verification, with low resource consumption and high event detection accuracy. At the same time, abnormal passenger behavior is graded for alarm. When general events occur, sound and light alarms, voice prompts, and pop-up window alarms are used to remind people. When serious events occur, in addition to alarm reminders, the escalator speed can be reduced to reduce casualties. The specific implementation steps are as follows:

[0086] See also Figure 2 In an escalator passenger abnormal behavior detection method, an escalator floor plate and step positions are obtained by an image sensor to obtain a first target list, the first target list at least including a human frame position, a human posture category and a human key point position, and the following steps are included:

[0087] Step 201: Acquire the escalator floor plate and step positions through an image sensor, and the detection device reads the video stream of the camera to obtain video stream image data;

[0088] Step 202: Read the video stream image data at a set time interval, and decode the video stream image data into a picture to obtain picture data;

[0089] Step 203: Use the posture detection model to infer the image data, and the output of the inference includes confidence, posture category, human frame position and human key point position;

[0090] Step 204, the key points of the human body at least include the center of the head, the left eye, the right eye, the left ear, the right ear, the left shoulder, the right shoulder, the left elbow, the right elbow, the left hand, the right hand, the left hip, the right hip, the left knee, the right knee, the left foot, and the right foot;

[0091] Step 205: Targets whose confidence level is greater than a threshold and are within the set floor plate and step area are considered valid targets, and a target list is obtained.

[0092] See also Figure 3 ,In an escalator passenger abnormal behavior detection method,

[0093] Collect pictures of squatting, falling, head sticking out, and hands sticking out. They can be pictures of real events happening in the escalator scene, or they can be staged pictures.

[0094] Use the material images from step 1 to annotate, including posture categories (standing, bending, squatting, falling, head extended, hands extended, etc.), human body frames, and human body key point locations.

[0095] Use the human posture detection algorithm and the material training model in step 2. During the application process, the model should continue to collect material pictures for iterative updates.

[0096] The camera is used to shoot the escalator floor and step positions on site, and the detection equipment reads the video stream of the camera. The video stream is read at the set time interval and decoded into pictures for reasoning. The output of model reasoning includes confidence, posture category, human frame position and human key point position (the key point position is the center of the head, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, right hip, left knee, right knee, left foot, right foot). Targets with confidence greater than the threshold and within the set floor and step area are considered valid. A list of qualified targets is obtained, including human frame position, human posture category, and human key point position.

[0097] Traverse the target list obtained in step 4 and make logical judgments on the relative position or angle of the key points. For example, when the posture category is squatting, the position of the human body's knee key point must be higher than the position of the hip key point. When the posture category is falling, the foot key point must be higher than the hip key point or the head key point must be lower than the shoulder key point. When the posture is the head extending, the angle between the head key point and the center line of the human frame must be greater than the set angle. When the posture is the hand extending, the distance between the hand and the center line of the human frame must be greater than the set value. Keep the target, otherwise delete the target, and get a list of all eligible targets. The purpose of this step is mainly to perform a secondary check on the results of model reasoning based on the relative position characteristics of the key points when the corresponding event occurs when observed by the human eye. After testing, some false detections can be effectively filtered out. At the same time, this part of the logic can be continuously optimized during on-site use.

[0098] Perform multi-target tracking on the results of step 5, record the time, ID, and posture category of the target, form a tracking list, and delete the target that is not detected in the set frame interval n from the tracking list. The purpose of setting the frame interval for deleting the tracked target here is to adjust the sensitivity of event detection. For example, when it is set that the target in the list is deleted when 1 frame is not detected, if the target frame is not detected continuously, it will be deleted, and the sensitivity of event detection is low at this time. On the contrary, when the set frame interval number is large, the system is allowed to consider that the target continues to exist when the frame interval -1 frame does not detect the target, and the sensitivity of event detection is higher.

[0099] Traverse the tracking list of step 6. When the target event (squatting, falling, head extension, hand extension) occurs for a time longer than the set value (this set value is used to measure the duration of the event), it is determined that the event has occurred. In particular, when the tracking list of step 6 is empty, it indicates that no event has occurred. If the remote speed reduction flag is 1, send a speed recovery command to the escalator.

[0100] Grading alarm: When step 7 determines that the event is squatting or hand extension, set the alarm level to general; when the event is a fall or head extension, set the alarm level to severe. Send to the sound and light alarm module, broadcast a voice alarm to remind passengers, and generate an alarm record and upload it to the platform. Managers can view event pictures and event videos on the platform in real time. After manual inspection and confirmation, they can remotely operate the escalator to emergency stop, slow stop, speed reduction, and speed recovery on the platform interface, and can also handle the event on site in time. Since falling or head extension may cause casualties to passengers, and since the incident happened suddenly, it is difficult to rescue in time on site. Therefore, when the event alarm is severe, send a deceleration command to the escalator to reduce the escalator running speed to reduce the possibility of casualties. Set the remote deceleration flag to 1.

[0101] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for detecting abnormal behavior of escalator passengers, characterized in that: The method for detecting abnormal passenger behavior comprises the following steps: Acquire passenger image data of an escalator scene, wherein the passenger image data of the escalator scene includes at least pictures of squatting, falling, extending the head, and extending the hands; Annotating the passenger image data of the escalator scene, wherein the annotation includes at least a posture category, a human body frame and a human body key point position, to obtain annotated image data; Using a human body posture detection algorithm and the labeled image data to train a model, to obtain a posture detection model; Acquire the escalator floor plate and step positions through an image sensor to obtain a first target list, wherein the first target list at least includes a human frame position, a human posture category, and a human key point position; Traversing the first target list, performing logical judgment on the relative positions or angles of key points, and obtaining a second target list; Perform multi-target tracking on the second target list, record the time, ID and posture category of the target appearance, and obtain a tracking list; The tracking list is traversed, and when the time for the target event to occur exceeds a set value, it is determined that the event has occurred.

2. The method for detecting abnormal behavior of escalator passengers according to claim 1, characterized in that: The escalator floor plate and step positions are acquired by an image sensor to obtain a first target list, wherein the first target list includes at least a human frame position, a human posture category, and a human key point position, including: The escalator floor plate and step positions are obtained through the image sensor, and the detection device reads the video stream of the camera to obtain the video stream image data; Read the video stream image data at a set time interval, and decode the video stream image data into a picture to obtain picture data; Performing reasoning on the image data using a posture detection model, wherein the output of the reasoning includes confidence, posture category, human frame position, and human key point position; The key points of the human body at least include the center of the head, the left eye, the right eye, the left ear, the right ear, the left shoulder, the right shoulder, the left elbow, the right elbow, the left hand, the right hand, the left hip, the right hip, the left knee, the right knee, the left foot, and the right foot; The targets whose confidence is greater than the threshold and are within the set floor plate and step area are valid targets, and a target list is obtained.

3. The method for detecting abnormal behavior of escalator passengers according to claim 1, characterized in that: The traversing the first target list and performing logical judgment on the relative positions or angles of the key points to obtain the second target list includes: Traversing the first target list, and performing logical judgment on the relative positions or angles of key points in the first target list; When the posture category is squatting, the knee key point position of the human body must be higher than the hip key point position; When the posture category is falling, the foot key point must be higher than the hip key point or the head key point must be lower than the shoulder key point; When the posture is head extended, the angle between the key point of the head and the middle line of the human frame must be greater than the set angle; When the gesture is that the hand is extended, the distance between the hand and the center line of the human body frame must be greater than the set value, and the target is retained; otherwise, the target is deleted to obtain the second target list.

4. The method for detecting abnormal behavior of escalator passengers according to claim 1, characterized in that: The performing multi-target tracking on the second target list, recording the time, ID and posture category of the target appearance, and obtaining a tracking list includes: Perform multi-target tracking on the second target list, and record the time, ID, and posture category of the target appearance; Targets that are not detected within the set frame interval n are deleted from the tracking list; The purpose of setting the frame interval for deleting the tracking target here is to make the sensitivity of event detection adjustable; When it is set that if one frame is not detected, the target in the list will be deleted. If the target frame is not detected continuously, it will be deleted. In this case, the sensitivity of event detection is low; On the contrary, when the set frame interval number is larger, the system is allowed to consider that the target continues to exist when the target is not detected in the frame interval -1 frame. The sensitivity of event detection is higher and a tracking list is obtained.

5. The method for detecting abnormal behavior of escalator passengers according to claim 1, characterized in that: The traversing the tracking list, when the time for the target event to occur exceeds a set value, determines that the event has occurred, includes: Traversing the tracking list, when the time when the target event occurs exceeds a set value, it is determined that the event occurs; The target events at least include squatting, falling, extending the head, and extending the hands; the set value is used to measure the duration of the event; When the tracking list is empty, it indicates that no event has occurred. If the remote speed reduction flag is 1, a speed recovery command is sent to the escalator; When the event is determined to be squatting or extending the hand according to the occurrence of the event, the warning level is set to general. When the event is falling or extending the head, the warning level is set to severe.

6. The method for detecting abnormal behavior of escalator passengers according to claim 5, characterized in that: When the event is determined to be squatting or hand extension, the warning level is set to general; when the event is falling or head extension, the warning level is set to severe, including: If the event is determined to be squatting or hand extension, the warning level is set to general; if the event is falling or head extension, the warning level is set to severe; The warning level is sent to the sound and light warning module as serious, and a voice warning is broadcast to remind passengers. At the same time, an alarm record is generated and uploaded to the platform; The management personnel can view the event pictures and videos in real time on the platform, and after manual inspection and confirmation, remotely operate the escalator to emergency stop, slow stop, speed reduction, and speed recovery on the platform interface; When the event warning is serious, a speed reduction command is sent to the escalator to reduce the escalator running speed and set the remote speed reduction flag to 1.

7. A system for implementing the method for detecting abnormal behavior of escalator passengers as claimed in claim 1, characterized in that: The system includes the following modules: A data acquisition module, used to acquire elevator passenger image data, wherein the elevator passenger image data at least includes pictures of squatting, falling, head sticking out, and hand sticking out; An image annotation module is used to annotate the elevator passenger image data, wherein the annotation includes at least a posture category, a human body frame and a human body key point position, to obtain annotated image data; A model training module, used to train a model using a human posture detection algorithm and the labeled image data to obtain a posture detection model; A target acquisition module, used to acquire the escalator floor plate and step positions through an image sensor to obtain a first target list, wherein the first target list at least includes a human frame position, a human posture category, and a human key point position; A logic judgment module, used for traversing the first target list, performing logic judgment on the relative positions or angles of key points, and obtaining a second target list; A target tracking module, used to track multiple targets in the second target list, record the time, ID and posture category of the target, and obtain a tracking list; An event determination module is used to traverse the tracking list and determine that an event occurs when the time when the target event occurs exceeds a set value; The event warning module is used to set the warning level to general when the event is determined to be squatting or hand extension according to the occurrence of the event, and to set the warning level to severe when the event is falling or head extension.

8. A system for implementing the method for detecting abnormal behavior of escalator passengers as claimed in claim 1, characterized in that: The system includes the following submodules: The acquisition submodule is used to obtain the escalator floor plate and step position through the image sensor, and the detection device reads the video stream of the camera to obtain the video stream image data; A decoding submodule, used for reading the video stream image data at a set time interval, and decoding the video stream image data into a picture to obtain picture data; An inference submodule, used to perform inference on the image data using a posture detection model, wherein the inference output includes confidence, posture category, human frame position, and human key point position; A key point submodule, used to determine the positions of the key points of the human body, including at least the center of the head, the left eye, the right eye, the left ear, the right ear, the left shoulder, the right shoulder, the left elbow, the right elbow, the left hand, the right hand, the left hip, the right hip, the left knee, the right knee, the left foot, and the right foot; The judgment submodule is used to determine the targets whose confidence is greater than the threshold and are within the set floor plate and step area as valid targets, and obtain a target list.

9. A system for implementing the method for detecting abnormal behavior of escalator passengers as claimed in claim 1, characterized in that: The system includes the following submodules: A traversal submodule, used for traversing the first target list and performing logical judgment on the relative positions or angles of key points in the first target list; The first judgment submodule is used to judge that when the posture category is squatting, the position of the knee key point of the human body must be higher than the position of the hip key point; The second judgment submodule is used to judge that when the posture category is falling, it must meet the requirements that the foot key point is higher than the hip key point or the head key point is lower than the shoulder key point; The third judgment submodule is used to judge that when the posture is the head extending, the angle between the key point of the head and the middle line of the human frame must be greater than the set angle; The fourth judgment submodule is used to judge that when the posture is the hand extending, the distance between the hand and the center line of the human body frame must be greater than a set value, and the target is retained; otherwise, the target is deleted to obtain a second target list.

Images